* ci: add Dockerfile and action to build celery worker image
* ci: build celery worker on push to jennifer/celery branch
* ci: also build celery worker for main branch
* ci: Add comment to celery Dockerfile
* chore: first stab at a celery/rabbitmq docker-compose
* feat: add celery configuration and test task / endpoint
* chore: run mq/celery containers for dev work
* chore: point to ghcr.io image for celery worker
* refactor: move XML parsing duties into XMLDraft
Move some PlaintextDraft methods into the Draft base class and
implement for the XMLDraft class. Use xml2rfc code from ietf.submit
as a model for the parsing.
This leaves some mismatch between the PlaintextDraft and the Draft
class spec for the get_author_list() method to be resolved.
* feat: add api_upload endpoint and beginnings of async processing
This adds an api_upload() that behaves analogously to the api_submit()
endpoint. Celery tasks to handle asynchronous processing are added but
are not yet functional enough to be useful.
* perf: index Submission table on submission_date
This substantially speeds up submission rate threshold checks.
* feat: remove existing files when accepting a new submission
After checking that a submission is not in progress, remove any files
in staging that have the same name/rev with any extension. This should
guard against stale files confusing the submission process if the
usual cleanup fails or is skipped for some reason.
* refactor: make clear that deduce_group() uses only the draft name
* refactor: extract only draft name/revision in clean() method
Minimizing the amount of validation done when accepting a file. The
data extraction will be moved to asynchronous processing.
* refactor: minimize checks and data extraction in api_upload() view
* ci: fix dockerfiles to match sandbox testing
* ci: tweak celery container docker-compose settings
* refactor: clean up Draft parsing API and usage
* remove get_draftname() from Draft api; set filename during init
* further XMLDraft work
- remember xml_version after parsing
- extract filename/revision during init
- comment out long broken get_abstract() method
* adjust form clean() method to use changed API
* feat: flesh out async submission processing
First basically working pass!
* feat: add state name for submission being validated asynchronously
* feat: cancel submissions that async processing can't handle
* refactor: simplify/consolidate async tasks and improve error handling
* feat: add api_submission_status endpoint
* refactor: return JSON from submission api endpoints
* refactor: reuse cancel_submission method
* refactor: clean up error reporting a bit
* feat: guard against cancellation of a submission while validating
Not bulletproof but should prevent
* feat: indicate that a submission is still being validated
* fix: do not delete submission files after creating them
* chore: remove debug statement
* test: add tests of the api_upload and api_submission_status endpoints
* test: add tests and stubs for async side of submission handling
* fix: gracefully handle (ignore) invalid IDs in async submit task
* test: test process_uploaded_submission method
* fix: fix failures of new tests
* refactor: fix type checker complaints
* test: test submission_status view of submission in "validating" state
* fix: fix up migrations
* fix: use the streamlined SubmissionBaseUploadForm for api_upload
* feat: show submission history event timestamp as mouse-over text
* fix: remove 'manual' as next state for 'validating' submission state
* refactor: share SubmissionBaseUploadForm code with Deprecated version
* fix: validate text submission title, update a couple comments
* chore: disable requirements updating when celery dev container starts
* feat: log traceback on unexpected error during submission processing
* feat: allow secretariat to cancel "validating" submission
* feat: indicate time since submission on the status page
* perf: check submission rate thresholds earlier when possible
No sense parsing details of a draft that is going to be dropped regardless
of those details!
* fix: create Submission before saving to reduce race condition window
* fix: call deduce_group() with filename
* refactor: remove code lint
* refactor: change the api_upload URL to api/submission
* docs: update submission API documentation
* test: add tests of api_submission's text draft consistency checks
* refactor: rename api_upload to api_submission to agree with new URL
* test: test API documentation and submission thresholds
* fix: fix a couple api_submission view renames missed in templates
* chore: use base image + add arm64 support
* ci: try to fix workflow_dispatch for celery worker
* ci: another attempt to fix workflow_dispatch
* ci: build celery image for submit-async branch
* ci: fix typo
* ci: publish celery worker to ghcr.io/painless-security
* ci: install python requirements in celery image
* ci: fix up requirements install on celery image
* chore: remove XML_LIBRARY references that crept back in
* feat: accept 'replaces' field in api_submission
* docs: update api_submission documentation
* fix: remove unused import
* test: test "replaces" validation for submission API
* test: test that "replaces" is set by api_submission
* feat: trap TERM to gracefully stop celery container
* chore: tweak celery/mq settings
* docs: update installation instructions
* ci: adjust paths that trigger celery worker image build
* ci: fix branches/repo names left over from dev
* ci: run manage.py check when initializing celery container
Driver here is applying the patches. Starting the celery workers
also invokes the check task, but this should cause a clearer failure
if something fails.
* docs: revise INSTALL instructions
* ci: pass filename to pip update in celery container
* docs: update INSTALL to include freezing pip versions
Will be used to coordinate package versions with the celery
container in production.
* docs: add explanation of frozen-requirements.txt
* ci: build image for sandbox deployment
* ci: add additional build trigger path
* docs: tweak INSTALL
* fix: change INSTALL process to stop datatracker before running migrations
* chore: use ietf.settings for manage.py check in celery container
* chore: set uid/gid for celery worker
* chore: create user/group in celery container if needed
* chore: tweak docker compose/init so celery container works in dev
* ci: build mq docker image
* fix: move rabbitmq.pid to writeable location
* fix: clear password when CELERY_PASSWORD is empty
Setting to an empty password is really not a good plan!
* chore: add shutdown debugging option to celery image
* chore: add django-celery-beat package
* chore: run "celery beat" in datatracker-celery image
* chore: fix docker image name
* feat: add task to cancel stale submissions
* test: test the cancel_stale_submissions task
* chore: make f-string with no interpolation a plain string
Co-authored-by: Nicolas Giard <github@ngpixel.com>
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
* feat: Add new page as requested in #4242 to list all area directors
and their current workload. Include links to the specific dashboards
for each area director. This new page is in doc/ad/.
* feat: Add new page as requested in #4242 to list all area directors
and their current workload. Include links to the specific dashboards
for each area director. This new page is in doc/ad/.
* Fixed issues from the previous commit by renaming hash to get_hash.
* Making outer () to be non matching
* Fixed RFC Ed Queue Internet-Draft to RFC Ed Queue
* refactor: split the /ad view apart from the /ad/name view.
* fix: make the new template html valid.
* test: start building a test for the new view
* refactor: make the view testable and test it.
* chore: remove unneeded commented lines
* fix: avoid parenthsized-string-looks-like-tuple bug.
* fix: repair bad closing tag in template
Co-authored-by: Tero Kivinen <kivinen@iki.fi>
* feat: add field MeetingRegistration.checkedin to track onsite checkin
* feat: add handling of ticket_type, checkedin to get_meeting_registation_data. Fixes#3672
* Fix: Update timezone on the interim create page
The interim creation screen will set the timezone to the timezone of
the browser. Unfortunately, the UI element was never refreshed, so
it remained on its loaded default, thus telling the user the wrong
value.
This change adds a timezone widget refresh to the end of the function that
loads the browser's local timezone into the timezone value.
This change also adds code to update the 'Local timezone' help text
for the start time and end time input bozes to
call out the selected timezone specifically.
Fixes#3898
* Fix: Change var to const for code review
Remove "-date" from data-sort fields for all dates, so that all columns
are now much better aligned.
Note: did not find an opened issue related to this after a quick search.
* refactor: avoid using select2 data-* attrs on html elements
Using "data-ajax--url" shadows explicit configuration in our
select2.js wrapper. Use "data-select2-ajax-url" to avoid this.
Also add ability to omit the ajax setup entirely by returning
None from ajax_url().
* chore: hook up a flag to disable ajax for SearchablePersonsField
* refactor: send select2 prefetch data as array and allow config of min input length
* feat: only offer IAB/IESG members for bofreq responsible leadership
* test: area directors/IAB members should be options for bofreq responsible leaders
* test: update tests to match changes to SearchableField
* fix: clean up SearchablePersonsField breakage when searching by email address
* chore: finish incomplete comment
* fix: Allow searching for non-alphanumeric characters
Fixes#3675
@rjsparks, this avoids us needing to fork list.js. If you prefer a fork, let me
know.
* Add a TODO
* fix: Don't run `striptags` on event descriptions that are emailed
Remove the `textify` filter, which is now unused.
Fixes#3600
* Update ietf_filters.py
* Update notification_email.txt
* Fix: Add 'back' button instead of 'cancel' to session request page
The 'Cancel' button that goes back to the same page was confusing, so change it out for a 'Back' button instead.
Fixes#4007
* Fix: Change absolute target for going back with history.back()
This form can be included from multiple pages, which have differing
navigation for how to land on the form. Therefore, history.back()
is the single way to navigate back to the previous page.
* feat(nomcom): Allow nomcom chair to turn volunteer acceptance on and off
Fixes#4105
Also allows chair to set date of first call for volunteers. If this should be read-only and set by the code, let me know.
* fix(nomcom): Correct helptext for accepting volunteers
* fix: corrects the IESG evaluation record summary statement for documents where the IESG state is approved (announced or not).
* fix: corrects IESG evaluation record summary for approved docs
This is a correction to a previous fix that didn't pass all the tests
for some documents
* fix: corrects IESG evaluation record summary for approved docs
Corrects the previous fix by removing commented out code. Sigh. At least
these are free.
* feat: Add opengraph information to document pages
* Apply suggestions from code review
Co-authored-by: Mark Nottingham <mnot@mnot.net>
* Add IRTF and IAB logo and make card PNGs for all
* Use the cards and restore stuff that somehow got dropped
* Try and make Twitter use the "large image" card format
Co-authored-by: Mark Nottingham <mnot@mnot.net>
* feat: Render the document shepherd writeup templates at two new URL.
Those being `/doc/shepherdwriteuptemplate/group` and
`/doc/shepherdwriteuptemplate/individual`.
* Address review comments from @jennifer-richards
* Fixes
* Remove debug statement
* Make bleach sanitizer not strip the `start` attribute of `ol` tags
Also rearrange the code a bit
* Don't sanitize the `python_markdown` output, it destroys wanted formatting
* Restore bleach
* Don't bleach tag `id`s.
* feat: Use d3 for doc dependencies
* Interim commit
* Progress
* Progress
* Auto pan and zoom
* Arrows
* Remove graphviz and the code that uses it
* More graphviz-related changes
* Interim commit
* Move things into place
* Add test
* Final touches
* Make SVG work in Chrome
* Get the docs more similarly to how the group doc page does it
* Reindent
* Add ability to download the SVG, and use bs fonts.
* Follow @rjsparks' advice on how to compute the reference list
* Interim commit
* Add legend
* Speed up simulation
* Fix tooltips
* fix: escape a period in a new url regex
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
