* chore: Use codespell to fix typos in code.
Second part of replacement of #4651
@rjsparks, I probably need to revert some things here, and I also
still need to add that new migration - how do I do that?
* Revert migrations
* Migrate "Whitelisted" to "Allowlisted"
* TEST_COVERAGE_MASTER_FILE -> TEST_COVERAGE_MAIN_FILE
* Fix permissions
* Add suggestions from @jennifer-richards
* feat: apis for attaching chatlogs and polls to session materials
* fix: anticipate becoming tzaware, and improve guard against attempts to provide docs for sessions that have no official timeslot assignment.
* fix: get chatlog upload to actually work
Modifications to several initial implementation decisions.
Updates to the fixtures.
* fix: test polls upload
Refactored test to reduce duplicate code
* fix: allow api keys to be created for the new endpoints
* feat: add ability to view chatlog and polls documents. Show links in session materials.
* fix: commit new template
* fix: typo in migration signatures
* feat: add main doc page handling for polls. Improve tests.
* feat: chat log vue component + embedded vue loader
* feat: render polls using Vue
* fix: address pug syntax review comments from Nick.
* fix: repair remaining mention of chat log from copymunging
* fix: use double-quotes in html attributes
* fix: provide missing choices update migration
* test: silence html validator empty attr warnings
* test: fix test_runner config
* fix: locate session when looking at a dochistory object for polls or chatlog
Co-authored-by: Nicolas Giard <github@ngpixel.com>
* ci: add Dockerfile and action to build celery worker image
* ci: build celery worker on push to jennifer/celery branch
* ci: also build celery worker for main branch
* ci: Add comment to celery Dockerfile
* chore: first stab at a celery/rabbitmq docker-compose
* feat: add celery configuration and test task / endpoint
* chore: run mq/celery containers for dev work
* chore: point to ghcr.io image for celery worker
* refactor: move XML parsing duties into XMLDraft
Move some PlaintextDraft methods into the Draft base class and
implement for the XMLDraft class. Use xml2rfc code from ietf.submit
as a model for the parsing.
This leaves some mismatch between the PlaintextDraft and the Draft
class spec for the get_author_list() method to be resolved.
* feat: add api_upload endpoint and beginnings of async processing
This adds an api_upload() that behaves analogously to the api_submit()
endpoint. Celery tasks to handle asynchronous processing are added but
are not yet functional enough to be useful.
* perf: index Submission table on submission_date
This substantially speeds up submission rate threshold checks.
* feat: remove existing files when accepting a new submission
After checking that a submission is not in progress, remove any files
in staging that have the same name/rev with any extension. This should
guard against stale files confusing the submission process if the
usual cleanup fails or is skipped for some reason.
* refactor: make clear that deduce_group() uses only the draft name
* refactor: extract only draft name/revision in clean() method
Minimizing the amount of validation done when accepting a file. The
data extraction will be moved to asynchronous processing.
* refactor: minimize checks and data extraction in api_upload() view
* ci: fix dockerfiles to match sandbox testing
* ci: tweak celery container docker-compose settings
* refactor: clean up Draft parsing API and usage
* remove get_draftname() from Draft api; set filename during init
* further XMLDraft work
- remember xml_version after parsing
- extract filename/revision during init
- comment out long broken get_abstract() method
* adjust form clean() method to use changed API
* feat: flesh out async submission processing
First basically working pass!
* feat: add state name for submission being validated asynchronously
* feat: cancel submissions that async processing can't handle
* refactor: simplify/consolidate async tasks and improve error handling
* feat: add api_submission_status endpoint
* refactor: return JSON from submission api endpoints
* refactor: reuse cancel_submission method
* refactor: clean up error reporting a bit
* feat: guard against cancellation of a submission while validating
Not bulletproof but should prevent
* feat: indicate that a submission is still being validated
* fix: do not delete submission files after creating them
* chore: remove debug statement
* test: add tests of the api_upload and api_submission_status endpoints
* test: add tests and stubs for async side of submission handling
* fix: gracefully handle (ignore) invalid IDs in async submit task
* test: test process_uploaded_submission method
* fix: fix failures of new tests
* refactor: fix type checker complaints
* test: test submission_status view of submission in "validating" state
* fix: fix up migrations
* fix: use the streamlined SubmissionBaseUploadForm for api_upload
* feat: show submission history event timestamp as mouse-over text
* fix: remove 'manual' as next state for 'validating' submission state
* refactor: share SubmissionBaseUploadForm code with Deprecated version
* fix: validate text submission title, update a couple comments
* chore: disable requirements updating when celery dev container starts
* feat: log traceback on unexpected error during submission processing
* feat: allow secretariat to cancel "validating" submission
* feat: indicate time since submission on the status page
* perf: check submission rate thresholds earlier when possible
No sense parsing details of a draft that is going to be dropped regardless
of those details!
* fix: create Submission before saving to reduce race condition window
* fix: call deduce_group() with filename
* refactor: remove code lint
* refactor: change the api_upload URL to api/submission
* docs: update submission API documentation
* test: add tests of api_submission's text draft consistency checks
* refactor: rename api_upload to api_submission to agree with new URL
* test: test API documentation and submission thresholds
* fix: fix a couple api_submission view renames missed in templates
* chore: use base image + add arm64 support
* ci: try to fix workflow_dispatch for celery worker
* ci: another attempt to fix workflow_dispatch
* ci: build celery image for submit-async branch
* ci: fix typo
* ci: publish celery worker to ghcr.io/painless-security
* ci: install python requirements in celery image
* ci: fix up requirements install on celery image
* chore: remove XML_LIBRARY references that crept back in
* feat: accept 'replaces' field in api_submission
* docs: update api_submission documentation
* fix: remove unused import
* test: test "replaces" validation for submission API
* test: test that "replaces" is set by api_submission
* feat: trap TERM to gracefully stop celery container
* chore: tweak celery/mq settings
* docs: update installation instructions
* ci: adjust paths that trigger celery worker image build
* ci: fix branches/repo names left over from dev
* ci: run manage.py check when initializing celery container
Driver here is applying the patches. Starting the celery workers
also invokes the check task, but this should cause a clearer failure
if something fails.
* docs: revise INSTALL instructions
* ci: pass filename to pip update in celery container
* docs: update INSTALL to include freezing pip versions
Will be used to coordinate package versions with the celery
container in production.
* docs: add explanation of frozen-requirements.txt
* ci: build image for sandbox deployment
* ci: add additional build trigger path
* docs: tweak INSTALL
* fix: change INSTALL process to stop datatracker before running migrations
* chore: use ietf.settings for manage.py check in celery container
* chore: set uid/gid for celery worker
* chore: create user/group in celery container if needed
* chore: tweak docker compose/init so celery container works in dev
* ci: build mq docker image
* fix: move rabbitmq.pid to writeable location
* fix: clear password when CELERY_PASSWORD is empty
Setting to an empty password is really not a good plan!
* chore: add shutdown debugging option to celery image
* chore: add django-celery-beat package
* chore: run "celery beat" in datatracker-celery image
* chore: fix docker image name
* feat: add task to cancel stale submissions
* test: test the cancel_stale_submissions task
* chore: make f-string with no interpolation a plain string
Co-authored-by: Nicolas Giard <github@ngpixel.com>
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
* fix: only use non-empty Q object as interim group filter
* refactor: add with_meetings queryset to GroupManager
* test: users can only request interims for managed groups
* fix: find managed groups from groupman_roles/authroles
* feat: let chair manage directorate groups
* test: remove debug statements and unused imports
* test: remove do-nothing code from test
* feat: restructure rfcedtype groups. add rpc, rswg, rsab.
* feat: show rfc editor groups on menu
* feat: add the editorial stream
* fix: repair broken html and extent dest coverage.
* chore: address Lars review comment
* chore: update copyright notices.
There is definitely still quite some breakage, my main focus during
the conflict resolution was not to drop any changes made by others.
- Legacy-Id: 19632
Add IAB Administrative Support Group group type. Add IAB groups to the group menu. Clearly delineate group types in the group menu. Fixes#3295 and #3296.
- Legacy-Id: 19328
Note: SVN reference [19307] has been migrated to Git commit d42730300a
Add ConstraintNames for chair, tech, and key participant conflicts. Replace temporary UI workaround with proper conflict type handling. Fixes#3083.
- Legacy-Id: 19123
Note: SVN reference [19103] has been migrated to Git commit 66b9c41dcc
Improve handling of submissions for closed working groups. Fixes#3058.
- Legacy-Id: 18807
Note: SVN reference [18798] has been migrated to Git commit 233bff8196
Email requesters when an interim moves into the announcement queue. Partially addresses #3016 and #3099.
- Legacy-Id: 18630
Note: SVN reference [18616] has been migrated to Git commit ff3f3bd99a
Track slide submissions even after acceptance or rejection. Fixes#2835.
- Legacy-Id: 18421
Note: SVN reference [18371] has been migrated to Git commit 21ba67fb79
Send extra mail to yangdoctors chairs when a draft with yang checks goes into IETF LC. Fixes#2419.
- Legacy-Id: 18363
Note: SVN reference [18324] has been migrated to Git commit cb58810705
Store Auth48 URL as DocumentURL and display in RFC-Editor state. Migrates old data when possible. Alternative to 17563. Fixes#2722.
- Legacy-Id: 18166
Note: SVN reference [18157] has been migrated to Git commit fff927b085
session when rescheduling a session after the schedule is made the
official meeting schedule.
Show both cancelled and rescheduled sessions as tombstones in the new
meeting schedule editor.
Add support for showing rescheduled tombstones in the meeting agenda
views.
Adjust the Secretariat session tool so that it's not possible to
(re)cancel cancelled or rescheduled tombstones.
- Legacy-Id: 18108
references directly in the database as HTML.
Rework the label update migration to take this into account (can be
rerun by running "ietf/manage.py migrate name 0011 --fake" first).
Add a little bit of support for the recently added constraints types -
the JS does not hint about them, but they do show up without looking
silly.
- Legacy-Id: 18033
Add used_roles to Group and default_used_roles to GroupFeatures. Dynamically build the group edit form to provide a field for all used_roles. Fixes#2785 and #2572. Partially addresses #2160.
- Legacy-Id: 17960
Note: SVN reference [17908] has been migrated to Git commit 2862727d02
Allow IAB programs to use normal meeting mechanics. Fixes#2970.
- Legacy-Id: 17773
Note: SVN reference [17764] has been migrated to Git commit fb8e5c6842
Turned autoescape off for another email fragment template (production database has also been changed). Fixes an issue where test_assign_reviewer() would sometimes fail due to html-escaped characters in email text.
- Legacy-Id: 17745
Note: SVN reference [17711] has been migrated to Git commit 8ec9e1cf77
upcoming automatic scheduling assistance:
. Added a management command to create a dummy IETF 999 meeting.
. Added display of new constraints and joint sessions to agenda builder
interface.
. The new timerange, time_relation and wg_adjacent constraints, along with
the joint_with_groups option, are now reflected in the special requests
field. This allows them to be taken into account while scheduling
sessions.
. Clarified the wording in the session request form regarding conflicts with
BOFs.
. Added support for structured entry and storage of joint sessions in
meetings:
- Also adds additional tests for the SessionForm
- Fixes a javascript error in session requests for non-WG groups,
that could cause incorrect form behaviour.
- Expands the tests added in [17289] a bit.
. Added support for the timerange, wg_adjacent and time_relation constraints.
This adds three new constraints to the database and relevant UIs:
- timerange: "This WG can't meet during these timeframes"
- wg_adjacent: "Schedule adjacent to another WG (directly following,
no breaks, same room)"
- time_relation: schedule the two sessions of one WG on subsequent
days or with at least one day seperation
- Legacy-Id: 17605
Note: SVN reference [17289] has been migrated to Git commit a227813dc5
for displaying violated constraints in the new schedule editor, with
the old of a new field, ConstraintName.editor_label.
Add support for displaying room capacity violations.
Add support for selecting a session and displaying information about
it similar to the existing scheduling editor.
Add support for sorting unassigned sessions.
Clean up markup and styles a bit, and fix some bugs.
Expand HTML-based test and add JS test that exercises the
Javascript-based functionality.
Switch to using Chrome driver instead of PhantomJS since the HTML
engine in PhantomJS is apparently too old to support the constructs in
the new schema editor.
Add a workaround for LiveServerTestCase clashing with IetfTestRunner
fixture loading.
- Legacy-Id: 17519
