* feat: django-rest-framework + Person/Email API (#8233)
* chore: djangorestframework -> requirements.txt
* chore: auth/perm/schema classes for drf
* chore: settings for drf and friends
* chore: comment that api/serializer.py is not DRF
* feat: URL router for DRF
* feat: simple api/v3/person/{id} endpoint
* fix: actually working demo endpoint
* chore: no auth for PersonViewSet
* ci: params in ci-run-tests.yml
* Revert "ci: params in ci-run-tests.yml"
This reverts commit 03808ddf94afe42b7382ddd3730959987389612b.
* feat: email addresses for person API
* feat: email update api (WIP)
* fix: working Email API endpoint
* chore: annotate address format in api schema
* chore: api adjustments
* feat: expose SpectacularAPIView
At least for now...
* chore: better schema_path_prefix
* feat: permissions for DRF API
* refactor: use permissions classes
* refactor: extract NewEmailForm validation for reuse
* refactor: ietfauth.validators module
* refactor: send new email conf req via helper
* feat: API call to issue new address request
* chore: move datatracker DRF api to /api/core/
* fix: unused import
* fix: lint
* test: drf URL names + API tests (#8248)
* refactor: better drf URL naming
* test: test person-detail view
* test: permissions
* test: add_email tests + stubs
* test: test email update
* test: test 404 vs 403
* fix: fix permissions
* test: test email partial update
* test: assert we have a nonexistent PK
* chore: disable DRF api for now
* chore: fix git inanity
* fix: lint
* test: disable tests of disabled code
* test: more lint
* feat: style admin site in dev mode
* refactor: eliminate base_site.html
* fix: remove debug
* fix: commit missing __init__.py
* refactor: make method static; fix tests
* refactor: move api init to AppConfig.ready()
Avoids interacting with the app registry before
it's ready.
* feat: IANA review email ingestor API
* refactor: Replace iana email api with generic one
* chore: Add type hint
* feat: Ingest ipr responses
* feat: Ingest nomcom feedback
* refactor: message -> msg
* fix: Typo
* feat: Send email on nomcom ingestion failure
* feat: Send email on IPR mail ingestion error
* feat: Check content type, handle more errs
* fix: drop additionalProperties: false
Unfortunately this does not mix well with
the conditional "year" property.
* test: Test ingest_email view
* Revert "test: Test ingest_email view"
This reverts commit e498022829f834a0d3cebcb0dafb0d5f5a5d162e.
* test: Test ingest_email view
* fix: pass new test
* test: Test ingest_review_email
* fix: Pass new test
* test: Test ipr ingest_response_email
* fix: pass new test
* test: test nomcom ingest_feedback_email
* chore: fix typo found in code reviw
* fix: De-lint
* feat: API to list role-holder addresses
* test: Test new API endpoint
* fix: role_holder_addresses gets own API token
* refactor: Move role_holder_addresses to ietf.api.views
* test: test for group.utils.role_holder_emails
* test: Clean up test_role_holder_addresses
* fix: Missed a change in urls.py
* refactor: Remove old view
* chore: Remove unused import
* chore: Remove unused import
* feat: DraftAliasGenerator class
Encapsulates logic from generate_draft_aliases.py
* refactor: Avoid circular imports
* feat: Add draft_aliases API endpoint
* feat: Add @requires_api_token decorator
Stolen from feat/rpc-api
* feat: Add token auth to draft_aliases endpoint
* feat: draft-aliases-from-json.py script
Parses output from the draft_aliases API call
* chore: Remove unused cruft
* refactor: Avoid shadowing "draft" name
* fix: Suppress empty lists from DraftAliasGenerator
* refactor: Use a GET instead of POST
* feat: GroupAliasGenerator class
* feat: group aliases API view
* fix: Handle domains array correctly
* fix: Suppress empty group aliases
* refactor: Generalize aliases-from-json.py script
* refactor: Same output fmt for draft and group alias apis
* feat: Sort addresses for stability
* fix: Add "anything" virtual alias
* test: Test requires_api_token decorator
* feat: Harden is_valid_token against misconfig
* test: Test is_valid_token
* test: Test draft_aliases view
* test: Test group_aliases view
* test: Test DraftAliasGenerator
* fix: ise group is type "ise" in test data
* test: Fix logic in testManagementCommand
The test was incorrect - and fails when fixed. :-(
* test: Test GroupAliasGenerator
Test currently fails
* fix: Suppress empty -ads alias
* test: Fix group acronym copy/paste error
I *think* this must be what had been intended. The
code does not look like it ever dealt with GroupHistory,
so I'm pretty sure it wasn't meant to have the same
acronym used by two different Groups at different
times.
* test: Check draft .notify alias generation
* test: Cover get_draft_notify_emails()
* refactor: remove import_audio_files() and related code
* refactor: move functions from proc_utils to meeting/utils
* refactor: remove secr/proceedings
* feat: apis for attaching chatlogs and polls to session materials
* fix: anticipate becoming tzaware, and improve guard against attempts to provide docs for sessions that have no official timeslot assignment.
* fix: get chatlog upload to actually work
Modifications to several initial implementation decisions.
Updates to the fixtures.
* fix: test polls upload
Refactored test to reduce duplicate code
* fix: allow api keys to be created for the new endpoints
* feat: add ability to view chatlog and polls documents. Show links in session materials.
* fix: commit new template
* fix: typo in migration signatures
* feat: add main doc page handling for polls. Improve tests.
* feat: chat log vue component + embedded vue loader
* feat: render polls using Vue
* fix: address pug syntax review comments from Nick.
* fix: repair remaining mention of chat log from copymunging
* fix: use double-quotes in html attributes
* fix: provide missing choices update migration
* test: silence html validator empty attr warnings
* test: fix test_runner config
* fix: locate session when looking at a dochistory object for polls or chatlog
Co-authored-by: Nicolas Giard <github@ngpixel.com>
* ci: add Dockerfile and action to build celery worker image
* ci: build celery worker on push to jennifer/celery branch
* ci: also build celery worker for main branch
* ci: Add comment to celery Dockerfile
* chore: first stab at a celery/rabbitmq docker-compose
* feat: add celery configuration and test task / endpoint
* chore: run mq/celery containers for dev work
* chore: point to ghcr.io image for celery worker
* refactor: move XML parsing duties into XMLDraft
Move some PlaintextDraft methods into the Draft base class and
implement for the XMLDraft class. Use xml2rfc code from ietf.submit
as a model for the parsing.
This leaves some mismatch between the PlaintextDraft and the Draft
class spec for the get_author_list() method to be resolved.
* feat: add api_upload endpoint and beginnings of async processing
This adds an api_upload() that behaves analogously to the api_submit()
endpoint. Celery tasks to handle asynchronous processing are added but
are not yet functional enough to be useful.
* perf: index Submission table on submission_date
This substantially speeds up submission rate threshold checks.
* feat: remove existing files when accepting a new submission
After checking that a submission is not in progress, remove any files
in staging that have the same name/rev with any extension. This should
guard against stale files confusing the submission process if the
usual cleanup fails or is skipped for some reason.
* refactor: make clear that deduce_group() uses only the draft name
* refactor: extract only draft name/revision in clean() method
Minimizing the amount of validation done when accepting a file. The
data extraction will be moved to asynchronous processing.
* refactor: minimize checks and data extraction in api_upload() view
* ci: fix dockerfiles to match sandbox testing
* ci: tweak celery container docker-compose settings
* refactor: clean up Draft parsing API and usage
* remove get_draftname() from Draft api; set filename during init
* further XMLDraft work
- remember xml_version after parsing
- extract filename/revision during init
- comment out long broken get_abstract() method
* adjust form clean() method to use changed API
* feat: flesh out async submission processing
First basically working pass!
* feat: add state name for submission being validated asynchronously
* feat: cancel submissions that async processing can't handle
* refactor: simplify/consolidate async tasks and improve error handling
* feat: add api_submission_status endpoint
* refactor: return JSON from submission api endpoints
* refactor: reuse cancel_submission method
* refactor: clean up error reporting a bit
* feat: guard against cancellation of a submission while validating
Not bulletproof but should prevent
* feat: indicate that a submission is still being validated
* fix: do not delete submission files after creating them
* chore: remove debug statement
* test: add tests of the api_upload and api_submission_status endpoints
* test: add tests and stubs for async side of submission handling
* fix: gracefully handle (ignore) invalid IDs in async submit task
* test: test process_uploaded_submission method
* fix: fix failures of new tests
* refactor: fix type checker complaints
* test: test submission_status view of submission in "validating" state
* fix: fix up migrations
* fix: use the streamlined SubmissionBaseUploadForm for api_upload
* feat: show submission history event timestamp as mouse-over text
* fix: remove 'manual' as next state for 'validating' submission state
* refactor: share SubmissionBaseUploadForm code with Deprecated version
* fix: validate text submission title, update a couple comments
* chore: disable requirements updating when celery dev container starts
* feat: log traceback on unexpected error during submission processing
* feat: allow secretariat to cancel "validating" submission
* feat: indicate time since submission on the status page
* perf: check submission rate thresholds earlier when possible
No sense parsing details of a draft that is going to be dropped regardless
of those details!
* fix: create Submission before saving to reduce race condition window
* fix: call deduce_group() with filename
* refactor: remove code lint
* refactor: change the api_upload URL to api/submission
* docs: update submission API documentation
* test: add tests of api_submission's text draft consistency checks
* refactor: rename api_upload to api_submission to agree with new URL
* test: test API documentation and submission thresholds
* fix: fix a couple api_submission view renames missed in templates
* chore: use base image + add arm64 support
* ci: try to fix workflow_dispatch for celery worker
* ci: another attempt to fix workflow_dispatch
* ci: build celery image for submit-async branch
* ci: fix typo
* ci: publish celery worker to ghcr.io/painless-security
* ci: install python requirements in celery image
* ci: fix up requirements install on celery image
* chore: remove XML_LIBRARY references that crept back in
* feat: accept 'replaces' field in api_submission
* docs: update api_submission documentation
* fix: remove unused import
* test: test "replaces" validation for submission API
* test: test that "replaces" is set by api_submission
* feat: trap TERM to gracefully stop celery container
* chore: tweak celery/mq settings
* docs: update installation instructions
* ci: adjust paths that trigger celery worker image build
* ci: fix branches/repo names left over from dev
* ci: run manage.py check when initializing celery container
Driver here is applying the patches. Starting the celery workers
also invokes the check task, but this should cause a clearer failure
if something fails.
* docs: revise INSTALL instructions
* ci: pass filename to pip update in celery container
* docs: update INSTALL to include freezing pip versions
Will be used to coordinate package versions with the celery
container in production.
* docs: add explanation of frozen-requirements.txt
* ci: build image for sandbox deployment
* ci: add additional build trigger path
* docs: tweak INSTALL
* fix: change INSTALL process to stop datatracker before running migrations
* chore: use ietf.settings for manage.py check in celery container
* chore: set uid/gid for celery worker
* chore: create user/group in celery container if needed
* chore: tweak docker compose/init so celery container works in dev
* ci: build mq docker image
* fix: move rabbitmq.pid to writeable location
* fix: clear password when CELERY_PASSWORD is empty
Setting to an empty password is really not a good plan!
* chore: add shutdown debugging option to celery image
* chore: add django-celery-beat package
* chore: run "celery beat" in datatracker-celery image
* chore: fix docker image name
* feat: add task to cancel stale submissions
* test: test the cancel_stale_submissions task
* chore: make f-string with no interpolation a plain string
Co-authored-by: Nicolas Giard <github@ngpixel.com>
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
* feat: add model to track session attendance
* feat: add model to track session attendance
* feat: add api to set session attendees
* fix: use user pk instead off person pk in the attended api.
* feat: calculate three of five from attended
* feat: management utility to populate Attended model history
* docs: document why nomcom calculations don't use Attended yet.
* fix: add migration to add new personalapikey endpoint to choices
* test: verify very old last login prevents api key use,
* chore: address review nits
* chore: comment on some idiosyncracies of the expected input to populate_attended
* fix: add unique_together constraint for the Attended model
* fix: correctly handle empty querysets passed to three_of_five_eligible functions.
Added new custom API endpoints for export of person data, restricted
to secretariat use. Added a test for the new custom API.
- Legacy-Id: 15543
Note: SVN reference [15263] has been migrated to Git commit 8e7e0faf529db057ac4f7496d0bc84fb05f0625d
Note: SVN reference [15264] has been migrated to Git commit e6549635650d4d9f9a9de7c5b4711f5a4a25c42c
Note: SVN reference [15265] has been migrated to Git commit 7c0e97f824b874763550adf1647841129017bf5a
Note: SVN reference [15287] has been migrated to Git commit 7431bdfcd9f2213c812a9004162ca576ae230a94
