* chore: remove unused setting
* feat: initial import of iesg minutes
* fix: let the meetings view show older iesg meetings
* feat: iesg narrative minutes
* feat: import bof coordination call minutes
* wip: import commands for iesg appeals and statements
* feat: import iesg statements.
* feat: import iesg artifacts
* feat: many fewer n+1 queries for the group meetings view
* fix: restore chain of elifs in views_doc
* fix: use self.stdout.write vs print in mgmt commands
* fix: use replace instead of astimezone when appropriate
* chore: refactor new migrations into one
* fix: transcode some old files into utf8
* fix: repair overzealous replace
* chore: black
* fix: address minro review comments
* fix: actually capture transcoding work
* fix: handle multiple iesg statements on the same day
* fix: better titles
* feat: pill badge replaced statements
* fix: consolodate source repos to one
* feat: liberal markdown for secretariat controlled content
* fix: handle (and clean) html narrative minutes
* feat: scrub harder
* fix: simplify and improve a scrubber
* chore: reorder migrations
* feat: DraftAliasGenerator class
Encapsulates logic from generate_draft_aliases.py
* refactor: Avoid circular imports
* feat: Add draft_aliases API endpoint
* feat: Add @requires_api_token decorator
Stolen from feat/rpc-api
* feat: Add token auth to draft_aliases endpoint
* feat: draft-aliases-from-json.py script
Parses output from the draft_aliases API call
* chore: Remove unused cruft
* refactor: Avoid shadowing "draft" name
* fix: Suppress empty lists from DraftAliasGenerator
* refactor: Use a GET instead of POST
* feat: GroupAliasGenerator class
* feat: group aliases API view
* fix: Handle domains array correctly
* fix: Suppress empty group aliases
* refactor: Generalize aliases-from-json.py script
* refactor: Same output fmt for draft and group alias apis
* feat: Sort addresses for stability
* fix: Add "anything" virtual alias
* test: Test requires_api_token decorator
* feat: Harden is_valid_token against misconfig
* test: Test is_valid_token
* test: Test draft_aliases view
* test: Test group_aliases view
* test: Test DraftAliasGenerator
* fix: ise group is type "ise" in test data
* test: Fix logic in testManagementCommand
The test was incorrect - and fails when fixed. :-(
* test: Test GroupAliasGenerator
Test currently fails
* fix: Suppress empty -ads alias
* test: Fix group acronym copy/paste error
I *think* this must be what had been intended. The
code does not look like it ever dealt with GroupHistory,
so I'm pretty sure it wasn't meant to have the same
acronym used by two different Groups at different
times.
* test: Check draft .notify alias generation
* test: Cover get_draft_notify_emails()
* refactor: change references from User to Person (#5821)
* refactor: Change CommunityList reference from User to Person
* refactor: Convert more user references to person
* refactor: Change augment_docs_and_user_with_user_info to person
* refactor: Change Nomination and Feedback references from User to Person
* refactor: Change a few test case function signatures to be more pythonic
* refactor: Harmonize how profile and photo views look up email_or_name
* refactor: Rework community views to operate on Person instead of User (#5859)
* test: Update tests to try all of the person's emails and aliases
* fix: Recode a test case to avoid an exception if there's Unicode in the URL
This only happens using the form-filling and submission feature of
WebTest, which is only used in this one test case, so just it rip out.
* test: Add duplicate-person tests
* fix: If there are multiple matching users, prefer the logged-in one.
* chore: We no longer use WebTest, so don't include it.
* fix: Address review comments
* fix: case-insensitive person name or email matching (#6096)
* chore: Renumber migrations
* fix: Update merged code so tests pass (#6887)
* fix: Use refactored method
* fix: Don't assume user has person
* fix: Use new view param name
* chore: Drop community lists w/o person; cleanup (#6896)
* fix: Don't assume user has person
* fix: user->person in update_community_list_index.py
* feat: Remove CommunityLists without Person
* refactor: Speed up nomcom migrations
---------
Co-authored-by: Paul Selkirk <paul@painless-security.com>
Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
* fix: don't create a volunteer object when a person has already volunteered
* fix: safer create
* refactor: use shorthand from django
* fix: also protect volunteer from datatracker from race
* feat: Capture volunteers from the registration system. Fixes#5938
* fix: allow withdrawn field to be null
* docs: add explanatory code comments
* fix: identify current nomcom by is_accepting_volunteers
* fix: use auto_now_add instead of auto_now
* fix: update migration
---------
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
* refactor: Remove is_rfc() - test type_id instead
* fix: Guard against unknown pub_date
This should not ever come up - we have a published_rfc event for
every rfc. Should investigate fixing pub_date() to always return a val.
* feat: Use session.id to specify session for api_set_session_video_url
* feat: Use session.id to specify session for api_upload_bluesheet
* refactor: Add audio/video stream and onsite tool URLs to Session model
* refactor: Get onsite tool/stream URLs for agenda from Session
* refactor: Use Session methods for onsite tool/stream a few more places
* refactor: Move hard-coded meetecho URLs into settings.py
* feat: Add has_onsite_flag to Session
* chore: Set has_onsite_tool for sessions that had meetecho UrlResources
* fix: Only show onsite tool URLs when Session.has_onsite_tool is True
* test: Update test_api_upload_bluesheet to test deprecated version
* fix: Fix test failure in api_upload_bluesheet view
* test: Add test of new api_upload_bluesheet view
* style: Apply Black style to test_api_upload_bluesheet
* fix: Fix test failures in api_upload_bluesheet()
* test: Update test_api_set_session_video_url to test deprecated version
* fix: Fix test failure in api_set_session_video_url view
* test: Add test of new api_set_session_video_url view
* style: Apply Black styling to new test
* fix: Fix test failures in api_set_session_video_url view
* test: Fix test_meeting_agenda; set has_onsite_tool in SessionFactory
* feat: Add has_onsite_tool to Session list in admin
* feat: Add has_onsite_tool flag to SessionDetailsForm
* feat: Add has_onsite_tool flag to sreq
* feat: Show has_onsite_tool flag on secr view for a submitted request
* feat: Only prompt for has_onsite_tool in sreq for non-wg type groups
* fix: Clean up styling of sreq view a bit
* chore: Renumber migrations
* refactor: remove import_audio_files() and related code
* refactor: move functions from proc_utils to meeting/utils
* refactor: remove secr/proceedings
* feat: insensitive username matching at django authentication
* feat: use iexact when using the User object manager
* fix: more places to ignore username case
* fix: remove unused management command
* fix: avoid get when probing for object existance
* fix: force lowercase new usernames in secr/rolodex
* fix: use explicit arguments when creating user
* chore: Use codespell to fix typos in code.
Second part of replacement of #4651
@rjsparks, I probably need to revert some things here, and I also
still need to add that new migration - how do I do that?
* Revert migrations
* Migrate "Whitelisted" to "Allowlisted"
* TEST_COVERAGE_MASTER_FILE -> TEST_COVERAGE_MAIN_FILE
* Fix permissions
* Add suggestions from @jennifer-richards
* feat: apis for attaching chatlogs and polls to session materials
* fix: anticipate becoming tzaware, and improve guard against attempts to provide docs for sessions that have no official timeslot assignment.
* fix: get chatlog upload to actually work
Modifications to several initial implementation decisions.
Updates to the fixtures.
* fix: test polls upload
Refactored test to reduce duplicate code
* fix: allow api keys to be created for the new endpoints
* feat: add ability to view chatlog and polls documents. Show links in session materials.
* fix: commit new template
* fix: typo in migration signatures
* feat: add main doc page handling for polls. Improve tests.
* feat: chat log vue component + embedded vue loader
* feat: render polls using Vue
* fix: address pug syntax review comments from Nick.
* fix: repair remaining mention of chat log from copymunging
* fix: use double-quotes in html attributes
* fix: provide missing choices update migration
* test: silence html validator empty attr warnings
* test: fix test_runner config
* fix: locate session when looking at a dochistory object for polls or chatlog
Co-authored-by: Nicolas Giard <github@ngpixel.com>
* refactor: replace datetime.now with timezone.now
* refactor: migrate model fields to use timezone.now as default
* refactor: replace datetime.today with timezone.now
datetime.datetime.today() is equivalent to datetime.datetime.now(); both
return a naive datetime with the current local time.
* refactor: rephrase datetime.now(tz) as timezone.now().astimezone(tz)
This is effectively the same, but is less likely to encourage accidental
use of naive datetimes.
* refactor: revert datetime.today() change to old migrations
* refactor: change a missed datetime.now to timezone.now
* chore: renumber timezone_now migration
* chore: renumber migrations
* ci: add Dockerfile and action to build celery worker image
* ci: build celery worker on push to jennifer/celery branch
* ci: also build celery worker for main branch
* ci: Add comment to celery Dockerfile
* chore: first stab at a celery/rabbitmq docker-compose
* feat: add celery configuration and test task / endpoint
* chore: run mq/celery containers for dev work
* chore: point to ghcr.io image for celery worker
* refactor: move XML parsing duties into XMLDraft
Move some PlaintextDraft methods into the Draft base class and
implement for the XMLDraft class. Use xml2rfc code from ietf.submit
as a model for the parsing.
This leaves some mismatch between the PlaintextDraft and the Draft
class spec for the get_author_list() method to be resolved.
* feat: add api_upload endpoint and beginnings of async processing
This adds an api_upload() that behaves analogously to the api_submit()
endpoint. Celery tasks to handle asynchronous processing are added but
are not yet functional enough to be useful.
* perf: index Submission table on submission_date
This substantially speeds up submission rate threshold checks.
* feat: remove existing files when accepting a new submission
After checking that a submission is not in progress, remove any files
in staging that have the same name/rev with any extension. This should
guard against stale files confusing the submission process if the
usual cleanup fails or is skipped for some reason.
* refactor: make clear that deduce_group() uses only the draft name
* refactor: extract only draft name/revision in clean() method
Minimizing the amount of validation done when accepting a file. The
data extraction will be moved to asynchronous processing.
* refactor: minimize checks and data extraction in api_upload() view
* ci: fix dockerfiles to match sandbox testing
* ci: tweak celery container docker-compose settings
* refactor: clean up Draft parsing API and usage
* remove get_draftname() from Draft api; set filename during init
* further XMLDraft work
- remember xml_version after parsing
- extract filename/revision during init
- comment out long broken get_abstract() method
* adjust form clean() method to use changed API
* feat: flesh out async submission processing
First basically working pass!
* feat: add state name for submission being validated asynchronously
* feat: cancel submissions that async processing can't handle
* refactor: simplify/consolidate async tasks and improve error handling
* feat: add api_submission_status endpoint
* refactor: return JSON from submission api endpoints
* refactor: reuse cancel_submission method
* refactor: clean up error reporting a bit
* feat: guard against cancellation of a submission while validating
Not bulletproof but should prevent
* feat: indicate that a submission is still being validated
* fix: do not delete submission files after creating them
* chore: remove debug statement
* test: add tests of the api_upload and api_submission_status endpoints
* test: add tests and stubs for async side of submission handling
* fix: gracefully handle (ignore) invalid IDs in async submit task
* test: test process_uploaded_submission method
* fix: fix failures of new tests
* refactor: fix type checker complaints
* test: test submission_status view of submission in "validating" state
* fix: fix up migrations
* fix: use the streamlined SubmissionBaseUploadForm for api_upload
* feat: show submission history event timestamp as mouse-over text
* fix: remove 'manual' as next state for 'validating' submission state
* refactor: share SubmissionBaseUploadForm code with Deprecated version
* fix: validate text submission title, update a couple comments
* chore: disable requirements updating when celery dev container starts
* feat: log traceback on unexpected error during submission processing
* feat: allow secretariat to cancel "validating" submission
* feat: indicate time since submission on the status page
* perf: check submission rate thresholds earlier when possible
No sense parsing details of a draft that is going to be dropped regardless
of those details!
* fix: create Submission before saving to reduce race condition window
* fix: call deduce_group() with filename
* refactor: remove code lint
* refactor: change the api_upload URL to api/submission
* docs: update submission API documentation
* test: add tests of api_submission's text draft consistency checks
* refactor: rename api_upload to api_submission to agree with new URL
* test: test API documentation and submission thresholds
* fix: fix a couple api_submission view renames missed in templates
* chore: use base image + add arm64 support
* ci: try to fix workflow_dispatch for celery worker
* ci: another attempt to fix workflow_dispatch
* ci: build celery image for submit-async branch
* ci: fix typo
* ci: publish celery worker to ghcr.io/painless-security
* ci: install python requirements in celery image
* ci: fix up requirements install on celery image
* chore: remove XML_LIBRARY references that crept back in
* feat: accept 'replaces' field in api_submission
* docs: update api_submission documentation
* fix: remove unused import
* test: test "replaces" validation for submission API
* test: test that "replaces" is set by api_submission
* feat: trap TERM to gracefully stop celery container
* chore: tweak celery/mq settings
* docs: update installation instructions
* ci: adjust paths that trigger celery worker image build
* ci: fix branches/repo names left over from dev
* ci: run manage.py check when initializing celery container
Driver here is applying the patches. Starting the celery workers
also invokes the check task, but this should cause a clearer failure
if something fails.
* docs: revise INSTALL instructions
* ci: pass filename to pip update in celery container
* docs: update INSTALL to include freezing pip versions
Will be used to coordinate package versions with the celery
container in production.
* docs: add explanation of frozen-requirements.txt
* ci: build image for sandbox deployment
* ci: add additional build trigger path
* docs: tweak INSTALL
* fix: change INSTALL process to stop datatracker before running migrations
* chore: use ietf.settings for manage.py check in celery container
* chore: set uid/gid for celery worker
* chore: create user/group in celery container if needed
* chore: tweak docker compose/init so celery container works in dev
* ci: build mq docker image
* fix: move rabbitmq.pid to writeable location
* fix: clear password when CELERY_PASSWORD is empty
Setting to an empty password is really not a good plan!
* chore: add shutdown debugging option to celery image
* chore: add django-celery-beat package
* chore: run "celery beat" in datatracker-celery image
* chore: fix docker image name
* feat: add task to cancel stale submissions
* test: test the cancel_stale_submissions task
* chore: make f-string with no interpolation a plain string
Co-authored-by: Nicolas Giard <github@ngpixel.com>
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
