* feat: basic blobstore infrastructure for dev
* refactor: (broken) attempt to put minio console behind nginx
* feat: initialize blobstore with boto3
* fix: abandon attempt to proxy minio. Use docker compose instead.
* feat: beginning of blob writes
* feat: storage utilities
* feat: test buckets
* chore: black
* chore: remove unused import
* chore: avoid f string when not needed
* fix: inform all settings files about blobstores
* fix: declare types for some settings
* ci: point to new target base
* ci: adjust test workflow
* fix: give the tests debug environment a blobstore
* fix: "better" name declarations
* ci: use devblobstore container
* chore: identify places to write to blobstorage
* chore: remove unreachable code
* feat: store materials
* feat: store statements
* feat: store status changes
* feat: store liaison attachments
* feat: store agendas provided with Interim session requests
* chore: capture TODOs
* feat: store polls and chatlogs
* chore: remove unneeded TODO
* feat: store drafts on submit and post
* fix: handle storage during doc expiration and resurrection
* fix: mirror an unlink
* chore: add/refine TODOs
* feat: store slide submissions
* fix: structure slide test correctly
* fix: correct sense of existence check
* feat: store some indexes
* feat: BlobShadowFileSystemStorage
* feat: shadow floorplans / host logos to the blob
* chore: remove unused import
* feat: strip path from blob shadow names
* feat: shadow photos / thumbs
* refactor: combine photo and photothumb blob kinds
The photos / thumbs were already dropped in the same
directory, so let's not add a distinction at this point.
* style: whitespace
* refactor: use kwargs consistently
* chore: migrations
* refactor: better deconstruct(); rebuild migrations
* fix: use new class in mack patch
* chore: add TODO
* feat: store group index documents
* chore: identify more TODO
* feat: store reviews
* fix: repair merge
* chore: remove unnecessary TODO
* feat: StoredObject metadata
* fix: deburr some debugging code
* fix: only set the deleted timestamp once
* chore: correct typo
* fix: get_or_create vs get and test
* fix: avoid the questionable is_seekable helper
* chore: capture future design consideration
* chore: blob store cfg for k8s
* chore: black
* chore: copyright
* ci: bucket name prefix option + run Black
Adds/uses DATATRACKER_BLOB_STORE_BUCKET_PREFIX option. Other changes
are just Black styling.
* ci: fix typo in bucket name expression
* chore: parameters in app-configure-blobstore
Allows use with other blob stores.
* ci: remove verify=False option
* fix: don't return value from __init__
* feat: option to log timing of S3Storage calls
* chore: units
* fix: deleted->null when storing a file
* style: Black
* feat: log as JSON; refactor to share code; handle exceptions
* ci: add ietf_log_blob_timing option for k8s
* test: --no-manage-blobstore option for running tests
* test: use blob store settings from env, if set
* test: actually set a couple more storage opts
* feat: offswitch (#8541)
* feat: offswitch
* fix: apply ENABLE_BLOBSTORAGE to BlobShadowFileSystemStorage behavior
* chore: log timing of blob reads
* chore: import Config from botocore.config
* chore(deps): import boto3-stubs / botocore
botocore is implicitly imported, but make it explicit
since we refer to it directly
* chore: drop type annotation that mypy loudly ignores
* refactor: add storage methods via mixin
Shares code between Document and DocHistory without
putting it in the base DocumentInfo class, which
lacks the name field. Also makes mypy happy.
* feat: add timeout / retry limit to boto client
* ci: let k8s config the timeouts via env
* chore: repair merge resolution typo
* chore: tweak settings imports
* chore: simplify k8s/settings_local.py imports
---------
Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
* feat: django-rest-framework + Person/Email API (#8233)
* chore: djangorestframework -> requirements.txt
* chore: auth/perm/schema classes for drf
* chore: settings for drf and friends
* chore: comment that api/serializer.py is not DRF
* feat: URL router for DRF
* feat: simple api/v3/person/{id} endpoint
* fix: actually working demo endpoint
* chore: no auth for PersonViewSet
* ci: params in ci-run-tests.yml
* Revert "ci: params in ci-run-tests.yml"
This reverts commit 03808ddf94afe42b7382ddd3730959987389612b.
* feat: email addresses for person API
* feat: email update api (WIP)
* fix: working Email API endpoint
* chore: annotate address format in api schema
* chore: api adjustments
* feat: expose SpectacularAPIView
At least for now...
* chore: better schema_path_prefix
* feat: permissions for DRF API
* refactor: use permissions classes
* refactor: extract NewEmailForm validation for reuse
* refactor: ietfauth.validators module
* refactor: send new email conf req via helper
* feat: API call to issue new address request
* chore: move datatracker DRF api to /api/core/
* fix: unused import
* fix: lint
* test: drf URL names + API tests (#8248)
* refactor: better drf URL naming
* test: test person-detail view
* test: permissions
* test: add_email tests + stubs
* test: test email update
* test: test 404 vs 403
* fix: fix permissions
* test: test email partial update
* test: assert we have a nonexistent PK
* chore: disable DRF api for now
* chore: fix git inanity
* fix: lint
* test: disable tests of disabled code
* test: more lint
* feat: Expose important library versions
Update `/api/version` to include
```
"other": {
"xml2rfc": "<version>",
"weasyprint": "<version>"
},
```
Fixes#3415
* fix: Use importlib
* chore: Reomve additional newline
* fix: Expose libraries that are important for document submission
* fix: Rename IMPORTANT_LIBRARIES as ADVERTISE_VERSIONS
* refactor: doc search via POST (WIP)
Changes the search view to use a POST instead of a GET. Refactors cache key computation to use cleaned data.
Still todo:
* refactor frontpage view to match
* refactor menubar search (?)
* refactor stats view that uses SearchForm
* revive or drop the "backwards compatibility" branch
* feat: convert GET search to POST
Still todo:
* refactor frontpage view to match
* refactor menubar search (?)
* refactor stats view that uses SearchForm
* fix: revert frontpage changes, search works
Still todo:
* refactor stats view that uses SearchForm
* fix: define vars in all branches
* refactor: update stats use of SearchForm
* chore: improve message
* fix: remove lint
* chore: comments re: QuerySetAny
* test: test query string search params
* style: Black
* test: refactor test_search()
* test: refactor test_search_became_rfc()
* test: use scroll_and_click helper
* Status WIP
* feat: Status
* fix: Status tests
* feat: status redirect
* chore: Status tests
* chore: Status tests
* feat: Status tests
* chore: Status playwright tests
* fix: PR feedback, mostly Vue and copyright dates
* fix: Status model migration tidy up
* chore: Status - one migration
* feat: status on doc/html pages
* chore: Resetting Status migration
* chore: removing unused FieldError
* fix: Update Status test to remove 'by'
* chore: fixing API test to exclude 'status'
* chore: fixing status_page test
* feat: Site Status PR feedback. URL coverage debugging
* Adding ietf.status to Tastypie omitted apps
* feat: Site Status PR feedback
* chore: correct copyright year on newly created files
* chore: repair merge damage
* chore: repair more merge damage
* fix: reconcile the api init refactor with ignoring apps
---------
Co-authored-by: Matthew Holloway <Matthew Holloway>
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
* feat: style admin site in dev mode
* refactor: eliminate base_site.html
* fix: remove debug
* fix: commit missing __init__.py
* refactor: make method static; fix tests
* refactor: move api init to AppConfig.ready()
Avoids interacting with the app registry before
it's ready.
* feat: IANA review email ingestor API
* refactor: Replace iana email api with generic one
* chore: Add type hint
* feat: Ingest ipr responses
* feat: Ingest nomcom feedback
* refactor: message -> msg
* fix: Typo
* feat: Send email on nomcom ingestion failure
* feat: Send email on IPR mail ingestion error
* feat: Check content type, handle more errs
* fix: drop additionalProperties: false
Unfortunately this does not mix well with
the conditional "year" property.
* test: Test ingest_email view
* Revert "test: Test ingest_email view"
This reverts commit e498022829f834a0d3cebcb0dafb0d5f5a5d162e.
* test: Test ingest_email view
* fix: pass new test
* test: Test ingest_review_email
* fix: Pass new test
* test: Test ipr ingest_response_email
* fix: pass new test
* test: test nomcom ingest_feedback_email
* chore: fix typo found in code reviw
* fix: De-lint
* feat: API to list role-holder addresses
* test: Test new API endpoint
* fix: role_holder_addresses gets own API token
* refactor: Move role_holder_addresses to ietf.api.views
* test: test for group.utils.role_holder_emails
* test: Clean up test_role_holder_addresses
* fix: Missed a change in urls.py
* refactor: Remove old view
* chore: Remove unused import
* chore: Remove unused import
* feat: Show bluesheets using Attended tables (#7094)
* feat: Show bluesheets using Attended tables (#6898)
* feat: Allow users to add themselves to session attendance (#6454)
* chore: Correct copyright year
* fix: Address review comments
* fix: Don't try to generate empty bluesheets
* refactor: Complete rewrite of bluesheet.html
* refactor: Fill in a few gaps, close a few holes
- Rename the live "bluesheet" to "attendance", add some explanatory text.
- Add attendance links in materials view and pre-finalized proceedings view.
- Don't allow users to add themselves after the corrections cutoff date.
* fix: Report file-save errors to caller
* fix: Address review comments
* fix: typo
* refactor: if instead of except; refactor gently
* refactor: Rearrange logic a little, add comment
* style: Black
* refactor: auto_now_add->default to allow override
* refactor: jsonschema to validate API payload
* feat: Handle new API data format
Not yet tested except that it falls back when the old
format is used.
* test: Split test into deprecated/new version
Have not yet touched the new version
* style: Black
* test: Test new add_session_attendees API
* fix: Fix bug uncovered by test
* refactor: Refactor affiliation lookup a bit
* fix: Order bluesheet by Attended.time
* refactor: Move helpers from views.py to utils.py
* test: Test that finalize calls generate_bluesheets
* test: test_bluesheet_data()
* fix: Clean up merge
* fix: Remove debug statement
* chore: comments
* refactor: Renumber migrations
---------
Co-authored-by: Paul Selkirk <paul@painless-security.com>
* chore: Remove unused import
* style: Black
* feat: Stub session update notify API
* feat: Add order & rev to slides JSON
* style: Black
* feat: Stub actual Meetecho slide deck mgmt API
* refactor: Limit reordering to type="slides"
* chore: Remove repository from meetecho API
(API changed on their end)
* feat: update Meetecho on slide reorder
* refactor: drop pytz from meetecho.py
* chore: Remove more repository refs
* refactor: Eliminate more pytz
* test: Test add_slide_deck api
* fix: Allow 202 status code / absent Content-Type
* test: Test delete_slide_deck api
* test: Test update_slide_decks api
* refactor: sessionpresentation_set -> presentations
* test: Test send_update()
* fix: Debug send_update()
* test: ajax_reorder_slides calls Meetecho API
* test: Test SldesManager.add()
* feat: Implement SlidesManager.add()
* test: Test that ajax_add_slides... calls API
* feat: Call Meetecho API when slides added to session
* test: Test SlidesManager.delete()
* feat: Implement SlidesManager.delete()
* test: ajax_remove_slides... calls Meetecho API
* feat: Call Meetecho API when slides removed
* chore: Update docstring
* feat: rudimentary debug mode for Meetecho API
* test: remove_sessionpresentation() calls Meetecho API
* feat: Call Meetecho API from remove_sessionpresentation()
* test: upload_slides() calls Meetecho API
* style: Black
* fix: Refactor/debug upload_session_slides
Avoids double-save of a SessionPresentation for the session
being updated and updates other sessions when apply_to_all
is set (previously it only created ones that did not exist,
so rev would never be updated).
* test: Fix test bug
* feat: Call Meetecho API when uploading session slides
* fix: Only replace slides actually linked to session
* fix: Delint
Removed some type checking rather than debugging it
* fix: Send get_versionless_href() as url for slides
* test: TZ-aware timestamps, please
* chore: Add comments
* feat: Call Meetecho API in edit_sessionpresentation
* feat: Call Meetecho API in remove_sessionpresentation
* feat: Call Meetecho API from add_sessionpresentation
* fix: Set order in add_sessionpresentation
* fix: Restrict API calls to "slides" docs
* feat: Call Meetecho API on title changes
* test: Check meetecho API calls in test_revise()
* fix: better Meetecho API "order" management
* fix: no PUT if there are no slides after DELETE
* feat: Catch exceptions from SlidesManager
Don't let errors in the MeetEcho slides API interfere with
the ability to modify slides for a session.
* feat: Limit which sessions we send notifications for
* fix: handle absence of request_timeout in api config
* test: always send slide notifications in tests
* fix: save slides before sending notification (#7172)
* fix: save slides before sending notification
* style: fix indentation
It's not a bug, it's a flourish!
---------
Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
Co-authored-by: Paul Selkirk <paul@painless-security.com>
* chore: remove unused setting
* feat: initial import of iesg minutes
* fix: let the meetings view show older iesg meetings
* feat: iesg narrative minutes
* feat: import bof coordination call minutes
* wip: import commands for iesg appeals and statements
* feat: import iesg statements.
* feat: import iesg artifacts
* feat: many fewer n+1 queries for the group meetings view
* fix: restore chain of elifs in views_doc
* fix: use self.stdout.write vs print in mgmt commands
* fix: use replace instead of astimezone when appropriate
* chore: refactor new migrations into one
* fix: transcode some old files into utf8
* fix: repair overzealous replace
* chore: black
* fix: address minro review comments
* fix: actually capture transcoding work
* fix: handle multiple iesg statements on the same day
* fix: better titles
* feat: pill badge replaced statements
* fix: consolodate source repos to one
* feat: liberal markdown for secretariat controlled content
* fix: handle (and clean) html narrative minutes
* feat: scrub harder
* fix: simplify and improve a scrubber
* chore: reorder migrations
* feat: DraftAliasGenerator class
Encapsulates logic from generate_draft_aliases.py
* refactor: Avoid circular imports
* feat: Add draft_aliases API endpoint
* feat: Add @requires_api_token decorator
Stolen from feat/rpc-api
* feat: Add token auth to draft_aliases endpoint
* feat: draft-aliases-from-json.py script
Parses output from the draft_aliases API call
* chore: Remove unused cruft
* refactor: Avoid shadowing "draft" name
* fix: Suppress empty lists from DraftAliasGenerator
* refactor: Use a GET instead of POST
* feat: GroupAliasGenerator class
* feat: group aliases API view
* fix: Handle domains array correctly
* fix: Suppress empty group aliases
* refactor: Generalize aliases-from-json.py script
* refactor: Same output fmt for draft and group alias apis
* feat: Sort addresses for stability
* fix: Add "anything" virtual alias
* test: Test requires_api_token decorator
* feat: Harden is_valid_token against misconfig
* test: Test is_valid_token
* test: Test draft_aliases view
* test: Test group_aliases view
* test: Test DraftAliasGenerator
* fix: ise group is type "ise" in test data
* test: Fix logic in testManagementCommand
The test was incorrect - and fails when fixed. :-(
* test: Test GroupAliasGenerator
Test currently fails
* fix: Suppress empty -ads alias
* test: Fix group acronym copy/paste error
I *think* this must be what had been intended. The
code does not look like it ever dealt with GroupHistory,
so I'm pretty sure it wasn't meant to have the same
acronym used by two different Groups at different
times.
* test: Check draft .notify alias generation
* test: Cover get_draft_notify_emails()
* refactor: change references from User to Person (#5821)
* refactor: Change CommunityList reference from User to Person
* refactor: Convert more user references to person
* refactor: Change augment_docs_and_user_with_user_info to person
* refactor: Change Nomination and Feedback references from User to Person
* refactor: Change a few test case function signatures to be more pythonic
* refactor: Harmonize how profile and photo views look up email_or_name
* refactor: Rework community views to operate on Person instead of User (#5859)
* test: Update tests to try all of the person's emails and aliases
* fix: Recode a test case to avoid an exception if there's Unicode in the URL
This only happens using the form-filling and submission feature of
WebTest, which is only used in this one test case, so just it rip out.
* test: Add duplicate-person tests
* fix: If there are multiple matching users, prefer the logged-in one.
* chore: We no longer use WebTest, so don't include it.
* fix: Address review comments
* fix: case-insensitive person name or email matching (#6096)
* chore: Renumber migrations
* fix: Update merged code so tests pass (#6887)
* fix: Use refactored method
* fix: Don't assume user has person
* fix: Use new view param name
* chore: Drop community lists w/o person; cleanup (#6896)
* fix: Don't assume user has person
* fix: user->person in update_community_list_index.py
* feat: Remove CommunityLists without Person
* refactor: Speed up nomcom migrations
---------
Co-authored-by: Paul Selkirk <paul@painless-security.com>
Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
* fix: don't create a volunteer object when a person has already volunteered
* fix: safer create
* refactor: use shorthand from django
* fix: also protect volunteer from datatracker from race
* feat: Capture volunteers from the registration system. Fixes#5938
* fix: allow withdrawn field to be null
* docs: add explanatory code comments
* fix: identify current nomcom by is_accepting_volunteers
* fix: use auto_now_add instead of auto_now
* fix: update migration
---------
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
* refactor: Remove is_rfc() - test type_id instead
* fix: Guard against unknown pub_date
This should not ever come up - we have a published_rfc event for
every rfc. Should investigate fixing pub_date() to always return a val.
