* feat: basic blobstore infrastructure for dev
* refactor: (broken) attempt to put minio console behind nginx
* feat: initialize blobstore with boto3
* fix: abandon attempt to proxy minio. Use docker compose instead.
* feat: beginning of blob writes
* feat: storage utilities
* feat: test buckets
* chore: black
* chore: remove unused import
* chore: avoid f string when not needed
* fix: inform all settings files about blobstores
* fix: declare types for some settings
* ci: point to new target base
* ci: adjust test workflow
* fix: give the tests debug environment a blobstore
* fix: "better" name declarations
* ci: use devblobstore container
* chore: identify places to write to blobstorage
* chore: remove unreachable code
* feat: store materials
* feat: store statements
* feat: store status changes
* feat: store liaison attachments
* feat: store agendas provided with Interim session requests
* chore: capture TODOs
* feat: store polls and chatlogs
* chore: remove unneeded TODO
* feat: store drafts on submit and post
* fix: handle storage during doc expiration and resurrection
* fix: mirror an unlink
* chore: add/refine TODOs
* feat: store slide submissions
* fix: structure slide test correctly
* fix: correct sense of existence check
* feat: store some indexes
* feat: BlobShadowFileSystemStorage
* feat: shadow floorplans / host logos to the blob
* chore: remove unused import
* feat: strip path from blob shadow names
* feat: shadow photos / thumbs
* refactor: combine photo and photothumb blob kinds
The photos / thumbs were already dropped in the same
directory, so let's not add a distinction at this point.
* style: whitespace
* refactor: use kwargs consistently
* chore: migrations
* refactor: better deconstruct(); rebuild migrations
* fix: use new class in mack patch
* chore: add TODO
* feat: store group index documents
* chore: identify more TODO
* feat: store reviews
* fix: repair merge
* chore: remove unnecessary TODO
* feat: StoredObject metadata
* fix: deburr some debugging code
* fix: only set the deleted timestamp once
* chore: correct typo
* fix: get_or_create vs get and test
* fix: avoid the questionable is_seekable helper
* chore: capture future design consideration
* chore: blob store cfg for k8s
* chore: black
* chore: copyright
* ci: bucket name prefix option + run Black
Adds/uses DATATRACKER_BLOB_STORE_BUCKET_PREFIX option. Other changes
are just Black styling.
* ci: fix typo in bucket name expression
* chore: parameters in app-configure-blobstore
Allows use with other blob stores.
* ci: remove verify=False option
* fix: don't return value from __init__
* feat: option to log timing of S3Storage calls
* chore: units
* fix: deleted->null when storing a file
* style: Black
* feat: log as JSON; refactor to share code; handle exceptions
* ci: add ietf_log_blob_timing option for k8s
* test: --no-manage-blobstore option for running tests
* test: use blob store settings from env, if set
* test: actually set a couple more storage opts
* feat: offswitch (#8541)
* feat: offswitch
* fix: apply ENABLE_BLOBSTORAGE to BlobShadowFileSystemStorage behavior
* chore: log timing of blob reads
* chore: import Config from botocore.config
* chore(deps): import boto3-stubs / botocore
botocore is implicitly imported, but make it explicit
since we refer to it directly
* chore: drop type annotation that mypy loudly ignores
* refactor: add storage methods via mixin
Shares code between Document and DocHistory without
putting it in the base DocumentInfo class, which
lacks the name field. Also makes mypy happy.
* feat: add timeout / retry limit to boto client
* ci: let k8s config the timeouts via env
* chore: repair merge resolution typo
* chore: tweak settings imports
* chore: simplify k8s/settings_local.py imports
---------
Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
* fix: keep draft-iesg state on expiration. Update action holders
* feat: task to repair docs in dead because expiry
* fix: restore all to-date flows through update_action_holders
* fix: Fetch the System user following more regular conventions
* fix: better signal test
* feat: remove "AD is watching" state
* chore: update names.json
* refactor: use idexists state, not dead
* fix: remove guidance to use watching state
* chore: remove references to 'watching' state
* feat: remove create_in_state from edit_info view
* test: update test
* style: Black + move class closer to use
* refactor: remove lint
* fix: restore missing admonition
---------
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
* wip: identify whats needed to obviate ghostlinkd
* fix: hardlink new charter files to ftp directory
* fix: hardlink new charter files to ftp directory (continued)
* chore: bring settings comment up to date
* chore: add archive and ftp dirs to setup of various environments
* fix: test charter submits write to ftp dir
* chore: remove debug
* fix: test charter approval writes to ftp dir
* fix: link review revisions into ftp dir
* fix: link to all archive and ftp on submission post
* chore: clean comments, move action to github issue
* fix: link idindex files to all archive and ftp
* chore: deflake
* chore: remove TODO comment
* fix: use settings
* chore: rename new setting
* fix: set tz when passing a date to timesince_days filter
* fix: fill in tz for a constructed datetime in idindex/index.py
* test: simplify double-negatives in test assertions
* test: fix I-D expiration test cases to be tz aware
* fix: use tz-aware comparisons for in_draft_expire_freeze method
* test: fix tz used for timesince_days filter test case
* refactor: replace datetime.now with timezone.now
* refactor: migrate model fields to use timezone.now as default
* refactor: replace datetime.today with timezone.now
datetime.datetime.today() is equivalent to datetime.datetime.now(); both
return a naive datetime with the current local time.
* refactor: rephrase datetime.now(tz) as timezone.now().astimezone(tz)
This is effectively the same, but is less likely to encourage accidental
use of naive datetimes.
* refactor: revert datetime.today() change to old migrations
* refactor: change a missed datetime.now to timezone.now
* chore: renumber timezone_now migration
* chore: renumber migrations
The new API requires at least one event and will automatically save a
snapshot of the document and related state. Document.save() will now
throw an exception if called directly, as the new API is intended to
ensure that documents are saved with both an appropriate snapsnot and
relevant history log, both of which are easily defeated by just
calling .save() directly.
To simplify things, the snapshot is generated after the changes to a
document have been made (in anticipation of coming changes), instead
of before as was usual.
While revising the existing code to work with this API, a couple of
missing events was discovered:
- In draft expiry, a "Document has expired" event was only generated
in case an IESG process had started on the document - now it's
always generated, as the document changes its state in any case
- Synchronization updates like title and abstract amendmends from the
RFC Editor were silently (except for RFC publication) applied and
not accompanied by a descriptive event - they now are
- do_replace in the Secretariat tools now adds an event
- Proceedings post_process in the Secretariat tools now adds an event
- do_withdraw in the Secretariat tools now adds an event
A migration is needed for snapshotting all documents, takes a while to
run. It turns out that a single document had a bad foreign key so the
migration fixes that too.
- Legacy-Id: 10101
doc/utils_charter.py) in favour of add_state_change_event which has
slightly better API but otherwise does basically the same except it
sets an event type we can later search for. Also expand tests slightly
to exercise three more templates.
- Legacy-Id: 7129
