* feat: basic blobstore infrastructure for dev
* refactor: (broken) attempt to put minio console behind nginx
* feat: initialize blobstore with boto3
* fix: abandon attempt to proxy minio. Use docker compose instead.
* feat: beginning of blob writes
* feat: storage utilities
* feat: test buckets
* chore: black
* chore: remove unused import
* chore: avoid f string when not needed
* fix: inform all settings files about blobstores
* fix: declare types for some settings
* ci: point to new target base
* ci: adjust test workflow
* fix: give the tests debug environment a blobstore
* fix: "better" name declarations
* ci: use devblobstore container
* chore: identify places to write to blobstorage
* chore: remove unreachable code
* feat: store materials
* feat: store statements
* feat: store status changes
* feat: store liaison attachments
* feat: store agendas provided with Interim session requests
* chore: capture TODOs
* feat: store polls and chatlogs
* chore: remove unneeded TODO
* feat: store drafts on submit and post
* fix: handle storage during doc expiration and resurrection
* fix: mirror an unlink
* chore: add/refine TODOs
* feat: store slide submissions
* fix: structure slide test correctly
* fix: correct sense of existence check
* feat: store some indexes
* feat: BlobShadowFileSystemStorage
* feat: shadow floorplans / host logos to the blob
* chore: remove unused import
* feat: strip path from blob shadow names
* feat: shadow photos / thumbs
* refactor: combine photo and photothumb blob kinds
The photos / thumbs were already dropped in the same
directory, so let's not add a distinction at this point.
* style: whitespace
* refactor: use kwargs consistently
* chore: migrations
* refactor: better deconstruct(); rebuild migrations
* fix: use new class in mack patch
* chore: add TODO
* feat: store group index documents
* chore: identify more TODO
* feat: store reviews
* fix: repair merge
* chore: remove unnecessary TODO
* feat: StoredObject metadata
* fix: deburr some debugging code
* fix: only set the deleted timestamp once
* chore: correct typo
* fix: get_or_create vs get and test
* fix: avoid the questionable is_seekable helper
* chore: capture future design consideration
* chore: blob store cfg for k8s
* chore: black
* chore: copyright
* ci: bucket name prefix option + run Black
Adds/uses DATATRACKER_BLOB_STORE_BUCKET_PREFIX option. Other changes
are just Black styling.
* ci: fix typo in bucket name expression
* chore: parameters in app-configure-blobstore
Allows use with other blob stores.
* ci: remove verify=False option
* fix: don't return value from __init__
* feat: option to log timing of S3Storage calls
* chore: units
* fix: deleted->null when storing a file
* style: Black
* feat: log as JSON; refactor to share code; handle exceptions
* ci: add ietf_log_blob_timing option for k8s
* test: --no-manage-blobstore option for running tests
* test: use blob store settings from env, if set
* test: actually set a couple more storage opts
* feat: offswitch (#8541)
* feat: offswitch
* fix: apply ENABLE_BLOBSTORAGE to BlobShadowFileSystemStorage behavior
* chore: log timing of blob reads
* chore: import Config from botocore.config
* chore(deps): import boto3-stubs / botocore
botocore is implicitly imported, but make it explicit
since we refer to it directly
* chore: drop type annotation that mypy loudly ignores
* refactor: add storage methods via mixin
Shares code between Document and DocHistory without
putting it in the base DocumentInfo class, which
lacks the name field. Also makes mypy happy.
* feat: add timeout / retry limit to boto client
* ci: let k8s config the timeouts via env
* chore: repair merge resolution typo
* chore: tweak settings imports
* chore: simplify k8s/settings_local.py imports
---------
Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
* fix: test file existance using metadata
* fix: use Path more
* fix: don't read the file to see if it exists
* fix: more conservative error handling
* chore: remove unused import
* wip: identify whats needed to obviate ghostlinkd
* fix: hardlink new charter files to ftp directory
* fix: hardlink new charter files to ftp directory (continued)
* chore: bring settings comment up to date
* chore: add archive and ftp dirs to setup of various environments
* fix: test charter submits write to ftp dir
* chore: remove debug
* fix: test charter approval writes to ftp dir
* fix: link review revisions into ftp dir
* fix: link to all archive and ftp on submission post
* chore: clean comments, move action to github issue
* fix: link idindex files to all archive and ftp
* chore: deflake
* chore: remove TODO comment
* fix: use settings
* chore: rename new setting
* chore: remove unused setting
* feat: initial import of iesg minutes
* fix: let the meetings view show older iesg meetings
* feat: iesg narrative minutes
* feat: import bof coordination call minutes
* wip: import commands for iesg appeals and statements
* feat: import iesg statements.
* feat: import iesg artifacts
* feat: many fewer n+1 queries for the group meetings view
* fix: restore chain of elifs in views_doc
* fix: use self.stdout.write vs print in mgmt commands
* fix: use replace instead of astimezone when appropriate
* chore: refactor new migrations into one
* fix: transcode some old files into utf8
* fix: repair overzealous replace
* chore: black
* fix: address minro review comments
* fix: actually capture transcoding work
* fix: handle multiple iesg statements on the same day
* fix: better titles
* feat: pill badge replaced statements
* fix: consolodate source repos to one
* feat: liberal markdown for secretariat controlled content
* fix: handle (and clean) html narrative minutes
* feat: scrub harder
* fix: simplify and improve a scrubber
* chore: reorder migrations
* chore: Unpin django-stubs / update mypy
* test: Use "app.model" for ManyToManyField
django-stubs requires "app.model" instead of just "model" for
ManyToManyField lazy model references.
See https://github.com/typeddjango/django-stubs/issues/1802
* fix: Wrap weasyprint to catch exceptions (#6324)
* test: Restore socket function after test
* test: Use mock instead of monkeying with sockets
* refactor: Log the error
* fix: Don't catch non-Exception interruptions
---------
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
* fix: clean up shadowed name in document_referenced_by.html
* fix: include refs to rfc's came_from_draft()
* fix: include refs to draft's became_rfc()
* fix: Count indirect refs by RFCs
* refactor: break indirect ref_by counting to its own fn
* fix: only count refs to pre-rfc draft, not post-draft rfc
(and rename a method)
* test: test referenced_by_rfcs methods
The test_referenced_by_rfcs_as_rfc_or_draft() test
fails because there's a bug!
* test: actually, do double-count refs to rfc/draft
Let's do include refs to an rfc and its precursor draft
as separate refs. This almost surely indicates a data
error because it would mean an rfc referenced both an
rfc and the draft that it came from. That should never
be allowed, so at least let some light fall on it if
it happens.
* chore: Add docstring to document_referenced_by view
* Added history entry when approving the slides.
Also changed os.rename to shutils.move as the submissions and
proceedings are on the separate filesystems on the docker image,
and this same thing might happen in the real environment in the
future.
* Add migrations for the docevents type.
* feat: support iab and iesg statements. Import iab statements. (#5895)
* feat: infrastructure for statements doctype
* chore: basic test framework
* feat: basic statement document view
* feat: show replaced statements
* chore: black
* fix: state help for statements
* fix: cleanout non-relevant email expansions
* feat: import iab statements, provide group statements tab
* fix: guard against running import twice
* feat: build redirect csv for iab statements
* fix: set document state on import
* feat: show published date on main doc view
* feat: handle pdf statements
* feat: create new and update statements
* chore: copyright block updates
* chore: remove flakes
* chore: black
* feat: add edit/new buttons for the secretariat
* fix: address PR #5895 review comments
* fix: pin pydantic until inflect catches up (#5901) (#5902)
* chore: re-un-pin pydantic
* refactor: Remove is_rfc() - test type_id instead
* fix: Guard against unknown pub_date
This should not ever come up - we have a published_rfc event for
every rfc. Should investigate fixing pub_date() to always return a val.
* fix: Fix friendly_state() for drafts in rfc state
* refactor: Change became-rfc to became_rfc for consistency
* refactor: Begin handling RFCs separatly in document_main
WIP
* chore: Hide reviews and conflict reviews for RFCs
* chore: Remove irrelevant fields from rfc doc view
* chore: Remove unused variables
* chore: Remove "Versions" field from document_rfc.html
* chore: Hide "Nits" button for RFC documents
* fix: Enforce naming of charter docs in submit()
* style: Reformat submit() with Black
* refactor: Remove redundant check of charter name
* style: Reformat charter_with_milestones_txt with Black
* refactor: Drop canonical_name, use Path in charter_with_milestones_txt
* style: Reformat review_announcement_text() with Black
* style: Reformat action_announcement_text() with Black
* refactor: Change uses of charter.canonical_name() to charter.name
* refactor: Skip docialias when retrieving charter
* refactor: Change canonical_name() to name in utils_charter.py
* refactor: Use Path in read_charter_text()
* refactor: Drop canonical_name, minor refactor of tests_charter.py
* refactor: charter.name instead of canonical_name in milestones.py
* refactor: charter.name instead of canonical_name in tests_info.py
* refactor: Remove unused functions in ietf/secr/utils/groups.py
* refactor: charter.canonical_name -> charter.name in templates
* refactor: Remove charter handling from canonical_name
Temporarily raise an exception for testing
* refactor: Refactor get_charter_text() without canonical_name
* refactor: Remove raise when canonical_name called on a charter
* fix: Add back missing ".txt" extension
* test: Test rejection of invalid charter names
