* feat: catch and report any <date> parsing error
* refactor: error handling in a more testable way
* fix: no bare `except`
* test: exception cases for test_parse_creation_date
* fix: explicitly reject non-numeric day/year
* test: suppress xml2rfc output in test
* refactor: context manager to capture xml2rfc output
* refactor: more capture_xml2rfc_output usage
* fix: capture_xml2rfc_output exception handling
* feat: basic blobstore infrastructure for dev
* refactor: (broken) attempt to put minio console behind nginx
* feat: initialize blobstore with boto3
* fix: abandon attempt to proxy minio. Use docker compose instead.
* feat: beginning of blob writes
* feat: storage utilities
* feat: test buckets
* chore: black
* chore: remove unused import
* chore: avoid f string when not needed
* fix: inform all settings files about blobstores
* fix: declare types for some settings
* ci: point to new target base
* ci: adjust test workflow
* fix: give the tests debug environment a blobstore
* fix: "better" name declarations
* ci: use devblobstore container
* chore: identify places to write to blobstorage
* chore: remove unreachable code
* feat: store materials
* feat: store statements
* feat: store status changes
* feat: store liaison attachments
* feat: store agendas provided with Interim session requests
* chore: capture TODOs
* feat: store polls and chatlogs
* chore: remove unneeded TODO
* feat: store drafts on submit and post
* fix: handle storage during doc expiration and resurrection
* fix: mirror an unlink
* chore: add/refine TODOs
* feat: store slide submissions
* fix: structure slide test correctly
* fix: correct sense of existence check
* feat: store some indexes
* feat: BlobShadowFileSystemStorage
* feat: shadow floorplans / host logos to the blob
* chore: remove unused import
* feat: strip path from blob shadow names
* feat: shadow photos / thumbs
* refactor: combine photo and photothumb blob kinds
The photos / thumbs were already dropped in the same
directory, so let's not add a distinction at this point.
* style: whitespace
* refactor: use kwargs consistently
* chore: migrations
* refactor: better deconstruct(); rebuild migrations
* fix: use new class in mack patch
* chore: add TODO
* feat: store group index documents
* chore: identify more TODO
* feat: store reviews
* fix: repair merge
* chore: remove unnecessary TODO
* feat: StoredObject metadata
* fix: deburr some debugging code
* fix: only set the deleted timestamp once
* chore: correct typo
* fix: get_or_create vs get and test
* fix: avoid the questionable is_seekable helper
* chore: capture future design consideration
* chore: blob store cfg for k8s
* chore: black
* chore: copyright
* ci: bucket name prefix option + run Black
Adds/uses DATATRACKER_BLOB_STORE_BUCKET_PREFIX option. Other changes
are just Black styling.
* ci: fix typo in bucket name expression
* chore: parameters in app-configure-blobstore
Allows use with other blob stores.
* ci: remove verify=False option
* fix: don't return value from __init__
* feat: option to log timing of S3Storage calls
* chore: units
* fix: deleted->null when storing a file
* style: Black
* feat: log as JSON; refactor to share code; handle exceptions
* ci: add ietf_log_blob_timing option for k8s
* test: --no-manage-blobstore option for running tests
* test: use blob store settings from env, if set
* test: actually set a couple more storage opts
* feat: offswitch (#8541)
* feat: offswitch
* fix: apply ENABLE_BLOBSTORAGE to BlobShadowFileSystemStorage behavior
* chore: log timing of blob reads
* chore: import Config from botocore.config
* chore(deps): import boto3-stubs / botocore
botocore is implicitly imported, but make it explicit
since we refer to it directly
* chore: drop type annotation that mypy loudly ignores
* refactor: add storage methods via mixin
Shares code between Document and DocHistory without
putting it in the base DocumentInfo class, which
lacks the name field. Also makes mypy happy.
* feat: add timeout / retry limit to boto client
* ci: let k8s config the timeouts via env
* chore: repair merge resolution typo
* chore: tweak settings imports
* chore: simplify k8s/settings_local.py imports
---------
Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
* chore: fix typo in log message
* fix: write new conflict reviews to the ftp directory
* fix: log exception description
* chore: avoid reusing a name for mypy
* fix: write non-meeting-related slides to ftp/slides
* fix: write status changes to ftp dir
* fix: copy draft yang modules to the ftp dir
the last commit above (see #8401) has no tests :(. There aren't tests of this yang scaffolding to add to. Adding a set of tests should be an independent larger effort.
* Clarify "Replace" in I-D submission form
Update help text In the "Replacement Information" section of
the "Status" tab
* refactor: update node, eslint, neostandard + fix esm (#8083)
* chore: update dependencies
* fix: eslint + neostandard
* fix: add corepack prompt env var to init script
* docs: Update README.md
---------
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
* ci: update base image target version to 20241114T1703
* ci: fix tests.yml workflow
* fix: clarify "Replace" in I-D submission form #8205
update quote syntax issue on previous update help text In the "Replacement Information" section of the "Status" tab to fix#8059
* Revert "refactor: update node, eslint, neostandard + fix esm (#8083)"
This reverts commit 649879efd745470f6e0cc6768d889f45640e1505.
* Revert "ci: update base image target version to 20241114T1703"
This reverts commit f11144017ed788bc7a38a1d028127434f0d50eb4.
* Revert "ci: fix tests.yml workflow"
This reverts commit 39231321c49291565a39608b98740e098c74dda7.
* Update forms.py
---------
Co-authored-by: Nicolas Giard <github@ngpixel.com>
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
Co-authored-by: NGPixel <NGPixel@users.noreply.github.com>
* feat: capture xml2rfc output on exception
* fix: chain exception properly
* feat: log xml2rfc stderr/stdout
May want to pass these back to the submitter, but
let's watch them to see what sorts of sanitizing we
should do first.
* Revert "feat: log xml2rfc stderr/stdout"
This reverts commit 959c54d30733a4a3df7ed0250fef347ed6f03a08.
* feat: log xml2rfc stderr/stdout
This time without reformatting all the imports
* feat: path sanitization for submission errors
* fix: parentheses in regex
* test: test SubmissionError sanitization
* style: consolidate imports
* fix: apply sanitization to all args
---------
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
This allows an exception to be raised if submission
files are missing, leading to a server error. That's
not pretty, but is better than ignoring the fail.
* feat: get tool versions without VersionInfo model
* chore: remove update_external_command_info call
* feat: get tool version without VersionInfo
* chore: Remove VersionInfo model
* chore: Migration to remove VersionInfo
* fix: handle errors better; ignore stderr
* fix: type annotation
* wip: identify whats needed to obviate ghostlinkd
* fix: hardlink new charter files to ftp directory
* fix: hardlink new charter files to ftp directory (continued)
* chore: bring settings comment up to date
* chore: add archive and ftp dirs to setup of various environments
* fix: test charter submits write to ftp dir
* chore: remove debug
* fix: test charter approval writes to ftp dir
* fix: link review revisions into ftp dir
* fix: link to all archive and ftp on submission post
* chore: clean comments, move action to github issue
* fix: link idindex files to all archive and ftp
* chore: deflake
* chore: remove TODO comment
* fix: use settings
* chore: rename new setting
* refactor: Update parsers/base.py for Python3
* style: Black
* refactor: Remove mime type check from FileParser
* refactor: Validate that submission is UTF-8
The mime check distinguished us-ascii from UTF-8,
but as far as I can tell the code relying on it
treated both as equally valid.
* feat: Clear error when file is not valid XML
* chore: Remove unused import
* test: Update tests to match changes
* fix: Count bytes starting from 1
* test: Add tests of FileParser validations
* fix: Fix / simplify regexp
* test: Test error caused by bad XML submission
---------
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
* fix: Show quieter alert for metadata error in canceled submission
* fix: Suppress metadata errors and show warning for canceled submissions
* style: Fix indentation in HTML
* fix: Give better notice of a processing error
* refactor: Eliminate _construct_creation_date helper
* fix: Use xml2rfc method for filling in missing date fields
* fix: Set options.date for xml2rfc writers
* test: Test handling of missing date element/fields
