* feat: basic blobstore infrastructure for dev
* refactor: (broken) attempt to put minio console behind nginx
* feat: initialize blobstore with boto3
* fix: abandon attempt to proxy minio. Use docker compose instead.
* feat: beginning of blob writes
* feat: storage utilities
* feat: test buckets
* chore: black
* chore: remove unused import
* chore: avoid f string when not needed
* fix: inform all settings files about blobstores
* fix: declare types for some settings
* ci: point to new target base
* ci: adjust test workflow
* fix: give the tests debug environment a blobstore
* fix: "better" name declarations
* ci: use devblobstore container
* chore: identify places to write to blobstorage
* chore: remove unreachable code
* feat: store materials
* feat: store statements
* feat: store status changes
* feat: store liaison attachments
* feat: store agendas provided with Interim session requests
* chore: capture TODOs
* feat: store polls and chatlogs
* chore: remove unneeded TODO
* feat: store drafts on submit and post
* fix: handle storage during doc expiration and resurrection
* fix: mirror an unlink
* chore: add/refine TODOs
* feat: store slide submissions
* fix: structure slide test correctly
* fix: correct sense of existence check
* feat: store some indexes
* feat: BlobShadowFileSystemStorage
* feat: shadow floorplans / host logos to the blob
* chore: remove unused import
* feat: strip path from blob shadow names
* feat: shadow photos / thumbs
* refactor: combine photo and photothumb blob kinds
The photos / thumbs were already dropped in the same
directory, so let's not add a distinction at this point.
* style: whitespace
* refactor: use kwargs consistently
* chore: migrations
* refactor: better deconstruct(); rebuild migrations
* fix: use new class in mack patch
* chore: add TODO
* feat: store group index documents
* chore: identify more TODO
* feat: store reviews
* fix: repair merge
* chore: remove unnecessary TODO
* feat: StoredObject metadata
* fix: deburr some debugging code
* fix: only set the deleted timestamp once
* chore: correct typo
* fix: get_or_create vs get and test
* fix: avoid the questionable is_seekable helper
* chore: capture future design consideration
* chore: blob store cfg for k8s
* chore: black
* chore: copyright
* ci: bucket name prefix option + run Black
Adds/uses DATATRACKER_BLOB_STORE_BUCKET_PREFIX option. Other changes
are just Black styling.
* ci: fix typo in bucket name expression
* chore: parameters in app-configure-blobstore
Allows use with other blob stores.
* ci: remove verify=False option
* fix: don't return value from __init__
* feat: option to log timing of S3Storage calls
* chore: units
* fix: deleted->null when storing a file
* style: Black
* feat: log as JSON; refactor to share code; handle exceptions
* ci: add ietf_log_blob_timing option for k8s
* test: --no-manage-blobstore option for running tests
* test: use blob store settings from env, if set
* test: actually set a couple more storage opts
* feat: offswitch (#8541)
* feat: offswitch
* fix: apply ENABLE_BLOBSTORAGE to BlobShadowFileSystemStorage behavior
* chore: log timing of blob reads
* chore: import Config from botocore.config
* chore(deps): import boto3-stubs / botocore
botocore is implicitly imported, but make it explicit
since we refer to it directly
* chore: drop type annotation that mypy loudly ignores
* refactor: add storage methods via mixin
Shares code between Document and DocHistory without
putting it in the base DocumentInfo class, which
lacks the name field. Also makes mypy happy.
* feat: add timeout / retry limit to boto client
* ci: let k8s config the timeouts via env
* chore: repair merge resolution typo
* chore: tweak settings imports
* chore: simplify k8s/settings_local.py imports
---------
Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
* chore: remove the effectively unused DATABASE_TEST_OPTIONS setting
* fix: simplify default settings_local so that db-include-fix.py is not needed in CI
* fix: simplify migrations in db-pg-migrate.sh
* chore: Use codespell to fix typos in code.
Second part of replacement of #4651
@rjsparks, I probably need to revert some things here, and I also
still need to add that new migration - how do I do that?
* Revert migrations
* Migrate "Whitelisted" to "Allowlisted"
* TEST_COVERAGE_MASTER_FILE -> TEST_COVERAGE_MAIN_FILE
* Fix permissions
* Add suggestions from @jennifer-richards
* fix: avoid readonly attribute on select element
* chore: remove now-unused CSS for select2
* test: don't ignore `readonly` on `select`
* refactor: simplify OutgoingLiaisonForm base class overrides
* feat: Use bs5 for htmlized doc (#4082)
Co-authored-by: Martin Thomson <mt@lowentropy.net>
* fix: Various fixes to HTMLized document view (#4501)
* fix: Pref labels were switched
* fix: Fix ToC for htmlized docs
* Replace datatracker button with document name link to datatracker
* ui: Make fonts even larger on larger window widths
* fix: Document format buttons open new tabs
* fix: Various suggestions from Jay
* fix: Don't show "htmlized" self-link under formats
* ui: Font size fix for iOS
* ui: More little tweaks
* feat/htmlize fixes and improvements (#4506)
* fix: Don't open htmlized view in new tab
* fix: Tests were failing
* feat: Add pref settings to cap max font size
* feat: Add ability to hide side panel
* fix: And more `feat/htmlize` fixes (#4511)
* fix: Remove superfluous scrollbars
* fix: Show email links for authors
* fix: Only show "email authors" button for latest reversion
* fix: Remove duplicate code, fix nav scrolling
* feat: Add RFCs to revision lists
* feat: Add pref option to control dependency inlining
* fix: Add analytical tags
* feat: Notify user of rendering inconsistencies
* feat: Split out bootstrap-icons when not inlining dependencies
* fix: Revision list and minor other fixes
* feat: Show stream logo when possible (#4516)
* fix: Remove superfluous scrollbars
* fix: Show email links for authors
* fix: Only show "email authors" button for latest reversion
* fix: Remove duplicate code, fix nav scrolling
* feat: Add RFCs to revision lists
* feat: Add pref option to control dependency inlining
* fix: Add analytical tags
* feat: Notify user of rendering inconsistencies
* feat: Split out bootstrap-icons when not inlining dependencies
* fix: Revision list and minor other fixes
* feat: Show stream logos when possible
* fix: Pick up CSS changes from https://github.com/martinthomson/rfc-txt-html (#4520)
* fix: Remove superfluous scrollbars
* fix: Show email links for authors
* fix: Only show "email authors" button for latest reversion
* fix: Remove duplicate code, fix nav scrolling
* feat: Add RFCs to revision lists
* feat: Add pref option to control dependency inlining
* fix: Add analytical tags
* feat: Notify user of rendering inconsistencies
* feat: Split out bootstrap-icons when not inlining dependencies
* fix: Revision list and minor other fixes
* feat: Show stream logos when possible
* fix: Pick up CSS changes from https://github.com/martinthomson/rfc-txt-html
* chore: add debug script to replicate GitHub Actions test environment
* chore: cleanup from merge of main
* fix: Fix PDFixation crash due to referencing renamed CSS asset (#4665)
* fix: Rename some CSS classes to handle recent xml2rfc changes (#4791)
* Merge main
* fix: Rename some CSS classes to handle recent xml2rfc changes
Fixes#4784.
* chore: repair merge damage
* chore: more merge corrections
* chore: one more merge correction
* fix: npm dependencies
* fix: Change default font size (#4817)
* Fix dependency issues
* Cap fontsize at 12pt by default
Co-authored-by: Nicolas Giard <github@ngpixel.com>
Co-authored-by: Lars Eggert <lars@eggert.org>
Co-authored-by: Martin Thomson <mt@lowentropy.net>
Co-authored-by: Nicolas Giard <github@ngpixel.com>
* ci: move HTML validation out of teardown so all failures are caught
Failures in the last batch were suppressed by the original implementation.
* test: rename template validation "test" so it is not auto-discovered
* test: run all HTML validation in a single batch at the end
Adds ~50 MB peak RAM usage during a full test run.
* test: refactor validation to collect failures of all kinds before exit
* feat: apis for attaching chatlogs and polls to session materials
* fix: anticipate becoming tzaware, and improve guard against attempts to provide docs for sessions that have no official timeslot assignment.
* fix: get chatlog upload to actually work
Modifications to several initial implementation decisions.
Updates to the fixtures.
* fix: test polls upload
Refactored test to reduce duplicate code
* fix: allow api keys to be created for the new endpoints
* feat: add ability to view chatlog and polls documents. Show links in session materials.
* fix: commit new template
* fix: typo in migration signatures
* feat: add main doc page handling for polls. Improve tests.
* feat: chat log vue component + embedded vue loader
* feat: render polls using Vue
* fix: address pug syntax review comments from Nick.
* fix: repair remaining mention of chat log from copymunging
* fix: use double-quotes in html attributes
* fix: provide missing choices update migration
* test: silence html validator empty attr warnings
* test: fix test_runner config
* fix: locate session when looking at a dochistory object for polls or chatlog
Co-authored-by: Nicolas Giard <github@ngpixel.com>
* refactor: replace datetime.now with timezone.now
* refactor: migrate model fields to use timezone.now as default
* refactor: replace datetime.today with timezone.now
datetime.datetime.today() is equivalent to datetime.datetime.now(); both
return a naive datetime with the current local time.
* refactor: rephrase datetime.now(tz) as timezone.now().astimezone(tz)
This is effectively the same, but is less likely to encourage accidental
use of naive datetimes.
* refactor: revert datetime.today() change to old migrations
* refactor: change a missed datetime.now to timezone.now
* chore: renumber timezone_now migration
* chore: renumber migrations
* fix: set vite base path during deploy build + allow google fonts
* ci: only set vite base path if build is for production release
* test: add html-validate ignore rules for vite generated content that is valid
* fix: show buttons on regular sessions without agenda materials + link to legacy agenda
* fix: temporarily hide left menu on agenda-neue and floor-plan-neue
* fix: set resize handler to whole app
* fix: apply html-validate ignore to agenda-neue only + deploy/build.sh fix
* test: change old agenda cypress to always take the same elements to avoid #3564
* fix: Fix some HTML validation errors that came in via #4035
Also make validation errors display a bit better.
* content -> color for mask-icon (caught by @jennifer-richards; thanks)
* Unicode messages are triggered by both db content and tests
* Make ids unique
* Avoid "No value found" message on page
* Strip HTML from history entries, it's often broken
* Check HTML sources for occurrences of "** No value found for" and fix them
* Fix another occurrence of "** No value found for"
* Fix more occurrences of "** No value found for"
* Fix document revision stripping
* Force breaks of long (garbage) words
* Check URL validity before urlizing them
* Handle some additional corner cases
* Linkify action items
* Don't create profile/email links for System
* Handle headings with HTML elements in them better
* Fix comment
* Fix another occurrence of "** No value found for"
* Better I-D URLization that handles more edge cases. Also, test for them.
* Remove print
* Handle charters better
* Cache for one day
* Update vnu.jar
* Fix py2 -> py3 issue
* Run pyupgrade
* test: Add default-jdk to images
* test: Add option to also validate HTML with vnu.jar
Since it's already installed in bin. Don't do this by default, since it
increases the time needed for tests by ~50%.
* fix: Stop the urlizer from urlizing in linkified mailto: text
* More HTML fixes
* More HTML validation fixes
* And more HTML fixes
* Fix floating badge
* Ignore unicode errors
* Only URLize docs that are existing
* Final fixes
* Don't URLize everything during test-crawl
* Feed HTML into vnu using python rather than Java to speed things up
* Allow test-crawl to start vnu on a different port
* Increase retry count to vnu. Restore batch size to 30.
* More HTML validation fixes
* Use urllib3 to make requests to vnu, since overriding requests_mock is tricky
* Undo commit of unmodified file
* Also urlize ftp links
* Fix matching of file name
* More HTML fixes
* Add `is_valid_url` filter
* weekday -> data-weekday
* urlencode URLs
* Add and use vnu_fmt_message. Bump vnu max buffer.
* Simplify doc_exists
* Don't add tab link to mail archive if the URL is invalid
* Run urlize_ietf_docs before linkify
Reduces the possibility of generating incorrect HTML
* Undo superfluous change
* Runner fixes
* Consolidate vnu message filtering into vnu_filter_message
* Correctly handle multiple persons with same name
* Minimze diff
* Fix HTML nits
* Print source snippet in vnu_fmt_message
* Only escape if there is something to escape
* Fix snippet
* Skip crufty old IPR declarations
* Only include modal when needed. Add handles.
* Fix wordwrap+linkification
* Update ietf/doc/templatetags/ietf_filters.py
* Update ietf/doc/templatetags/tests_ietf_filters.py
* Don't right-align second column
* ci: Also check generated HTML with the "vnu" validator
Because HTML Tidy apparently misses a bunch of errors.
* thead -> tbody
* More fixes
* More fixes
* Start checker in test runner
