* feat: basic blobstore infrastructure for dev
* refactor: (broken) attempt to put minio console behind nginx
* feat: initialize blobstore with boto3
* fix: abandon attempt to proxy minio. Use docker compose instead.
* feat: beginning of blob writes
* feat: storage utilities
* feat: test buckets
* chore: black
* chore: remove unused import
* chore: avoid f string when not needed
* fix: inform all settings files about blobstores
* fix: declare types for some settings
* ci: point to new target base
* ci: adjust test workflow
* fix: give the tests debug environment a blobstore
* fix: "better" name declarations
* ci: use devblobstore container
* chore: identify places to write to blobstorage
* chore: remove unreachable code
* feat: store materials
* feat: store statements
* feat: store status changes
* feat: store liaison attachments
* feat: store agendas provided with Interim session requests
* chore: capture TODOs
* feat: store polls and chatlogs
* chore: remove unneeded TODO
* feat: store drafts on submit and post
* fix: handle storage during doc expiration and resurrection
* fix: mirror an unlink
* chore: add/refine TODOs
* feat: store slide submissions
* fix: structure slide test correctly
* fix: correct sense of existence check
* feat: store some indexes
* feat: BlobShadowFileSystemStorage
* feat: shadow floorplans / host logos to the blob
* chore: remove unused import
* feat: strip path from blob shadow names
* feat: shadow photos / thumbs
* refactor: combine photo and photothumb blob kinds
The photos / thumbs were already dropped in the same
directory, so let's not add a distinction at this point.
* style: whitespace
* refactor: use kwargs consistently
* chore: migrations
* refactor: better deconstruct(); rebuild migrations
* fix: use new class in mack patch
* chore: add TODO
* feat: store group index documents
* chore: identify more TODO
* feat: store reviews
* fix: repair merge
* chore: remove unnecessary TODO
* feat: StoredObject metadata
* fix: deburr some debugging code
* fix: only set the deleted timestamp once
* chore: correct typo
* fix: get_or_create vs get and test
* fix: avoid the questionable is_seekable helper
* chore: capture future design consideration
* chore: blob store cfg for k8s
* chore: black
* chore: copyright
* ci: bucket name prefix option + run Black
Adds/uses DATATRACKER_BLOB_STORE_BUCKET_PREFIX option. Other changes
are just Black styling.
* ci: fix typo in bucket name expression
* chore: parameters in app-configure-blobstore
Allows use with other blob stores.
* ci: remove verify=False option
* fix: don't return value from __init__
* feat: option to log timing of S3Storage calls
* chore: units
* fix: deleted->null when storing a file
* style: Black
* feat: log as JSON; refactor to share code; handle exceptions
* ci: add ietf_log_blob_timing option for k8s
* test: --no-manage-blobstore option for running tests
* test: use blob store settings from env, if set
* test: actually set a couple more storage opts
* feat: offswitch (#8541)
* feat: offswitch
* fix: apply ENABLE_BLOBSTORAGE to BlobShadowFileSystemStorage behavior
* chore: log timing of blob reads
* chore: import Config from botocore.config
* chore(deps): import boto3-stubs / botocore
botocore is implicitly imported, but make it explicit
since we refer to it directly
* chore: drop type annotation that mypy loudly ignores
* refactor: add storage methods via mixin
Shares code between Document and DocHistory without
putting it in the base DocumentInfo class, which
lacks the name field. Also makes mypy happy.
* feat: add timeout / retry limit to boto client
* ci: let k8s config the timeouts via env
* chore: repair merge resolution typo
* chore: tweak settings imports
* chore: simplify k8s/settings_local.py imports
---------
Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
* chore: nginx log is s, not ms
* chore: log seconds from gunicorn too
* chore: drop X-Real-IP header / log
* style: Black
* style: single -> double quotes
* feat: add is-authenticated header
* feat: log is-authenticated header
* chore: update nginx-auth.conf to match
* chore: increase nginx proxy_read_timeout
* refactor: speed up bluesheet_data
The affiliation helper was extremely slow. Using
queryset annotation speeds it up by almost an order
of magnitude.
* chore: delint
* feat: nginx + robots.txt
* feat: minimal /health/ endpoint
* ci: startupProbe for datatracker pod
* ci: probe auth pod; set timeoutSeconds
* feat: add CSP and other headers to nginx
* fix: typo in nginx.conf
* feat: split auth/dt nginx confs
* test: test health endpoint
* ci: auth service on port 80
We'll remove http-old (8080) in the future.
* ci: rename auth container/nginx cfg
* refactor: always use console log handler
* refactor: json for k8s, plain otherwise
* chore: remove syslog from wsgi.py
* chore: remove debug.log()
* chore: drop syslog from settings.py
* refactor: use log.log() in person.utils
* refactor: fetch_meeting_attendance->log.log()
* chore: gunicorn logs as JSON (wip)
* feat: better json log formatting
* refactor: improve log config
* feat: gunicorn access log fields
* fix: remove type hints
The gunicorn logger plays tricks with the
LogRecord args parameter to let it have string
keys instead of being a simple tuple.
The mypy tests rightly flag this. Rather
than fighting the typing, just remove the
hints and leave a comment warning not to
use the gunicorn-specific formatter with
other loggers.
* refactor: generate group aliases on the fly
* chore: remove group alias file check
* chore: drop group alias settings, fix lint
* refactor: rename var to hint it's ignored
* test: update tests
* refactor: move utility to utils
* test: add test
---------
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
* refactor: compute draft aliases on demand
n.b., very slow for full set of aliases
* refactor: simplify and cache email_aliases
The name != "" case is, as far as I can see, unused.
* chore: remove draft alias checks
* chore: remove draft alias/virtual settings
* chore: remove lint
* test: update tests
* test: better mocking
* refactor: move utility to utils
* test: add tests
* chore: handle TERM in datatracker-start.sh
* chore: delay celery start if migration needed
* chore: skip-checks when migrating
* chore: label beat/celery as deleteBeforeUpgrade
Used by the infra-k8s deployment process to flag
these as needing to be shut down before a new
release rolls out.
* chore: increase termination grace periods
* ci: remove stray serviceName
* ci: volumeClaimTemplate name -> volumeMount, not volume
* ci: datatracker listens on containerPort 8000
* ci: services/containers have dt- prefix
* chore: adjust indent for k8s yaml
* ci: use a secret for CELERY_PASSWORD
* fix: touched wrong CELERY_PASSWORD setting
* ci: get rid of the celery pw secretGenerator
* ci: use DB_PASS instead of DBPASS (etc) for k8s
* ci: Fill in django-config.yaml from env vars
* ci: add vault-mappings.txt
* ci: use $CELERY_PASSWORD in rabbitmq.yaml
* ci: moving vault-mappings.txt out of this repo
* Revert "ci: Fill in django-config.yaml from env vars"
This reverts commit 75cd181deb390d3ab21d6887b091d66c80e1d18e.
* Revert "ci: use $CELERY_PASSWORD in rabbitmq.yaml"
This reverts commit f251f9920d07c65413f72fd165cc06acd562c2c7.
* ci: parameterize db OPTIONS setting
