Commit graph

35 commits

Author SHA1 Message Date
Jennifer Richards f921cdba5d
fix: disable nginx body size check (#7803)
* fix: increase nginx client_max_body_size

* fix: entirely disable nginx body size check
2024-08-07 10:37:26 -05:00
Jennifer Richards b13a606a24
feat: recognize HTTPS via proxy (#7765)
* feat: set SECURE_PROXY_SSL_HEADER

* chore: update comment
2024-08-05 09:00:15 -05:00
Jennifer Richards 247361b7dd
ci: better access logs+redirect auth URLs+fix X-Request-Start header (#7700)
* fix: silence nginx healthcheck logs

* fix: nginx logs in JSON

* fix: typos in nginx conf

* refactor: repeat less nginx config

* fix: log more req headers from gunicorn

* fix: redirect auth->datatracker, not deny

* feat: log X-Forwarded-Proto
2024-07-30 20:55:07 -03:00
Jennifer Richards 30510d97d8
ci: fix migration container secrets (#7694) 2024-07-15 17:57:25 -05:00
Jennifer Richards 18bb793b2d
feat: add nginx, robots.txt, HTTP headers (#7683)
* feat: nginx + robots.txt

* feat: minimal /health/ endpoint

* ci: startupProbe for datatracker pod

* ci: probe auth pod; set timeoutSeconds

* feat: add CSP and other headers to nginx

* fix: typo in nginx.conf

* feat: split auth/dt nginx confs

* test: test health endpoint

* ci: auth service on port 80

We'll remove http-old (8080) in the future.

* ci: rename auth container/nginx cfg
2024-07-15 16:45:51 -05:00
Nicolas Giard c9dab33f83
ci: migrate to using secret instead of configmap (#7685) 2024-07-15 16:32:47 -05:00
Jennifer Richards 9c95689baf
ci: run migrations via init container (#7663)
* chore: split auth/dt/migration start

* chore: migrations via init container

* chore: no checks before migration

* chore: indicate celery start

* chore: additional log output
2024-07-10 15:33:03 -05:00
Jennifer Richards 39961374d9
ci: persistent db connection options (#7664) 2024-07-10 15:32:33 -05:00
Jennifer Richards 4a6749b5a6
ci: add auth pod 2024-07-09 11:51:57 -03:00
Jennifer Richards 6a96a69234
feat: improve / clean up logging (#7591)
* refactor: always use console log handler

* refactor: json for k8s, plain otherwise

* chore: remove syslog from wsgi.py

* chore: remove debug.log()

* chore: drop syslog from settings.py

* refactor: use log.log() in person.utils

* refactor: fetch_meeting_attendance->log.log()

* chore: gunicorn logs as JSON (wip)

* feat: better json log formatting

* refactor: improve log config

* feat: gunicorn access log fields

* fix: remove type hints

The gunicorn logger plays tricks with the
LogRecord args parameter to let it have string
keys instead of being a simple tuple.
The mypy tests rightly flag this. Rather
than fighting the typing, just remove the
hints and leave a comment warning not to
use the gunicorn-specific formatter with
other loggers.
2024-06-26 14:53:05 -05:00
Jennifer Richards 66b3095878
ci: writeable /home/datatracker 2024-06-20 11:34:57 -03:00
Jennifer Richards b3fb20cf97
ci: /var/cache/xml2rfc-cache for celery 2024-06-20 11:27:21 -03:00
Jennifer Richards a28bf98402
ci: writable /var/cache/xml2rfc 2024-06-20 11:22:03 -03:00
Jennifer Richards ad9fb85551
ci: replace htpasswd2 with /bin/true (#7567) 2024-06-19 16:24:45 -05:00
Jennifer Richards 74cdc15853
ci: get registration API key from env (#7564) 2024-06-19 13:43:43 -05:00
Nicolas Giard 7a1863317a
chore: Update k8s/datatracker.yaml 2024-06-18 18:46:03 -04:00
Jennifer Richards 0ac2ae12dc
refactor: don't use filesystem for group aliases (#7556)
* refactor: generate group aliases on the fly

* chore: remove group alias file check

* chore: drop group alias settings, fix lint

* refactor: rename var to hint it's ignored

* test: update tests

* refactor: move utility to utils

* test: add test

---------

Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
2024-06-18 08:28:09 -05:00
Jennifer Richards 6338f4594f
refactor: don't use filesystem for draft aliases (#7555)
* refactor: compute draft aliases on demand

n.b., very slow for full set of aliases

* refactor: simplify and cache email_aliases

The name != "" case is, as far as I can see, unused.

* chore: remove draft alias checks

* chore: remove draft alias/virtual settings

* chore: remove lint

* test: update tests

* test: better mocking

* refactor: move utility to utils

* test: add tests
2024-06-18 08:13:10 -05:00
Jennifer Richards 7541c21486
chore: scoutapm shutdown fix (#7538) 2024-06-14 15:51:21 -05:00
Jennifer Richards c1941df7e7
chore: adjustments for k8s deployment (#7531)
* chore: handle TERM in datatracker-start.sh

* chore: delay celery start if migration needed

* chore: skip-checks when migrating

* chore: label beat/celery as deleteBeforeUpgrade

Used by the infra-k8s deployment process to flag
these as needing to be shut down before a new
release rolls out.

* chore: increase termination grace periods
2024-06-14 09:28:14 -05:00
Jennifer Richards bdc4b618bb
chore: configure logging for k8s (#7525) 2024-06-13 09:35:43 -03:00
Jennifer Richards b744f2b318 chore: switch back to uid/gid=1000 2024-05-13 21:41:36 -04:00
Jennifer Richards 212a915bd3 ci: don't set APP_API_TOKENS in django-config 2024-05-13 21:41:36 -04:00
Jennifer Richards 5486345ab0 ci: remove unnecessary helper
b64decode defaults to validate=false, which discards chars
outside the base64 alphabet. That includes whitespace.
2024-05-13 21:41:36 -04:00
Jennifer Richards c1a7a60eb9 ci: handle b64-encoded APP_API_TOKENS 2024-05-13 21:41:36 -04:00
Jennifer Richards 49a3cdc43c ci: "true" 2024-05-13 21:41:36 -04:00
Jennifer Richards 8b3d330bff ci: metrics.portName annotation 2024-05-13 21:41:36 -04:00
Jennifer Richards 3a4939cc0b ci: collect memcached metrics (#7410)
* ci: collect memcached metrics

* ci: use nobody/nobody for metrics container
2024-05-13 21:41:36 -04:00
Jennifer Richards dd46a8af6f ci: use ietfa uid/gid for datatracker user (#7407)
* ci: use ietfa uid/gid for datatracker user

* chore: add comment
2024-05-13 21:41:36 -04:00
Jennifer Richards 867360e96f ci: k8s fixup (#7401)
* ci: remove stray serviceName

* ci: volumeClaimTemplate name -> volumeMount, not volume

* ci: datatracker listens on containerPort 8000

* ci: services/containers have dt- prefix

* chore: adjust indent for k8s yaml

* ci: use a secret for CELERY_PASSWORD

* fix: touched wrong CELERY_PASSWORD setting

* ci: get rid of the celery pw secretGenerator

* ci: use DB_PASS instead of DBPASS (etc) for k8s

* ci: Fill in django-config.yaml from env vars

* ci: add vault-mappings.txt

* ci: use $CELERY_PASSWORD in rabbitmq.yaml

* ci: moving vault-mappings.txt out of this repo

* Revert "ci: Fill in django-config.yaml from env vars"

This reverts commit 75cd181deb390d3ab21d6887b091d66c80e1d18e.

* Revert "ci: use $CELERY_PASSWORD in rabbitmq.yaml"

This reverts commit f251f9920d07c65413f72fd165cc06acd562c2c7.

* ci: parameterize db OPTIONS setting
2024-05-13 21:41:36 -04:00
Nicolas Giard d075404fdb refactor: add beat + celery deployments 2024-05-13 21:41:36 -04:00
Nicolas Giard 05bd47cbad chore: add rabbitmq vol claim template 2024-05-13 21:41:36 -04:00
Nicolas Giard 24309c2b04 refactor: move nfs entry upstream 2024-05-13 21:41:36 -04:00
Nicolas Giard 0a3bb9e381 refactor: move node affinity to upstream 2024-05-13 21:41:36 -04:00
Nicolas Giard 3ea70f2ceb refactor: helm to kustomize (wip) 2024-05-13 21:41:36 -04:00