altf4arnold/datatracker
1
0
Fork 0
Code Issues Pull requests Actions 3 Packages Projects Releases Wiki Activity
16179 commits 1 branch 0 tags 2.6 GiB
Commit graph

16179 commits

This branch
This branch
All branches
Author SHA1 Message Date
Jennifer Richards ffb9eb12ff
refactor: generate I-D bibxml files via celery (#7426) 
* refactor: task to generate_draft_bibxml_files

* test: test task/utility methods

* chore: add periodic task

* chore: remove generate_draft_bibxml_files.py

* chore: further prune /bin/hourly
 2024-05-16 11:37:29 -05:00
Robert Sparks de8b3b5ce3
ci: merge pull request #7416 from ietf-tools/feat/k8s 
ci: k8s deployment files
 2024-05-16 10:58:53 -05:00
Jennifer Richards d9c6ae7b7e
chore: remove syslog handler from settings.py (#7427) 
System-level integration like this really needs
to be in settings_local. This was causing problems
when running in the dev environment.
 2024-05-16 10:55:23 -05:00
Jennifer Richards 0d68646fdc
chore: suppress pyflakes check 2024-05-15 20:47:00 -03:00
Jennifer Richards 1c3825bdec
test: ensure logs only go to console during tests 2024-05-15 18:25:59 -03:00
Jennifer Richards 06b99fa64b
chore: remove unused import 2024-05-15 16:13:47 -03:00
Jennifer Richards 2b816630ef
Merge branch 'refs/heads/main' into feat/k8s 
# Conflicts:
#	ietf/settings.py
#	ietf/utils/__init__.py
#	ietf/utils/log.py
 2024-05-15 15:35:32 -03:00
Jennifer Richards c59d6122d9
refactor: send_nomcom_reminders via celery task (#7424) 
* refactor: send_reminders.py -> celery task

* chore: add PeriodicTask

* chore: remove management command and tests
 2024-05-15 13:25:15 -05:00
Jennifer Richards c9f35987bc
refactor: expire last calls via celery (#7417) 
* feat: expire_last_calls_task

* feat: create periodic task for last calls

* test: test new task

* chore: remove expire-last-calls script
 2024-05-15 12:04:47 -05:00
Jennifer Richards 46a00acefc
refactor: sync to RFC Editor queue via celery (#7415) 
* feat: rfc_editor_queue_updates_task

* refactor: use rfc_editor_queue_updates_task()

* chore: remove now-unused scripts

* test: test new task

* chore: de-lint
 2024-05-14 18:56:14 -05:00
Jennifer Richards a4e0354090
feat: get tool versions without VersionInfo model (#7418) 
* feat: get tool versions without VersionInfo model

* chore: remove update_external_command_info call

* feat: get tool version without VersionInfo

* chore: Remove VersionInfo model

* chore: Migration to remove VersionInfo

* fix: handle errors better; ignore stderr

* fix: type annotation
 2024-05-14 18:53:31 -05:00
Jennifer Richards 48e0aa23f5
refactor: clean up logging (#7419) 
* fix: log to stdout/stderr in json format

* chore: remove UTILS_LOGGER_LEVELS

This is not used (there _is_ a setting for the
django.security logger in settings_local.py on
production, but it is redundant with the
settings.LOGGING configuration and is not doing
anything).

* chore: revert to debug_console django logging

* chore: log.log to syslog via datatracker logger

* chore: remove unused imports

---------

Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
 2024-05-14 18:47:40 -05:00
Jennifer Richards 235ac8b2a6
refactor: idnits2 mgmt cmds -> tasks (#7421) 
* feat: tasks for generate_idnits2_rfc* mgmt cmds

* chore: create periodic tasks

* chore: remove mgmt cmds from bin/hourly

* test: test new tasks

* chore: remove now-unused scripts

* refactor: unitize Idnits2SupportTests
 2024-05-14 18:46:12 -05:00
Jennifer Richards ea0648d75f
Merge branch 'main' into feat/k8s 2024-05-14 11:18:31 -03:00
Jennifer Richards c0a12fa8b2
chore(deps): fix requirements.txt versions (#7414) 
* not ready for django-stubs==5
 * pin types-pytz to match existing pytz pin
 2024-05-14 09:08:18 -05:00
Nicolas Giard 6cb0fa5ed4
chore: Remove helm config from .editorconfig 2024-05-13 21:42:21 -04:00
Nicolas Giard 635ca28ec1 chore: remove helm stuff 2024-05-13 21:41:36 -04:00
Jennifer Richards 38b0b2c035 feat: get tool versions without VersionInfo model (#7393) (#7403) 
* chore: remove update_external_command_info call

* feat: get tool version without VersionInfo

* chore: Remove VersionInfo model

* chore: Migration to remove VersionInfo

* fix: handle errors better; ignore stderr
 2024-05-13 21:41:36 -04:00
Jennifer Richards b744f2b318 chore: switch back to uid/gid=1000 2024-05-13 21:41:36 -04:00
Jennifer Richards 212a915bd3 ci: don't set APP_API_TOKENS in django-config 2024-05-13 21:41:36 -04:00
Jennifer Richards 5486345ab0 ci: remove unnecessary helper 
b64decode defaults to validate=false, which discards chars
outside the base64 alphabet. That includes whitespace.
 2024-05-13 21:41:36 -04:00
Jennifer Richards c1a7a60eb9 ci: handle b64-encoded APP_API_TOKENS 2024-05-13 21:41:36 -04:00
Jennifer Richards 49a3cdc43c ci: "true" 2024-05-13 21:41:36 -04:00
Jennifer Richards 8b3d330bff ci: metrics.portName annotation 2024-05-13 21:41:36 -04:00
Jennifer Richards 3a4939cc0b ci: collect memcached metrics (#7410) 
* ci: collect memcached metrics

* ci: use nobody/nobody for metrics container
 2024-05-13 21:41:36 -04:00
Jennifer Richards dd46a8af6f ci: use ietfa uid/gid for datatracker user (#7407) 
* ci: use ietfa uid/gid for datatracker user

* chore: add comment
 2024-05-13 21:41:36 -04:00
Jennifer Richards 7e56b2e923 ci: drop helm chart GHA step 2024-05-13 21:41:36 -04:00
Jennifer Richards 867360e96f ci: k8s fixup (#7401) 
* ci: remove stray serviceName

* ci: volumeClaimTemplate name -> volumeMount, not volume

* ci: datatracker listens on containerPort 8000

* ci: services/containers have dt- prefix

* chore: adjust indent for k8s yaml

* ci: use a secret for CELERY_PASSWORD

* fix: touched wrong CELERY_PASSWORD setting

* ci: get rid of the celery pw secretGenerator

* ci: use DB_PASS instead of DBPASS (etc) for k8s

* ci: Fill in django-config.yaml from env vars

* ci: add vault-mappings.txt

* ci: use $CELERY_PASSWORD in rabbitmq.yaml

* ci: moving vault-mappings.txt out of this repo

* Revert "ci: Fill in django-config.yaml from env vars"

This reverts commit 75cd181deb390d3ab21d6887b091d66c80e1d18e.

* Revert "ci: use $CELERY_PASSWORD in rabbitmq.yaml"

This reverts commit f251f9920d07c65413f72fd165cc06acd562c2c7.

* ci: parameterize db OPTIONS setting
 2024-05-13 21:41:36 -04:00
Nicolas Giard d075404fdb refactor: add beat + celery deployments 2024-05-13 21:41:36 -04:00
Nicolas Giard 05bd47cbad chore: add rabbitmq vol claim template 2024-05-13 21:41:36 -04:00
Nicolas Giard 24309c2b04 refactor: move nfs entry upstream 2024-05-13 21:41:36 -04:00
Nicolas Giard 0a3bb9e381 refactor: move node affinity to upstream 2024-05-13 21:41:36 -04:00
Nicolas Giard 3ea70f2ceb refactor: helm to kustomize (wip) 2024-05-13 21:41:36 -04:00
Jennifer Richards e35b46eed8 ci: fix celery scout env var names (#7373) 2024-05-13 21:41:36 -04:00
Jennifer Richards d2623de615 feat: config celery logging via Django (#7371) 
* feat: config celery logger via Django

* feat: Disable celery's logging config
 2024-05-13 21:41:36 -04:00
Jennifer Richards 8f87573144 ci: Fill in settings_local for docker 2024-05-13 21:41:36 -04:00
Jennifer Richards 4e2b9ce7a6 ci: Non-root for celery containers (#7368) 2024-05-13 21:41:36 -04:00
Jennifer Richards 70c32254a9 ci: non-root user for scout containers 2024-05-13 21:41:36 -04:00
Jennifer Richards c8ee43da95 ci: run datatracker pod as non-root user (#7366) 
* feat: patch_libraries management command

* ci: Patch libraries in docker img build

* ci: non-root datatracker user

* ci: securityContext for datatracker pod
 2024-05-13 21:41:36 -04:00
Jennifer Richards 30a4a5a77b ci: run rabbitmq as non-root (#7362) 
* ci: securityContext for rabbitmq

* ci: logs from rabbitmq as json to console

* ci: tmp volume for rabbitmq

Needed since rootfs is now read-only

* ci: fix permissions on /var/lib/rabbitmq vol

Rabbitmq needs to be able to write to the fs at
/var/lib/rabbitmq. It may be possible to get rid
of the initContainer and use fsGroup in the pod
securityContext to manage this, but that does not
work for the hostVolume mounts I use for dev.
The solution here moves the actual mount to the
rabbitmq/ directory in the rabbitmq-data volume
and uses an initContainer to set the permissions
on that. That should work for any volume type.
 2024-05-13 21:41:36 -04:00
Jennifer Richards 2c9c61d878 ci: more memcached mem; set securityContext (#7356) 2024-05-13 21:41:36 -04:00
Jennifer Richards 9379bbad7d ci: remove reminder comment 2024-05-13 21:41:36 -04:00
Jennifer Richards b50e60b05f ci: default to static.ietf.org in helm chart 2024-05-13 21:41:36 -04:00
Jennifer Richards 26f2306316 ci: add scout container to celery pod (#7354) 
* ci: Add scout container to celery pod

* ci: Refactor scoutapm settings in values.yaml
 2024-05-13 21:41:36 -04:00
Jennifer Richards 6ccde89a68 ci: do not re-run yarn / collectstatics (#7353) 2024-05-13 21:41:36 -04:00
Jennifer Richards d14cbd10e9 ci: refactor scout sidecar (#7338) 
* ci: remove stray comma in settings_local.py

* ci: move scout sidecar to containers

initContainers sidecars not supported until kubernetes 1.29
 2024-05-13 21:41:36 -04:00
Jennifer Richards 90dc303293 ci: Add scoutapm sidecar container (#7330) 
* ci: Add scoutapm sidecar container

* ci: Configure ScoutAPM via helm
 2024-05-13 21:41:36 -04:00
Jennifer Richards ffcf74bd03 chore: Remove accidental commits... (#7329) 2024-05-13 21:41:36 -04:00
Jennifer Richards 64441be330 chore: Remove accidentally committed file 2024-05-13 21:41:36 -04:00
Jennifer Richards ac5155122c ci: label PVCs 2024-05-13 21:41:36 -04:00