Commit graph

332 commits

Author SHA1 Message Date
Nicolas Giard cf21c4129a
docs: Update README.md 2025-03-08 00:59:26 -05:00
Robert Sparks 554182ef8a
feat: run the docker container as dev (#8606)
* feat: run the docker container as dev

* fix: $@ -> $*

Old bug, but might as well fix it now

---------

Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
2025-03-04 11:42:04 -06:00
Jennifer Richards 041fa83d21
chore: handle errors in app-configure-blobstore.py (#8567) 2025-02-20 14:45:22 -06:00
Robert Sparks 997239a2ea
feat: write objects to blob storage (#8557)
* feat: basic blobstore infrastructure for dev

* refactor: (broken) attempt to put minio console behind nginx

* feat: initialize blobstore with boto3

* fix: abandon attempt to proxy minio. Use docker compose instead.

* feat: beginning of blob writes

* feat: storage utilities

* feat: test buckets

* chore: black

* chore: remove unused import

* chore: avoid f string when not needed

* fix: inform all settings files about blobstores

* fix: declare types for some settings

* ci: point to new target base

* ci: adjust test workflow

* fix: give the tests debug environment a blobstore

* fix: "better" name declarations

* ci: use devblobstore container

* chore: identify places to write to blobstorage

* chore: remove unreachable code

* feat: store materials

* feat: store statements

* feat: store status changes

* feat: store liaison attachments

* feat: store agendas provided with Interim session requests

* chore: capture TODOs

* feat: store polls and chatlogs

* chore: remove unneeded TODO

* feat: store drafts on submit and post

* fix: handle storage during doc expiration and resurrection

* fix: mirror an unlink

* chore: add/refine TODOs

* feat: store slide submissions

* fix: structure slide test correctly

* fix: correct sense of existence check

* feat: store some indexes

* feat: BlobShadowFileSystemStorage

* feat: shadow floorplans / host logos to the blob

* chore: remove unused import

* feat: strip path from blob shadow names

* feat: shadow photos / thumbs

* refactor: combine photo and photothumb blob kinds

The photos / thumbs were already dropped in the same
directory, so let's not add a distinction at this point.

* style: whitespace

* refactor: use kwargs consistently

* chore: migrations

* refactor: better deconstruct(); rebuild migrations

* fix: use new class in mack patch

* chore: add TODO

* feat: store group index documents

* chore: identify more TODO

* feat: store reviews

* fix: repair merge

* chore: remove unnecessary TODO

* feat: StoredObject metadata

* fix: deburr some debugging code

* fix: only set the deleted timestamp once

* chore: correct typo

* fix: get_or_create vs get and test

* fix: avoid the questionable is_seekable helper

* chore: capture future design consideration

* chore: blob store cfg for k8s

* chore: black

* chore: copyright

* ci: bucket name prefix option + run Black

Adds/uses DATATRACKER_BLOB_STORE_BUCKET_PREFIX option. Other changes
are just Black styling.

* ci: fix typo in bucket name expression

* chore: parameters in app-configure-blobstore

Allows use with other blob stores.

* ci: remove verify=False option

* fix: don't return value from __init__

* feat: option to log timing of S3Storage calls

* chore: units

* fix: deleted->null when storing a file

* style: Black

* feat: log as JSON; refactor to share code; handle exceptions

* ci: add ietf_log_blob_timing option for k8s

* test: --no-manage-blobstore option for running tests

* test: use blob store settings from env, if set

* test: actually set a couple more storage opts

* feat: offswitch (#8541)

* feat: offswitch

* fix: apply ENABLE_BLOBSTORAGE to BlobShadowFileSystemStorage behavior

* chore: log timing of blob reads

* chore: import Config from botocore.config

* chore(deps): import boto3-stubs / botocore

botocore is implicitly imported, but make it explicit
since we refer to it directly

* chore: drop type annotation that mypy loudly ignores

* refactor: add storage methods via mixin

Shares code between Document and DocHistory without
putting it in the base DocumentInfo class, which
lacks the name field. Also makes mypy happy.

* feat: add timeout / retry limit to boto client

* ci: let k8s config the timeouts via env

* chore: repair merge resolution typo

* chore: tweak settings imports

* chore: simplify k8s/settings_local.py imports

---------

Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
2025-02-19 17:41:10 -06:00
Robert Sparks ad56d42d9c
ci: dev/test blobstore image (#8431)
* ci: dev/test blobstore image

* chore: removed QEMU step
2025-01-17 12:35:26 -06:00
Robert Sparks e5c4a9f298
feat: additional filesystem monitoring (#8405)
* feat: additional filesystem monitoring

* chore: rename setting for tmp directory

* fix: restructure path to new endpoint

---------

Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
2025-01-09 13:07:51 -06:00
Robert Sparks 2ec7a71edf
chore: remove unused setting from various settings_local templates (#8311) 2024-12-09 10:34:53 -06:00
Nicolas Giard bdf753e052
chore: set gitattributes for normalizing line endings (#8245)
* chore: add proper .gitattributes

* chore: normalize file endings

* chore: fix .gitattributes

* chore: normalize file endings (2)
2024-11-20 16:48:00 -06:00
Kesara Rathnayake a345673c0d
build: Install fonts from xml2rfc-fonts (#6300)
* build: Install fonts from xml2rfc-fonts

Fixes #6294

* fixup! build: Install fonts from xml2rfc-fonts
2024-10-01 10:24:21 -05:00
Lars Eggert 35074660dc
chore: Prevent test suite artifact creation in work directory (#6438)
* chore: Prevent test suite artifact creation in work directory

Also remove a few other stale test assets while I'm here.

* Try and fix CI

* Change IDSUBMIT_REPOSITORY_PATH

* Make the dir

* test: clean up media/nomcom directories

* test: de-dup settings_temp_path_overrides

* chore: remove debug

* refactor: avoid premature import of test_utils

* refactor: drop useless lambda wrapper

---------

Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
2024-09-18 14:08:01 -05:00
Kesara Rathnayake b8c6cb34dd
chore: Remove obsolete version attribute (#7931) 2024-09-12 16:28:44 -05:00
Rich Salz 475cf2fc85
fix: Ensure called from the docker directory (#7689) 2024-07-15 16:32:28 -05:00
Robert Sparks 26ca821f31
Added a note on updating older clones 2024-07-15 12:16:13 -05:00
Nicolas Giard 6fbd8473f2
ci: Update db.Dockerfile to postgres 16 2024-06-23 19:28:10 -04:00
Matthew Holloway f01ef0c915 Adding Linux Docker Desktop install docs 2024-05-29 14:47:49 +12:00
Nicolas Giard 96902bf3b8
chore: fix app-init.sh chown check 2024-05-24 16:41:19 -04:00
Nicolas Giard b951c80a4d
chore: update app-init.sh with linux host chown check 2024-05-24 16:34:41 -04:00
Robert Sparks cedd58f950
feat: obviate ghostlinkd (#7336)
* wip: identify whats needed to obviate ghostlinkd

* fix: hardlink new charter files to ftp directory

* fix: hardlink new charter files to ftp directory (continued)

* chore: bring settings comment up to date

* chore: add archive and ftp dirs to setup of various environments

* fix: test charter submits write to ftp dir

* chore: remove debug

* fix: test charter approval writes to ftp dir

* fix: link review revisions into ftp dir

* fix: link to all archive and ftp on submission post

* chore: clean comments, move action to github issue

* fix: link idindex files to all archive and ftp

* chore: deflake

* chore: remove TODO comment

* fix: use settings

* chore: rename new setting
2024-04-19 16:18:52 -05:00
Jennifer Richards 74ceac7aea
chore: Bump default geckodriver version (#7076)
* style: CRLF -> LF

* chore: Bump default geckodriver version
2024-02-20 16:37:17 -06:00
Lars Eggert 3d44825333
ci: Switch to using geckodriver (#6541)
* Switch to using geckodriver

* Switch to selenium 4

* Undo

* Remove comment

* Fixes

* Restore non-standard line endings to minimize diff

* Undo

* Remove comment

* test: Fix test_upcoming_view_time_zone_selection

The inputs are hidden and managed by JS, so click
the visible elements instead.

* test: Clumsy fix to test_upcoming_materials_modal

Waiting for the button to be clickable does not
work because the modal is still fading in, so does
not actually close. Would be better to check for it
responding, but I haven't found the right way to do
that yet.

* test: Fix test_add_author_forms

Sending '\n' does not seem to work as it did before,
so click the option instead.

Also reverted some fixme hacks that seem to be obe.

* ci: Update base.Dockerfile

* test: add resource limits to dev/tests/debug.sh env

* ci: add upload of geckodriver.log on failure

* ci: run tests as user 1001

* ci: run app-create-dirs as sudo

* ci: set home folder to root to run tests

---------

Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
Co-authored-by: Nicolas Giard <github@ngpixel.com>
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
2023-11-21 15:30:50 -06:00
Nicolas Giard 5edd06fd04
chore: fix app.Dockerfile 2023-11-08 14:28:44 -05:00
Nicolas Giard fab078d14a
ci: Update base.Dockerfile 2023-11-07 21:26:39 -05:00
Nicolas Giard a49daddfad
ci: Update base.Dockerfile 2023-11-07 21:22:47 -05:00
Nicolas Giard 3ae3f3a7fe
ci: Update base.Dockerfile 2023-11-07 21:19:28 -05:00
Jennifer Richards 85927c083d
chore: Move IDSUBMIT_REPOSITORY_PATH for dev (#6218) 2023-08-26 16:14:02 -05:00
Robert Sparks 73137364cc
chore: rename test settings and shorten runserver instructions (#6131) 2023-08-11 11:21:38 -05:00
Robert Sparks dbe1749438
feat: IAB statements (#5940)
* feat: support iab and iesg statements. Import iab statements. (#5895)

* feat: infrastructure for statements doctype

* chore: basic test framework

* feat: basic statement document view

* feat: show replaced statements

* chore: black

* fix: state help for statements

* fix: cleanout non-relevant email expansions

* feat: import iab statements, provide group statements tab

* fix: guard against running import twice

* feat: build redirect csv for iab statements

* fix: set document state on import

* feat: show published date on main doc view

* feat: handle pdf statements

* feat: create new and update statements

* chore: copyright block updates

* chore: remove flakes

* chore: black

* feat: add edit/new buttons for the secretariat

* fix: address PR #5895 review comments

* fix: pin pydantic until inflect catches up (#5901) (#5902)

* chore: re-un-pin pydantic
2023-07-23 11:00:24 -07:00
Nicolas Giard a38b6c1bde
docs: update docker/README.md 2023-07-21 18:59:29 -04:00
Roman Beltiukov a1a88c5244
docs: update VS Code extensions names (#5997)
[ci skip]
2023-07-20 12:26:54 -05:00
Nicolas Giard added8603a
chore(dev): increase dev nginx timeout to 1d 2023-06-19 16:58:22 -04:00
Nicolas Giard 4179d08dbb
chore(dev): silence browserlist warnings 2023-06-15 23:30:10 -04:00
Nicolas Giard f3831de3ef
chore(dev): fix internal static proxy + add /pgadmin to reverse-proxy (#5809) 2023-06-13 10:56:11 -05:00
Rich Salz 32b90b99bc
fix: Make sure "./run" is the invocation (#5724)
* fix: Make sure "./run" is the invocation

* Use $(dirname $0) instead of hacky test

* chore: Update docs
2023-06-08 15:27:23 -05:00
Lars Eggert 860e661cf5
fix: port 8000 -> 8001 (#5788) 2023-06-08 14:15:46 -05:00
Nicolas Giard 52e175b39b
chore: start devcontainer services on re-open (#5776) 2023-06-06 17:07:06 -05:00
Nicolas Giard fd2303044c
chore: rewrite host header in dev nginx proxy config (#5772) 2023-06-06 16:11:32 -05:00
Nicolas Giard cfaf966492
fix: add internal static.ietf.org var for pdfized render (#5765) 2023-06-05 20:18:35 -05:00
Nicolas Giard 055651f5f7
chore: fix app-init.sh 2023-06-05 18:06:19 -04:00
Nicolas Giard 03a192b54c
chore: add nginx reverse proxy to dev environment (#5745)
* chore: add nginx reverse proxy to dev environment

* chore: replace settings_local in docker and backup old
2023-06-05 16:47:06 -05:00
Jennifer Richards 1922a6066e
Merge pull request #5700 from ietf-tools/main
chore: main -> django4
2023-05-31 10:59:09 -03:00
Nicolas Giard 0a11892414
chore: use variable for static.ietf.org references (#5692) 2023-05-25 17:23:18 -05:00
Jennifer Richards 264ff60cd2
Merge pull request #5601 from jennifer-richards/django4
chore: Upgrade to Django 3.0
2023-05-11 11:04:36 -04:00
Nicolas Giard 2163a01adc
chore: move git safe directory command to top 2023-05-10 15:40:22 -04:00
Nicolas Giard 871afb1b86
chore: add git safe directory to docker init script 2023-05-10 15:20:48 -04:00
Jennifer Richards ed571ae50b
chore: Rename DB engine to drop the deprecated "_psycopg2" suffix 2023-05-10 12:40:37 -03:00
Robert Sparks 400a830661
chore: add bibxml-ids dir to container build. (#5590) 2023-05-08 15:16:31 -05:00
Nicolas Giard 82d0d42b28
chore: add pgadmin to dev environment 2023-04-25 18:02:34 -04:00
Nicolas Giard 86a183ec21
chore: cleanup of legacy scripts 2023-04-25 16:42:46 -04:00
Robert Sparks be47b76128
fix: more search_path tweaking (#5535) 2023-04-25 13:35:33 -05:00
Nicolas Giard 7f2043336b
chore: remove alter search_path from app-init.sh 2023-04-25 14:17:17 -04:00