Commit graph

35 commits

Author SHA1 Message Date
Robert Sparks 997239a2ea
feat: write objects to blob storage (#8557)
* feat: basic blobstore infrastructure for dev

* refactor: (broken) attempt to put minio console behind nginx

* feat: initialize blobstore with boto3

* fix: abandon attempt to proxy minio. Use docker compose instead.

* feat: beginning of blob writes

* feat: storage utilities

* feat: test buckets

* chore: black

* chore: remove unused import

* chore: avoid f string when not needed

* fix: inform all settings files about blobstores

* fix: declare types for some settings

* ci: point to new target base

* ci: adjust test workflow

* fix: give the tests debug environment a blobstore

* fix: "better" name declarations

* ci: use devblobstore container

* chore: identify places to write to blobstorage

* chore: remove unreachable code

* feat: store materials

* feat: store statements

* feat: store status changes

* feat: store liaison attachments

* feat: store agendas provided with Interim session requests

* chore: capture TODOs

* feat: store polls and chatlogs

* chore: remove unneeded TODO

* feat: store drafts on submit and post

* fix: handle storage during doc expiration and resurrection

* fix: mirror an unlink

* chore: add/refine TODOs

* feat: store slide submissions

* fix: structure slide test correctly

* fix: correct sense of existence check

* feat: store some indexes

* feat: BlobShadowFileSystemStorage

* feat: shadow floorplans / host logos to the blob

* chore: remove unused import

* feat: strip path from blob shadow names

* feat: shadow photos / thumbs

* refactor: combine photo and photothumb blob kinds

The photos / thumbs were already dropped in the same
directory, so let's not add a distinction at this point.

* style: whitespace

* refactor: use kwargs consistently

* chore: migrations

* refactor: better deconstruct(); rebuild migrations

* fix: use new class in mack patch

* chore: add TODO

* feat: store group index documents

* chore: identify more TODO

* feat: store reviews

* fix: repair merge

* chore: remove unnecessary TODO

* feat: StoredObject metadata

* fix: deburr some debugging code

* fix: only set the deleted timestamp once

* chore: correct typo

* fix: get_or_create vs get and test

* fix: avoid the questionable is_seekable helper

* chore: capture future design consideration

* chore: blob store cfg for k8s

* chore: black

* chore: copyright

* ci: bucket name prefix option + run Black

Adds/uses DATATRACKER_BLOB_STORE_BUCKET_PREFIX option. Other changes
are just Black styling.

* ci: fix typo in bucket name expression

* chore: parameters in app-configure-blobstore

Allows use with other blob stores.

* ci: remove verify=False option

* fix: don't return value from __init__

* feat: option to log timing of S3Storage calls

* chore: units

* fix: deleted->null when storing a file

* style: Black

* feat: log as JSON; refactor to share code; handle exceptions

* ci: add ietf_log_blob_timing option for k8s

* test: --no-manage-blobstore option for running tests

* test: use blob store settings from env, if set

* test: actually set a couple more storage opts

* feat: offswitch (#8541)

* feat: offswitch

* fix: apply ENABLE_BLOBSTORAGE to BlobShadowFileSystemStorage behavior

* chore: log timing of blob reads

* chore: import Config from botocore.config

* chore(deps): import boto3-stubs / botocore

botocore is implicitly imported, but make it explicit
since we refer to it directly

* chore: drop type annotation that mypy loudly ignores

* refactor: add storage methods via mixin

Shares code between Document and DocHistory without
putting it in the base DocumentInfo class, which
lacks the name field. Also makes mypy happy.

* feat: add timeout / retry limit to boto client

* ci: let k8s config the timeouts via env

* chore: repair merge resolution typo

* chore: tweak settings imports

* chore: simplify k8s/settings_local.py imports

---------

Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
2025-02-19 17:41:10 -06:00
Robert Sparks 50aa3995ee
chore: reconfigure pylint in dev containers (#6817) 2023-12-20 16:41:33 -06:00
Nicolas Giard f3831de3ef
chore(dev): fix internal static proxy + add /pgadmin to reverse-proxy (#5809) 2023-06-13 10:56:11 -05:00
Nicolas Giard 52e175b39b
chore: start devcontainer services on re-open (#5776) 2023-06-06 17:07:06 -05:00
Nicolas Giard 03a192b54c
chore: add nginx reverse proxy to dev environment (#5745)
* chore: add nginx reverse proxy to dev environment

* chore: replace settings_local in docker and backup old
2023-06-05 16:47:06 -05:00
Nicolas Giard 0a11892414
chore: use variable for static.ietf.org references (#5692) 2023-05-25 17:23:18 -05:00
Nicolas Giard 82d0d42b28
chore: add pgadmin to dev environment 2023-04-25 18:02:34 -04:00
Nicolas Giard ad5280ba29
chore: update devcontainer.json 2023-04-25 16:55:13 -04:00
Robert Sparks ac6f96185e
Merge remote-tracking branch 'ietf-tools/main' into feat/postgres 2023-02-13 17:23:09 -06:00
Robert Sparks eaa7a7ed96
fix: remove unused and now breaking docker-in-docker feature. 2023-02-13 17:20:31 -06:00
Nicolas Giard a60d52253e
chore: use prebuilt pg db image in dev (#4890) 2022-12-15 14:00:51 -06:00
Nicolas Giard f2e9701302
ci: pg db nightly build script 2022-12-13 20:29:57 +00:00
Nicolas Giard deb01f2d83
test: remove cypress + migrate existing tests to playwright (#4781)
* test: remove cypress + migrate existing tests to playwright + update workflows

* ci: remove leftover condition in ci-run-tests workflow

* ci: add missing matrix to tests-playwright-legacy

* ci: run playwright legacy as user 1001

* ci: set home var for firefox

* ci: allow firefox to run as root for tests
2022-11-22 16:26:19 -06:00
Nicolas Giard 9e55ec1fd4
chore: improve devcontainer build (#4241) 2022-07-23 10:33:41 -04:00
Nicolas Giard 6ba91145ed
fix(agenda-neue): responsive UI fixes + no materials warning icon (#4229)
* fix: agenda-neue - go to current meeting in URL if not provided

* fix: agenda neue - various responsive UI fixes + no materials icon
2022-07-19 11:06:07 -05:00
Nicolas Giard aa9490faf6
feat(ui): new dynamic agenda view (#4086)
* feat: agenda page in vue (wip)

* feat: scroll to agenda day

* fix: vue 3 composition api + eslint settings

* fix: agenda day scroll match indicator

* fix: convert vite deps to yarn

* fix: missing lodash + legacy build step

* fix: agenda - move calendar into drawer

* fix: improve agenda filter UI

* fix: download ics + move agenda into own component

* feat: use fullcalendar for agenda calendar view (wip)

* feat: add events to agenda calendar

* feat: agenda filter UI improvements

* feat: agenda add to calendar dropdown

* feat: agenda calendar filter + timezone + event coloring

* feat: agenda calendar color improvements

* chore: exclude dist-neue from git

* feat: agenda calendar event modal

* fix: rebuild yarn deps

* chore: add run migration task to vscode

* fix: agenda buttons display flag

* feat: agenda event modal component

* feat: show calendar event quick info on hover

* fix: clear calendar quick info on timezone change

* feat: agenda list view improvements

* feat: agenda list row coloring

* feat: agenda list note

* feat: agenda list icons for office hours + hackathon

* fix: agenda top links

* refactor: use pinia as store for agenda components

* feat: agenda jump to now

* fix: agenda mobile improvements

* feat: agenda search

* feat: agenda search improvements

* feat: agenda event recordings buttons for post-meeting

* fix: agenda switch to meeting timezone on load

* feat: agenda pre & live session buttons

* fix: remove agenda utc + personalize links in top menu

* feat: add pre-vue loading state on page load

* feat: filter from agenda picker mode

* fix: agenda UI improvements

* fix: django-vite non-dev mode

* chore: update yarn dependencies for vue + vite

* feat: agenda settings panel + UI improvements

* feat: agenda settings colors + import/export feature

* feat: agenda color assignments + responsive UI improvements

* feat: agenda realtime red line + debug datetime offset

* feat: agenda add aria labels for settings

* feat: add new agenda path + pages/menu

* fix: bring base/menu.html up to main

* fix: agenda various fixes

* test: add new agenda item to meetings menu for item count

* chore: restore devcontainer extensions list

* fix: agenda UI improvements + montserrat default font

* feat: agenda bolder text + hide event icons options

* feat: agenda warning badge

* fix: agenda various UI improvements + intersectionObserver fix

* feat: agenda floorplan page + various UI improvements

* feat: agenda floor plan pin

* feat: view floor plan room from agenda

* feat: agenda floor plan mobile optimization

* feat: adjust calendar options + default calendar view in settings

* feat: agenda persist picked events + change base font only on new agenda page

* feat: agenda mobile view optimizations

* fix: add .vite to cached volumes

* fix: mobile view for filters, calendar, settings panels

* test: upgrade cypress existing tests to work on bs5 + update dependencies

* fix: use named url patterns to avoid hardcoded URLs. Add rudimentary test coverage for the neue views.

Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
2022-07-13 16:20:23 -05:00
Nicolas Giard 5d5fc75958
chore: fix missing devcontainer dependencies (#4024) 2022-05-31 16:47:06 -05:00
Nicolas Giard e02ce05612
chore: make /assets a named volume for dev container (#4018)
* chore: make /assets a static volume for dev container

* chore: fix create-dirs script to point to /assets
2022-05-31 16:43:29 -05:00
Nicolas Giard 742ce5a08e
chore: make dev container use base image 2022-05-24 16:49:37 -04:00
Nicolas Giard f1472ffcac
chore: run devcontainer as user vscode instead of root (#3948) 2022-05-13 14:13:50 -05:00
Nicolas Giard 8ea883cb7e
chore: switch to Yarn 3 (#3819)
* refactor: switch to yarn 3

* chore: add other os+cpu to yarn supported architectures

* chore: add gitattributes

* chore: add missing yarn native modules rebuild step

* chore: replace npx with yarn in test_runner + update readme

* chore: remove bootstrap-datepicker github dependency, use local instead
2022-04-25 15:54:05 -05:00
Nicolas Giard aba2aa2b59
chore: add vscode extensions volume to reduce ext reinstalls (#3833) 2022-04-20 10:49:04 -05:00
Nicolas Giard 562742d653
chore: add memcached + docker-cli to dev container (#3691) 2022-03-19 13:05:59 -04:00
Nicolas Giard ec3b525567
chore: remove django settings module from devcontainer config (#3673) 2022-03-18 16:35:49 -04:00
NGPixel 1771d6ea77
chore: improve dev docker for offline + other fixes 2022-03-15 22:22:30 -04:00
Nicolas Giard e6584bb717
chore: bring docker-compose to root level (#3642) 2022-03-14 17:02:53 -05:00
NGPixel 9ae07e44e6
chore: fix node_modules volume for docker dev 2022-03-10 13:03:45 -05:00
nick b98c7ec3bd feat: add nomcom questionnaires tabs tests
Commit ready for merge.
 - Legacy-Id: 19698
2021-11-22 19:34:22 +00:00
nick db34d83feb feat: cypress JS testing for agenda meetings + weekview swimlane (WIP)
- Legacy-Id: 19693
2021-11-18 23:48:23 +00:00
Robert Sparks 164ee7792c Adjust docker volume mounting strategy.
- Legacy-Id: 19638
2021-11-11 17:20:20 +00:00
nick d52ad4fbc2 misc: import docker improvements from 7.39.1.dev2
- Legacy-Id: 19629
2021-11-10 21:51:55 +00:00
nick c2979adc2b misc: devcontainer - django debug toolbar profile + dockerfile cleanup
- Legacy-Id: 19529
2021-11-02 18:08:04 +00:00
nick 9c903f1b01 misc: improve devcontainer environment setup
- Legacy-Id: 19481
2021-10-27 20:42:03 +00:00
nick fe502da612 misc: devcontainer docker setup fix
- Legacy-Id: 19479
2021-10-27 18:44:19 +00:00
nick 1e7b44787f misc: devcontainer configuration for vscode
- Legacy-Id: 19477
2021-10-27 16:41:02 +00:00