997239a2ea
* feat: basic blobstore infrastructure for dev * refactor: (broken) attempt to put minio console behind nginx * feat: initialize blobstore with boto3 * fix: abandon attempt to proxy minio. Use docker compose instead. * feat: beginning of blob writes * feat: storage utilities * feat: test buckets * chore: black * chore: remove unused import * chore: avoid f string when not needed * fix: inform all settings files about blobstores * fix: declare types for some settings * ci: point to new target base * ci: adjust test workflow * fix: give the tests debug environment a blobstore * fix: "better" name declarations * ci: use devblobstore container * chore: identify places to write to blobstorage * chore: remove unreachable code * feat: store materials * feat: store statements * feat: store status changes * feat: store liaison attachments * feat: store agendas provided with Interim session requests * chore: capture TODOs * feat: store polls and chatlogs * chore: remove unneeded TODO * feat: store drafts on submit and post * fix: handle storage during doc expiration and resurrection * fix: mirror an unlink * chore: add/refine TODOs * feat: store slide submissions * fix: structure slide test correctly * fix: correct sense of existence check * feat: store some indexes * feat: BlobShadowFileSystemStorage * feat: shadow floorplans / host logos to the blob * chore: remove unused import * feat: strip path from blob shadow names * feat: shadow photos / thumbs * refactor: combine photo and photothumb blob kinds The photos / thumbs were already dropped in the same directory, so let's not add a distinction at this point. * style: whitespace * refactor: use kwargs consistently * chore: migrations * refactor: better deconstruct(); rebuild migrations * fix: use new class in mack patch * chore: add TODO * feat: store group index documents * chore: identify more TODO * feat: store reviews * fix: repair merge * chore: remove unnecessary TODO * feat: StoredObject metadata * fix: deburr some debugging code * fix: only set the deleted timestamp once * chore: correct typo * fix: get_or_create vs get and test * fix: avoid the questionable is_seekable helper * chore: capture future design consideration * chore: blob store cfg for k8s * chore: black * chore: copyright * ci: bucket name prefix option + run Black Adds/uses DATATRACKER_BLOB_STORE_BUCKET_PREFIX option. Other changes are just Black styling. * ci: fix typo in bucket name expression * chore: parameters in app-configure-blobstore Allows use with other blob stores. * ci: remove verify=False option * fix: don't return value from __init__ * feat: option to log timing of S3Storage calls * chore: units * fix: deleted->null when storing a file * style: Black * feat: log as JSON; refactor to share code; handle exceptions * ci: add ietf_log_blob_timing option for k8s * test: --no-manage-blobstore option for running tests * test: use blob store settings from env, if set * test: actually set a couple more storage opts * feat: offswitch (#8541) * feat: offswitch * fix: apply ENABLE_BLOBSTORAGE to BlobShadowFileSystemStorage behavior * chore: log timing of blob reads * chore: import Config from botocore.config * chore(deps): import boto3-stubs / botocore botocore is implicitly imported, but make it explicit since we refer to it directly * chore: drop type annotation that mypy loudly ignores * refactor: add storage methods via mixin Shares code between Document and DocHistory without putting it in the base DocumentInfo class, which lacks the name field. Also makes mypy happy. * feat: add timeout / retry limit to boto client * ci: let k8s config the timeouts via env * chore: repair merge resolution typo * chore: tweak settings imports * chore: simplify k8s/settings_local.py imports --------- Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
65 lines
3.2 KiB
Docker
65 lines
3.2 KiB
Docker
FROM ghcr.io/ietf-tools/datatracker-app-base:latest
|
|
LABEL maintainer="IETF Tools Team <tools-discuss@ietf.org>"
|
|
|
|
ENV DEBIAN_FRONTEND=noninteractive
|
|
|
|
# Install needed packages and setup non-root user.
|
|
ARG USERNAME=dev
|
|
ARG USER_UID=1000
|
|
ARG USER_GID=$USER_UID
|
|
COPY docker/scripts/app-setup-debian.sh /tmp/library-scripts/docker-setup-debian.sh
|
|
RUN sed -i 's/\r$//' /tmp/library-scripts/docker-setup-debian.sh && chmod +x /tmp/library-scripts/docker-setup-debian.sh
|
|
|
|
# Add Postgresql Apt Repository to get 14
|
|
RUN echo "deb http://apt.postgresql.org/pub/repos/apt $(. /etc/os-release && echo "$VERSION_CODENAME")-pgdg main" | tee /etc/apt/sources.list.d/pgdg.list
|
|
RUN wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
|
|
|
|
RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
|
|
&& apt-get install -y --no-install-recommends postgresql-client-14 pgloader \
|
|
# Remove imagemagick due to https://security-tracker.debian.org/tracker/CVE-2019-10131
|
|
&& apt-get purge -y imagemagick imagemagick-6-common \
|
|
# Install common packages, non-root user
|
|
# Syntax: ./docker-setup-debian.sh [install zsh flag] [username] [user UID] [user GID] [upgrade packages flag] [install Oh My Zsh! flag] [Add non-free packages]
|
|
&& bash /tmp/library-scripts/docker-setup-debian.sh "true" "${USERNAME}" "${USER_UID}" "${USER_GID}" "false" "true" "true"
|
|
|
|
# Setup default python tools in a venv via pipx to avoid conflicts
|
|
ENV PIPX_HOME=/usr/local/py-utils \
|
|
PIPX_BIN_DIR=/usr/local/py-utils/bin
|
|
ENV PATH=${PATH}:${PIPX_BIN_DIR}
|
|
COPY docker/scripts/app-setup-python.sh /tmp/library-scripts/docker-setup-python.sh
|
|
RUN sed -i 's/\r$//' /tmp/library-scripts/docker-setup-python.sh && chmod +x /tmp/library-scripts/docker-setup-python.sh
|
|
RUN bash /tmp/library-scripts/docker-setup-python.sh "none" "/usr/local" "${PIPX_HOME}" "${USERNAME}"
|
|
|
|
# Setup nginx
|
|
COPY docker/scripts/app-setup-nginx.sh /tmp/library-scripts/docker-setup-nginx.sh
|
|
RUN sed -i 's/\r$//' /tmp/library-scripts/docker-setup-nginx.sh && chmod +x /tmp/library-scripts/docker-setup-nginx.sh
|
|
RUN bash /tmp/library-scripts/docker-setup-nginx.sh
|
|
COPY docker/configs/nginx-proxy.conf /etc/nginx/sites-available/default
|
|
COPY docker/configs/nginx-502.html /var/www/html/502.html
|
|
|
|
# Remove library scripts for final image
|
|
RUN rm -rf /tmp/library-scripts
|
|
|
|
# Copy the startup file
|
|
COPY docker/scripts/app-init.sh /docker-init.sh
|
|
COPY docker/scripts/app-start.sh /docker-start.sh
|
|
RUN sed -i 's/\r$//' /docker-init.sh && chmod +rx /docker-init.sh
|
|
RUN sed -i 's/\r$//' /docker-start.sh && chmod +rx /docker-start.sh
|
|
|
|
# Fix user UID / GID to match host
|
|
RUN groupmod --gid $USER_GID $USERNAME \
|
|
&& usermod --uid $USER_UID --gid $USER_GID $USERNAME \
|
|
&& chown -R $USER_UID:$USER_GID /home/$USERNAME \
|
|
|| exit 0
|
|
|
|
# Switch to local dev user
|
|
USER dev:dev
|
|
|
|
# Install current datatracker python dependencies
|
|
COPY requirements.txt /tmp/pip-tmp/
|
|
RUN pip3 --disable-pip-version-check --no-cache-dir install --user --no-warn-script-location -r /tmp/pip-tmp/requirements.txt
|
|
RUN pip3 --disable-pip-version-check --no-cache-dir install --user --no-warn-script-location pylint pylint-common pylint-django
|
|
RUN sudo rm -rf /tmp/pip-tmp
|
|
|
|
VOLUME [ "/assets" ]
|