============================================================================== IETF Datatracker ============================================================================== ------------------------------------------------------------------------------ Installation Instructions ------------------------------------------------------------------------------ General Instructions for Deployment of a New Release ==================================================== 0. Prepare to hold different roles at different stages of the instructions below. You will need to be root, wwwrun, and some user in group docker. Consider using separate shells for the wwwrun and other roles. These instructions are written assuming you will only use one shell. 1. Make a directory to hold the new release as wwwrun:: sudo su - -s /bin/bash wwwrun mkdir /a/www/ietf-datatracker/${releasenumber} cd /a/www/ietf-datatracker/${releasenumber} 2. Fetch the release tarball from github (see https://github.com/ietf-tools/datatracker/releases):: wget https://github.com/ietf-tools/datatracker/releases/download/${releasenumber}/release.tar.gz tar xzvf release.tar.gz 3. Copy ietf/settings_local.py from previous release:: cp ../web/ietf/settings_local.py ietf/ 4. Setup a new virtual environment and install requirements:: python3.9 -mvenv env source env/bin/activate pip install -r requirements.txt pip freeze > frozen-requirements.txt (The pip freeze command records the exact versions of the Python libraries that pip installed. This is used by the celery docker container to ensure it uses the same library versions as the datatracker service.) 5. Move static files into place for CDN (/a/www/www6s/lib/dt): ietf/manage.py collectstatic 6. Run system checks (which patches the just installed modules):: ietf/manage.py check 7. Switch to the docker directory and update images as a user in group docker: exit cd /a/docker/datatracker docker image tag ghcr.io/ietf-tools/datatracker-celery:latest datatracker-celery-fallback docker image tag ghcr.io/ietf-tools/datatracker-mq:latest datatracker-mq-fallback docker-compose pull 8. Stop and remove the async task containers: Wait for this to finish cleanly. Usually this will only be a few seconds, but it may take up to about 10 minutes for the 'down' command to complete if a long-running task is in progress. docker-compose down 9. Stop the datatracker and remove the web link so cron or other applications don't run code in the older deployment. sudo systemctl stop datatracker.socket datatracker.service rm /a/www/ietf-datatracker/web 10. Return to the release directory and run migrations as wwwrun: sudo su - -s /bin/bash wwwrun cd /a/www/ietf-datatracker/${releasenumber} ietf/manage.py migrate Take note if any migrations were executed. 11. Back out one directory level, then re-point the 'web' symlink:: cd .. ln -s ${releasenumber} web 12. Start the datatracker service (it is no longer necessary to restart apache) :: exit sudo systemctl start datatracker.service datatracker.socket 13. Start async task worker and message broker: cd /a/docker/datatracker bash startcommand 14. Verify operation: http://datatracker.ietf.org/ 15. If install failed and there were no migrations at step 9, revert web symlink and docker update and repeat the restart in steps 11 and 12. To revert the docker update: cd /a/docker/datatracker docker-compose down docker image rm ghcr.io/ietf-tools/datatracker-celery:latest ghcr.io/ietf-tools/datatracker-mq:latest docker image tag datatracker-celery-fallback ghcr.io/ietf-tools/datatracker-celery:latest docker image tag datatracker-mq-fallback ghcr.io/ietf-tools/datatracker-mq:latest cd - If there were migrations at step 10, they will need to be reversed before the restart at step 12. If it's not obvious what to do to reverse the migrations, contact the dev team. Patching a Production Release ============================= Sometimes it can prove necessary to patch an existing release. The following process should be used: 1. Code and test the patch on an copy of the release with any previously applied patches put in place. 2. Produce a patch file, named with date and subject:: $ git diff > 2013-03-25-ballot-calculation.patch 3. Move the patch file to the production server, and place it in '/a/www/ietf-datatracker/patches/' 4. Make a recursive copy of the production code to a new directory, named with a patch number. /a/www/ietf-datatracker $ rsync -a web/ ${releasenumber}.p1/ 5. Apply the patch:: /a/www/ietf-datatracker $ cd ${releasenumber}.p1/ /a/www/ietf-datatracker/${releasnumber}.p1 $ patch -p1 \ < ../patches/2013-03-25-ballot-calculation.patch This must not produce any messages about failing to apply any chunks; if it does, go back to 1. and figure out why. 6. Edit ``.../ietf/__init__.py`` in the new patched release to indicate the patch version in the ``__patch__`` string. 7. Stop the async task container (this may take a few minutes if tasks are in progress): cd /a/docker/datatracker docker-compose down 8. Change the 'web' symlink, reload etc. as described in `General Instructions for Deployment of a New Release`_. 9. Start async task worker: cd /a/docker/datatracker bash startcommand