diff --git a/dev/INSTALL b/dev/INSTALL index be3eef109..eee75494b 100644 --- a/dev/INSTALL +++ b/dev/INSTALL @@ -11,272 +11,91 @@ General Instructions for Deployment of a New Release ==================================================== - 1. In order to fetch a new release of the django datatracker code, simply - check out the appropriate tag from svn:: + 1. Make a directory to hold the new release:: + sudo su - -s /bin/bash wwwrun + mkdir /a/www/ietf-datatracker/${releasenumber} + cd /a/www/ietf-datatracker/${releasenumber} - svn co http://svn.tools.ietf.org/svn/tools/ietfdb/tags/$releasenumber + 2. Fetch the release tarball from github + (see https://github.com/ietf-tools/datatracker/releases):: - 2. Don't forget to copy $releasenumber/ietf/settings_local.py from the - old release to the new one; otherwise things won't work! - :: + wget https://github.com/ietf-tools/datatracker/archive/refs/tags/${releasenumber}.tar.gz + tar xzvf ${releasenumber}.tar.gz + + 3. Copy ietf/settings_local.py from previous release:: - cp $oldreleasenumber/ietf/settings_local.py $releasenumber/ietf/ + cp ../web/ietf/settings_local.py ietf/ - 3. Change into the directory of the new release:: - - cd $releasenumber - - It is recommended set up a Python virtual environment - under $releasenumber/env/:: - - virtualenv env # optional - source env/bin/activate # optional - - 4. Install requirements (make sure your pip is reasonably fresh first). - The following will install required python modules locally if you - are using a virtualenv, or globally if you are not:: + 4. Setup a new virtual environment and install requirements:: + python3.6 -mvenv env + source env/bin/activate + pip install --upgrade setuptools pip install -r requirements.txt - 5. Move static files to the appropriate direcrory for serving via CDN:: + 5. Run system checks (which patches the just installed modules):: - ietf/manage.py collectstatic + ietf/manage.py check - 6. Run migrations. Some migrations create files which need to be writeable - by the web servers, so make sure that you're running as wwwrun when - doing this:: + 6. Move static files into place for CDN (/a/www/www6s/lib/dt): + + ietf/manage.py collectstatic --noinput --ignore=bower.json --ignore='README.*' --ignore=rev | grep -v "Found another file with the destination path" + + 7. Run migrations: - sudo su wwwrun -s /bin/bash - source env/bin/activate ietf/manage.py migrate - ^D - - 7. Run some basic datatracker system checks:: - - ietf/manage.py check - + 8. Back out one directory level, then re-point the 'web' symlink:: cd .. - rm ./web - ln -s $releasenumber web + rm ./web; ln -s ${releasenumber} web - 9. Reload apache:: + 9. Reload both apache and the datatracker service :: - sudo service apache2 reload + exit # or CTRL-D, back to root level shell + systemctl restart apache2 datatracker.service - 10. It's now also a good idea to go to the datatracker front page:: + 10. Verify operation: http://datatracker.ietf.org/ - and then check that it's alive and kicking, and displaying the new release - number at the bottom of the left-side menubar :-) - - 11. If things **aren't** cool, revert the symlink step, re-pointing the - symlink to the release that was running before the new release, and restart - apache again to roll back to that. - - -Installing from Scratch -======================= - -In addition to the new release deployment instructions above, the -settings_local.py file has to be set up properly, and Apache has to be -configured. Since the IETF datatracker is only intended to be deployed from -scratch once, these instructions don't cover this scenario in any more detail. -The general Django depoloyment instructions are relevant, however. - + 11. If install failed, revert web symlink and repeat the restart in step 9. + Patching a Production Release ============================= -Sometimes it can prove necessary to patch an existing release. The following -process should be used: +Sometimes it can prove necessary to patch an existing release. +The following process should be used: - 1. Code and test the patch on a development system copy of the production - release which has no other code changes (or on trunk, with no uncommitted - code changes, if it's sufficiently close). + 1. Code and test the patch on an copy of the release with any + previously applied patches put in place. 2. Produce a patch file, named with date and subject:: - ~/src/ietfdb/working $ svn diff > 2013-03-25-ballot-calculation.patch + $ git diff > 2013-03-25-ballot-calculation.patch 3. Move the patch file to the production server, and place it in '/a/www/ietf-datatracker/patches/' - 4. Make a recursive copy of the production code to a new directory, named - with a patch number. Assuming the production code is in 4.43/, and we - have web -> 4.43/:: + 4. Make a recursive copy of the production code to a new directory, named with a patch number. - /a/www/ietf-datatracker $ rsync -a web/ 4.43.p1/ - - (you could use 'cp -rp' instead, but rsync seems to do this faster). + /a/www/ietf-datatracker $ rsync -a web/ ${releasenumber}.p1/ 5. Apply the patch:: - /a/www/ietf-datatracker $ cd 4.43.p1/ - /a/www/ietf-datatracker/4.43.p1 $ patch -p0 \ + /a/www/ietf-datatracker $ cd ${releasenumber}.p1/ + /a/www/ietf-datatracker/${releasnumber}.p1 $ patch -p1 \ < ../patches/2013-03-25-ballot-calculation.patch - This should not produce any messages about failing to apply any chunks; - if it does, you probably should go back to 1. and figure - out why. + This must not produce any messages about failing to apply any chunks; + if it does, go back to 1. and figure out why. 6. Edit ``.../ietf/__init__.py`` in the new patched release to indicate the patch version in the ``__patch__`` string. 7. Change the 'web' symlink, reload etc. as described in - `General Instructions for Deployment of a New Release`_ . + `General Instructions for Deployment of a New Release`_. -Additional Version-Specific Instructions -======================================== - -Version 4.50 ------------- - -Before you run step 3 (migration) of the general instructions, please run some specific -initial migrations with the a --fake argument: - - cd $releasenumber - PYTHONPATH=$PWD ietf/manage.py migrate group 0001 --fake - cd .. - - -Version 4.42 ------------- - - - In order to serve the secretariat tools static pages (such as image, css and js - files) the exceptions to which urls need to be handled by the python-handler - must be updated. - - In the datatracker test server, the following configuration has been used to - make apache handle the static files, and Django the dynamic pages. The new - part is the which mentions /secr/. Adapt as needed for ietfa:: - - ---------------- - # Logging and document root settings omitted - - - PythonPath "['/srv/www/ietfdb'] + sys.path" - SetHandler python-program - PythonHandler django.core.handlers.modpython - SetEnv DJANGO_SETTINGS_MODULE ietf.settings - PythonDebug On - - - - SetHandler None - - - # New for secretariat tools: - - SetHandler None - - - - AuthType Digest - AuthName "IETF" - AuthUserFile /var/local/loginmgr/digest - AuthGroupFile /var/local/loginmgr/groups - AuthDigestDomain http://tools.ietf.org/ - Require valid-user - - - # Caching and compression settings omitted - ---------------- - -Version 4.40 ------------- - - - (DONE) Add ianasync user with an auth role in the "iana" group and an - rfceditorsync user with an auth role in the "rfceditor" group (don't - think Group(acronym="rfceditor") exists yet); IANA and RFC Editor need - to know the passwords for the poke mechanism - - - - (DONE) Make sure mailing list for iab-stream@iab.org is up (since we're now - emailing that) - - - (DONE) Set rfc_must_published_later_than date in bin/iana-protocols-updates to today - - - (DONE) Run the 3 new doc South migrations - - - (DONE) New polling scripts, to be run every hour:: - - web/ietf/bin/iana-changes-updates - web/ietf/bin/iana-protocols-updates - web/ietf/bin/rfc-editor-index-updates (replaces mirror_rfc_index) - web/ietf/bin/rfc-editor-queue-updates (replaces mirror_rfc_queue) - - - (DONE) Import old events from IANA:: - - bin/iana-changes-updates --from="2005-01-01 00:00:00" --to="2013-01-31 00:00:00" --no-email - - - (DONE) Pipe IANA review emails to the datatracker. There used to be an action to pipe - such mails to henrik@levkowetz.com, for testing this feature, but I haven't seen - any in a little while, so I don't know if this has broken. Anyway, the iana review - emails should be piped into:: - - /www/ietf-datatracker/web/ietf/bin/iana-review-email - - - - (DONE) Tell IANA we're doing this for real now. - - -Version 4.34 ------------- - -The migration step you do as a part of the release sequence is going to take -quite some time -- probably minutes. It will generate some output while it's -working, too. As long as it doesn't halt and say that something failed or -gave an error, this is as expected, and when it terminates, you should be OK -to continue. - -Version 4.21 ------------- - -This release will you to run migrations before moving the link to the new -version and doing the apache reload. I know you have a routine for the steps -needed to deploy a new release by now, but thought I'd mention it, anyway. - -If there is any problem at all doing the migrations, then you'll need to -do a fake initial migration, as follows:: - - web $ PYTHONPATH=PWD ietf/manage.py migrate --fake meeting 0001 - -and then to the regular migration again. - -Version 4.20 ------------- - -Some one-time actions that need to be taken are as follows:: - - Assuming that the release has been checked out in /a/www/ietf-datatracker/4.20: - - cd /a/www/ietf-datatracker/4.20 - - PYTHONPATH=$PWD ietf/manage.py migrate --fake doc 0001 - PYTHONPATH=$PWD ietf/manage.py migrate --fake name 0001 - - PYTHONPATH=$PWD ietf/manage.py dbshell <<< "delete from django_content_type where app_label='doc' - and model='groupballotpositiondocevent';" - - PYTHONPATH=$PWD ietf/manage.py migrate doc || { \ - PYTHONPATH=$PWD ietf/manage.py dbshell <<< 'CREATE TABLE `doc_groupballotpositiondocevent` - (`block_comment` longtext NOT NULL, `comment` longtext NOT NULL, - `ad_id` integer NOT NULL, `comment_time` datetime NULL, - `block_comment_time` datetime NULL, `pos_id` varchar(8) NOT NULL DEFAULT "norecord", - `docevent_ptr_id` integer NOT NULL PRIMARY KEY);' - PYTHONPATH=$PWD ietf/manage.py dbshell <<< 'DROP TABLE `doc_ballottype` CASCADE;' - PYTHONPATH=$PWD ietf/manage.py dbshell <<< 'DROP TABLE `doc_ballottype_positions` CASCADE;' - PYTHONPATH=$PWD ietf/manage.py dbshell <<< 'DROP TABLE `doc_ballotdocevent` CASCADE;' - PYTHONPATH=$PWD ietf/manage.py dbshell <<< 'ALTER TABLE `doc_ballotpositiondocevent` - DROP COLUMN `ballot_id` CASCADE;' - } - - PYTHONPATH=$PWD ietf/manage.py migrate doc - PYTHONPATH=$PWD ietf/manage.py migrate name - PYTHONPATH=$PWD python ietf/wgcharter/migrate.py | tee -a ~/charter-migration.log -