184 lines
8.2 KiB
ReStructuredText
184 lines
8.2 KiB
ReStructuredText
================================================================================
|
|
Serving Static Datatracker Files via a CDN
|
|
================================================================================
|
|
|
|
Intro
|
|
=====
|
|
|
|
With release 6.4.0, the way that the static files used by the datatracker are
|
|
handled changes substantially. Static files were previously versioned under a
|
|
top-level ``static/`` directory, but this is not the case any more. External
|
|
files (such as for instance ``jquery.min.js``) are now placed under
|
|
``ietf/externals/static/`` and updated using a tool called bower_, while
|
|
datatracker-specific files (images, css, js, etc.) are located under
|
|
``ietf/static/ietf/`` and ``ietf/secr/static/secr/`` respectively.
|
|
|
|
The following sections provide more details about handling of internals,
|
|
externals, and how deployment is done.
|
|
|
|
|
|
Serving Static Files via CDN
|
|
============================
|
|
|
|
Production Mode
|
|
---------------
|
|
|
|
If resources served over a CDN and/or with a high max-age don't have different
|
|
URLs for different versions, then any component upgrade which is accompanied
|
|
by a change in template functionality will have a long transition time
|
|
during which the new pages are served with old components, with possible
|
|
breakage. We want to avoid this.
|
|
|
|
The intention is that after a release has been checked out, but before it is
|
|
deployed, the standard django 'collectstatic' management command will be
|
|
run, resulting in all static files being collected from their working
|
|
directory location and placed in an appropiate location for serving via CDN.
|
|
This location will have the datatracker release version as part of its URL,
|
|
so that after the deployment of a new release, the CDN will be forced to fetch
|
|
the appropriate static files for that release.
|
|
|
|
An important part of this is to set up the ``STATIC_ROOT`` and ``STATIC_URL``
|
|
settings appropriately. In 6.4.0, the setting is as follows in production
|
|
mode::
|
|
|
|
STATIC_URL = "https://www.ietf.org/lib/dt/%s/"%__version__
|
|
STATIC_ROOT = CDN_ROOT + "/a/www/www6s/lib/dt/%s/"%__version__
|
|
|
|
The result is that all static files collected via the ``collectstatic``
|
|
command will be placed in a location served via CDN, with the release
|
|
version being part of the URL.
|
|
|
|
Development Mode
|
|
----------------
|
|
|
|
In development mode, ``STATIC_URL`` is set to ``/static/``, and Django's
|
|
``staticfiles`` infrastructure makes the static files available under that
|
|
local URL root (unless you set
|
|
``settings.SERVE_CDN_FILES_LOCALLY_IN_DEV_MODE`` to ``False``). It is not
|
|
necessary to actually populate the ``static/`` directory by running
|
|
``collectstatic`` in order for static files to be served when running
|
|
``ietf/manage.py runserver`` -- the ``runserver`` command has extra support
|
|
for finding and serving static files without running collectstatic.
|
|
|
|
In order to work backwards from a file served in development mode to the
|
|
location from which it is served, the mapping is as follows::
|
|
|
|
============================== ==============================
|
|
Development URL Working copy location
|
|
============================== ==============================
|
|
localhost:8000/static/ietf/* ietf/static/ietf/*
|
|
localhost:8000/static/secr/* ietf/secr/static/secr/*
|
|
localhost:8000/static/* ietf/externals/static/*
|
|
============================== ==============================
|
|
|
|
Handling of External Javascript and CSS Components
|
|
==================================================
|
|
|
|
In order to make it easy to keep track of and upgrade external components,
|
|
these are now handled by a tool called ``bower``, via a new management
|
|
command ``bower_install``. Each external component is listed in a file
|
|
``ietf/bower.json``. In order to update the version of a component listed in
|
|
``ietf/bower.json``, or add a new one, you should edit ``bower.json``, and
|
|
then run the management command::
|
|
|
|
$ ietf/manage.py bower_install
|
|
|
|
(Not surprisingly, you need to have bower_ installed in order to use this
|
|
management command.)
|
|
|
|
That command will fetch the required version of each external component listed
|
|
in ``bower.json`` (actually, it will do this for *all* ``bower.json`` files
|
|
found in the ``static/`` directories of all ``INSTALLED_APPS`` and the
|
|
directories in ``settings.STATICFILES_DIRS``), saving them temporarily under
|
|
``.tmp/bower_components/``; it will then extract the relevant production
|
|
``js`` and ``css`` files and place them in an appropriately named directory
|
|
under ``ietf/externals/static/``. The latter location is taken from
|
|
``COMPONENT_ROOT`` in ``settings.py``.
|
|
|
|
Managing external components via bower has the additional benefit of
|
|
managing dependencies -- components that have dependencies will pull in
|
|
these, so that they also are placed under ``ietf/externals/static/``.
|
|
You still have to manually add the necessary stylesheet and/or javascript
|
|
references to your templates, though.
|
|
|
|
The ``bower_install`` command is not run automatically by ``bin/mkrelease``,
|
|
since it needs an updated ``bower.json`` in order to do anything interesting.
|
|
So when you're intending to update an external web asset to a newer version,
|
|
you need to edit the ``bower.json`` file, run ``manage.py bower_install``,
|
|
verify that the new version doesn't break things, and then commit the new
|
|
files under ``ietf/externals/static/`` and the updated ``bower.json``.
|
|
|
|
.. _bower: http://bower.io/
|
|
|
|
The ``ietf/externals/static/`` Directory
|
|
-----------------------------------------
|
|
|
|
The directory ``ietf/externals/static/`` holds a number of subdirectories
|
|
which hold distribution files for external client-side components, collected
|
|
by ``bower_install`` as described above. Currently
|
|
(01 Aug 2015) this means ``js`` and ``css`` components and fonts.
|
|
|
|
These components each reside in their own subdirectory, which is named with
|
|
the component name::
|
|
|
|
henrik@zinfandel $ ls -l ietf/externals/static/
|
|
total 40
|
|
drwxr-xr-x 6 henrik henrik 4096 Jul 25 15:25 bootstrap
|
|
drwxr-xr-x 4 henrik henrik 4096 Jul 25 15:25 bootstrap-datepicker
|
|
drwxr-xr-x 4 henrik henrik 4096 Jul 25 15:25 font-awesome
|
|
drwxr-xr-x 2 henrik henrik 4096 Jul 25 15:25 jquery
|
|
drwxr-xr-x 2 henrik henrik 4096 Jul 25 15:25 jquery.cookie
|
|
drwxr-xr-x 2 henrik henrik 4096 Jul 25 15:24 ptmono
|
|
drwxr-xr-x 2 henrik henrik 4096 Jul 25 15:24 ptsans
|
|
drwxr-xr-x 2 henrik henrik 4096 Jul 25 15:24 ptserif
|
|
drwxr-xr-x 2 henrik henrik 4096 Jul 25 15:25 select2
|
|
drwxr-xr-x 2 henrik henrik 4096 Jul 25 15:25 select2-bootstrap-css
|
|
|
|
The ``pt*`` fonts are an exception, in that there is no bower component
|
|
available for these fonts, so they have been put in place manually.
|
|
|
|
|
|
Handling of Internal Static Files
|
|
=================================
|
|
|
|
Previous to this release, internal static files were located under
|
|
``static/``, mixed together with the external components. They are now
|
|
located under ``ietf/static/ietf/`` and ``ietf/secr/static/secr``, and will be
|
|
collected for serving via CDN by the ``collectstatic`` command. Any static
|
|
files associated with a particular app will be handled the same way (which
|
|
means that all ``admin/`` static files automatically will be handled correctly, too).
|
|
|
|
Handling of Customised Bootstrap Files
|
|
======================================
|
|
|
|
We are using a customised version of Bootstrap_, which is handled specially,
|
|
by a SVN externals definition in ``ietf/static/ietf``. That pulls the content
|
|
of the ``bootstrap/dist/`` directory (which is generated by running ``grunt``
|
|
in the ``bootstrap/`` directory) into ``ietf/static/ietf/bootstrap``, from
|
|
where it is collected by ``collectstatic``.
|
|
|
|
Changes to Template Files
|
|
=========================
|
|
|
|
In order to make the template files refer to the correct versioned CDN URL
|
|
(as given by the STATIC_URL root) all references to static files in the
|
|
templates have been updated to use the ``static`` template tag when referring
|
|
to static files. This will automatically result in both serving static files
|
|
from the right place in development mode, and referring to the correct
|
|
versioned URL in production mode and the simpler ``/static/`` urls in
|
|
development mode.
|
|
|
|
.. _bootstrap: http://getbootstrap.com/
|
|
|
|
Deployment
|
|
==========
|
|
|
|
During deployment, it is now necessary to run the management command::
|
|
|
|
$ ietf/manage.py collectstatic
|
|
|
|
before activating a new release.
|
|
|
|
The deployment ``README`` file at ``/a/www/ietf-datatracker/README`` has been
|
|
updated accordingly.
|