IETF Datatracker [![Release](https://img.shields.io/github/release/ietf-tools/datatracker.svg?style=flat&maxAge=300)](https://github.com/ietf-tools/datatracker/releases) [![License](https://img.shields.io/github/license/ietf-tools/datatracker)](https://github.com/ietf-tools/datatracker/blob/main/LICENSE) [![Code Coverage](https://codecov.io/gh/ietf-tools/datatracker/branch/feat/bs5/graph/badge.svg?token=V4DXB0Q28C)](https://codecov.io/gh/ietf-tools/datatracker) [![Python Version](https://img.shields.io/badge/python-3.9-blue?logo=python&logoColor=white)](#prerequisites) [![Django Version](https://img.shields.io/badge/django-4.x-51be95?logo=django&logoColor=white)](#prerequisites) [![Node Version](https://img.shields.io/badge/node.js-16.x-green?logo=node.js&logoColor=white)](#prerequisites) [![MariaDB Version](https://img.shields.io/badge/postgres-16-blue?logo=postgresql&logoColor=white)](#prerequisites) ##### The day-to-day front-end to the IETF database for people who work on IETF standards.
- [**Production Website**](https://datatracker.ietf.org) - [Changelog](https://github.com/ietf-tools/datatracker/releases) - [Contributing](https://github.com/ietf-tools/.github/blob/main/CONTRIBUTING.md) - [Getting Started](#getting-started) - *[ tl;dr ](#the-tldr-to-get-going)* - [Creating a Fork](#creating-a-fork) - [Git Cloning Tips](#git-cloning-tips) - [Docker Dev Environment](docker/README.md) - [Database & Assets](#database--assets) - [Old Datatracker Branches](https://github.com/ietf-tools/old-datatracker-branches/branches/all) - [Frontend Development](#frontend-development) - [Intro](#intro) - [Vite](#vite-vue-3) - [Parcel](#parcel-legacyjquery) - [Bootstrap](#bootstrap) - [Serving Static Files via CDN](#serving-static-files-via-cdn) - [Handling of External Javascript and CSS Components](#handling-of-external-javascript-and-css-components) - [Handling of Internal Static Files](#handling-of-internal-static-files) - [Changes to Template Files](#changes-to-template-files) - [Deployment](#deployment) - [Running Tests](#running-tests) - [Python](#python-tests) - [Frontend](#frontend-tests) - [Diff Tool](#diff-tool) --- ### Getting Started This project is following the standard **Git Feature Workflow** development model. Learn about all the various steps of the development workflow, from creating a fork to submitting a pull request, in the [Contributing](https://github.com/ietf-tools/.github/blob/main/CONTRIBUTING.md) guide. > [!TIP] > Make sure to read the [Styleguides](https://github.com/ietf-tools/.github/blob/main/CONTRIBUTING.md#styleguides) section to ensure a cohesive code format across the project. You can submit bug reports, enhancement and new feature requests in the [discussions](https://github.com/ietf-tools/datatracker/discussions) area. Accepted tickets will be converted to issues. #### Creating a Fork Click the Fork button in the top-right corner of the repository to create a personal copy that you can work on. > [!NOTE] > Some GitHub Actions might be enabled by default in your fork. You should disable them by going to **Settings** > **Actions** > **General** and selecting **Disable actions** (then Save). #### Git Cloning Tips As outlined in the [Contributing](https://github.com/ietf-tools/.github/blob/main/CONTRIBUTING.md) guide, you will first want to create a fork of the datatracker project in your personal GitHub account before cloning it. Windows developers: [Start with WSL2 from the beginning](https://github.com/ietf-tools/.github/blob/main/docs/windows-dev.md). Because of the extensive history of this project, cloning the datatracker project locally can take a long time / disk space. You can speed up the cloning process by limiting the history depth, for example *(replace `USERNAME` with your GitHub username)*: - To fetch only up to the 10 latest commits: ```sh git clone --depth=10 https://github.com/USERNAME/datatracker.git ``` - To fetch only up to a specific date: ```sh git clone --shallow-since=DATE https://github.com/USERNAME/datatracker.git ``` #### The tl;dr to get going Note that you will have to have cloned the datatracker code locally - please read the above sections. Datatracker development is performed using Docker containers. You will need to be able to run docker (and docker-compose) on your machine to effectively develop. It is possible to get a purely native install working, but it is _very complicated_ and typically takes a first time datatracker developer a full day of setup, where the docker setup completes in a small number of minutes. Many developers are using [VS Code](https://code.visualstudio.com/) and taking advantage of VS Code's ability to start a project in a set of containers. If you are using VS Code, simply start VS Code in your clone and inside VS Code choose `Restart in container`. If VS Code is not available to you, in your clone, type `cd docker; ./run` Once the containers are started, run the tests to make sure your checkout is a good place to start from (all tests should pass - if any fail, ask for help at tools-help@). Inside the app container's shell type: ```sh ietf/manage.py test --settings=settings_test ``` Note that we recently moved the datatracker onto PostgreSQL - you may still find older documentation that suggests testing with settings_sqlitetest. That will no longer work. For a more detailed description of getting going, see [docker/README.md](docker/README.md). #### Overview of the datatracker models A beginning of a [walkthrough of the datatracker models](https://notes.ietf.org/iab-aid-datatracker-database-overview) was prepared for the IAB AID workshop. #### Docker Dev Environment In order to simplify and reduce the time required for setup, a preconfigured docker environment is available. Read the [Docker Dev Environment](docker/README.md) guide to get started. ### Database & Assets Nightly database dumps of the datatracker are available as Docker images: `ghcr.io/ietf-tools/datatracker-db:latest` > [!TIP] > In order to update the database in your dev environment to the latest version, you should run the `docker/cleandb` script. ### Blob storage for dev/test The dev and test environments use [minio](https://github.com/minio/minio) to provide local blob storage. See the settings files for how the app container communicates with the blobstore container. If you need to work with minio directly from outside the containers (to interact with its api or console), use `docker compose` from the top level directory of your clone to expose it at an ephemeral port. ``` $ docker compose port blobstore 9001 0.0.0.0: $ curl -I http://localhost: HTTP/1.1 200 OK ... ``` The minio container exposes the minio api at port 9000 and the minio console at port 9001 ### Frontend Development #### Intro We now use `yarn` to manage assets for the Datatracker, and `vite`/`parcel` to package them. `yarn` maintains its `node` packages under the `.yarn` directory. The datatracker uses 2 different build systems, depending on the use case: - [**Vite**](https://vitejs.dev/) for Vue 3 pages / components - [**Parcel**](https://parceljs.org/) for legacy pages / jQuery #### Vite *(Vue 3)* Pages will gradually be updated to Vue 3 components. These components are located under the `/client` directory. Each Vue 3 app has its own sub-directory. For example, the agenda app is located under `/client/agenda`. The datatracker makes use of the Django-Vite plugin to point to either the Vite.js server or the precompiled production files. The `DJANGO_VITE_DEV_MODE` flag, found in the `ietf/settings_local.py` file determines whether the Vite.js server is used or not. In development mode, you must start the Vite.js development server, in addition to the usual Datatracker server: ```sh yarn dev ``` Any changes made to the files under `/client` will automatically trigger a hot-reload of the modified components. To generate production assets, run the build command: ```sh yarn build ``` This will create packages under `ietf/static/dist-neue`, which are then served by the Django development server, and which must be uploaded to the CDN. #### Parcel *(Legacy/jQuery)* The Datatracker includes these packages from the various Javascript and CSS files in `ietf/static/js` and `ietf/static/css` respectively, bundled using Parcel. Static images are likewise in `ietf/static/images`. Whenever changes are made to the files under `ietf/static`, you must re-run the build command to package them: ``` shell yarn legacy:build ``` This will create packages under `ietf/static/dist/ietf`, which are then served by the Django development server, and which must be uploaded to the CDN. #### Bootstrap The "new" datatracker uses Twitter Bootstrap for the UI. Get familiar with and use those UI elements, CSS classes, etc. instead of cooking up your own. Some ground rules: - Think hard before tweaking the bootstrap CSS, it will make it harder to upgrade to future releases. - No `