Commit graph

316 commits

Author SHA1 Message Date
Henrik Levkowetz b196542143 Added a utility script to dump a normalized names.json fixture
- Legacy-Id: 17775
2020-05-11 14:23:21 +00:00
Henrik Levkowetz 695b6e0e86 Tweaked test-crawl to not visit all 180.000 /html/ pages.
- Legacy-Id: 17763
2020-05-08 18:49:33 +00:00
Henrik Levkowetz 85f397149e Tweaked bin/mergeready with a hint in an error situation
- Legacy-Id: 17730
2020-05-04 19:40:51 +00:00
Henrik Levkowetz 78963ecdef Added request profiler and a management command to purge profiler records.
- Legacy-Id: 17648
2020-04-15 20:48:08 +00:00
Henrik Levkowetz f10ddadc0e Merged in changes from trunk up to r17584.
- Legacy-Id: 17593
2020-04-07 16:02:52 +00:00
Henrik Levkowetz 8faf272010 Added a check for availability of 'bower' in bin/mkrelease.
- Legacy-Id: 17567
2020-03-31 10:25:06 +00:00
Henrik Levkowetz dd2e5c124e Cleaned up the contributors list in bin/mkrelease a bit.
- Legacy-Id: 17566
2020-03-31 10:08:56 +00:00
Henrik Levkowetz 768a4dfa0a Another tweak to bin/mergeready
- Legacy-Id: 17502
2020-03-22 12:37:44 +00:00
Henrik Levkowetz 532e4d2d05 Tweaked the bin/mergeready script
- Legacy-Id: 17500
2020-03-22 11:22:30 +00:00
Henrik Levkowetz bd89c7f5be Added a --dry-run option to bin/mkdevbranch, and added some exception handling
- Legacy-Id: 17467
2020-03-19 22:39:43 +00:00
Henrik Levkowetz 52b6009289 Tweaked bin/mkpatch some for -c handling
- Legacy-Id: 17444
2020-03-16 18:33:44 +00:00
Henrik Levkowetz 376d5a2ea0 Additional tweaks to bin/mkpatch; removing buggy -r option.
- Legacy-Id: 17441
2020-03-14 19:18:08 +00:00
Henrik Levkowetz 64111d3ac6 Added automatic naming to bin/mkpatch when changeset or revision range is given.
- Legacy-Id: 17440
2020-03-14 19:00:27 +00:00
Henrik Levkowetz c650a763b2 Tweaked bin/mkrelease
- Legacy-Id: 17398
2020-03-06 20:54:57 +00:00
Henrik Levkowetz 726fcbf27d Removed all __future__ imports.
- Legacy-Id: 17391
2020-03-05 23:53:42 +00:00
Henrik Levkowetz 2133cb010c Changed the release script to not pick up other email addresses than those of contributors from the release notes.
- Legacy-Id: 17369
2020-02-28 15:34:59 +00:00
Henrik Levkowetz d9b11f31d9 More Py2/3 adaptations fetched from production.
- Legacy-Id: 17333
2020-02-23 16:27:30 +00:00
Henrik Levkowetz 3728b37629 Added a cront script to be called every 15 minutes
- Legacy-Id: 17331
2020-02-23 16:21:41 +00:00
Henrik Levkowetz cf56beb8cc Cronjob script changes pulled from the production server after the switch to python 3.
- Legacy-Id: 17326
2020-02-23 01:28:16 +00:00
Henrik Levkowetz 293e505bbb Hardcoded an rsync target path, and added a bash trap for errors in hourly and daily cron scripts.
- Legacy-Id: 17264
2020-01-28 15:41:47 +00:00
Henrik Levkowetz 445afbe5c5 Hardcoded an rsync target path, and added a bash trap for errors in hourly and daily cron scripts.
- Legacy-Id: 17263
2020-01-28 15:40:42 +00:00
Henrik Levkowetz e80daea644 Minor tweak of output
- Legacy-Id: 17214
2020-01-10 12:32:47 +00:00
Henrik Levkowetz 82e270fc82 Added a check for uncomitted changes to bin/mergedevbranch.
- Legacy-Id: 17213
2020-01-10 12:29:54 +00:00
Henrik Levkowetz ba3010e3b0 Added fetch and use of the RFC-Editor's errata.json file in order to set a new 'verified-errata' document tag, and use that to show the link to inline-errata documents only when there is verified errata. Fixes issue #2861.
- Legacy-Id: 17212
2020-01-10 12:22:33 +00:00
Henrik Levkowetz c979b39bad Added some thoughts on using flake8 as part of the merge process.
- Legacy-Id: 17197
2020-01-08 14:09:00 +00:00
Henrik Levkowetz 722a60882a Tweaked the bin/mergedevbranch utility command
- Legacy-Id: 17193
2020-01-07 17:32:36 +00:00
Henrik Levkowetz 3594283549 Added Jay to the release notification email addressees
- Legacy-Id: 17098
2019-11-23 03:30:36 +00:00
Henrik Levkowetz 6070101b4d Fixed a typo.
- Legacy-Id: 17056
2019-11-19 09:38:29 +00:00
Henrik Levkowetz d798196f5a Merged in [16703] from sasha@dashcare.nl:
Ref #2231 - Fix send-review-reminders and add it to daily cron
This fixes a syntax error and a Python 3 incompatibility, and adds
send-review-reminders to the daily cron script.
Important notes:
- I have not tested to what degree the existing reminders work as they
  should, as that's out of scope. It does have tests.
- I can't assess whether the virtualenv activation works in the
  production setup, and it may be obsolete as bin/daily also
  activates the virtualenv.
- The same Python 3 incompatibility (execfile() no longer exists) seems
  to exist in various other scripts.
 - Legacy-Id: 16821
Note: SVN reference [16703] has been migrated to Git commit 296b126c70
2019-10-04 13:04:37 +00:00
Henrik Levkowetz eafaa0d742 Added an --ignore-resources switch to bin/mkrelease, as a stopgap measure before moving to yarn.
- Legacy-Id: 16717
2019-09-08 14:58:17 +00:00
Sasha Romijn 296b126c70 Ref #2231 - Fix send-review-reminders and add it to daily cron
This fixes a syntax error and a Python 3 incompatibility, and adds
send-review-reminders to the daily cron script.

Important notes:
- I have not tested to what degree the existing reminders work as they
  should, as that's out of scope. It does have tests.
- I can't assess whether the virtualenv activation works in the
  production setup, and it may be obsolete as bin/daily also
  activates the virtualenv.
- The same Python 3 incompatibility (execfile() no longer exists) seems
  to exist in various other scripts.
  
Commit ready for merge.
 - Legacy-Id: 16703
2019-09-05 10:50:39 +00:00
Henrik Levkowetz 506dcb3472 Merged in [16604] from rjsparks@nostrum.com:
Look at all scheduled sessions for a group when deciding to display a cancelled banner. Fixes #2764.
 - Legacy-Id: 16691
Note: SVN reference [16604] has been migrated to Git commit aea3c1a55b1b14f9da90d0cfb03891e7129f8632
2019-09-03 11:08:59 +00:00
Henrik Levkowetz a9b9d30eab Tweaked mergeready, and added/changed some items in the merge support files.
- Legacy-Id: 16605
2019-07-31 21:31:55 +00:00
Henrik Levkowetz 9b415e5826 More Py2/3 compatibility changes
- Legacy-Id: 16582
2019-07-23 17:29:10 +00:00
Henrik Levkowetz 3d0cca885d Py2/3 compatibility changes
- Legacy-Id: 16574
2019-07-22 17:53:26 +00:00
Henrik Levkowetz 4ce6768399 Fixed a bug in check-copyright
- Legacy-Id: 16457
2019-07-15 19:12:58 +00:00
Henrik Levkowetz 25af6fbfad Updated the test crawler for python3.
- Legacy-Id: 16438
2019-07-08 19:37:10 +00:00
Henrik Levkowetz d4fc890655 Tweaked check-copyright
- Legacy-Id: 16431
2019-07-08 11:44:17 +00:00
Henrik Levkowetz 1bba64330b Bumped check-copyright version number.
- Legacy-Id: 16430
2019-07-08 11:42:54 +00:00
Henrik Levkowetz 5d7815f3b0 Added an option to specify a copyright line pattern to bin/check-copyright.
- Legacy-Id: 16429
2019-07-08 11:41:53 +00:00
Henrik Levkowetz b7e9fd14b0 Updated check-copyright to python3.7 and added a --patch switch
- Legacy-Id: 16372
2019-07-04 13:00:55 +00:00
Henrik Levkowetz cfd4025a9c Merged from trunk
- Legacy-Id: 16371
2019-07-04 12:58:58 +00:00
Henrik Levkowetz 4bb1cec337 Improved support for the mixed-release switch in bin/mkrelease.
- Legacy-Id: 16270
2019-06-16 14:21:52 +00:00
Henrik Levkowetz 426870b766 Merged in the Document and DocAlias primary key change and m2m work in ^/personal/henrik/6.96.1-docalias.
- Legacy-Id: 16261
2019-06-15 12:00:48 +00:00
Henrik Levkowetz a2cafded25 Changed DocAlias to be many-to-many, in order to be able to handle STD aliases referring to multiple RFCs.
- Legacy-Id: 16249
2019-06-14 20:38:14 +00:00
Henrik Levkowetz 0bef48de2d Tweaked the patch name to include the commit number if available.
- Legacy-Id: 16234
2019-06-07 12:29:16 +00:00
Henrik Levkowetz 6e5b2f11dc Removed Matt from the release email recipients.
- Legacy-Id: 16215
2019-05-23 14:15:05 +00:00
Henrik Levkowetz 8d1d0cda97 Added a no-follow option to the test crawler, in order to be able to easily test a specific list of URLs.
- Legacy-Id: 16188
2019-05-06 13:35:29 +00:00
Henrik Levkowetz 82672519df Undid previous merge
- Legacy-Id: 16113
2019-03-28 11:05:25 +00:00
Henrik Levkowetz 6a159ebdfd Merged in [16094] from magnus.westerlund@ericsson.com:
Added sortable columns to Review Team Reviewers page. Partially addresses issue #2656
 - Legacy-Id: 16112
Note: SVN reference [16094] has been migrated to Git commit 0c0e75861e
2019-03-28 11:02:52 +00:00
Henrik Levkowetz b19370b98c Tweaked mkdevbranch to take and optional extra message chunk.
- Legacy-Id: 15958
2019-02-18 12:14:19 +00:00
Henrik Levkowetz 5d87de67cd Tweaked bin/mergeready to look also for ../ready-for-merge
- Legacy-Id: 15821
2018-12-14 21:21:38 +00:00
Henrik Levkowetz 1c2f9904e6 Tweaked the bin/mergedevbranch to add a pause to permit fixes to merged code before running tests.
- Legacy-Id: 15740
2018-11-09 17:37:28 +00:00
Henrik Levkowetz 08df854c5d Updated the release email distribution list in the mkrelease script.
- Legacy-Id: 15520
2018-10-03 12:23:15 +00:00
Henrik Levkowetz 6a69551e33 The nomcom send_reminders command was not being run daily. Fixed that.
- Legacy-Id: 15519
2018-10-03 12:22:11 +00:00
Henrik Levkowetz 780c71aed2 Tweaked the rsync arguments for IANA YANG models.
- Legacy-Id: 15284
2018-07-04 20:28:52 +00:00
Henrik Levkowetz 1bbe5cf9ef Added a guard to bin/mkpatch to avoid overwriting existing patches.
- Legacy-Id: 14983
2018-04-01 15:19:48 +00:00
Henrik Levkowetz b48caef487 Tweaked the test-crawler to not follow redirects to www.ietf.org. Asking the test client for non-datatracker URLs doesn't give back anything meaningful ,:-)
- Legacy-Id: 14930
2018-03-26 13:01:38 +00:00
Henrik Levkowetz 893015b8d1 Tweaked mergeready to accept an additional hold-for-merge file outside the repository.
- Legacy-Id: 14894
2018-03-21 13:12:18 +00:00
Henrik Levkowetz 278f5928ce Tweaked bin/commitlog
- Legacy-Id: 14747
2018-03-08 18:46:50 +00:00
Henrik Levkowetz ef99946ca9 Fixed a bug in the handling of checks failures.
- Legacy-Id: 14477
2017-12-30 18:46:13 +00:00
Henrik Levkowetz 6567e707ce Merged in ^/personal/henrik/6.64.2-ballotapi@14426. This provides personal API keys and a ballot position API at /api/iesg/position. Also added an endpoint description at /api/.
- Legacy-Id: 14430
2017-12-17 23:43:44 +00:00
Henrik Levkowetz e7209c6e50 Added a new personal event table to keep track of personal API key logins, and a management command to send out reports about activity to users with API keys. Added a weekly cronjob script to trigger weekly reports, and a monthly script for future use. Added a @require_api_key decorator to validate API keys for API key views and log in the API key owner. Modified the API key management urls to use create and disable rather than add and delete. Updated the API key list view. Added an API placeholder view function for ballot position setting, for test purposes. Added tests for the decorator and management command.
- Legacy-Id: 14426
2017-12-16 18:37:52 +00:00
Henrik Levkowetz 383b8b16b9 Corrected daily and hourly cron script descriptions.
- Legacy-Id: 14425
2017-12-16 18:27:06 +00:00
Henrik Levkowetz 47a0b9d3ce Changed handling of yang modeules from drafts with errors, to not place them in separate directories, due to possible follow-on effects. Added use of IANA-held modules when validating, and regular rsyncing from IANA.
- Legacy-Id: 14389
2017-12-04 14:37:15 +00:00
Henrik Levkowetz 2c95e0ed4a Added a check for new migrations which haven't been added to the repository.
- Legacy-Id: 14208
2017-10-13 13:58:44 +00:00
Robert Sparks 3a7d0d6255 script to process the id-archive and add Document objects for drafts that are currently missing from the datatracker. Fixes #1316. Commit ready for merge.
- Legacy-Id: 14138
2017-09-18 20:07:48 +00:00
Henrik Levkowetz 1b09ce419d Tweaked mkrelease
- Legacy-Id: 14020
2017-08-01 10:56:40 +00:00
Henrik Levkowetz 5bcecc7c54 Fixed a bug and added an url exception for some redirected urls in the test crawler.
- Legacy-Id: 13992
2017-07-28 12:50:39 +00:00
Henrik Levkowetz fa408ce7f6 Removed fenner@ from release email list, after Bill's wishes.
- Legacy-Id: 13986
2017-07-27 15:08:13 +00:00
Henrik Levkowetz 65e9a63c8d Fixed a bug in mkdevbranch
- Legacy-Id: 13977
2017-07-26 19:53:14 +00:00
Henrik Levkowetz fe7d484e27 Added the necessary switch argument for the registration importer's --latest switch.
- Legacy-Id: 13960
2017-07-21 12:55:27 +00:00
Henrik Levkowetz 303c3fd911 Fixed a path in bin/mkrelease.
- Legacy-Id: 13840
2017-07-12 12:34:37 +00:00
Henrik Levkowetz 6b69660559 Fixed a wrong virtualenv path in bin/mm_hourly
- Legacy-Id: 13705
2017-06-22 14:59:28 +00:00
Henrik Levkowetz eb610d2d94 Increased the test crawlers verbose output.
- Legacy-Id: 13685
2017-06-19 23:31:53 +00:00
Henrik Levkowetz ad57b107f2 Tweaked count helper.
- Legacy-Id: 13672
2017-06-18 17:21:14 +00:00
Henrik Levkowetz f90a7a721b Reverted coverage data. Tweaked mkrelease.
- Legacy-Id: 13646
2017-06-16 09:19:54 +00:00
Henrik Levkowetz 4fc2c6f960 Merged in ^/personal/henrik/6.54.2-yang@13630:
Added a new yang checker, 'yanglint', to the existing Yang checker class, in
addition to the existing 'pyang' checker.

Added modal overlay displays showing the yang check results every place the
yin/yang symbol is shown (red or green) to indicate the presencee and result
of yang checks.  Added a Yang Validation: line in the document
meta-information section on the document's page in the datatracker.

Added the result of the xym extaction to the yang check results, to make
extration failures visible.

Added the version of the used xym, pyang, and yanglint commands to the check
results.

Added an action to move successfully extracted and validated modules to the
module library directories immediately on submission.

Added the xym and pyang repositories as svn:external components, rather than
listing them in requirements.txt, as there has been delays of many months
between essential features in the repositories, and an actual release.  We may
get occasional buildbot failures if broken code is pulled in from the
repository, but better that than the functionality failure of severely
outdated componets.

Added a new management command to re-run yang validation for active drafts for
which yang modules were found at submission time, in order to pick up imported
models which may have arrived in the model libraries after the draft's
submission.  Run daily from bin/daily.

Added a table to hold version information for external commands.  The yang
checker output should include the version information of the used checkers,
but seems unnecessary to run each command with its --version switch every
time we check a module...

Added a new management command to collect version information for external
commands on demand.  To be run daily from bin/daily.

Added tests to verify that xym, pyang and yanglint information is available
on the submission confirmation page, and updated the yang module contained in
the test document to validate under both pyang and yanglint.

Updated admin.py and resource.py files as needed.
 - Legacy-Id: 13634
2017-06-15 20:16:48 +00:00
Henrik Levkowetz d98054c103 Added a new yang checker, 'yanglint', to the existing Yang checker class, in
addition to the existing 'pyang' checker.

Added modal overlay displays showing the yang check results every place the
yin/yang symbol is shown (red or green) to indicate the presencee and result
of yang checks.  Added a Yang Validation: line in the document
meta-information section on the document's page in the datatracker.

Added the result of the xym extaction to the yang check results, to make
extration failures visible.

Added the version of the used xym, pyang, and yanglint commands to the check
results.

Added an action to move successfully extracted and validated modules to the
module library directories immediately on submission.

Added the xym and pyang repositories as svn:external components, rather than
listing them in requirements.txt, as there has been delays of many months
between essential features in the repositories, and an actual release.  We may
get occasional buildbot failures if broken code is pulled in from the
repository, but better that than the functionality failure of severely
outdated componets.

Added a new management command to re-run yang validation for active drafts for
which yang modules were found at submission time, in order to pick up imported
models which may have arrived in the model libraries after the draft's
submission.  Run daily from bin/daily.

Added a table to hold version information for external commands.  The yang
checker output should include the version information of the used checkers,
but seems unnecessary to run each command with its --version switch every
time we check a module...

Added a new management command to collect version information for external
commands on demand.  To be run daily from bin/daily.

Added tests to verify that xym, pyang and yanglint information is available
on the submission confirmation page, and updated the yang module contained in
the test document to validate under both pyang and yanglint.

Updated admin.py and resource.py files as needed.
 - Legacy-Id: 13630
2017-06-15 16:09:28 +00:00
Henrik Levkowetz bf6b8d8ca1 Tweaked bin/mergedevbranch
- Legacy-Id: 13590
2017-06-12 22:26:49 +00:00
Henrik Levkowetz 5926f8832c Told svn to ignore some additional files under bin/
- Legacy-Id: 13557
2017-06-07 12:30:04 +00:00
Henrik Levkowetz 023a32715d Merged in ^/branch/iola/author-stats-r13145 from olau@iola.com, with additional features from ^/personal/henrik/6.52.1-authorstats.
- Legacy-Id: 13550
2017-06-06 18:36:59 +00:00
Henrik Levkowetz 4544b0643f Tweaks to count.c
- Legacy-Id: 13537
2017-06-06 07:26:17 +00:00
Henrik Levkowetz b3fe24d76b Merged in [13503] from rjsparks@nostrum.com:
Add usernames to the email sent to the secretariat when a possible duplicate person has been detected. Fixes #2307.
 - Legacy-Id: 13521
Note: SVN reference [13503] has been migrated to Git commit d330b8ded453b61eabc59589d44785d4fc09c767
2017-06-04 21:49:43 +00:00
Henrik Levkowetz d31c58973d Tweaked mergedevbranch to use bin/count
- Legacy-Id: 13516
2017-06-04 16:31:10 +00:00
Henrik Levkowetz 7ae80407db Tweaked version of count.c
- Legacy-Id: 13514
2017-06-04 16:23:23 +00:00
Henrik Levkowetz ed1e9cd8db A tiny utility to count test dots and periodically report the count.
- Legacy-Id: 13512
2017-06-04 15:28:03 +00:00
Henrik Levkowetz 219c3015d7 Added the meeting attendance fetcher to the daily cronjobs
- Legacy-Id: 13491
2017-06-01 20:58:54 +00:00
Henrik Levkowetz 25bb9a1c0f Tweaked bin/mkdevbranch
- Legacy-Id: 13477
2017-05-31 16:26:48 +00:00
Henrik Levkowetz afc932feaa Removed the table-list.json file and support for it; there are better ways of cleaning out extra local tables (see bin/drop-new-tables for a starting point)
- Legacy-Id: 13459
2017-05-30 00:03:26 +00:00
Henrik Levkowetz d8cf039f6a Tweaked the use of the -p otpion of bin/mkrelease. Added sbirkholz@amsl.com to the release email addressees.
- Legacy-Id: 13222
2017-04-14 14:49:05 +00:00
Henrik Levkowetz 02c6692f70 bin/mkrelease bugfix
- Legacy-Id: 13216
2017-04-14 12:35:23 +00:00
Henrik Levkowetz 828cfb89f7 Added a switch to bin/mkrelease to permit mixed data and schema migrations (applicable when the data migrations are as tiny and swift as schema migrations, such as when adding a new entry to a name model).
- Legacy-Id: 13215
2017-04-14 12:33:19 +00:00
Henrik Levkowetz 4c8f47ce73 Trying to re-apply already run migrations after loading a new databases dump runs into the problem that new tables created by the migrations linger, even if the tables loaded from the dump match the non-migrated state. Having lists of the tables matching the latest release available makes it easy to construct an sql command to dump the extra tables, so the migrations can be run. Adding generation and saving of a table list to bin/mkrelease.
- Legacy-Id: 13199
2017-04-12 18:16:19 +00:00
Henrik Levkowetz e557c2113e Merged in [13118] from lars@netapp.com:
Update vnu.jar and fix various HTML5 nits it found during a test crawl.
 - Legacy-Id: 13152
Note: SVN reference [13118] has been migrated to Git commit 76a3c8bdc0
2017-03-27 17:35:38 +00:00
Lars Eggert 76a3c8bdc0 Update vnu.jar and fix various HTML5 nits it found during a test crawl.
Commit ready for merge.
 - Legacy-Id: 13118
2017-03-25 20:21:14 +00:00
Henrik Levkowetz 044b9b8de0 Refined the selection of commit log info lines.
- Legacy-Id: 13110
2017-03-25 18:53:22 +00:00
Henrik Levkowetz 5bb9518b5f Added some new exceptions to the test-crawler; files which are known to not exist, and files with known html character problems.
- Legacy-Id: 13037
2017-03-20 13:46:23 +00:00
Henrik Levkowetz 6f47675f6a Minor tweak
- Legacy-Id: 13029
2017-03-20 11:45:08 +00:00
Henrik Levkowetz 90c61f87ba Added a check to bin/mkrelease, preventing releases with both schema change migrations and data migrations. Data migrations often take some time, which will delay the time between the schema changes and deployment of the matching new code.
- Legacy-Id: 13028
2017-03-20 11:42:50 +00:00