Commit graph

79 commits

Author SHA1 Message Date
Jennifer Richards 46a00acefc
refactor: sync to RFC Editor queue via celery (#7415)
* feat: rfc_editor_queue_updates_task

* refactor: use rfc_editor_queue_updates_task()

* chore: remove now-unused scripts

* test: test new task

* chore: de-lint
2024-05-14 18:56:14 -05:00
Jennifer Richards 0616b07d2d
feat: email ingestion API (#7342)
* feat: IANA review email ingestor API

* refactor: Replace iana email api with generic one

* chore: Add type hint

* feat: Ingest ipr responses

* feat: Ingest nomcom feedback

* refactor: message -> msg

* fix: Typo

* feat: Send email on nomcom ingestion failure

* feat: Send email on IPR mail ingestion error

* feat: Check content type, handle more errs

* fix: drop additionalProperties: false

Unfortunately this does not mix well with
the conditional "year" property.

* test: Test ingest_email view

* Revert "test: Test ingest_email view"

This reverts commit e498022829f834a0d3cebcb0dafb0d5f5a5d162e.

* test: Test ingest_email view

* fix: pass new test

* test: Test ingest_review_email

* fix: Pass new test

* test: Test ipr ingest_response_email

* fix: pass new test

* test: test nomcom ingest_feedback_email

* chore: fix typo found in code reviw

* fix: De-lint
2024-04-23 09:07:50 -05:00
Jennifer Richards cf21b8f236
fix: Only POST to rfceditor in production (#7241)
* fix: Only POST to rfceditor in production

* test: Test server mode checking
2024-03-20 21:43:10 -05:00
Jennifer Richards b4cf04a09d
feat: celery tasks to replace ietf/bin scripts (#6971)
* refactor: Change import style for clarity

* feat: Add iana_changes_updates_task()

* chore: Squelch lint warning

My linter does not like variables defined outside
of __init__()

* feat: Add PeriodicTask for iana_changes_updates_task

* refactor: tasks instead of scripts on sync.views.notify()

* test: Test iana_changes_updates_task

* refactor: rename task for consistency

* feat: Add iana_protocols_update_task

* feat: Add PeriodicTask for iana protocols sync

* refactor: Use protocol sync task instead of script in view

* refactor: itertools.batched() not available until py312

* test: test iana_protocols_update_task

* feat: Add idindex_update_task()

Calls idindex generation functions and does the file
update dance to put them in place.

* chore: Add comments to bin/hourly

* fix: annotate types and fix bug

* feat: Create PeriodicTask for idindex_update_task

* refactor: Move helpers into a class

More testable this way

* refactor: Make TempFileManager a context mgr

* test: Test idindex_update_task

* test: Test TempFileManager

* fix: Fix bug in TestFileManager

yay testing

* feat: Add expire_ids_task()

* feat: Create PeriodicTask for expire_ids_task

* test: Test expire_ids_task

* test: Test request timeout in iana_protocols_update_task

* refactor: do not re-raise timeout exception

Not sure this is the right thing to do, but it's the
same as rfc_editor_index_update_task

* feat: Add notify_expirations_task

* feat: Add "weekly" celery beat crontab

* refactor: Reorder crontab fields

This matches the crontab file field order

* feat: Add PeriodicTask for notify_expirations

* test: Test notify_expirations_task

* test: Add annotation to satisfy mypy
2024-01-31 15:24:20 -06:00
Jennifer Richards 36c43c8520
chore: add task tests; move message task to message app (#6964)
* test: Test send_review_reminders_task

* refactor: Move send_scheduled_mail_task to message app

* chore: Remove unused import

* test: Add Message/SendQueue factories

* test: Test send_scheduled_mail_task

* test: Reset mocks before reuse

* test: Cover error conditions

* test: Return non-empty change set

* test: Test SMTPException handling

* test: Test fetch_attendance_from_meetings()

* test: Test RuntimeError handling

* test: RFC index sync should populate authors
2024-01-24 08:53:42 -06:00
Jennifer Richards 7dbb7e36d3
fix: Add testing, fix bug in fetch_meeting_attendance_task (#6961)
* test: Test rfc_editor_index_update_task

* chore: Add docstring to test

* fix: Reuse stats instead of fetching twice

* test: Test fetch_meeting_attendance_task

* chore: Remove outdated tasks

* Revert "chore: Add docstring to test"

This reverts commit 0395410d665c0d310248fd151386f013357c5d13.

* Revert "test: Test rfc_editor_index_update_task"

This reverts commit 4dd9dd131723497db3d2aa76166169fd32c42fdd.

* test: Test rfc_editor_index_update_task

This time without reformatting the entire file...

* chore: Remove accidentally committed fragment

* test: Annotate function to satisfy mypy

* chore: Remove unused imports
2024-01-23 16:14:37 -06:00
Jennifer Richards 8351b4528f
test: RFC pub date is in RPC_TZINFO (#6747) 2023-12-07 11:57:24 -04:00
Jennifer Richards 4ac1153d7d
fix: Label draft change log with rfc number (#6659)
* fix: Label draft change log with rfc num

* test: Update test
2023-11-17 15:49:34 -04:00
Robert Sparks 195926ada7
fix:bring rfc-index sync test into sync with function changes 2023-11-05 14:47:43 +01:00
Robert Sparks 1b32fde588
fix: cleaner sync test (#6372) 2023-09-26 17:11:10 -05:00
Robert Sparks 8732dcb706
fix: bring tests up current. add subseries names to rfc view. fix html. 2023-09-22 12:40:15 -05:00
Robert Sparks 96fae0c436
feat: remove DocAlias 2023-08-19 13:17:21 -05:00
Robert Sparks 5489fea9f7
fix: untangle stats and iana sync 2023-08-19 12:58:01 -05:00
Jennifer Richards 80a361c3c8
test: Check that no logs are emitted in test_rfc_index 2023-08-14 14:27:26 -03:00
Jennifer Richards b565f104d8
test: Don't mix stream states in test_rfc_index()
Make the test a better simulation of actual practice.
2023-08-14 14:09:37 -03:00
Jennifer Richards f9ca1bc9d6
test: Test behavior of verified-errata tag
Also add comments flagging the near-miss between doc-id
and RFC number in the test_rfc_index() test data. It's
unclear whether these were intended to match, but the
test was self-consistent so I am not changing it.
2023-08-14 12:15:09 -03:00
Jennifer Richards 92ad44e70c
test: Test more fields on the rfc doc 2023-08-10 17:53:35 -03:00
Jennifer Richards 112ac906dc
test: Confirm changes are returned for draft and rfc 2023-08-10 16:01:34 -03:00
Jennifer Richards 4437796bc3
test: Update test_rfc_index() 2023-08-10 15:43:10 -03:00
Jennifer Richards bb7445575b
test: Remove unused assignment lint 2023-07-28 10:57:16 -07:00
Robert Sparks 189c32194d
test: construct the right kind of document for the rfc-index tests 2023-07-20 13:25:12 -05:00
Robert Sparks 51c22d8e6c
fix: repairs reflecting removal of DocAlias from the RelatedDocument models 2023-07-20 13:03:58 -05:00
Jennifer Richards 4084d7d557
fix: record and interpret RFC pub dates in correct timezone (#4421)
* fix: use PST8PDT for published_rfc event timestamps

* fix: find RFCs by PST8PDT year in RfcFeed

* refactor: add const RPC_TZINFO to represent RFC publication timezone

* chore: remove (rather than fix) unused template tags

* fix: always return RPC_TZINFO-local date from Document.pub_date()

* refactor: convert 'published' flag to a Boolean to reflect its usage

* fix: display doc publication dates in correct time zones

* fix: fix various small issues breaking tests
2022-09-08 14:51:19 -03:00
Jennifer Richards 32054111df
fix: fix failing tests and eliminate naive datetime warnings (#4402)
* test: fix timestamp construction in several doc tests

* refactor: rename date2datetime to datetime_from_date and clarify code

* chore: helper to get tzinfo for PRODUCTION_TIMEZONE

* fix: fix timezone handling in make_last_call()

* test: fix datetime generation in doc.tests_charter

* refactor: remove PRODUCTION_TIMEZONE setting

Replaces the PRODUCTION_TIMEZONE setting with a constant,
DEADLINE_TZINFO, in ietf.utils.timezone.

* test: be more careful about timezone in tests_charter.py

* test: be more careful about timezone in doc/tests.py

* fix: fix timezone handling affecting doc.tests_draft

* fix: fix timezone handling affecting tests_irsg_ballot.py

* fix: fix timezone handling affecting tests_review.py

* fix: fix timezone handling affecting last ietf.doc tests

* fix: fix timezone handling affecting last ietf.group tests

* fix: fix timezone handling affecting ietf.iesg tests

* fix: handle timezones in get_8989_eligibility_querysets

* fix: handle timezones affecting ietfauth tests

* fix: return tz-aware datetime from utc_from_string

* fix: specify timezone for constants in ipr_rfc_number()

* fix: specify tz for ipr deadlines

* fix: handle timezones affecting liaisons tests

* fix: treat leap day in get_8989_eligibility_querysets()

Manual cherry-pick of 248d6474

* test: treat leap day properly in nomcom tests

* fix: fix timezone handling affecting nomcom tests

* test: fix timezone handling in review tests

* fix: fix timezone handling affecting secr.meetings tests

* fix: handle both pytz and zoneinfo timezones in ietf.utils.timezone

* fix: fix timezone handling affecting secr.proceedings tests

* refactor: use make_aware() helper in secr.meetings tests

* test: fix timezone handling in secr.telechat tests

* fix: fix timezone handling affecting stats tests

* fix: eliminate tz-naive helpers affecting sync email parsing

* fix: include timezone data when serializing DeletedEvent data

* fix: fix timezone handling affecting sync tests

* style: remove unused import
2022-09-01 13:07:28 -03:00
Jennifer Richards ebebdbed3e
refactor: replace datetime.now and datetime.today with timezone.now (#4211)
* refactor: replace datetime.now with timezone.now

* refactor: migrate model fields to use timezone.now as default

* refactor: replace datetime.today with timezone.now

datetime.datetime.today() is equivalent to datetime.datetime.now(); both
return a naive datetime with the current local time.

* refactor: rephrase datetime.now(tz) as timezone.now().astimezone(tz)

This is effectively the same, but is less likely to encourage accidental
use of naive datetimes.

* refactor: revert datetime.today() change to old migrations

* refactor: change a missed datetime.now to timezone.now

* chore: renumber timezone_now migration

* chore: renumber migrations
2022-08-25 13:45:16 -03:00
Lars Eggert a5cbf5307e More fixes
- Legacy-Id: 19835
2022-01-12 11:54:00 +00:00
Jennifer Richards 81d9234d54 Use temporary directories instead of "real" filesystem for tests. Fixes #3414. Commit ready for merge.
- Legacy-Id: 19555
2021-11-05 19:45:57 +00:00
Jennifer Richards e11583a87f Allow assignment of Person as "action holder" for a Doc, plus rudimentary automation of assignment. Fixes #3146. Commit ready for merge.
- Legacy-Id: 18829
2021-02-12 20:31:00 +00:00
Robert Sparks 3697180cc1 Reverted merge of timezone-aware migration efforts.
- Legacy-Id: 18792
2021-01-12 16:54:20 +00:00
Henrik Levkowetz 774e752a54 Snapshot of timezone-aware datatracker code. Tests pass, and the test-crawler shows only expected differences. Trunk changes merged in up to r18768.
- Legacy-Id: 18770
2020-12-16 23:53:37 +00:00
Jennifer Richards fff927b085 Store Auth48 URL as DocumentURL and display in RFC-Editor state. Migrates old data when possible. Alternative to 17563. Fixes #2722. Commit ready for merge.
- Legacy-Id: 18157
2020-07-13 15:34:01 +00:00
Henrik Levkowetz 726fcbf27d Removed all __future__ imports.
- Legacy-Id: 17391
2020-03-05 23:53:42 +00:00
Henrik Levkowetz ba3010e3b0 Added fetch and use of the RFC-Editor's errata.json file in order to set a new 'verified-errata' document tag, and use that to show the link to inline-errata documents only when there is verified errata. Fixes issue #2861.
- Legacy-Id: 17212
2020-01-10 12:22:33 +00:00
Henrik Levkowetz 96f6f47758 Updated rfc-editor sync code to recognize the new Tooling Issue queue state.
- Legacy-Id: 16730
2019-09-11 16:20:25 +00:00
Henrik Levkowetz 8c6eb3a30a Python2/3 compatibility: Changed the use of open() and StringIO to io.open() etc.
- Legacy-Id: 16458
2019-07-15 19:14:04 +00:00
Henrik Levkowetz 864fc1a814 Firmed up the handling of non-ascii names for a test email templat, tweaked some asserts, and removed unused imports.
- Legacy-Id: 16392
2019-07-04 20:53:05 +00:00
Henrik Levkowetz 15e74e89cb Converted instances of assertTrue(text in response.content) to the recommended assertContains(response, text), and similar for assertFalse(... in ...)
- Legacy-Id: 16336
2019-06-29 13:43:31 +00:00
Henrik Levkowetz d7f5c84182 Initial 2to3 patch with added copyright statement updates.
- Legacy-Id: 16309
2019-06-27 14:40:54 +00:00
Henrik Levkowetz 426870b766 Merged in the Document and DocAlias primary key change and m2m work in ^/personal/henrik/6.96.1-docalias.
- Legacy-Id: 16261
2019-06-15 12:00:48 +00:00
Henrik Levkowetz a2cafded25 Changed DocAlias to be many-to-many, in order to be able to handle STD aliases referring to multiple RFCs.
- Legacy-Id: 16249
2019-06-14 20:38:14 +00:00
Henrik Levkowetz 815602351f This is a series of 50 migrations that changes the Document and DocAlias
primary keys from character strings to integers, and makes corresponding code
changes.

This was prompted by database limitations discovered when trying to make
DocAlias use a m2m document field; with 255 long strings as primary keys for
Document and DocAlias this violated the MySQL database limitations.

Changing the primary keys to integers should also improve efficiency.  

Due to the data migrations which create the new integer primary keys and adds
corresponding integer foreign keys matching the previous string foreign keys
in all tables having foreign keys to Document and DocAlias, some of these
migrations take a long time.  The total set of migrations are expected to have
a runtime on the order of 2 hours.
 - Legacy-Id: 16237
2019-06-10 11:32:46 +00:00
Robert Sparks 6118975620 Added an explicit ID-Exists state for the IESG state machine. Reworked code so that the IESG state machine always has a state. Added the ability to release a document from a working group, research group, or the independent stream. Releasing a document removes all stream state, and sets the document to have no stream.
- Legacy-Id: 15809
2018-12-11 22:38:05 +00:00
Henrik Levkowetz ef3cbf4684 Merged in [15595] from rjsparks@nostrum.com:
Converted stats, submit, sync. and utils/test to use factories. Excluded utils/test_data from coverage checks.
 - Legacy-Id: 15599
Note: SVN reference [15595] has been migrated to Git commit 39e76a836996ec9ed8ea449c92bdfae91c3dbaae
2018-10-19 12:54:58 +00:00
Henrik Levkowetz 14c748df98 Changed the creation of temporary test dirs to use names which include the test class name, in order to make it possible to run test cases in parallel. Disabled coverage testing when running in parallel (those classes cannot be pickled and distributed to the test runner threads, and also won't pass coverage data back). Tweaked the TestCase.tempdir() method.
- Legacy-Id: 13157
2017-03-27 21:41:18 +00:00
Lars Eggert 8a70e9773d Fix various "RemovedInDjango20Warning" warnings. Commit ready for merge.
- Legacy-Id: 13112
2017-03-25 19:16:45 +00:00
Henrik Levkowetz 4daf66d11c Added DocEvent rev= parameter througout the code.
- Legacy-Id: 12978
2017-03-06 15:08:21 +00:00
Henrik Levkowetz 4b0a9360f0 Merged in ^/branch/iola/event-saving-refactor-r10291, which refactors document saving to always use doc.save_with_history(events), and requires accompanying events. This branch also provides refactoring of recurring regexes in url patterns into a dictionary. As part of the merge, also refactored new code which didn't use the save_with_history() method.
- Legacy-Id: 11840
2016-08-23 10:52:08 +00:00
Henrik Levkowetz b00b0c3db9 Moved some production-path checks to the checks module, and fixed up tests which changed some settings without restoring them.
- Legacy-Id: 10808
2016-02-09 18:50:37 +00:00
Henrik Levkowetz cf7d794b09 Modified the test data to use some AD and WG chair names that contain non-ascii characters, and fixed test issues found.
- Legacy-Id: 10418
2015-11-04 07:34:56 +00:00
Ole Laursen 86c3a430d1 Merge in ^/branch/iola/event-saving-refactor-r10076, fixing a few problems
- Legacy-Id: 10298
2015-10-27 10:37:06 +00:00