Commit graph

34 commits

Author SHA1 Message Date
Jennifer Richards a35aa67c70
chore: clarify transaction-related comments (#8180) 2024-11-08 11:55:47 +00:00
Jennifer Richards afbe6aa429
fix: queue tasks in on_commit hook (#8149)
* fix: dispatch tasks in on_commit hook

* test: fix test
2024-11-06 08:49:54 +00:00
Jennifer Richards 6f4e328334
fix: remove is_secure() check (#7579)
The real work happened in a previous commit,
just nudging the patch version with this CC
message.
2024-06-21 09:50:35 -05:00
Jennifer Richards c8471d47d3
chore: remove is_secure() check (#7575) 2024-06-21 09:33:00 -05:00
Jennifer Richards 46a00acefc
refactor: sync to RFC Editor queue via celery (#7415)
* feat: rfc_editor_queue_updates_task

* refactor: use rfc_editor_queue_updates_task()

* chore: remove now-unused scripts

* test: test new task

* chore: de-lint
2024-05-14 18:56:14 -05:00
Jennifer Richards 0616b07d2d
feat: email ingestion API (#7342)
* feat: IANA review email ingestor API

* refactor: Replace iana email api with generic one

* chore: Add type hint

* feat: Ingest ipr responses

* feat: Ingest nomcom feedback

* refactor: message -> msg

* fix: Typo

* feat: Send email on nomcom ingestion failure

* feat: Send email on IPR mail ingestion error

* feat: Check content type, handle more errs

* fix: drop additionalProperties: false

Unfortunately this does not mix well with
the conditional "year" property.

* test: Test ingest_email view

* Revert "test: Test ingest_email view"

This reverts commit e498022829f834a0d3cebcb0dafb0d5f5a5d162e.

* test: Test ingest_email view

* fix: pass new test

* test: Test ingest_review_email

* fix: Pass new test

* test: Test ipr ingest_response_email

* fix: pass new test

* test: test nomcom ingest_feedback_email

* chore: fix typo found in code reviw

* fix: De-lint
2024-04-23 09:07:50 -05:00
Jennifer Richards b4cf04a09d
feat: celery tasks to replace ietf/bin scripts (#6971)
* refactor: Change import style for clarity

* feat: Add iana_changes_updates_task()

* chore: Squelch lint warning

My linter does not like variables defined outside
of __init__()

* feat: Add PeriodicTask for iana_changes_updates_task

* refactor: tasks instead of scripts on sync.views.notify()

* test: Test iana_changes_updates_task

* refactor: rename task for consistency

* feat: Add iana_protocols_update_task

* feat: Add PeriodicTask for iana protocols sync

* refactor: Use protocol sync task instead of script in view

* refactor: itertools.batched() not available until py312

* test: test iana_protocols_update_task

* feat: Add idindex_update_task()

Calls idindex generation functions and does the file
update dance to put them in place.

* chore: Add comments to bin/hourly

* fix: annotate types and fix bug

* feat: Create PeriodicTask for idindex_update_task

* refactor: Move helpers into a class

More testable this way

* refactor: Make TempFileManager a context mgr

* test: Test idindex_update_task

* test: Test TempFileManager

* fix: Fix bug in TestFileManager

yay testing

* feat: Add expire_ids_task()

* feat: Create PeriodicTask for expire_ids_task

* test: Test expire_ids_task

* test: Test request timeout in iana_protocols_update_task

* refactor: do not re-raise timeout exception

Not sure this is the right thing to do, but it's the
same as rfc_editor_index_update_task

* feat: Add notify_expirations_task

* feat: Add "weekly" celery beat crontab

* refactor: Reorder crontab fields

This matches the crontab file field order

* feat: Add PeriodicTask for notify_expirations

* test: Test notify_expirations_task

* test: Add annotation to satisfy mypy
2024-01-31 15:24:20 -06:00
Robert Sparks c38ade6e1b
feat: treat django auth username as case insensitive throughout the datatracker (#5165)
* feat: insensitive username matching at django authentication

* feat: use iexact when using the User object manager

* fix: more places to ignore username case

* fix: remove unused management command

* fix: avoid get when probing for object existance

* fix: force lowercase new usernames in secr/rolodex

* fix: use explicit arguments when creating user
2023-02-21 10:01:03 -06:00
Jennifer Richards ebebdbed3e
refactor: replace datetime.now and datetime.today with timezone.now (#4211)
* refactor: replace datetime.now with timezone.now

* refactor: migrate model fields to use timezone.now as default

* refactor: replace datetime.today with timezone.now

datetime.datetime.today() is equivalent to datetime.datetime.now(); both
return a naive datetime with the current local time.

* refactor: rephrase datetime.now(tz) as timezone.now().astimezone(tz)

This is effectively the same, but is less likely to encourage accidental
use of naive datetimes.

* refactor: revert datetime.today() change to old migrations

* refactor: change a missed datetime.now to timezone.now

* chore: renumber timezone_now migration

* chore: renumber migrations
2022-08-25 13:45:16 -03:00
Robert Sparks 3697180cc1 Reverted merge of timezone-aware migration efforts.
- Legacy-Id: 18792
2021-01-12 16:54:20 +00:00
Henrik Levkowetz 774e752a54 Snapshot of timezone-aware datatracker code. Tests pass, and the test-crawler shows only expected differences. Trunk changes merged in up to r18768.
- Legacy-Id: 18770
2020-12-16 23:53:37 +00:00
Henrik Levkowetz 57938b039d Converted many cases of plain-text 403 messages to use a properly styled page instead, with a login link when appropriate. Also changed some API endpoint 400 responses to a more correct 403.
- Legacy-Id: 18339
2020-08-06 10:59:52 +00:00
Henrik Levkowetz a2318d9f2c Backported patches from production
- Legacy-Id: 17370
2020-02-28 20:46:20 +00:00
Henrik Levkowetz d2ad86c5df Python 2/3 changes pulled from the production server after the switch to python 3.
- Legacy-Id: 17327
2020-02-23 01:34:52 +00:00
Lars Eggert 8a70e9773d Fix various "RemovedInDjango20Warning" warnings. Commit ready for merge.
- Legacy-Id: 13112
2017-03-25 19:16:45 +00:00
Henrik Levkowetz 4b0a9360f0 Merged in ^/branch/iola/event-saving-refactor-r10291, which refactors document saving to always use doc.save_with_history(events), and requires accompanying events. This branch also provides refactoring of recurring regexes in url patterns into a dictionary. As part of the merge, also refactored new code which didn't use the save_with_history() method.
- Legacy-Id: 11840
2016-08-23 10:52:08 +00:00
Henrik Levkowetz 6f5234969e If an external command fails, capture stderr and raise Exception.
This handles the RFC-Editor and IANA sync external commands, but
need to be generalised, and applied to all external commands: TODO.
 - Legacy-Id: 11646
2016-07-13 13:12:08 +00:00
Ole Laursen e1f0917659 Summary: Add new document saving API, Document.save_with_history(events).
The new API requires at least one event and will automatically save a
snapshot of the document and related state. Document.save() will now
throw an exception if called directly, as the new API is intended to
ensure that documents are saved with both an appropriate snapsnot and
relevant history log, both of which are easily defeated by just
calling .save() directly.

To simplify things, the snapshot is generated after the changes to a
document have been made (in anticipation of coming changes), instead
of before as was usual.

While revising the existing code to work with this API, a couple of
missing events was discovered:

- In draft expiry, a "Document has expired" event was only generated
  in case an IESG process had started on the document - now it's
  always generated, as the document changes its state in any case

- Synchronization updates like title and abstract amendmends from the
  RFC Editor were silently (except for RFC publication) applied and
  not accompanied by a descriptive event - they now are

- do_replace in the Secretariat tools now adds an event

- Proceedings post_process in the Secretariat tools now adds an event

- do_withdraw in the Secretariat tools now adds an event

A migration is needed for snapshotting all documents, takes a while to
run. It turns out that a single document had a bad foreign key so the
migration fixes that too.
 - Legacy-Id: 10101
2015-09-28 14:01:03 +00:00
Henrik Levkowetz 2f54511763 Changed all explicitly set HttpResponse content_types to specify charset.
Django sets charset if you don't specify a content_type; if you specify
one you also must add a charset specification (if you want one).  
Fixes issue #1647.
 - Legacy-Id: 9348
2015-03-26 21:35:21 +00:00
Henrik Levkowetz 8e9f835651 Made ietf/sync pyflakes-clean.
- Legacy-Id: 7498
2014-03-16 07:49:21 +00:00
Ole Laursen 42ade6273c Turn on CSRF protection - all forms must from now on have a {% csrf_token %}
- Legacy-Id: 6963
2013-12-17 12:50:24 +00:00
Ole Laursen d66ac15fc2 Rewrite a bunch of HttpResponseRedirect + url reverse calls to instead
take advantage of the redirect shortcut (introduced in Django 1.1)
 - Legacy-Id: 6960
2013-12-17 10:23:42 +00:00
Ole Laursen e63ac36b8f Remove shim layer from ietfauth, delete some dead code and migrate
remaining importers of ietfauth.decorators to ietfauth.utils
 - Legacy-Id: 6797
2013-12-05 16:24:03 +00:00
Ole Laursen 0f69f87a56 Skip IANA - Review Needed and Version Changed - Review Needed changes
from IANA as it turns out that the Datatracker is the authoritative
source on these states. Also improve logging so that the raw JSON from
IANA is dumped, the parsed JSON on imported changes is dumped and we
write to syslog before starting a sync script in the notification view.
 - Legacy-Id: 5800
2013-06-21 16:48:05 +00:00
Henrik Levkowetz 88e1d725c5 Moved settings for RFC-Editor sync to settings.py.
- Legacy-Id: 5775
2013-06-09 17:50:19 +00:00
Henrik Levkowetz e4fd951f2c Use IANA URL settings from settings.py in some more places.
- Legacy-Id: 5676
2013-04-21 14:38:20 +00:00
Henrik Levkowetz 5345b17eab Added a missing import.
- Legacy-Id: 5455
2013-02-24 20:15:20 +00:00
Ole Laursen 9c571e65ef Undo actual state when doing an RFC Editor undo, this is only relevant
if the removed entry is the latest entry
 - Legacy-Id: 5116
2012-12-11 15:10:51 +00:00
Ole Laursen 45cf853524 Dump redacted events in DeletedEvent table
- Legacy-Id: 5025
2012-11-07 14:59:17 +00:00
Ole Laursen 793856c217 Add RFC Editor undo page, missing test and dump model
- Legacy-Id: 5013
2012-11-06 17:06:10 +00:00
Ole Laursen 2ab4161daf Return 500 on sync notify errors
- Legacy-Id: 4877
2012-09-21 13:22:04 +00:00
Ole Laursen 2415ae8687 Support blank password in sync notify view (for testing)
- Legacy-Id: 4874
2012-09-21 09:01:00 +00:00
Ole Laursen ff077f4b1f Rename /sync/iana|rfceditor/update/ to /sync/iana|rfceditor/notify/,
and put the thing to notify in the URL, like
/sync/iana/notify/changes/, instead of using POST parameters; refactor
the two sync views to use the same for IANA and RFC Editor; require
Secretariat or IANA/RFC Editor roles for triggering the updates and
support passing username and password for auth to make it easier in
the other end once we move away from HTTP Auth
 - Legacy-Id: 4868
2012-09-20 11:53:56 +00:00
Ole Laursen 85d2cdddc2 Add POST interface for IANA/RFC Editor for triggering updates, add
discrepancies report for seeing differences between the
Datatracker/RFC Editor/IANA, add tests of IANA/RFC Editor integration,
add script for weekly discrepancies emails
 - Legacy-Id: 4851
2012-09-17 15:57:48 +00:00