Commit graph

59 commits

Author SHA1 Message Date
Jennifer Richards 32054111df
fix: fix failing tests and eliminate naive datetime warnings (#4402)
* test: fix timestamp construction in several doc tests

* refactor: rename date2datetime to datetime_from_date and clarify code

* chore: helper to get tzinfo for PRODUCTION_TIMEZONE

* fix: fix timezone handling in make_last_call()

* test: fix datetime generation in doc.tests_charter

* refactor: remove PRODUCTION_TIMEZONE setting

Replaces the PRODUCTION_TIMEZONE setting with a constant,
DEADLINE_TZINFO, in ietf.utils.timezone.

* test: be more careful about timezone in tests_charter.py

* test: be more careful about timezone in doc/tests.py

* fix: fix timezone handling affecting doc.tests_draft

* fix: fix timezone handling affecting tests_irsg_ballot.py

* fix: fix timezone handling affecting tests_review.py

* fix: fix timezone handling affecting last ietf.doc tests

* fix: fix timezone handling affecting last ietf.group tests

* fix: fix timezone handling affecting ietf.iesg tests

* fix: handle timezones in get_8989_eligibility_querysets

* fix: handle timezones affecting ietfauth tests

* fix: return tz-aware datetime from utc_from_string

* fix: specify timezone for constants in ipr_rfc_number()

* fix: specify tz for ipr deadlines

* fix: handle timezones affecting liaisons tests

* fix: treat leap day in get_8989_eligibility_querysets()

Manual cherry-pick of 248d6474

* test: treat leap day properly in nomcom tests

* fix: fix timezone handling affecting nomcom tests

* test: fix timezone handling in review tests

* fix: fix timezone handling affecting secr.meetings tests

* fix: handle both pytz and zoneinfo timezones in ietf.utils.timezone

* fix: fix timezone handling affecting secr.proceedings tests

* refactor: use make_aware() helper in secr.meetings tests

* test: fix timezone handling in secr.telechat tests

* fix: fix timezone handling affecting stats tests

* fix: eliminate tz-naive helpers affecting sync email parsing

* fix: include timezone data when serializing DeletedEvent data

* fix: fix timezone handling affecting sync tests

* style: remove unused import
2022-09-01 13:07:28 -03:00
Jennifer Richards ebebdbed3e
refactor: replace datetime.now and datetime.today with timezone.now (#4211)
* refactor: replace datetime.now with timezone.now

* refactor: migrate model fields to use timezone.now as default

* refactor: replace datetime.today with timezone.now

datetime.datetime.today() is equivalent to datetime.datetime.now(); both
return a naive datetime with the current local time.

* refactor: rephrase datetime.now(tz) as timezone.now().astimezone(tz)

This is effectively the same, but is less likely to encourage accidental
use of naive datetimes.

* refactor: revert datetime.today() change to old migrations

* refactor: change a missed datetime.now to timezone.now

* chore: renumber timezone_now migration

* chore: renumber migrations
2022-08-25 13:45:16 -03:00
Jennifer Richards 6f3fb6930f Add timeouts to requests library calls. Fixes #3498. Commit ready for merge.
- Legacy-Id: 19839
2022-01-13 15:06:28 +00:00
Jennifer Richards e11583a87f Allow assignment of Person as "action holder" for a Doc, plus rudimentary automation of assignment. Fixes #3146. Commit ready for merge.
- Legacy-Id: 18829
2021-02-12 20:31:00 +00:00
Robert Sparks 3697180cc1 Reverted merge of timezone-aware migration efforts.
- Legacy-Id: 18792
2021-01-12 16:54:20 +00:00
Henrik Levkowetz 774e752a54 Snapshot of timezone-aware datatracker code. Tests pass, and the test-crawler shows only expected differences. Trunk changes merged in up to r18768.
- Legacy-Id: 18770
2020-12-16 23:53:37 +00:00
Henrik Levkowetz c9e99e3436 Refined the code in ietf.sync.rfceditor that deals with errata to distinguish between rejected errata and other errata.
- Legacy-Id: 18279
2020-07-26 14:11:04 +00:00
Jennifer Richards fff927b085 Store Auth48 URL as DocumentURL and display in RFC-Editor state. Migrates old data when possible. Alternative to 17563. Fixes #2722. Commit ready for merge.
- Legacy-Id: 18157
2020-07-13 15:34:01 +00:00
Henrik Levkowetz 6d7c5b52ee Django 2.0 conversion. Test suite passes, except for javascript tests.
- Legacy-Id: 17762
2020-05-08 13:07:07 +00:00
Henrik Levkowetz f10ddadc0e Merged in changes from trunk up to r17584.
- Legacy-Id: 17593
2020-04-07 16:02:52 +00:00
Henrik Levkowetz 726fcbf27d Removed all __future__ imports.
- Legacy-Id: 17391
2020-03-05 23:53:42 +00:00
Henrik Levkowetz ab173cb2eb Removed further six usage: six.string_types, six.integer_types, six.moves.urllib instances.
- Legacy-Id: 17390
2020-03-05 20:46:58 +00:00
Henrik Levkowetz e9a37d8ac8 Removed six.text_type(), changed six.moves.urllib to plain urllib, and removed now unused six imports.
- Legacy-Id: 17385
2020-03-05 14:41:41 +00:00
Henrik Levkowetz a14e38a1a1 Replaced most cases of using of urlopen(), instead using the higher-level 'requests' module where it simplifies the code.
- Legacy-Id: 17380
2020-03-03 17:34:42 +00:00
Henrik Levkowetz 07747b26c3 Fixed an issue with tests mocking urlopen() responses, and tweaked error messages on bad results in sync/rfceditor.py
- Legacy-Id: 17356
2020-02-27 15:05:27 +00:00
Henrik Levkowetz ebdad3a5a5 Added charset decoding of data read from an urlopen() object in a few places, and removed some use-once functions that were now dead code.
- Legacy-Id: 17355
2020-02-27 13:39:45 +00:00
Henrik Levkowetz 32e9692f54 Fixed a py2/py3 issue in sync.rfceditor.post_approved_draft(), and enhanced tests to execute the failing code (with Mock functions) instead of skipping the code.
- Legacy-Id: 17265
2020-01-28 22:37:50 +00:00
Henrik Levkowetz cb93ba8369 Fixed a py2/py3 issue in post_approved_draft()
- Legacy-Id: 17258
2020-01-27 20:31:50 +00:00
Henrik Levkowetz 09b83a0ab8 Fixed a Py2/3 compatibility issue with the verified errata code.
- Legacy-Id: 17220
2020-01-11 00:20:37 +00:00
Henrik Levkowetz ba3010e3b0 Added fetch and use of the RFC-Editor's errata.json file in order to set a new 'verified-errata' document tag, and use that to show the link to inline-errata documents only when there is verified errata. Fixes issue #2861.
- Legacy-Id: 17212
2020-01-10 12:22:33 +00:00
Henrik Levkowetz 96f6f47758 Updated rfc-editor sync code to recognize the new Tooling Issue queue state.
- Legacy-Id: 16730
2019-09-11 16:20:25 +00:00
Henrik Levkowetz 3e67a395b5 Updated ietf.sync.rfceditor.parse_index() to use the new rfc-index schema (changed placement of the page-count element).
- Legacy-Id: 16729
2019-09-11 12:48:19 +00:00
Henrik Levkowetz bdc73e771a Python2/3 compatibility: replaced six.ensure_text() with either six.text_type or django's force_text(), depending on the case, and fixed a variable scope issue.
- Legacy-Id: 16461
2019-07-16 13:20:05 +00:00
Henrik Levkowetz 8c6eb3a30a Python2/3 compatibility: Changed the use of open() and StringIO to io.open() etc.
- Legacy-Id: 16458
2019-07-15 19:14:04 +00:00
Henrik Levkowetz 0589d0b313 Changed a bunch of regexes to use r strings; also miscellaneous smaller fixes.
- Legacy-Id: 16376
2019-07-04 15:51:05 +00:00
Henrik Levkowetz d7f5c84182 Initial 2to3 patch with added copyright statement updates.
- Legacy-Id: 16309
2019-06-27 14:40:54 +00:00
Henrik Levkowetz 426870b766 Merged in the Document and DocAlias primary key change and m2m work in ^/personal/henrik/6.96.1-docalias.
- Legacy-Id: 16261
2019-06-15 12:00:48 +00:00
Henrik Levkowetz a2cafded25 Changed DocAlias to be many-to-many, in order to be able to handle STD aliases referring to multiple RFCs.
- Legacy-Id: 16249
2019-06-14 20:38:14 +00:00
Henrik Levkowetz e37303b441 Merged in [15822] and [15809] from rjsparks@nostrum.com:
Added an explicit ID-Exists state for the IESG state machine.
    Reworked code so that the IESG state machine always has a state.
    Added the ability to release a document from a working group,
    research group, or the independent stream. Releasing a document
    removes all stream state, and sets the document to have no stream.
 - Legacy-Id: 15823
Note: SVN reference [15809] has been migrated to Git commit 6118975620

Note: SVN reference [15822] has been migrated to Git commit f6886f6105
2018-12-14 22:17:58 +00:00
Henrik Levkowetz 75b0f0dacc Fixed a bug in the RFC Index parsing that caused extraction of only one paragraph of an abstract, even if there were multiple.
- Legacy-Id: 15820
2018-12-14 21:05:18 +00:00
Robert Sparks 6118975620 Added an explicit ID-Exists state for the IESG state machine. Reworked code so that the IESG state machine always has a state. Added the ability to release a document from a working group, research group, or the independent stream. Releasing a document removes all stream state, and sets the document to have no stream.
- Legacy-Id: 15809
2018-12-11 22:38:05 +00:00
Henrik Levkowetz d6565f0450 Added exception logging for ietf/bin/rfc-editor-queue-updates. Changed the logging in the ietf/bin/rfc-editor-* scripts to use ietf.utils.log.log(). Reordered some imports.
- Legacy-Id: 15318
2018-07-12 12:52:58 +00:00
Henrik Levkowetz 26bf387356 Replaced direct assignment to m2m fields with .clear() and .set()
- Legacy-Id: 14666
2018-02-21 23:58:25 +00:00
Henrik Levkowetz 4daf66d11c Added DocEvent rev= parameter througout the code.
- Legacy-Id: 12978
2017-03-06 15:08:21 +00:00
Henrik Levkowetz 476ad780e3 Fixed a bug in the rfceditor index sync introduced by the event saving refactoring.
- Legacy-Id: 11915
2016-09-01 08:24:05 +00:00
Ole Laursen 86c3a430d1 Merge in ^/branch/iola/event-saving-refactor-r10076, fixing a few problems
- Legacy-Id: 10298
2015-10-27 10:37:06 +00:00
Henrik Levkowetz 42f58a3074 Provide a bit more information in rfc-editor sync warnings.
- Legacy-Id: 10142
2015-10-07 18:52:11 +00:00
Ole Laursen e1f0917659 Summary: Add new document saving API, Document.save_with_history(events).
The new API requires at least one event and will automatically save a
snapshot of the document and related state. Document.save() will now
throw an exception if called directly, as the new API is intended to
ensure that documents are saved with both an appropriate snapsnot and
relevant history log, both of which are easily defeated by just
calling .save() directly.

To simplify things, the snapshot is generated after the changes to a
document have been made (in anticipation of coming changes), instead
of before as was usual.

While revising the existing code to work with this API, a couple of
missing events was discovered:

- In draft expiry, a "Document has expired" event was only generated
  in case an IESG process had started on the document - now it's
  always generated, as the document changes its state in any case

- Synchronization updates like title and abstract amendmends from the
  RFC Editor were silently (except for RFC publication) applied and
  not accompanied by a descriptive event - they now are

- do_replace in the Secretariat tools now adds an event

- Proceedings post_process in the Secretariat tools now adds an event

- do_withdraw in the Secretariat tools now adds an event

A migration is needed for snapshotting all documents, takes a while to
run. It turns out that a single document had a bad foreign key so the
migration fixes that too.
 - Legacy-Id: 10101
2015-09-28 14:01:03 +00:00
Ryan Cross 39fd739fad Have RFC sync change IESG state of draft to RFC Ed Queue when draft appears in queue. Fixes #1750. Commit ready for merge.
- Legacy-Id: 9788
2015-07-18 12:55:05 +00:00
Henrik Levkowetz acc2c59920 Updated a bunch of links from http: to https: based on a patch from lars@netapp.com.
- Legacy-Id: 9556
2015-04-24 21:47:12 +00:00
Henrik Levkowetz b8cd069070 Merged [7557] from rjsparks@nostrum.com: Update reference relationships (asyncronously) for new rfcs found when syncing with the rfc index. Fixes ticket #1347.
- Legacy-Id: 7572
Note: SVN reference [7557] has been migrated to Git commit 9b43e159623ed1fb53438541b25a1605922943dd
2014-04-06 15:01:12 +00:00
Henrik Levkowetz 8e9f835651 Made ietf/sync pyflakes-clean.
- Legacy-Id: 7498
2014-03-16 07:49:21 +00:00
Henrik Levkowetz 02031a01b9 Cleaned out some (arbitrary) submodule name imports into ietf.utils, and made the corresponding import statements import the names directly from the correct submodules.
- Legacy-Id: 7495
2014-03-16 07:09:38 +00:00
Ole Laursen 32874a2c89 Move draft files to archive when setting the draft state to RFC,
fixes #1254.
 - Legacy-Id: 7157
2014-01-23 11:41:22 +00:00
Ole Laursen 4525c22b54 Remove Python 2.6 compatibility hack
- Legacy-Id: 7156
2014-01-23 11:27:29 +00:00
Henrik Levkowetz 942efacb08 Added a guard to prevent duplicate RFC-Editor notifications for approved documents, and added logging to the notification routine.
- Legacy-Id: 5852
2013-07-23 22:28:19 +00:00
Henrik Levkowetz e678659b56 Merged in a lot of shim-layer removals from olau@iola.dk
(-r5194:5465 from branch/iola/shimfree).  Copying relevant
commit messages here:

  - Removed .related many to many relationship, it's not really useful
    since we always have to restrict on the relationship type anyway,
    instead add two helpers for doing the necessary queries (in both
    directions)

  - Added migration for transforming the .desc on the new_revision events
    into something more akin to what is actually shown in the history page

  - Added migration for blanking IESG notes that just consist of "RFC
    XXXX", these have been superfluous for some time

  - Grant stream chairs access to changing the stream on a draft

  - Hacked the format_history_text filter to be less weird, using the same
    formatting for snippets and full text, also link up legacy ballot set
    events

  - Moved the decoraters + utilities to new ietfauth/utils.py file

  - Added simple helper to Email to identify invalid email addresses (from
    legacy author entries)

  - Used new new_revision .desc format for when drafts are submitted

  - Improved the looks of the button class by adding extra contrast and a
    linear gradient. Currently the gradient is only visible in fairly
    recent browsers.

  - Rewrote draft and RFC tabs in terms of the new schema, porting
    write-up and history tabs as well

  - Fixed two bugs in RFC Editor syncing: make sure documents we don't know
    beforehand get a "draft" type and make sure individually submitted
    drafts get the type="individ" group instead of NULL

  - Made the CSS-styled button feel a bit nicer to use by flattening the
    active state, also introduce some temporary styles until browsers
    catch up with the standard syntax

  - Added migrations for fixing 1) a dummy RFC entry, 2) three stand-alone
    RFCs that didn't get their doc.type set, 3) a big bunch of historic
    stand-alone RFCs that have doc.group=None - set these to the
    individual submission "none" group for the time being so the view code
    doesn't have to deal with a special case.

    In some cases this is wrong since there actually was a WG associated
    but unfortunately fixing them properly requires detective work
    (probably parsing the RFCs) and in at least some cases recreating
    historic WGs. In case someone ends up doing this, the documents to
    check can still be found with

    Document.objects.filter(name__startswith="rfc", group__type="individ")
    since there are almost no new RFCs that didn't went through the I-D
    process.

  - Merged the I-D and RFC views by showing I-D information on RFCs too.
    I-Ds that have been published as RFCs redirect to the RFC URL. Also
    support alias URLs so e.g. /doc/bcpXXXX redirects to /doc/rfcXXXX.

  - Fixed revision augmentation so events after RFC publication gets a "RFC"
    designation

  - Fixed a bug with tabs not using provided name but rather doc.name

  - Displaying draft-iesg state rather than doc.friendly_state as IESG state,
    also show a notice that the IESG state refers to post-RFC processing
    if it does, like the old separate RFC page did

  - Fixed the RFC number doc.note migration to catch combined "RFC XXX; BCP
    XXX" notes too, use the opportunity to remove inserted HTML tags from
    notes and rely on linebreaksbr filter instead (the other thing was a
    left-over from the Perl days), update the various uses of the note to
    reflect that

  - Refactored slightly to make views_doc.py independent of other idrfc code

  - Moveed idrfc/views_doc.py to doc/ with associated templates, replace the
    somewhat fragile simple URL tests for views_doc.py with ordinary unit
    tests. The new tests are still fairly basic but at least test more
    than the URL tests did.

  - Made sure RFC's (and BCP/STD/FYI) are stored as RFC123 instead of
    RFC0123 in the alias table with a new migration and a change to the
    RFC Editor sync, this in turn makes /doc/std1/ do the right thing

  - Now /doc/std1/ works, we can actually do a local link in
    urlize_ietf_docs rather than linking to the tools.ietf.org server

  - Fixed history text formatter: sanitize HTML before adding linebreaks and
    non-breaking spaces, this cuts the time to render a history page with
    long comments in half

  - Added a test crawler that walks through the crawlable part of the site,
    reporting errors and slow pages

  - Got rid of initial "No record" positions when showing old positions,
    it's just noise

  - Added a .select_related() to the document main tab to reduce the number
    of DB queries, unfortunately it seems it doesn't really help with
    Django 1.2.x due to a bug (Document inherits from DocumentInfo which
    makes things a bit more complicated)

  - Introduced a simple cache in doc.get_state so repeated reads don't
    cause a DB query

  - Cleaned up the search code in preparation for removal of the shim-layer;
    use a static button and don't send extraneous GET parameters

  - Removed dead code in several places
 - Legacy-Id: 5830
2013-07-15 20:55:24 +00:00
Henrik Levkowetz a4d486ab26 Use argument 'draft' for document instead of 'approved_draft_name' when sending sync notifications to the RFC-Editor.
- Legacy-Id: 5776
2013-06-09 17:55:47 +00:00
Henrik Levkowetz 88e1d725c5 Moved settings for RFC-Editor sync to settings.py.
- Legacy-Id: 5775
2013-06-09 17:50:19 +00:00
Henrik Levkowetz 8eec9ccb15 Don't actually do a post to the rfc-editor's state sync interface for a publication request when not in production mode.
- Legacy-Id: 5773
2013-06-03 20:49:40 +00:00