Commit graph

221 commits

Author SHA1 Message Date
Lars Eggert 6eabd4a3a1
chore: Use codespell to fix typos in comments. (#4794)
First part of replacement of #4651
2022-11-28 10:36:36 -06:00
Jennifer Richards a653582cf7
fix: timezone fixes from last few comparisons with tzaware-obe 2022-09-28 13:21:50 -03:00
Jennifer Richards c5e5a00ff3
fix: fill in tzinfo for datetime.fromtimestamp() calls
Unix timestamp is defined to be UTC and fromtimetamp correctly turns
it into whatever timezone you request. In most cases, just need to
specify *something* to avoid creating naive datetimes.
2022-09-22 12:36:35 -03:00
Jennifer Richards 4084d7d557
fix: record and interpret RFC pub dates in correct timezone (#4421)
* fix: use PST8PDT for published_rfc event timestamps

* fix: find RFCs by PST8PDT year in RfcFeed

* refactor: add const RPC_TZINFO to represent RFC publication timezone

* chore: remove (rather than fix) unused template tags

* fix: always return RPC_TZINFO-local date from Document.pub_date()

* refactor: convert 'published' flag to a Boolean to reflect its usage

* fix: display doc publication dates in correct time zones

* fix: fix various small issues breaking tests
2022-09-08 14:51:19 -03:00
Jennifer Richards 32054111df
fix: fix failing tests and eliminate naive datetime warnings (#4402)
* test: fix timestamp construction in several doc tests

* refactor: rename date2datetime to datetime_from_date and clarify code

* chore: helper to get tzinfo for PRODUCTION_TIMEZONE

* fix: fix timezone handling in make_last_call()

* test: fix datetime generation in doc.tests_charter

* refactor: remove PRODUCTION_TIMEZONE setting

Replaces the PRODUCTION_TIMEZONE setting with a constant,
DEADLINE_TZINFO, in ietf.utils.timezone.

* test: be more careful about timezone in tests_charter.py

* test: be more careful about timezone in doc/tests.py

* fix: fix timezone handling affecting doc.tests_draft

* fix: fix timezone handling affecting tests_irsg_ballot.py

* fix: fix timezone handling affecting tests_review.py

* fix: fix timezone handling affecting last ietf.doc tests

* fix: fix timezone handling affecting last ietf.group tests

* fix: fix timezone handling affecting ietf.iesg tests

* fix: handle timezones in get_8989_eligibility_querysets

* fix: handle timezones affecting ietfauth tests

* fix: return tz-aware datetime from utc_from_string

* fix: specify timezone for constants in ipr_rfc_number()

* fix: specify tz for ipr deadlines

* fix: handle timezones affecting liaisons tests

* fix: treat leap day in get_8989_eligibility_querysets()

Manual cherry-pick of 248d6474

* test: treat leap day properly in nomcom tests

* fix: fix timezone handling affecting nomcom tests

* test: fix timezone handling in review tests

* fix: fix timezone handling affecting secr.meetings tests

* fix: handle both pytz and zoneinfo timezones in ietf.utils.timezone

* fix: fix timezone handling affecting secr.proceedings tests

* refactor: use make_aware() helper in secr.meetings tests

* test: fix timezone handling in secr.telechat tests

* fix: fix timezone handling affecting stats tests

* fix: eliminate tz-naive helpers affecting sync email parsing

* fix: include timezone data when serializing DeletedEvent data

* fix: fix timezone handling affecting sync tests

* style: remove unused import
2022-09-01 13:07:28 -03:00
Jennifer Richards 8b52d27b02
refactor: refactor timestamp handling so tests in meeting app pass (#4371)
* refactor: replace datetime.now with timezone.now

* refactor: migrate model fields to use timezone.now as default

* refactor: replace datetime.today with timezone.now

datetime.datetime.today() is equivalent to datetime.datetime.now(); both
return a naive datetime with the current local time.

* refactor: rephrase datetime.now(tz) as timezone.now().astimezone(tz)

This is effectively the same, but is less likely to encourage accidental
use of naive datetimes.

* refactor: revert datetime.today() change to old migrations

* refactor: change a missed datetime.now to timezone.now

* chore: renumber timezone_now migration

* chore: add migration to change timestamps to UTC

* refactor: move tz instantiation/caching from TimeSlot to Meeting

* fix: assume utc if meeting.time_zone is blank

* chore: make datetime.combine() calls tz aware in the meeting app

* ci: correctly use meeting.tz in TimeSlotFactory

* chore: compute TimeSlot utc / local times assuming tz-aware times

* chore: use tzaware math for agenda editor timeslot layout

* chore: fill in Meeting.time_zone where it is blank

Nearly all interim meetings on or before 2016-07-01 have blank
time_zone values. This migration fills these in with PST8PDT.

* chore: disallow blank Meeting.time_zone value

* refactor: no need to handle blank time_zone case in TZ migration

* refactor: remove now-unnecessary checks that meeting has time_zone

* chore: fix timezone handling in agenda.ics and Meeting.updated()

* chore: fix tz handling in interim_request_details, exercise in tests

* chore: fix timezone handling for test_interim_send_announcement

* chore: fix timezone handling in agenda_json()

* chore: fix timezone handling in old agenda

* chore: fix timezone handling for EditTimeslotsTests

* refactor: refactor a few fixes for more consistent timezone handling

* chore: add timezone info to timestamps in fixtures

* chore: remove naive datetime warnings found in meetings.tests_views

* chore: fix a few more test failures in meetings.tests_views

All tests in meetings.tests_views now passing

* chore: remove unused import

* chore: fix timezone handling in test_schedule_generator.py

* chore: fix timezone handling affecting meeting.tests_js

* chore: fix timeslot test bug when local date != UTC date

* test: fix a few failing tests, all meetings tests now pass

(for me, anyway)

* chore: renumber migrations

* chore: update timestamp conversion migration

The django-celery-beat package introduces tables with timestamp
columns. These columns are stored in CELERY_TIMEZONE. Because we run with
this set to UTC, the migration ignores these columns.

* chore: fix pytz-related change in migration

* chore: remove duplicate migrations

* chore: remove CELERY_BEAT_TZ_AWARE setting now that USE_TZ is True

* test: avoid failure in test with bogus timezone
2022-08-26 16:53:19 -03:00
Jennifer Richards ebebdbed3e
refactor: replace datetime.now and datetime.today with timezone.now (#4211)
* refactor: replace datetime.now with timezone.now

* refactor: migrate model fields to use timezone.now as default

* refactor: replace datetime.today with timezone.now

datetime.datetime.today() is equivalent to datetime.datetime.now(); both
return a naive datetime with the current local time.

* refactor: rephrase datetime.now(tz) as timezone.now().astimezone(tz)

This is effectively the same, but is less likely to encourage accidental
use of naive datetimes.

* refactor: revert datetime.today() change to old migrations

* refactor: change a missed datetime.now to timezone.now

* chore: renumber timezone_now migration

* chore: renumber migrations
2022-08-25 13:45:16 -03:00
Lars Eggert 5598762608
fix: add more HTML validation & fixes (#3891)
* Update vnu.jar

* Fix py2 -> py3 issue

* Run pyupgrade

* test: Add default-jdk to images

* test: Add option to also validate HTML with vnu.jar

Since it's already installed in bin. Don't do this by default, since it
increases the time needed for tests by ~50%.

* fix: Stop the urlizer from urlizing in linkified mailto: text

* More HTML fixes

* More HTML validation fixes

* And more HTML fixes

* Fix floating badge

* Ignore unicode errors

* Only URLize docs that are existing

* Final fixes

* Don't URLize everything during test-crawl

* Feed HTML into vnu using python rather than Java to speed things up

* Allow test-crawl to start vnu on a different port

* Increase retry count to vnu. Restore batch size to 30.

* More HTML validation fixes

* Use urllib3 to make requests to vnu, since overriding requests_mock is tricky

* Undo commit of unmodified file

* Also urlize ftp links

* Fix matching of file name

* More HTML fixes

* Add `is_valid_url` filter

* weekday -> data-weekday

* urlencode URLs

* Add and use vnu_fmt_message. Bump vnu max buffer.

* Simplify doc_exists

* Don't add tab link to mail archive if the URL is invalid

* Run urlize_ietf_docs before linkify

Reduces the possibility of generating incorrect HTML

* Undo superfluous change

* Runner fixes

* Consolidate vnu message filtering into vnu_filter_message

* Correctly handle multiple persons with same name

* Minimze diff

* Fix HTML nits

* Print source snippet in vnu_fmt_message

* Only escape if there is something to escape

* Fix snippet

* Skip crufty old IPR declarations

* Only include modal when needed. Add handles.

* Fix wordwrap+linkification

* Update ietf/doc/templatetags/ietf_filters.py

* Update ietf/doc/templatetags/tests_ietf_filters.py

* Don't right-align second column
2022-05-03 13:55:48 -05:00
Jennifer Richards fd0df6f619 Add ability to import session minutes from notes.ietf.org. Mock out calls to the requests library in tests. Call markdown library through a util method. Fixes #3489. Commit ready for merge.
- Legacy-Id: 19763
2021-12-09 17:16:19 +00:00
Jennifer Richards 81d9234d54 Use temporary directories instead of "real" filesystem for tests. Fixes #3414. Commit ready for merge.
- Legacy-Id: 19555
2021-11-05 19:45:57 +00:00
Robert Sparks 3697180cc1 Reverted merge of timezone-aware migration efforts.
- Legacy-Id: 18792
2021-01-12 16:54:20 +00:00
Henrik Levkowetz 59b49c846b Fixed a number of TimeSlot-related issues, in particular related to getting the correct date from a timeslot. Also fixed the displayed slot times in the new schedule editors to show local times, not UTC times.
- Legacy-Id: 18785
2020-12-19 22:55:11 +00:00
Henrik Levkowetz 774e752a54 Snapshot of timezone-aware datatracker code. Tests pass, and the test-crawler shows only expected differences. Trunk changes merged in up to r18768.
- Legacy-Id: 18770
2020-12-16 23:53:37 +00:00
Henrik Levkowetz 5d26af8e40 Merged in [18587] from rcross@amsl.com:
Fix section 5 of secretariat progress report. Fixes #3063.
 - Legacy-Id: 18594
Note: SVN reference [18587] has been migrated to Git commit c2a0bd613b
2020-10-11 14:56:30 +00:00
Henrik Levkowetz f9b3e2e477 Merged in ^/branch/iola/meeting-improvement-r18382@18466. This provides a beta-level snapshot of the new schedule editor work. All known issues should be fixed, but there might be adjustments needed after experience with using the new edit view.
- Legacy-Id: 18467
2020-09-04 14:28:25 +00:00
Henrik Levkowetz c95ffe42e7 Fixed a pyflakes issue
- Legacy-Id: 18393
2020-08-20 13:47:27 +00:00
Henrik Levkowetz 6dd0b7c81e Updated an initial migration to reflect actual database tables present.
- Legacy-Id: 18390
2020-08-20 12:35:37 +00:00
Henrik Levkowetz 48126ba505 - Legacy-Id: 18389 2020-08-20 12:34:09 +00:00
Henrik Levkowetz b4cc53fea0 Migration for deleted model
- Legacy-Id: 18388
2020-08-20 12:31:52 +00:00
Ole Laursen a39fa7f967 New branch from trunk @ r18382 for meeting improvements, with iola/meeting-improvement-r17835 merged in
- Legacy-Id: 18385
2020-08-20 09:36:41 +00:00
Henrik Levkowetz 50d559f05f Removed an unused model.
- Legacy-Id: 18374
2020-08-15 09:34:25 +00:00
Ole Laursen c78ffbcd18 Introduce support for setting a base schedule on a schedule. All
assignments on the base schedule are shown in the pages for the
schedule, read-only.

This allows managing things like breaks and misc sessions separately
from the regular WG sessions.

Base schedules are not allowed to be the base of other base schedules
(the hierarchy can only be one level deep) to simplify the mental
model and the code.

Add link for creating new schedules instead of relying on copying
Empty-Schedule and change the meeting creation code to no longer
create the special Empty-Schedule. Instead a "base" schedule is
created and a first schedule with the name and permissions of the user
creating the meeting, using "base" as base.

Speed up a couple of the Secretariat/AD agenda views by adding
prefetches.
 - Legacy-Id: 18355
2020-08-11 17:34:32 +00:00
Robert Sparks 2b10465aa7 Convert IRTF area groups into their own group type rather than attempting to overload AG. Fixes #3027. Commit ready for merge.
- Legacy-Id: 18298
2020-07-28 18:54:44 +00:00
Henrik Levkowetz 3a76e4a935 Added an API endpoint for bluesheet information upload. Factored out code in common with manual upload_session_bluesheets(). Tweaked ietf.secr.proceedings.utils.handle_upload_file() to handle temp files in addition to http file objects, to support the API endpoint.
- Legacy-Id: 18067
2020-06-26 14:49:13 +00:00
Henrik Levkowetz 9aeda11a29 Removed some Py2 compatibility decorators.
- Legacy-Id: 17673
2020-04-21 13:54:31 +00:00
Henrik Levkowetz f10ddadc0e Merged in changes from trunk up to r17584.
- Legacy-Id: 17593
2020-04-07 16:02:52 +00:00
Ryan Cross 5649bfdfd1 Remove drafts secretariat tool because this functionality is now provided by the core Datatracker. Moved ID reports to proceedings tool. Fixes #1655. Commit ready for merge.
- Legacy-Id: 17488
2020-03-21 19:52:12 +00:00
Henrik Levkowetz 726fcbf27d Removed all __future__ imports.
- Legacy-Id: 17391
2020-03-05 23:53:42 +00:00
Henrik Levkowetz ab173cb2eb Removed further six usage: six.string_types, six.integer_types, six.moves.urllib instances.
- Legacy-Id: 17390
2020-03-05 20:46:58 +00:00
Ole Laursen 568670c060 Turn sessions into regular sessions and non-sessions into misc.
sessions in the UI and code to avoid ambiguity.

This doesn't change the data in the DB except for uses of
TimeSlotTypeName where 'session' is now 'regular'.
 - Legacy-Id: 17128
2019-12-06 20:02:26 +00:00
Ole Laursen 3957743b85 Move Session.status, .requested, and .requested_by to a new SchedulingEvent
- Legacy-Id: 17122
2019-12-05 12:41:09 +00:00
Ole Laursen 0555879777 Rename Meeting.agenda to Meeting.schedule together with a bunch of
related internal things
 - Legacy-Id: 17051
2019-11-18 17:22:19 +00:00
Henrik Levkowetz 4ed73d2ba9 Removed unused .py file
- Legacy-Id: 16767
2019-09-30 15:35:03 +00:00
Henrik Levkowetz b2be7975c9 Merged in [16600] from rjsparks@nostrum.com:
Fixed a subtle bug in the code that converts slides to PDF that was writing back into settings.
 - Legacy-Id: 16645
Note: SVN reference [16600] has been migrated to Git commit b4714f95579efcb5051438239a5df002036c9975
2019-08-10 18:06:40 +00:00
Henrik Levkowetz 8c6eb3a30a Python2/3 compatibility: Changed the use of open() and StringIO to io.open() etc.
- Legacy-Id: 16458
2019-07-15 19:14:04 +00:00
Henrik Levkowetz ea8c84d2f6 Python2/3 compatibility: used @python_2_unicode_compatible to provide __unicode__() versions of __str__() methods.
- Legacy-Id: 16455
2019-07-15 18:01:26 +00:00
Henrik Levkowetz a5e31c3f40 Py2/3 compatibility: Added __future__ import.
- Legacy-Id: 16452
2019-07-15 15:52:18 +00:00
Henrik Levkowetz 2d7c9629aa Added 'from __future__' imports all over the place, to bring code behaviour into closer alignment between python2 and python3
- Legacy-Id: 16446
2019-07-15 15:40:51 +00:00
Henrik Levkowetz f33ecaa1a4 Fixed a test that compared an int with a meeting number string. Changed meeting test data to use meeting number 72 instead of 42, because proceedings code gives 404 for meetings before 65. Changed a number of StringIO() to BytesIO().
- Legacy-Id: 16378
2019-07-04 16:01:32 +00:00
Henrik Levkowetz f5ae2541ad Replaced multiple asserts with assertContains().
- Legacy-Id: 16353
2019-07-01 14:09:12 +00:00
Henrik Levkowetz 484dcece5f Changed all model __unicode__() methods to __str__()
- Legacy-Id: 16330
2019-06-28 21:07:08 +00:00
Henrik Levkowetz d7f5c84182 Initial 2to3 patch with added copyright statement updates.
- Legacy-Id: 16309
2019-06-27 14:40:54 +00:00
Henrik Levkowetz 426870b766 Merged in the Document and DocAlias primary key change and m2m work in ^/personal/henrik/6.96.1-docalias.
- Legacy-Id: 16261
2019-06-15 12:00:48 +00:00
Henrik Levkowetz a2cafded25 Changed DocAlias to be many-to-many, in order to be able to handle STD aliases referring to multiple RFCs.
- Legacy-Id: 16249
2019-06-14 20:38:14 +00:00
Henrik Levkowetz 815602351f This is a series of 50 migrations that changes the Document and DocAlias
primary keys from character strings to integers, and makes corresponding code
changes.

This was prompted by database limitations discovered when trying to make
DocAlias use a m2m document field; with 255 long strings as primary keys for
Document and DocAlias this violated the MySQL database limitations.

Changing the primary keys to integers should also improve efficiency.  

Due to the data migrations which create the new integer primary keys and adds
corresponding integer foreign keys matching the previous string foreign keys
in all tables having foreign keys to Document and DocAlias, some of these
migrations take a long time.  The total set of migrations are expected to have
a runtime on the order of 2 hours.
 - Legacy-Id: 16237
2019-06-10 11:32:46 +00:00
Robert Sparks 25cc00f925 Refactored DocumentInfo to address overloading the external_url field with strings that are not URLs. Commit ready for merge.
- Legacy-Id: 15864
2019-01-02 22:55:00 +00:00
Henrik Levkowetz 5cba7fc8a8 Added exclusion of cancelled session when doing audio file import. Fixes an server 500 error when a cancelled session is shown in the same room and timeslot as an actual session.
- Legacy-Id: 15752
2018-11-11 14:33:45 +00:00
Robert Sparks 87bda66740 Converted several secr apps to use factories for tests. Commit ready for merge.
- Legacy-Id: 15568
2018-10-12 22:08:31 +00:00
Henrik Levkowetz fcbbe08eba Removed video url import from youtube, which is not needed any more given the /api/meeting/session/video/url API endpoint. This also prevents duplicate video URLs.
- Legacy-Id: 15424
2018-08-10 12:54:27 +00:00
Henrik Levkowetz 0ef66ae9e2 Added optional arguments title and user to create_recording(), to make it possible to affect those attributes.
- Legacy-Id: 14966
2018-03-29 11:02:58 +00:00