Commit graph

166 commits

Author SHA1 Message Date
Jennifer Richards 3ddd8c9cce
fix: Fix/adjust periodic_tasks management command (#7108)
* fix: Skip midnight partial rfced sync

* fix: call iana protocols update task correctly
2024-02-28 08:15:04 -06:00
Jennifer Richards b4cf04a09d
feat: celery tasks to replace ietf/bin scripts (#6971)
* refactor: Change import style for clarity

* feat: Add iana_changes_updates_task()

* chore: Squelch lint warning

My linter does not like variables defined outside
of __init__()

* feat: Add PeriodicTask for iana_changes_updates_task

* refactor: tasks instead of scripts on sync.views.notify()

* test: Test iana_changes_updates_task

* refactor: rename task for consistency

* feat: Add iana_protocols_update_task

* feat: Add PeriodicTask for iana protocols sync

* refactor: Use protocol sync task instead of script in view

* refactor: itertools.batched() not available until py312

* test: test iana_protocols_update_task

* feat: Add idindex_update_task()

Calls idindex generation functions and does the file
update dance to put them in place.

* chore: Add comments to bin/hourly

* fix: annotate types and fix bug

* feat: Create PeriodicTask for idindex_update_task

* refactor: Move helpers into a class

More testable this way

* refactor: Make TempFileManager a context mgr

* test: Test idindex_update_task

* test: Test TempFileManager

* fix: Fix bug in TestFileManager

yay testing

* feat: Add expire_ids_task()

* feat: Create PeriodicTask for expire_ids_task

* test: Test expire_ids_task

* test: Test request timeout in iana_protocols_update_task

* refactor: do not re-raise timeout exception

Not sure this is the right thing to do, but it's the
same as rfc_editor_index_update_task

* feat: Add notify_expirations_task

* feat: Add "weekly" celery beat crontab

* refactor: Reorder crontab fields

This matches the crontab file field order

* feat: Add PeriodicTask for notify_expirations

* test: Test notify_expirations_task

* test: Add annotation to satisfy mypy
2024-01-31 15:24:20 -06:00
Jennifer Richards 0dadeb2af8
fix: Fix task specs in periodic_tasks.py (#6965) 2024-01-24 11:31:34 -06:00
Robert Sparks d9cc26be96
feat: replace references to User with references to Person (#6024)
* refactor: change references from User to Person (#5821)

* refactor: Change CommunityList reference from User to Person

* refactor: Convert more user references to person

* refactor: Change augment_docs_and_user_with_user_info to person

* refactor: Change Nomination and Feedback references from User to Person

* refactor: Change a few test case function signatures to be more pythonic

* refactor: Harmonize how profile and photo views look up email_or_name

* refactor: Rework community views to operate on Person instead of User (#5859)

* test: Update tests to try all of the person's emails and aliases

* fix: Recode a test case to avoid an exception if there's Unicode in the URL

This only happens using the form-filling and submission feature of
WebTest, which is only used in this one test case, so just it rip out.

* test: Add duplicate-person tests

* fix: If there are multiple matching users, prefer the logged-in one.

* chore: We no longer use WebTest, so don't include it.

* fix: Address review comments

* fix: case-insensitive person name or email matching (#6096)

* chore: Renumber migrations

* fix: Update merged code so tests pass (#6887)

* fix: Use refactored method

* fix: Don't assume user has person

* fix: Use new view param name

* chore: Drop community lists w/o person; cleanup (#6896)

* fix: Don't assume user has person

* fix: user->person in update_community_list_index.py

* feat: Remove CommunityLists without Person

* refactor: Speed up nomcom migrations

---------

Co-authored-by: Paul Selkirk <paul@painless-security.com>
Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
2024-01-24 11:00:19 -06:00
Jennifer Richards 36c43c8520
chore: add task tests; move message task to message app (#6964)
* test: Test send_review_reminders_task

* refactor: Move send_scheduled_mail_task to message app

* chore: Remove unused import

* test: Add Message/SendQueue factories

* test: Test send_scheduled_mail_task

* test: Reset mocks before reuse

* test: Cover error conditions

* test: Return non-empty change set

* test: Test SMTPException handling

* test: Test fetch_attendance_from_meetings()

* test: Test RuntimeError handling

* test: RFC index sync should populate authors
2024-01-24 08:53:42 -06:00
Jennifer Richards 8d12071bf5
refactor: Move cron jobs to celery tasks (#6926)
* refactor: Factor out helper from fetch_meeting_attendance.py

* feat: Define fetch_meeting_attendance_task task

Equivalent to the fetch_meeting_attendance management command

* chore: Disable fetch_meeting_attendance in bin/daily

* feat: Log errors in fetch_meeting_attendance_task

* feat: Enable a result backend for celery

Ignore results by default, but enable the backend so we
can manage tasks

* feat: Define daily task in ietf.utils.tasks

* refactor: Make bin/send-review-reminders into a task

* refactor: Make bin/send-scheduled-mail into a task

* chore: Update copyright years

* refactor: Make bin/rfc-editor-index-updates into a task

* refactor: Accept date type in rfc index update fn

* chore: Update comment

* fix: Annotate param as Optional

* fix: Revert treating skip_older_than_date as str

Misunderstood the comment, "fixed" a non-bug. Oops.

* feat: mgmt command to create periodic tasks

* feat: add summary of tasks to mgmt cmd

* style: black

* fix: Remove debug statements

* feat: Enable/disable tasks

* chore: Disable periodic tasks by default

* chore: Revert changes to daily and every15m

* fix: Call intended function

* chore: Add task descriptions
2024-01-22 12:04:16 -06:00
Robert Sparks 383899c5c4
chore: checkpoint: docalias mostly removed 2023-08-15 16:49:17 -05:00
Jennifer Richards 8f0b459050
refactor: Replace is_rfc() tests (#5925)
* refactor: Remove is_rfc() - test type_id instead

* fix: Guard against unknown pub_date

This should not ever come up - we have a published_rfc event for
every rfc. Should investigate fixing pub_date() to always return a val.
2023-07-06 12:07:53 -03:00
Jennifer Richards 264ff60cd2
Merge pull request #5601 from jennifer-richards/django4
chore: Upgrade to Django 3.0
2023-05-11 11:04:36 -04:00
Robert Sparks f8113cb862
fix: close open things (#5593)
* fix: close open things

* fix: clean up test created files

* fix: remove one close too many
2023-05-10 11:19:34 -05:00
Jennifer Richards 07e26dd52e
refactor: Replace deprecated force_text with force_str 2023-05-09 15:21:50 -03:00
Robert Sparks c38ade6e1b
feat: treat django auth username as case insensitive throughout the datatracker (#5165)
* feat: insensitive username matching at django authentication

* feat: use iexact when using the User object manager

* fix: more places to ignore username case

* fix: remove unused management command

* fix: avoid get when probing for object existance

* fix: force lowercase new usernames in secr/rolodex

* fix: use explicit arguments when creating user
2023-02-21 10:01:03 -06:00
Lars Eggert f8b48f4c43
fix: use Internet-Draft more consistently across the UI (#5104)
* s/Internet Draft/Internet-Draft/i

* s/draft/Internet-Draft/i or s/draft/I-D/i

* s/ID/I-D/

* Fix tests

* a -> an

* Undo case-change to ASCII

* Address code review comments

* Add migrations

* Add merged migration

* fix: straighten out migrations

* fix: finish straightening out migrations

---------

Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
2023-02-11 10:09:28 -06:00
Lars Eggert 220be21998
chore: Use codespell to fix typos in code. (#4797)
* chore: Use codespell to fix typos in code.

Second part of replacement of #4651

@rjsparks, I probably need to revert some things here, and I also
still need to add that new migration - how do I do that?

* Revert migrations

* Migrate "Whitelisted" to "Allowlisted"

* TEST_COVERAGE_MASTER_FILE -> TEST_COVERAGE_MAIN_FILE

* Fix permissions

* Add suggestions from @jennifer-richards
2022-12-07 15:10:35 -06:00
Lars Eggert 6eabd4a3a1
chore: Use codespell to fix typos in comments. (#4794)
First part of replacement of #4651
2022-11-28 10:36:36 -06:00
Jennifer Richards ebebdbed3e
refactor: replace datetime.now and datetime.today with timezone.now (#4211)
* refactor: replace datetime.now with timezone.now

* refactor: migrate model fields to use timezone.now as default

* refactor: replace datetime.today with timezone.now

datetime.datetime.today() is equivalent to datetime.datetime.now(); both
return a naive datetime with the current local time.

* refactor: rephrase datetime.now(tz) as timezone.now().astimezone(tz)

This is effectively the same, but is less likely to encourage accidental
use of naive datetimes.

* refactor: revert datetime.today() change to old migrations

* refactor: change a missed datetime.now to timezone.now

* chore: renumber timezone_now migration

* chore: renumber migrations
2022-08-25 13:45:16 -03:00
Robert Sparks b988850985
fix: avoid mutables as defaults. Compute date default arguments at runtime rather than loadtime. (#4144) 2022-07-06 14:39:36 -05:00
Lars Eggert 159f89ebf3
chore: Remove six and pathlib2 from deps (#4031)
Fixes: #3895 and #3892
2022-05-31 09:12:45 -05:00
Robert Sparks a60c31e45d
chore: remove svn/trac infrastructure (#3649)
* chore: Remove svn/trac related infrastructure

* chore: Remove more svn/trac infrastructure

* fix: remove commented out Trac requirement
2022-03-16 11:08:47 -05:00
Robert Sparks 95cb3f041e Merged in [19965] from rjsparks@nostrum.com:
Remove unused imports.
 - Legacy-Id: 19966
Note: SVN reference [19965] has been migrated to Git commit 70cd714fac087b5b5257708541d8ae4de8c5a6a6
2022-02-22 23:12:21 +00:00
Robert Sparks ec71083d1d Only keep the first and most recent yang validator SubmissionCheck for any given submission. Fixes #3542. Commit ready for merge.
- Legacy-Id: 19952
2022-02-18 22:46:40 +00:00
Jennifer Richards 968b775315 Create management command base class that sends emails on exceptions. Fixes #3356 and #3357. Commit ready for merge.
- Legacy-Id: 19493
2021-10-29 01:58:21 +00:00
Robert Sparks 3697180cc1 Reverted merge of timezone-aware migration efforts.
- Legacy-Id: 18792
2021-01-12 16:54:20 +00:00
Henrik Levkowetz 774e752a54 Snapshot of timezone-aware datatracker code. Tests pass, and the test-crawler shows only expected differences. Trunk changes merged in up to r18768.
- Legacy-Id: 18770
2020-12-16 23:53:37 +00:00
Henrik Levkowetz 606cedd796 Merged in the ExtResource work from rjsparks@nostrum.com, based on a feature request and contributions from rsalz@akamai.com. This provides external resource models for Person, Group, and Document models, as a generalisation of the previous related-URL concept. This provides a consistent labelling and classification of URLs and other resources related to a Group, Document, or Person. The GroupURL (and similar) classes will be removed in a later step. Some DocumentURL instances (such as auth48 URLs) remain to convert, as they have come in with other merged-in work while the ExtResource work was in transit.
- Legacy-Id: 18192
2020-07-20 01:58:26 +00:00
Henrik Levkowetz d8b11c4293 Removed an unused and unnecessary API endpoint, now that gather.town has gotten OpenID Connect authentication working.
- Legacy-Id: 18190
2020-07-17 19:33:15 +00:00
Robert Sparks 99df95d542 Cleaned up created names. Improved validation during migration. Cleaned up migration output. Cleaned the last of the awp includes from secr views. Removed now unused secr templates. Build extresource urls correctly during submission and when creating group wikis.
- Legacy-Id: 18158
2020-07-13 16:03:39 +00:00
Henrik Levkowetz 0924640676 Removed the command to purge request_profiler entries, given that request_profiler is incompatible with Django 2.2
- Legacy-Id: 18107
2020-06-30 14:34:15 +00:00
Henrik Levkowetz cb0817175e Tweaked the showloggers command help text.
- Legacy-Id: 17979
2020-06-13 21:28:32 +00:00
Henrik Levkowetz 77524a333b Added a management command to list registered loggers, in order to see which can be configured in settings_local with the UTILS_LOGGER_LEVELS settings.
- Legacy-Id: 17974
2020-06-12 15:00:15 +00:00
Henrik Levkowetz 3a963eaf29 Removed a future import
- Legacy-Id: 17818
2020-05-18 10:59:59 +00:00
Henrik Levkowetz 6d7c5b52ee Django 2.0 conversion. Test suite passes, except for javascript tests.
- Legacy-Id: 17762
2020-05-08 13:07:07 +00:00
Henrik Levkowetz c5a7cfaf10 Changed the verbosity of the purge_request_profiler_records command to require -v2 to show records deleted.
- Legacy-Id: 17704
2020-04-28 18:04:00 +00:00
Henrik Levkowetz 78963ecdef Added request profiler and a management command to purge profiler records.
- Legacy-Id: 17648
2020-04-15 20:48:08 +00:00
Henrik Levkowetz 5a0a502ff9 Merged in the py3clean work, which removes usage of six and __future__ imports, along with other py2/3 compatibility code.
- Legacy-Id: 17619
2020-04-14 19:06:53 +00:00
Henrik Levkowetz f10ddadc0e Merged in changes from trunk up to r17584.
- Legacy-Id: 17593
2020-04-07 16:02:52 +00:00
Henrik Levkowetz a0d1ffa7be Fixed a pyflakes issue.
- Legacy-Id: 17472
2020-03-20 18:59:16 +00:00
Henrik Levkowetz ea1d21a45c Added another check to the check_draft_event_revision_integrity management command, and refined it somewhat.
- Legacy-Id: 17469
2020-03-19 22:44:08 +00:00
Henrik Levkowetz 480301f020 Refactored and extended check_draft_event_revision_integrity a bit.
- Legacy-Id: 17445
2020-03-17 18:11:43 +00:00
Henrik Levkowetz c10706286b Added a management command to check draft event revision numbers. To be extended for other checks.
- Legacy-Id: 17423
2020-03-11 22:02:30 +00:00
Henrik Levkowetz 50e8918a56 Added a progress bar for verbosity=1 of the community list index update command.
- Legacy-Id: 17414
2020-03-09 18:03:09 +00:00
Henrik Levkowetz 2669afe69b Tweaked the check_referential_integrity management command verbose output.
- Legacy-Id: 17397
2020-03-06 20:37:41 +00:00
Henrik Levkowetz 726fcbf27d Removed all __future__ imports.
- Legacy-Id: 17391
2020-03-05 23:53:42 +00:00
Henrik Levkowetz ab173cb2eb Removed further six usage: six.string_types, six.integer_types, six.moves.urllib instances.
- Legacy-Id: 17390
2020-03-05 20:46:58 +00:00
Henrik Levkowetz ab944725d5 Py2/3 compatibility tweaks for pyflakes.
- Legacy-Id: 17375
2020-03-02 18:00:41 +00:00
Henrik Levkowetz b6d8bade44 Fixed a Py2/3 issue in the pyflakes management command, and tweaked the verbose output format.
- Legacy-Id: 17372
2020-03-02 12:31:02 +00:00
Henrik Levkowetz d2ad86c5df Python 2/3 changes pulled from the production server after the switch to python 3.
- Legacy-Id: 17327
2020-02-23 01:34:52 +00:00
Henrik Levkowetz aabd237656 Fixed some pyflakes issues.
- Legacy-Id: 17304
2020-02-18 19:39:03 +00:00
Henrik Levkowetz 7b5bebc727 Added a management command to dump data based on table names and primary keys derived from a list of SQL 'INSERT INTO' commands representing the items to be dumped. The output formats are the same as for dumpdata, and are suitable for later use with the loaddata or mergedata management commands.
- Legacy-Id: 17300
2020-02-16 21:40:01 +00:00
Henrik Levkowetz e0c6f3abe2 Added a management command to merge data from one of two divergent django databases to the other, taking care to insert or update records as appropriate, and update foreign keys and many-to-many keys appropriately. Accepts the dump formats generated by the dumpdata command.
- Legacy-Id: 17299
2020-02-16 21:39:08 +00:00