Commit graph

1655 commits

Author SHA1 Message Date
Jennifer Richards 48e0aa23f5
refactor: clean up logging (#7419)
* fix: log to stdout/stderr in json format

* chore: remove UTILS_LOGGER_LEVELS

This is not used (there _is_ a setting for the
django.security logger in settings_local.py on
production, but it is redundant with the
settings.LOGGING configuration and is not doing
anything).

* chore: revert to debug_console django logging

* chore: log.log to syslog via datatracker logger

* chore: remove unused imports

---------

Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
2024-05-14 18:47:40 -05:00
Robert Sparks cedd58f950
feat: obviate ghostlinkd (#7336)
* wip: identify whats needed to obviate ghostlinkd

* fix: hardlink new charter files to ftp directory

* fix: hardlink new charter files to ftp directory (continued)

* chore: bring settings comment up to date

* chore: add archive and ftp dirs to setup of various environments

* fix: test charter submits write to ftp dir

* chore: remove debug

* fix: test charter approval writes to ftp dir

* fix: link review revisions into ftp dir

* fix: link to all archive and ftp on submission post

* chore: clean comments, move action to github issue

* fix: link idindex files to all archive and ftp

* chore: deflake

* chore: remove TODO comment

* fix: use settings

* chore: rename new setting
2024-04-19 16:18:52 -05:00
Robert Sparks e6138ca126
feat: session apis (#7173)
* feat: Show bluesheets using Attended tables (#7094)

* feat: Show bluesheets using Attended tables (#6898)

* feat: Allow users to add themselves to session attendance (#6454)

* chore: Correct copyright year

* fix: Address review comments

* fix: Don't try to generate empty bluesheets

* refactor: Complete rewrite of bluesheet.html

* refactor: Fill in a few gaps, close a few holes

- Rename the live "bluesheet" to "attendance", add some explanatory text.
- Add attendance links in materials view and pre-finalized proceedings view.
- Don't allow users to add themselves after the corrections cutoff date.

* fix: Report file-save errors to caller

* fix: Address review comments

* fix: typo

* refactor: if instead of except; refactor gently

* refactor: Rearrange logic a little, add comment

* style: Black

* refactor: auto_now_add->default to allow override

* refactor: jsonschema to validate API payload

* feat: Handle new API data format

Not yet tested except that it falls back when the old
format is used.

* test: Split test into deprecated/new version

Have not yet touched the new version

* style: Black

* test: Test new add_session_attendees API

* fix: Fix bug uncovered by test

* refactor: Refactor affiliation lookup a bit

* fix: Order bluesheet by Attended.time

* refactor: Move helpers from views.py to utils.py

* test: Test that finalize calls generate_bluesheets

* test: test_bluesheet_data()

* fix: Clean up merge

* fix: Remove debug statement

* chore: comments

* refactor: Renumber migrations

---------

Co-authored-by: Paul Selkirk <paul@painless-security.com>

* chore: Remove unused import

* style: Black

* feat: Stub session update notify API

* feat: Add order & rev to slides JSON

* style: Black

* feat: Stub actual Meetecho slide deck mgmt API

* refactor: Limit reordering to type="slides"

* chore: Remove repository from meetecho API

(API changed on their end)

* feat: update Meetecho on slide reorder

* refactor: drop pytz from meetecho.py

* chore: Remove more repository refs

* refactor: Eliminate more pytz

* test: Test add_slide_deck api

* fix: Allow 202 status code / absent Content-Type

* test: Test delete_slide_deck api

* test: Test update_slide_decks api

* refactor: sessionpresentation_set -> presentations

* test: Test send_update()

* fix: Debug send_update()

* test: ajax_reorder_slides calls Meetecho API

* test: Test SldesManager.add()

* feat: Implement SlidesManager.add()

* test: Test that ajax_add_slides... calls API

* feat: Call Meetecho API when slides added to session

* test: Test SlidesManager.delete()

* feat: Implement SlidesManager.delete()

* test: ajax_remove_slides... calls Meetecho API

* feat: Call Meetecho API when slides removed

* chore: Update docstring

* feat: rudimentary debug mode for Meetecho API

* test: remove_sessionpresentation() calls Meetecho API

* feat: Call Meetecho API from remove_sessionpresentation()

* test: upload_slides() calls Meetecho API

* style: Black

* fix: Refactor/debug upload_session_slides

Avoids double-save of a SessionPresentation for the session
being updated and updates other sessions when apply_to_all
is set (previously it only created ones that did not exist,
so rev would never be updated).

* test: Fix test bug

* feat: Call Meetecho API when uploading session slides

* fix: Only replace slides actually linked to session

* fix: Delint

Removed some type checking rather than debugging it

* fix: Send get_versionless_href() as url for slides

* test: TZ-aware timestamps, please

* chore: Add comments

* feat: Call Meetecho API in edit_sessionpresentation

* feat: Call Meetecho API in remove_sessionpresentation

* feat: Call Meetecho API from add_sessionpresentation

* fix: Set order in add_sessionpresentation

* fix: Restrict API calls to "slides" docs

* feat: Call Meetecho API on title changes

* test: Check meetecho API calls in test_revise()

* fix: better Meetecho API "order" management

* fix: no PUT if there are no slides after DELETE

* feat: Catch exceptions from SlidesManager

Don't let errors in the MeetEcho slides API interfere with
the ability to modify slides for a session.

* feat: Limit which sessions we send notifications for

* fix: handle absence of request_timeout in api config

* test: always send slide notifications in tests

* fix: save slides before sending notification (#7172)

* fix: save slides before sending notification

* style: fix indentation

It's not a bug, it's a flourish!

---------

Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
Co-authored-by: Paul Selkirk <paul@painless-security.com>
2024-03-12 10:22:24 -05:00
Robert Sparks 7287e98709
feat: upload narrative minutes (#7125)
* feat: upload narrative minutes

* chore: cover other new URL path
2024-03-04 16:48:02 -06:00
Robert Sparks 8cb7f3dcae
feat: Import IESG artifacts into the datatracker (#6908)
* chore: remove unused setting

* feat: initial import of iesg minutes

* fix: let the meetings view show older iesg meetings

* feat: iesg narrative minutes

* feat: import bof coordination call minutes

* wip: import commands for iesg appeals and statements

* feat: import iesg statements.

* feat: import iesg artifacts

* feat: many fewer n+1 queries for the group meetings view

* fix: restore chain of elifs in views_doc

* fix: use self.stdout.write vs print in mgmt commands

* fix: use replace instead of astimezone when appropriate

* chore: refactor new migrations into one

* fix: transcode some old files into utf8

* fix: repair overzealous replace

* chore: black

* fix: address minro review comments

* fix: actually capture transcoding work

* fix: handle multiple iesg statements on the same day

* fix: better titles

* feat: pill badge replaced statements

* fix: consolodate source repos to one

* feat: liberal markdown for secretariat controlled content

* fix: handle (and clean) html narrative minutes

* feat: scrub harder

* fix: simplify and improve a scrubber

* chore: reorder migrations
2024-02-20 16:35:08 -06:00
Paul Selkirk 52a707628f
feat: Links to chatlogs and session recordings on proceedings page (#7042)
* feat: Links to chatlogs and session recordings on proceedings page (#6791)

* refactor: Add a url template and convenience function for session_recording url

* refactor: Avoid using the walrus operator
2024-02-20 09:40:31 -06:00
Robert Sparks 1e2a53e0e0
fix: restore explicit STATIC_ROOT (#6970) 2024-01-25 17:03:36 -06:00
Robert Sparks ca425790ea
fix: correct static url pattern 2024-01-24 13:06:58 -06:00
Jennifer Richards 8d12071bf5
refactor: Move cron jobs to celery tasks (#6926)
* refactor: Factor out helper from fetch_meeting_attendance.py

* feat: Define fetch_meeting_attendance_task task

Equivalent to the fetch_meeting_attendance management command

* chore: Disable fetch_meeting_attendance in bin/daily

* feat: Log errors in fetch_meeting_attendance_task

* feat: Enable a result backend for celery

Ignore results by default, but enable the backend so we
can manage tasks

* feat: Define daily task in ietf.utils.tasks

* refactor: Make bin/send-review-reminders into a task

* refactor: Make bin/send-scheduled-mail into a task

* chore: Update copyright years

* refactor: Make bin/rfc-editor-index-updates into a task

* refactor: Accept date type in rfc index update fn

* chore: Update comment

* fix: Annotate param as Optional

* fix: Revert treating skip_older_than_date as str

Misunderstood the comment, "fixed" a non-bug. Oops.

* feat: mgmt command to create periodic tasks

* feat: add summary of tasks to mgmt cmd

* style: black

* fix: Remove debug statements

* feat: Enable/disable tasks

* chore: Disable periodic tasks by default

* chore: Revert changes to daily and every15m

* fix: Call intended function

* chore: Add task descriptions
2024-01-22 12:04:16 -06:00
Robert Sparks b38e8c38ae
chore: shadowbox with mypy 2024-01-19 14:21:09 -06:00
Robert Sparks fc2673c13c
refactor: use statics.ietf.org for all statics 2024-01-18 16:51:09 -06:00
Robert Sparks 03e161bae9
chore: merge branch 'main' into feat/rfc 2023-12-05 09:12:11 -06:00
Jennifer Richards 62775bc56e
chore: Squelch celery deprecation warning (#6662) 2023-11-22 08:59:53 -06:00
Lars Eggert 3d44825333
ci: Switch to using geckodriver (#6541)
* Switch to using geckodriver

* Switch to selenium 4

* Undo

* Remove comment

* Fixes

* Restore non-standard line endings to minimize diff

* Undo

* Remove comment

* test: Fix test_upcoming_view_time_zone_selection

The inputs are hidden and managed by JS, so click
the visible elements instead.

* test: Clumsy fix to test_upcoming_materials_modal

Waiting for the button to be clickable does not
work because the modal is still fading in, so does
not actually close. Would be better to check for it
responding, but I haven't found the right way to do
that yet.

* test: Fix test_add_author_forms

Sending '\n' does not seem to work as it did before,
so click the option instead.

Also reverted some fixme hacks that seem to be obe.

* ci: Update base.Dockerfile

* test: add resource limits to dev/tests/debug.sh env

* ci: add upload of geckodriver.log on failure

* ci: run tests as user 1001

* ci: run app-create-dirs as sudo

* ci: set home folder to root to run tests

---------

Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
Co-authored-by: Nicolas Giard <github@ngpixel.com>
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
2023-11-21 15:30:50 -06:00
Robert Sparks 28c3e1bd5d
chore: merge branch 'main' into feat/rfc 2023-11-01 10:54:29 -05:00
Ryan Cross 4575864bfe
refactor: remove unused secr/areas app (#6509)
* refactor: remove unused secr/areas app
2023-10-23 09:55:54 -05:00
Robert Sparks 621528ad03
ci: Merge branch 'main' into feat/rfc 2023-09-15 11:48:44 -05:00
Paul Selkirk febdeff85f
feat: Capture that IPR disclosures are removed under the Objectively False IPR Disclosure Policy (#6231)
* feat: Capture that IPR disclosures are removed under the Objectively False IPR Disclosure Policy (#6088)

* chore: Move PUBLISH_IPR_STATES from settings_local to settings

* fix: Add migration for removed_objfalse in IprEventTypeName

* fix: De-conflict migration

* fix: De-conflict migration

* style: Move PUBLISH_IPR_STATES ahead of not-production block
2023-09-11 13:14:46 -05:00
Robert Sparks f9c9644263
chore: merge main and fix conflicts 2023-08-30 16:35:20 -05:00
Robert Sparks 383899c5c4
chore: checkpoint: docalias mostly removed 2023-08-15 16:49:17 -05:00
Jennifer Richards 82d91d371a
fix: Cast cache key to str before calling encode (#6100) 2023-08-08 17:38:59 -05:00
Robert Sparks baa5127d25
fix: refactor how settings handles cache definitions (#6099)
* fix: refactor how settings handles cache definitions

* chore: more english-speaker readable expression
2023-08-08 17:31:59 -05:00
Jennifer Richards 7e852aebfc
fix: Hash cache key for default memcached cache (#6089) 2023-08-08 13:21:19 -05:00
Paul Selkirk 06c9f06d55
feat: Reclassify nomcom feedback (#6002)
* fix: Clean up view_feedback_pending

- Remove "Unclassified" column header, which caused misalignment in the table body.

- Show the message author - previously displayed as `(None)`.

* feat: Reclassify nomcom feedback (#4669)

- There's a new `Chair/Advisor Tasks` menu item `Reclassify feedback`.

- I overloaded `view_feedback*` URLs with a `?reclassify` parameter.

- This adds a checkbox to each feedback message, and a `Reclassify` button
at the bottom of each feedback page.

- "Reclassifying" basically de-classifies the feedback, and punts it back
to the "Pending emails" view for reclassification.

- If a feedback has been applied to multiple nominees, declassifying it
from one nominee removes it from all.

* fix: Remove unused local variables

* fix: Fix some missing and mis-nested html

* test: Add tests for reclassifying feedback

* refactor: Substantial redesign of feedback reclassification

- Break out reclassify_feedback* as their own URLs and views,
  and revert changes to view_feedback*.html.

- Replace checkboxes with a Reclassify button on each message.

* fix: Remember to clear the feedback associations when reclassifying

* feat: Add an 'Overcome by events' feedback type

* refactor: When invoking reclassification from a view-feedback page, load the corresponding reclassify-feedback page

* fix: De-conflict migration with 0004_statements

Also change the coding style to match, and add a reverse migration.

* fix: Fix a test case to account for new feedback type

* fix: 842e730 broke the Back button

* refactor: Reclassify feedback directly instead of putting it back in the work queue

* fix: Adjust tests to new workflow

* refactor: Further refine reclassification to avoid redirects

* refactor: Impose a FeedbackTypeName ordering

Also add FeedbackTypeName.legend field, rather than synthesizing it every
time we classify or reclassify feedback.

In the reclassification forms, only show the relevant feedback types.

* refactor: Merge reclassify_feedback_* back into view_feedback_*

This means the "Reclassify" button is always present, but eliminates some
complexity.

* refactor: Add filter(used=True) on FeedbackTypeName querysets

* refactor: Add the new FeedbackTypeName to the reclassification success message

* fix: Secure reclassification against rogue nomcom members
2023-08-08 12:33:17 -05:00
Jennifer Richards 48d4072eeb
Merge branch 'feat/rfc' into main-merge
# Conflicts:
#	ietf/doc/factories.py
#	ietf/doc/tests_status_change.py
#	ietf/name/fixtures/names.json
#	ietf/templates/doc/document_info.html
2023-08-03 18:34:06 -03:00
Robert Sparks dbe1749438
feat: IAB statements (#5940)
* feat: support iab and iesg statements. Import iab statements. (#5895)

* feat: infrastructure for statements doctype

* chore: basic test framework

* feat: basic statement document view

* feat: show replaced statements

* chore: black

* fix: state help for statements

* fix: cleanout non-relevant email expansions

* feat: import iab statements, provide group statements tab

* fix: guard against running import twice

* feat: build redirect csv for iab statements

* fix: set document state on import

* feat: show published date on main doc view

* feat: handle pdf statements

* feat: create new and update statements

* chore: copyright block updates

* chore: remove flakes

* chore: black

* feat: add edit/new buttons for the secretariat

* fix: address PR #5895 review comments

* fix: pin pydantic until inflect catches up (#5901) (#5902)

* chore: re-un-pin pydantic
2023-07-23 11:00:24 -07:00
Robert Sparks d6186af2cc
Merge branch 'main' into feat/rfc 2023-07-20 09:35:58 -05:00
Lars Eggert d8c0ae5faf
fix: Add numeric localhost addresses to CSRF_TRUSTED_ORIGINS (#5989) 2023-07-19 09:55:22 -05:00
Jennifer Richards f2c2957925
refactor: Represent RFCs as their own DocType (#5835)
* feat: Add RFC DocTypeName, StateType, and States

* refactor: Rename rfc_number() to deprecated_rfc_number()

* feat: Add rfc_number field to DocInfo

* feat: Add DocRelationshipName "became-rfc"

* chore: First-pass migration to create rfc Documents

* chore: create_rfc_documents migration depends on new names

* refactor: Rename variable

* fix: Fix revname / downcase name for "became-rfc" DocRelationshipName

* chore: Remove debugging print statements

* feat: Point rfc aliases at rfc Documents

* test: Refactor RFC factories

* refactor: Rewrite is_rfc() in terms of type_id

* test: Use RfcFactory as base for IndividualRfcFactory

* refactor: Replace calls to deprecated_rfc_number()

* refactor: Remove deprecated_rfc_number() method

* test: Import WgRfcFactory
2023-06-20 15:28:16 -03:00
Jennifer Richards 6f361f9574
chore: Keep default CSRF_TRUSTED_ORIGINS when in debug mode (#5827) 2023-06-15 09:04:43 -05:00
Jennifer Richards 9cee31613b
chore: Set CSRF_TRUSTED_HOSTS to localhost for dev (#5771) 2023-06-06 15:58:06 -05:00
Jennifer Richards edd72826e6
fix: Prevent errors when memcached object size is exceeded (#5769)
* feat: Subclass PyMemcacheCache to tolerate too-large object errors

* fix: Use LenientMemcacheCache to prevent search errors

* fix: Re-raise errors other than "object too large"
2023-06-06 14:35:41 -05:00
Nicolas Giard cfaf966492
fix: add internal static.ietf.org var for pdfized render (#5765) 2023-06-05 20:18:35 -05:00
Jennifer Richards 2d661a1a5f
chore: Use permissive cross-origin-opener-policy setting 2023-06-01 17:12:15 -03:00
Jennifer Richards 1922a6066e
Merge pull request #5700 from ietf-tools/main
chore: main -> django4
2023-05-31 10:59:09 -03:00
Nicolas Giard 0a11892414
chore: use variable for static.ietf.org references (#5692) 2023-05-25 17:23:18 -05:00
Jennifer Richards e973672c25
Merge branch 'feat/django4' into dev/merge-main-5679 2023-05-23 12:11:17 -03:00
Jennifer Richards 2eaea55ce8
chore: Move log out suppression to settings,py 2023-05-19 15:25:35 -03:00
Jennifer Richards b6a791511f
chore: Use DjangoDivFormRenderer to opt in to new default 2023-05-19 12:52:26 -03:00
Jennifer Richards b06fc7acc4
chore: Suppress warning about CryptPasswordHasher 2023-05-19 11:28:58 -03:00
Jennifer Richards 25b9976460
chore: Switch to JSONSerializer
PickleSerializer is deprecated
2023-05-18 18:37:25 -03:00
Robert Sparks d0f8e1148a
chore: keep csp config in sync with production configuration (#5656) 2023-05-18 16:29:30 -05:00
Robert Sparks 34d714c4ec
chore: remove gather from CRSF_TRUSTED_ORIGINS (#5657) 2023-05-18 16:29:18 -05:00
Jennifer Richards c71f44fdb2
chore: Suppress deprecation warning for oidc_provider 2023-05-18 18:00:46 -03:00
Jennifer Richards 6df5d4c67f
chore: Suppress CICharField deprecation warning 2023-05-18 18:00:46 -03:00
Jennifer Richards 171a5bec73
chore: Update setuptools version and suppress warnings
pkg_resources warning is caused by a few packages
(django-simple-history. django-widget-tweaks, etc). The datetime_safe
warning is in tastypie, as indicated. The others are deprecated
settings we already have tickets for.
2023-05-18 18:00:45 -03:00
Jennifer Richards c26c9c71e4
chore: Switch to PyMemcacheCache backend 2023-05-18 15:18:01 -03:00
Jennifer Richards f85978fe24
chore: Disable L10N localization 2023-05-17 12:52:34 -03:00
Jennifer Richards 88452a2db1
chore: Add USE_DEPRECATED_PYTZ to settings.py 2023-05-16 16:20:51 -03:00
Jennifer Richards 76fa01b817
chore: Suppress deprecation warning for oidc_provider AppConfig 2023-05-16 13:21:12 -03:00