Commit graph

1099 commits

Author SHA1 Message Date
Lars Eggert 3d44825333
ci: Switch to using geckodriver (#6541)
* Switch to using geckodriver

* Switch to selenium 4

* Undo

* Remove comment

* Fixes

* Restore non-standard line endings to minimize diff

* Undo

* Remove comment

* test: Fix test_upcoming_view_time_zone_selection

The inputs are hidden and managed by JS, so click
the visible elements instead.

* test: Clumsy fix to test_upcoming_materials_modal

Waiting for the button to be clickable does not
work because the modal is still fading in, so does
not actually close. Would be better to check for it
responding, but I haven't found the right way to do
that yet.

* test: Fix test_add_author_forms

Sending '\n' does not seem to work as it did before,
so click the option instead.

Also reverted some fixme hacks that seem to be obe.

* ci: Update base.Dockerfile

* test: add resource limits to dev/tests/debug.sh env

* ci: add upload of geckodriver.log on failure

* ci: run tests as user 1001

* ci: run app-create-dirs as sudo

* ci: set home folder to root to run tests

---------

Co-authored-by: Jennifer Richards <jennifer@staff.ietf.org>
Co-authored-by: Nicolas Giard <github@ngpixel.com>
Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
2023-11-21 15:30:50 -06:00
Robert Sparks 925e22006b
ci: change test coverage fudge factor (#6649) 2023-11-16 12:46:36 -06:00
Jennifer Richards dc14308700
refactor: Drop submission validation via libmagic (#6500)
* refactor: Update parsers/base.py for Python3

* style: Black

* refactor: Remove mime type check from FileParser

* refactor: Validate that submission is UTF-8

The mime check distinguished us-ascii from UTF-8,
but as far as I can tell the code relying on it
treated both as equally valid.

* feat: Clear error when file is not valid XML

* chore: Remove unused import

* test: Update tests to match changes

* fix: Count bytes starting from 1

* test: Add tests of FileParser validations

* fix: Fix / simplify regexp

* test: Test error caused by bad XML submission

---------

Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
2023-10-23 10:00:04 -05:00
Lars Eggert 7dbfc0208a
feat: Remove document notes from the datatracker (#6387)
* Remove doc.note

* Address comments from @rjsparks

* Consistently display doc notes
2023-10-09 12:15:18 -05:00
Lars Eggert 8abbaae0c0
Update ietf/utils/test_runner.py 2023-09-28 11:34:03 +03:00
Lars Eggert abe93a42b2
feat: Add the ability to rerun a test suite until a failure occurs
More quickly than looping in the shell around a non-zero error code.
This only incurs all the setup and scaffolding overheads once.
2023-09-28 11:31:16 +03:00
Jennifer Richards 69fd3a059e
fix: Validate string before calling int() (#6268) 2023-09-13 15:29:16 -05:00
Robert Sparks f2dc32a310
chore: remove unused scipts from ietf/bin (#6259) 2023-09-01 14:31:23 -05:00
Robert Sparks c8c1ebdf12
chore: remove unnecessary .gitignore files (#6262) 2023-08-31 17:12:01 -05:00
Jennifer Richards 60dc60234d
fix: Better match xml2rfc date parsing (#5914)
* refactor: Split up get_creation_date to ease testing

* test: Add tests of parse_creation_date()

Note change in expected output when date_elt is None:
instead of returning None, this expects today's date.

* fix: Return today instead of None when date is absent

* fix: Handle empty string for day attribute

* test: Check a couple more parse_creation_date cases

* fix: Revert to returning None when date elt is absent

* style: black
2023-07-05 08:23:37 -05:00
Jennifer Richards 2bf7e8250c
fix: Import DEFAULT_TIMEOUT correctly (#5803) 2023-06-12 13:50:14 -05:00
Jennifer Richards 12f1ac10ad
fix: Fix signature for LenientMemcacheCache.set() (#5800) 2023-06-12 12:48:32 -05:00
Robert Sparks b75b560dea
feat: import IAB minutes from the IAB wordpress site (#5689)
* feat: import IAB minutes from the IAB wordpress site

Work in Progress

* fix: refactor as management commands

* fix: use datetime.timezone.utc for comparisons

* fix: coerce an argument. Command now runs

* chore: extend queryset in a better way

* fix: fetch kesara's repo and move files into place.

* fix: add a DocEvent explaining where the Documents came from

* fix: small optimizations and typo fixes

* fix: performance compromise for very old meetings

* fix: ensure variable is initialized

* fix: avoid operating on null by changing method signature - address review comments

* fix: repair html typo and address pyflake complaint

* chore: run black on a new and changed things
2023-06-09 17:07:31 -05:00
Jennifer Richards edd72826e6
fix: Prevent errors when memcached object size is exceeded (#5769)
* feat: Subclass PyMemcacheCache to tolerate too-large object errors

* fix: Use LenientMemcacheCache to prevent search errors

* fix: Re-raise errors other than "object too large"
2023-06-06 14:35:41 -05:00
Robert Sparks 4a062f5250
Merge pull request #5757 from ietf-tools/release
ci: merge release commit and patch back into main
2023-06-05 08:14:03 -05:00
Jennifer Richards d33a6f3c0c
fix: Handle missing date fields in XML submissions (#5744)
* refactor: Eliminate _construct_creation_date helper

* fix: Use xml2rfc method for filling in missing date fields

* fix: Set options.date for xml2rfc writers

* test: Test handling of missing date element/fields
2023-06-02 14:40:52 -05:00
Jennifer Richards c8f9739604
Merge pull request #5738 from ietf-tools/main
chore: Merge main into feat/django4
2023-06-01 17:32:27 -03:00
Jennifer Richards 5a2708283b
feat: Extract document creation date from XML draft (#5733)
* fix: Extract document creation date from XML draft

* test: Fix test
2023-06-01 09:58:55 -05:00
Jennifer Richards 8d4780d304
fix: Ignore failure to extract text draft title unless it is needed (#5730)
* fix: Accept a Path as source for a PlaintextDraft

* fix: Guard against failure to extract PlaintextDraft title

* fix: Ignore failure to extract text draft title unless it is needed
2023-06-01 09:39:59 -05:00
Jennifer Richards 58182fd7f6
test: Fix selectors in selenium tests 2023-05-19 17:54:24 -03:00
Jennifer Richards 19abdfe5e7
refactor: Inject tests without using deprecated extra_tests 2023-05-19 11:09:05 -03:00
Jennifer Richards 55fb502179
test: Iterate over template.nodelist in apply_template_test 2023-05-18 15:18:02 -03:00
Jennifer Richards cdec6cc0aa
Merge pull request #5655 from ietf-tools/main
chore: Merge main into feat/django4
2023-05-18 15:15:42 -03:00
Jennifer Richards cbabb864c2
test: Use issubclass, not isinstance, to ID view_class 2023-05-17 16:09:56 -03:00
Jennifer Richards d519bca12c
test: Fix ignore_pattern so Redirect/TemplateViews are ignored again 2023-05-17 15:59:28 -03:00
Eric Vyncke 2a27a2bffc
feat: Can we provide a better review assignment email subject. #3760 (#5415)
* Specific email subject + requester/secretary in cc

* Send the deadline in the subject

* Use unicode rather than ASCII for reviewer's name

* More log info in the test

* Fix the closing parenthesis

* Fix the email test  when review is assigned

* chore: address review comment

---------

Co-authored-by: Robert Sparks <rjsparks@nostrum.com>
2023-05-17 12:23:20 -05:00
Jennifer Richards 8cf609bfa9
refactor: Implement require_api_key with functools.wraps
The @decorator mechanism does not seem to work with @method_decorator
in Django 4.0, have not tracked down why.
2023-05-17 09:45:07 -03:00
Jennifer Richards c840d53cb2
test: Suppress mypy error on import of _lazy_re_compile() 2023-05-15 11:04:29 -03:00
Jennifer Richards 7ad74c99e4
refactor: import from django.urls instead of django.conf.urls 2023-05-12 20:29:11 -03:00
Jennifer Richards 075aed7e9a
refactor: Replace deprecated request.is_ajax() 2023-05-12 18:26:56 -03:00
Jennifer Richards aa4c04126c
chore: Use re_path() instead of url(), its deprecated synonym, 2023-05-12 17:09:32 -03:00
Jennifer Richards 264ff60cd2
Merge pull request #5601 from jennifer-richards/django4
chore: Upgrade to Django 3.0
2023-05-11 11:04:36 -04:00
Jennifer Richards 69eb6340fd
test: Do not misuse django.conf.settings for HTML validation params 2023-05-10 15:41:36 -03:00
Robert Sparks f8113cb862
fix: close open things (#5593)
* fix: close open things

* fix: clean up test created files

* fix: remove one close too many
2023-05-10 11:19:34 -05:00
Jennifer Richards da168395fc
refactor: Replace deprecated smart_text with smart_str 2023-05-09 15:23:33 -03:00
Jennifer Richards 07e26dd52e
refactor: Replace deprecated force_text with force_str 2023-05-09 15:21:50 -03:00
Robert Sparks 400a830661
chore: add bibxml-ids dir to container build. (#5590) 2023-05-08 15:16:31 -05:00
Robert Sparks 01b4a91bfe
Merge remote-tracking branch 'ietf-tools/main' into feat/postgres 2023-03-29 10:41:47 +09:00
Jennifer Richards 72954c24fc
chore: Add type annotations to satisfy linter (#5435) 2023-03-29 09:06:14 +09:00
Jennifer Richards 6b826fd985
refactor: Purge old migrations and update ci to build on a clean schema (#5431)
* refactor: Remove all existing migrations

* refactor: Create clean set of migrations

* chore: Skip check_statetype_slugs when DB is not yet populated

* fix: Do not cache active_groups_menu on module import

* fix: Do not patch timezone awareness out of oidc-provider

* refactor: Migrate to create postgres schema, only use pgloader for data

* ci: Use migration scripts from feat/pg-migrations branch

* Revert "ci: Use migration scripts from feat/pg-migrations branch"

This reverts commit c82f64c614241ccede4865a50d494725c8a47c15.

* ci: Run check before migrate

* fix: Remove redundant migration caused by merge error

* chore: Add casts/ALTER TABLEs to eliminate pgloader errors/warnings

* chore: Change schema name to match docker image assumptions

* chore: Clear out schema so we get a clean start in case of a retry
2023-03-28 18:06:07 +09:00
Jennifer Richards bc9c74ef50
fix: Render complete iCalendar event when show_location is False (#5394)
* test: More carefully test event syntax in ical file

* test: Test agenda.ics view with all meeting sessions

* fix: Render complete iCalendar event when show_location is False

* chore: Fix confusing comment
2023-03-26 14:13:58 +09:00
Lars Eggert c3bb142c98
Rebase (#5416) 2023-03-25 17:37:33 +09:00
Robert Sparks f010766654
Merge remote-tracking branch 'ietf-tools/main' into feat/postgres 2023-02-27 12:55:47 -06:00
Robert Sparks b9baa6270e
chore: remove a stray space that crashes black (#5185) 2023-02-23 08:50:24 -06:00
Robert Sparks c38ade6e1b
feat: treat django auth username as case insensitive throughout the datatracker (#5165)
* feat: insensitive username matching at django authentication

* feat: use iexact when using the User object manager

* fix: more places to ignore username case

* fix: remove unused management command

* fix: avoid get when probing for object existance

* fix: force lowercase new usernames in secr/rolodex

* fix: use explicit arguments when creating user
2023-02-21 10:01:03 -06:00
Lars Eggert 182158b5c0
fix: Also extract document names from XML seriesInfo attributes and XInclude URLs (#5037)
* fix: Also extract document names from XML seriesInfo attributes

The old code only looked in the anchor string for the document names of
references, which doesn't work if the anchor uses a mnemonic. This caused lots
of missed references for many documents.

* No need to import lxml anymore

* Add tests

* Handle xinclude to bibxml URLs

* Wrap line

* Apply suggestion from @rjsparks

* Undo erroneous additions

* Address suggestion from @rjsparks
2023-02-14 17:07:54 -06:00
Jennifer Richards a10464a250
chore: Fix reference to misspelled migration
The migration with the misspelled name already ran on production, so
we have to recreate the typo in the reference to it.
2023-02-14 12:58:45 -04:00
Robert Sparks 7a86a1c2cb
chore: tighten migration dependencies 2023-02-13 17:25:54 -06:00
Jennifer Richards 5abfcd4d71
chore: Update migration numberings/deps 2023-02-13 16:26:25 -04:00
Robert Sparks e1206fa91e
Merge remote-tracking branch 'upstream/main' into feat/postgres 2023-02-13 09:17:05 -06:00