Commit graph

1166 commits

Author SHA1 Message Date
Henrik Levkowetz 0df0a87a48 Added a workaround for the current libmagic which quite easily can mischaracterise text/plain documents as text/x-Algol68. Fixes issues #2941 and #2956.
- Legacy-Id: 17594
2020-04-07 18:33:24 +00:00
Henrik Levkowetz f10ddadc0e Merged in changes from trunk up to r17584.
- Legacy-Id: 17593
2020-04-07 16:02:52 +00:00
Henrik Levkowetz f86b87e2ed Merged in [17555] from rjsparks@nostrum.com:
Remove the not-quite-working customization widgets from /meeting/upcoming and /meeting/past. Simplify those views. Correct the list of sessions on those pages when one interim has more than one session. Fixes #2938. Partially addresses #2937.
 - Legacy-Id: 17556
Note: SVN reference [17555] has been migrated to Git commit e9756d8987
2020-03-28 15:38:10 +00:00
Robert Sparks e9756d8987 Remove the not-quite-working customization widgets from /meeting/upcoming and /meeting/past. Simplify those views. Correct the list of sessions on those pages when one interim has more than one session. Fixes #2938. Partially addresses #2937. Commit ready for merge.
- Legacy-Id: 17555
2020-03-27 22:47:38 +00:00
Henrik Levkowetz 827ab69705 Merged in [17521] from housley@vigilsec.com:
Improve performance of log.assertion() and log.unreachable()
 - Legacy-Id: 17535
Note: SVN reference [17521] has been migrated to Git commit 333abcab6d
2020-03-24 21:47:59 +00:00
Russ Housley 333abcab6d Improve performance of log.assertion() and log.unreachable()
- Legacy-Id: 17521
2020-03-23 22:02:58 +00:00
Ole Laursen e5943f814d Add support for displaying constraint hints when scheduling a task and
for displaying violated constraints in the new schedule editor, with
the old of a new field, ConstraintName.editor_label.

Add support for displaying room capacity violations.

Add support for selecting a session and displaying information about
it similar to the existing scheduling editor.

Add support for sorting unassigned sessions.

Clean up markup and styles a bit, and fix some bugs.

Expand HTML-based test and add JS test that exercises the
Javascript-based functionality.

Switch to using Chrome driver instead of PhantomJS since the HTML
engine in PhantomJS is apparently too old to support the constructs in
the new schema editor.

Add a workaround for LiveServerTestCase clashing with IetfTestRunner
fixture loading.
 - Legacy-Id: 17519
2020-03-23 17:55:36 +00:00
Henrik Levkowetz a0d1ffa7be Fixed a pyflakes issue.
- Legacy-Id: 17472
2020-03-20 18:59:16 +00:00
Henrik Levkowetz ea1d21a45c Added another check to the check_draft_event_revision_integrity management command, and refined it somewhat.
- Legacy-Id: 17469
2020-03-19 22:44:08 +00:00
Henrik Levkowetz d2243ba93d Added a utility function to convert objects to dictionaries (for comparison, for instance)
- Legacy-Id: 17468
2020-03-19 22:42:43 +00:00
Henrik Levkowetz 480301f020 Refactored and extended check_draft_event_revision_integrity a bit.
- Legacy-Id: 17445
2020-03-17 18:11:43 +00:00
Henrik Levkowetz 5211480895 Added code to show a call-in button on the agenda page if the session agenda-note contains and IETF webex URL.
- Legacy-Id: 17427
2020-03-13 14:01:54 +00:00
Henrik Levkowetz c10706286b Added a management command to check draft event revision numbers. To be extended for other checks.
- Legacy-Id: 17423
2020-03-11 22:02:30 +00:00
Henrik Levkowetz 50e8918a56 Added a progress bar for verbosity=1 of the community list index update command.
- Legacy-Id: 17414
2020-03-09 18:03:09 +00:00
Henrik Levkowetz 2669afe69b Tweaked the check_referential_integrity management command verbose output.
- Legacy-Id: 17397
2020-03-06 20:37:41 +00:00
Henrik Levkowetz 726fcbf27d Removed all __future__ imports.
- Legacy-Id: 17391
2020-03-05 23:53:42 +00:00
Henrik Levkowetz ab173cb2eb Removed further six usage: six.string_types, six.integer_types, six.moves.urllib instances.
- Legacy-Id: 17390
2020-03-05 20:46:58 +00:00
Henrik Levkowetz 1c808bf63b Removed further six usage.
- Legacy-Id: 17387
2020-03-05 15:54:32 +00:00
Henrik Levkowetz e9a37d8ac8 Removed six.text_type(), changed six.moves.urllib to plain urllib, and removed now unused six imports.
- Legacy-Id: 17385
2020-03-05 14:41:41 +00:00
Henrik Levkowetz ab944725d5 Py2/3 compatibility tweaks for pyflakes.
- Legacy-Id: 17375
2020-03-02 18:00:41 +00:00
Henrik Levkowetz b6d8bade44 Fixed a Py2/3 issue in the pyflakes management command, and tweaked the verbose output format.
- Legacy-Id: 17372
2020-03-02 12:31:02 +00:00
Henrik Levkowetz fc03063e9e Made email Message-ID fields generated by the datatracker more uniform.
- Legacy-Id: 17360
2020-02-27 20:23:20 +00:00
Henrik Levkowetz fb740ff6bf Added saving of outgoing emails as Message instances, with accompanying test.
- Legacy-Id: 17348
2020-02-26 18:04:44 +00:00
Henrik Levkowetz 2d724427cf Updated patch.py to avoid deprecation and pyflakes warnings.
- Legacy-Id: 17346
2020-02-26 17:22:12 +00:00
Henrik Levkowetz 03bd9c0594 Fixed log.log() to hand on the right type (str) to underlying functions.
- Legacy-Id: 17343
2020-02-26 17:16:35 +00:00
Henrik Levkowetz 8f2280130b Moved patch.py into project tree -- it's not being maintained any more.
- Legacy-Id: 17330
2020-02-23 16:20:43 +00:00
Henrik Levkowetz d2ad86c5df Python 2/3 changes pulled from the production server after the switch to python 3.
- Legacy-Id: 17327
2020-02-23 01:34:52 +00:00
Henrik Levkowetz fa9427769a Added cleaning of the session request form's 'comments' field, to convert any html entered to text. Related to [17322].
- Legacy-Id: 17324
Note: SVN reference [17322] has been migrated to Git commit eb88abc394
2020-02-21 21:36:18 +00:00
Henrik Levkowetz aabd237656 Fixed some pyflakes issues.
- Legacy-Id: 17304
2020-02-18 19:39:03 +00:00
Henrik Levkowetz 91d5d72a9d Tweaked send_mail_mime() to recognize settings.SERVER_MODE='repair', in order to avoid mail floods when running the 'mergedata' management command. It is assumed that the appropriate mails went out when the data was entered into the system the first time.
- Legacy-Id: 17301
2020-02-16 21:42:37 +00:00
Henrik Levkowetz 7b5bebc727 Added a management command to dump data based on table names and primary keys derived from a list of SQL 'INSERT INTO' commands representing the items to be dumped. The output formats are the same as for dumpdata, and are suitable for later use with the loaddata or mergedata management commands.
- Legacy-Id: 17300
2020-02-16 21:40:01 +00:00
Henrik Levkowetz e0c6f3abe2 Added a management command to merge data from one of two divergent django databases to the other, taking care to insert or update records as appropriate, and update foreign keys and many-to-many keys appropriately. Accepts the dump formats generated by the dumpdata command.
- Legacy-Id: 17299
2020-02-16 21:39:08 +00:00
Henrik Levkowetz 58d8c2fb47 Updated the check_referential_integrity command, adding a --delete command to remove dangling references to removed records, and also adding colorized success/fail indications for each FK and m2m key inspected.
- Legacy-Id: 17298
2020-02-16 21:05:12 +00:00
Henrik Levkowetz 8f0eca5032 Added a newline to test run output, to separate dot progress indication from teardown message.
- Legacy-Id: 17253
2020-01-22 23:17:44 +00:00
Henrik Levkowetz 288f4135e3 Added (currently inactive) code to trigger exceptions on failure to resolve template variables during tests, and added fixes for a couple of places that triggered such exceptions.
- Legacy-Id: 17226
2020-01-13 17:16:43 +00:00
Peter E. Yee bb7e504d14 12/13 merged into 6.113.1.dev0 with migration ordering failure
- Legacy-Id: 17149
2019-12-13 19:53:45 +00:00
Henrik Levkowetz fcb6806d17 Merged in work from sasha@dashcare.nl on Review Queue Managemnt:
This abstracts queue management, making it possible to implement different
policies for each team. It provides two concrete policies:
RotateAlphabeticallyReviewerQueuePolicy, which rotates an alphabetically
ordered reviewer list with consideration for skip indications, and is the
default policy; and LeastRecentlyUsedReviewerQueuePolicy, a simple
least-recently-used policy.  Also see issues #2721 and #2656.
 - Legacy-Id: 17121
2019-12-04 23:02:52 +00:00
Henrik Levkowetz ac6b664fa5 Added normalization of draft title extracted from submitted XML.
- Legacy-Id: 17119
2019-12-02 16:24:51 +00:00
Henrik Levkowetz 2c28cb3eb0 When group secretary email addresses occurred in some mailtrigger To: entries, they were given with both name and address parts, while when occurring in Cc: entries, no names were present. Fixed one case of this, and adjusted a test case for consistency. Also added a more general utility function to exclude selected email addresses from email address lists.
- Legacy-Id: 17107
2019-11-27 21:49:22 +00:00
Henrik Levkowetz 6333e7d842 Added an instance of force_text() to ensure non_ascii mail subjects are unicode when used for string interpolation, rather than bytes (which will trigger ascii decode()).
- Legacy-Id: 17055
2019-11-19 09:37:55 +00:00
Henrik Levkowetz c233f07b5d Added a management command to generate draft bibxml files, and also a trial version of datatracker draft bibxml pages.
- Legacy-Id: 16962
2019-11-05 18:10:29 +00:00
Henrik Levkowetz d5b6292060 Fixed a pyflakes issue.
- Legacy-Id: 16955
2019-11-04 18:11:59 +00:00
Henrik Levkowetz feaa5038a0 Rewrote the ietf/bin/fix-ambiguous-timestamps command as a management command (fix_ambiguous_timestamps) and made it look at all instances of DateTimeField by means of model introspection.
- Legacy-Id: 16954
2019-11-04 12:56:52 +00:00
Henrik Levkowetz 3c82dc6184 Merged in [16922] from sasha@dashcare.nl:
Fix #2584 - Add additional content validation for uploaded texts.
Permitted MIME types are now text/plain, text/markdown and text/x-rst.
This applies to all usages of get_cleaned_text_file_content(),
including reviews, but also other similar places where text can either
be written either into a textarea or uploaded.
 - Legacy-Id: 16930
Note: SVN reference [16922] has been migrated to Git commit fd53f98854
2019-10-26 15:42:58 +00:00
Sasha Romijn fd53f98854 Fix #2584 - Add additional content validation for uploaded texts.
Permitted MIME types are now text/plain, text/markdown and text/x-rst.
This applies to all usages of get_cleaned_text_file_content(),
including reviews, but also other similar places where text can either
be written either into a textarea or uploaded.

Commit ready for merge.
 - Legacy-Id: 16922
2019-10-23 20:15:30 +00:00
Henrik Levkowetz 45465f1174 Merged in [16818] from rjsparks@nostrum.com:
Make hybrid pdfs when converting ppt materials to pdf. Fixes #2769.
 - Legacy-Id: 16880
Note: SVN reference [16818] has been migrated to Git commit 84270a71d0e7373c5e05ddc834e91c445a1296ed
2019-10-21 13:17:33 +00:00
Henrik Levkowetz 447525de73 Tweaked the output of mypy_test().
- Legacy-Id: 16865
2019-10-15 16:48:23 +00:00
Henrik Levkowetz 24ede9a1ae In wordwrap(), consider lines consisting entirely of some non-alphanumeric characters like ---- or === to be block (paragraph) separators. Fixes issue #2806.
- Legacy-Id: 16790
2019-10-01 11:08:41 +00:00
Henrik Levkowetz 902f7769d1 Fixed incorrect type hints for Migration.dependencies.
- Legacy-Id: 16776
2019-09-30 16:25:58 +00:00
Henrik Levkowetz 33e8733b91 Fixed up mypy issues or added type:ignore comments as needed for a clean mypy run.
- Legacy-Id: 16772
2019-09-30 15:42:18 +00:00
Henrik Levkowetz 2d858f2e0a Added mypy and django-stubs to Py3 requirements, and added a mypy test case.
- Legacy-Id: 16768
2019-09-30 15:37:32 +00:00
Henrik Levkowetz 8e313e4ed5 Adjusted migrations for Python2/3 compatibility.
- Legacy-Id: 16633
2019-08-08 17:11:14 +00:00
Henrik Levkowetz 2418459afc Prevent login case issues by generating apache permissions file entries for both uppercase and lowercase logins.
- Legacy-Id: 16597
2019-07-29 19:20:38 +00:00
Henrik Levkowetz 0a591e4b2e Merged in [16562] from rcross@amsl.com:
Added encoding='utf-8' parameter to io.open operations in tests that starting failing since adding unicode_literals.
 - Legacy-Id: 16588
Note: SVN reference [16562] has been migrated to Git commit d332bfd91d
2019-07-24 23:44:36 +00:00
Henrik Levkowetz 07c5e33bc4 Added conditional decoding of bytes subject text, before interpolating into a new unicode string.
- Legacy-Id: 16577
2019-07-22 18:19:55 +00:00
Ryan Cross d332bfd91d Added encoding='utf-8' parameter to io.open operations in tests that starting failing since adding unicode_literals. Commit ready to merge
- Legacy-Id: 16562
2019-07-20 22:54:14 +00:00
Henrik Levkowetz b3244242aa Fixed some issues with the Py2/3 changes to the populate_yang_model_dirs management command.
- Legacy-Id: 16520
2019-07-18 18:53:14 +00:00
Henrik Levkowetz fb3d4ddb0b Added information about versions (Datatracker, Python, and Django) at the beginning of test suite runs.
- Legacy-Id: 16516
2019-07-17 20:54:51 +00:00
Henrik Levkowetz 618d6021b0 Read pdf files as binary.
- Legacy-Id: 16510
2019-07-17 16:05:19 +00:00
Henrik Levkowetz db47cb6b0f Replaced the error on bad Submission.submitter email address with syslog logging.
- Legacy-Id: 16509
2019-07-17 14:30:57 +00:00
Henrik Levkowetz 627f8a7eab Modified code to write release coverage json data to gzip file under both Py2 and Py3.
- Legacy-Id: 16475
2019-07-16 21:58:38 +00:00
Henrik Levkowetz ad1821f63a Enabled the wiki management command under py2
- Legacy-Id: 16471
2019-07-16 19:12:00 +00:00
Henrik Levkowetz e03784132d Merged changes from current trunk to Py3 branch.
- Legacy-Id: 16468
2019-07-16 15:36:16 +00:00
Henrik Levkowetz bdc73e771a Python2/3 compatibility: replaced six.ensure_text() with either six.text_type or django's force_text(), depending on the case, and fixed a variable scope issue.
- Legacy-Id: 16461
2019-07-16 13:20:05 +00:00
Henrik Levkowetz 46aec6eb85 Python2/3 compatibility: tentative version of create_group_wikis; may need more work.
- Legacy-Id: 16459
2019-07-16 13:17:12 +00:00
Henrik Levkowetz 8c6eb3a30a Python2/3 compatibility: Changed the use of open() and StringIO to io.open() etc.
- Legacy-Id: 16458
2019-07-15 19:14:04 +00:00
Henrik Levkowetz 44f51c1d64 The beginning of a dev file download helper.
- Legacy-Id: 16456
2019-07-15 19:03:35 +00:00
Henrik Levkowetz 831fb18f8a Python2/3 compatibility: Added force_str or force_bytes in some places, to ensure the argument right type
- Legacy-Id: 16450
2019-07-15 15:49:35 +00:00
Henrik Levkowetz eb42394534 Py2/3 compatibility: Added __future__ import.
- Legacy-Id: 16449
2019-07-15 15:47:31 +00:00
Henrik Levkowetz 855716e1d5 Py2/3 compatibility: Added __future__ import
- Legacy-Id: 16448
2019-07-15 15:46:06 +00:00
Henrik Levkowetz 2d7c9629aa Added 'from __future__' imports all over the place, to bring code behaviour into closer alignment between python2 and python3
- Legacy-Id: 16446
2019-07-15 15:40:51 +00:00
Henrik Levkowetz 6ace81b25d Corrected the order of string prefix 'r' and 'b' under python2.
- Legacy-Id: 16445
2019-07-15 15:37:48 +00:00
Henrik Levkowetz 58eca93f9a Added validation of email addresses extracted by get_email_addresses_from_text().
- Legacy-Id: 16439
2019-07-09 13:02:49 +00:00
Henrik Levkowetz 6732e7acff Fixed an issue with a gzip file mode.
- Legacy-Id: 16436
2019-07-08 18:30:34 +00:00
Henrik Levkowetz f481f5c3e6 Replaced use of six with the equivalent pure python3 constructs.
- Legacy-Id: 16428
2019-07-08 10:43:47 +00:00
Henrik Levkowetz 671b4035ea Fixed a syntax issue and removed debug calls.
- Legacy-Id: 16421
2019-07-07 13:04:54 +00:00
Henrik Levkowetz 472a3ed039 Refined our parseaddr() to deal correctly with both bytes and str
- Legacy-Id: 16419
2019-07-07 13:03:11 +00:00
Henrik Levkowetz 40ab3c949f Added missing code to skip coverage measurement for skippable tests.
- Legacy-Id: 16406
2019-07-06 17:27:49 +00:00
Henrik Levkowetz 0679eaa8d4 Removed unused imports.
- Legacy-Id: 16402
2019-07-04 21:06:57 +00:00
Henrik Levkowetz f480799af9 Undid unintentional bulk commit
- Legacy-Id: 16401
2019-07-04 21:04:46 +00:00
Henrik Levkowetz fc09a59950 Added decode() of command pipe output.
- Legacy-Id: 16400
2019-07-04 21:01:39 +00:00
Henrik Levkowetz 3a35dfcee4 Changed our parseaddr() and formataddr() to work correctly with python3.
- Legacy-Id: 16399
2019-07-04 21:00:48 +00:00
Henrik Levkowetz a3d40ace9b Fixed a str/bytes issue with a hashlib function call, and removed an unused import.
- Legacy-Id: 16398
2019-07-04 20:59:40 +00:00
Henrik Levkowetz f33ecaa1a4 Fixed a test that compared an int with a meeting number string. Changed meeting test data to use meeting number 72 instead of 42, because proceedings code gives 404 for meetings before 65. Changed a number of StringIO() to BytesIO().
- Legacy-Id: 16378
2019-07-04 16:01:32 +00:00
Henrik Levkowetz 0589d0b313 Changed a bunch of regexes to use r strings; also miscellaneous smaller fixes.
- Legacy-Id: 16376
2019-07-04 15:51:05 +00:00
Henrik Levkowetz fb8d12ac3c Fixed a pyflakes issue.
- Legacy-Id: 16358
2019-07-01 19:27:08 +00:00
Henrik Levkowetz 8c66e95a57 Test changes to match production group type changes for some special groups.
- Legacy-Id: 16357
2019-07-01 18:30:59 +00:00
Henrik Levkowetz c8e75c554a Various changes to make yang-related management commands quieter
- Legacy-Id: 16355
2019-07-01 18:29:03 +00:00
Henrik Levkowetz f5e9583f59 Fixed str/bytes issues with hashlib function arguments.
- Legacy-Id: 16354
2019-07-01 14:09:53 +00:00
Henrik Levkowetz 97a555936d Changed a large number of test assertions to use assertContains() and removed separate, now unnecessary,status_code checks. Also changed many assertTrue(... in ...) to use assertIn().
- Legacy-Id: 16346
2019-06-30 21:03:34 +00:00
Henrik Levkowetz 20c44a9e1a Adapted the pipe() code to python3; API changes and str/byte handling.
- Legacy-Id: 16345
2019-06-30 21:01:44 +00:00
Henrik Levkowetz 36cac48063 Fixed a bunch of str/bytes issues.
- Legacy-Id: 16341
2019-06-30 20:54:09 +00:00
Henrik Levkowetz 70c00950b6 Fixed several email handling str/bytes issues.
- Legacy-Id: 16334
2019-06-29 13:30:36 +00:00
Henrik Levkowetz ea109f2b1b Fixed API changes in SMTPChannel
- Legacy-Id: 16329
2019-06-28 20:52:15 +00:00
Henrik Levkowetz 1e86ccb4ee Builtin function file() --> open()
- Legacy-Id: 16328
2019-06-28 20:51:13 +00:00
Henrik Levkowetz 318bd0d5ea Changed regex strings to r strings.
- Legacy-Id: 16320
2019-06-28 13:32:50 +00:00
Henrik Levkowetz e39ac52071 Removed 2to3-generated list() around .items() iterator in for loops.
- Legacy-Id: 16315
2019-06-27 18:11:17 +00:00
Henrik Levkowetz 06daf6ac65 Changed regex strings to raw strings.
- Legacy-Id: 16312
2019-06-27 18:08:02 +00:00
Henrik Levkowetz 3ec7e864be Converted leading tabs to spaces in ietf/**/*.py
- Legacy-Id: 16310
2019-06-27 14:51:02 +00:00
Henrik Levkowetz d7f5c84182 Initial 2to3 patch with added copyright statement updates.
- Legacy-Id: 16309
2019-06-27 14:40:54 +00:00
Henrik Levkowetz 7270d5f67a Added the option to order API resource lists by primary key, for stable ordering when paging through a list.
- Legacy-Id: 16284
2019-06-18 12:40:19 +00:00
Henrik Levkowetz ba528a0c5e Adjusted the test_runner to generate migration coverage data also when permitting mixed migrations.
- Legacy-Id: 16280
2019-06-17 18:24:17 +00:00
Henrik Levkowetz 0e68f4bb4b Added a management command to update the index table used by some community list search rules.
- Legacy-Id: 16273
2019-06-17 13:02:57 +00:00
Henrik Levkowetz 426870b766 Merged in the Document and DocAlias primary key change and m2m work in ^/personal/henrik/6.96.1-docalias.
- Legacy-Id: 16261
2019-06-15 12:00:48 +00:00
Henrik Levkowetz a2cafded25 Changed DocAlias to be many-to-many, in order to be able to handle STD aliases referring to multiple RFCs.
- Legacy-Id: 16249
2019-06-14 20:38:14 +00:00
Henrik Levkowetz 815602351f This is a series of 50 migrations that changes the Document and DocAlias
primary keys from character strings to integers, and makes corresponding code
changes.

This was prompted by database limitations discovered when trying to make
DocAlias use a m2m document field; with 255 long strings as primary keys for
Document and DocAlias this violated the MySQL database limitations.

Changing the primary keys to integers should also improve efficiency.  

Due to the data migrations which create the new integer primary keys and adds
corresponding integer foreign keys matching the previous string foreign keys
in all tables having foreign keys to Document and DocAlias, some of these
migrations take a long time.  The total set of migrations are expected to have
a runtime on the order of 2 hours.
 - Legacy-Id: 16237
2019-06-10 11:32:46 +00:00
Henrik Levkowetz ca1f3340c0 Changed a ReviewAssignmentDocEvent type from 'closed_review_request' to 'closed_review_assignment', for consistency and readability.
- Legacy-Id: 16181
2019-05-02 12:38:41 +00:00
Henrik Levkowetz 7b93709a3b Tweaked the greate_group_wikis management command to accept trailing fileglob wildcards on adhoc wiki group acronyms. Added a generic NomCom wiki to the list of adhoc wikis to be created and maintained, with admins from active nomcom* groups.
- Legacy-Id: 16176
2019-04-29 12:30:36 +00:00
Henrik Levkowetz 682ddee8e4 Reverted earlier application of on_behalf_of() at mail sending call points, and instead did all on_behalf_of processing in condition_message(). Added insertion of Reply-To header fields in this case. Changed all use of the mail-sending 'extra' parameters to use value lists. Updated a bunch of tests accordingly.
- Legacy-Id: 16020
2019-03-11 17:01:33 +00:00
Henrik Levkowetz fc18cd93d4 Tweaked coverage_changes command. I suspect this would benefit from a rewrite.
- Legacy-Id: 16019
2019-03-11 16:52:09 +00:00
Henrik Levkowetz d698eb98df Changed the on-behalf-of name template from 'On behalf of ...' to '... via Datatracker' to make name come first.
- Legacy-Id: 16009
2019-03-07 14:42:11 +00:00
Henrik Levkowetz d57088b1f3 Modified one call and one exception point related to on_behalf_of() use for email sending.
- Legacy-Id: 16008
2019-03-06 19:13:15 +00:00
Henrik Levkowetz 8a669909bc Changed ietf/utils/mail.py to use unicode strings, and changed a call in ietf/reviv/utils.py to use unicode for mail Subject, to fix a server 500 issue.
- Legacy-Id: 16007
2019-03-06 16:57:00 +00:00
Henrik Levkowetz 5c6ab60f85 Changed a log() call to use unicode string interpolation to handle unicode email address names.
- Legacy-Id: 15999
2019-03-05 17:29:43 +00:00
Henrik Levkowetz ae8c1f2b91 Updated test_data email addresses.
- Legacy-Id: 15991
2019-03-04 20:13:02 +00:00
Henrik Levkowetz 849a3dcc97 Added another exception class to a catch instance in a function, triggered by a new usage case.
- Legacy-Id: 15990
2019-03-04 20:12:30 +00:00
Henrik Levkowetz 2b811b3d3d Updated the tests to always use @ietf.org addresses where appropriate, and not do so when not appropriate. All test user login addresses have been changed away from @ietf.org.
- Legacy-Id: 15988
2019-03-04 20:08:48 +00:00
Henrik Levkowetz 46bb4539c5 Added two new functions to replace email From: of users with On-behalf-of addresses, with associated values in settings. Added a catchall stanza in the mail sending pipeline to catch anything not already changed to On-behalf-of addresses.
- Legacy-Id: 15987
2019-03-04 20:06:44 +00:00
Henrik Levkowetz fd62a25f4e Added a missing variable initialization.
- Legacy-Id: 15925
2019-02-04 11:26:29 +00:00
Henrik Levkowetz ef97f2a821 Merged in GroupFeatures refactoring and expansion from ^/personal/henrik/6.89.4.dev0@15911.
- Legacy-Id: 15912
2019-01-24 15:25:01 +00:00
Henrik Levkowetz 57a4c9f41f Added 9 new group features, and changed list-like char fields to json fields, to get better support for using the values as lists. Modified code to use the group features instead of explicit lists of group types in many places in the code.
- Legacy-Id: 15908
2019-01-22 18:11:46 +00:00
Henrik Levkowetz d19228110c Applied a patch from dkg@fifthhorseman.net: py3 compatibility: fix another instance of integer division
- Legacy-Id: 15896
2019-01-15 17:50:33 +00:00
Henrik Levkowetz 98a74bd7f3 Moved __future__ imports down so as not to obscure the module docstring. Fixes inability to run '$ ietf/utils/draft.py -h'.
- Legacy-Id: 15894
2019-01-14 22:28:52 +00:00
Henrik Levkowetz 910d3d7723 Applied a patch from dkg@fifthhorseman.net: py3 compatibility: Use a list of dictionary keys
In python3, dict.keys() produces a dict_keys object, not a list.
    Since this code treats it as a list, we'll just be explicit about
    that.
 - Legacy-Id: 15893
2019-01-14 21:06:08 +00:00
Henrik Levkowetz c8f98e125c Applied a patch from dkg@fifthhorseman.net: Fix regex manipulation for word characters.
in python 3.7, re.sub() started treating unknown escape sequences in
     as errors.  Fix this by sending an escaped \ where we mean to
    pass it through raw.
    
    https://docs.python.org/3/library/re.html#re.sub
 - Legacy-Id: 15892
2019-01-14 21:03:28 +00:00
Henrik Levkowetz e39358312b Applied a patch from dkg@fifthhorseman.net: py3 compatibility: Use // for explicit integer division
Without this fix, in modern versions of python, the changed line
    produces:
    
    TypeError: 'float' object cannot be interpreted as an integer
 - Legacy-Id: 15891
2019-01-14 21:02:01 +00:00
Henrik Levkowetz e718272e71 Applied a patch from dkg@fifthhorseman.net: py3 compatibility: Use modern form of exception handling
- Legacy-Id: 15890
2019-01-14 21:00:50 +00:00
Henrik Levkowetz 8840efaef4 Applied a patch from dkg@fifthhorseman.net: py3 compatibility: use print function.
- Legacy-Id: 15889
2019-01-14 20:56:59 +00:00
Henrik Levkowetz 00544eb534 Merged in [15846] from rjsparks@nostrum.com:
Simple tests of some log utilities.
 - Legacy-Id: 15875
Note: SVN reference [15846] has been migrated to Git commit 1e1d92aa0d
2019-01-07 22:09:19 +00:00
Henrik Levkowetz 8e60e19d01 Merged in [15845] from rjsparks@nostrum.com:
simple test of BowerStorageFinder.
 - Legacy-Id: 15874
Note: SVN reference [15845] has been migrated to Git commit a4f334195e
2019-01-07 21:34:52 +00:00
Henrik Levkowetz 0cc7f73d58 Additional tweaks to submission yang testing and the test runner
- Legacy-Id: 15870
2019-01-07 13:38:44 +00:00
Robert Sparks 1e1d92aa0d Simple tests of some log utilities. Commit ready for merge.
- Legacy-Id: 15846
2018-12-20 20:33:02 +00:00
Robert Sparks a4f334195e simple test of BowerStorageFinder. Commit ready for merge.
- Legacy-Id: 15845
2018-12-20 19:13:35 +00:00
Henrik Levkowetz 2d0c503d33 Fixed pyflakes issues
- Legacy-Id: 15792
2018-11-30 00:07:49 +00:00
Henrik Levkowetz 0e8f63951e Added two new management commands to make it easier to load back selected objects that have been removed by mistake (providing they are available in a full database dump or backup that can be loaded and worked with): dumprelated and loadrelated.
- Legacy-Id: 15790
2018-11-29 15:17:46 +00:00
Henrik Levkowetz 2397f97da3 Updated test_unicode() to work with the latest version of the unidecode lib.
- Legacy-Id: 15780
2018-11-20 16:08:51 +00:00
Henrik Levkowetz 1d604b7cd9 Another name test tweak
- Legacy-Id: 15773
2018-11-13 21:50:28 +00:00
Henrik Levkowetz fad6311a13 Fixed a test problem.
- Legacy-Id: 15772
2018-11-13 21:12:25 +00:00
Henrik Levkowetz d6caf615e0 Added testing of test_unidecode() and another name_parts() test case, and removed some debug.statements.
- Legacy-Id: 15771
2018-11-13 19:31:18 +00:00
Henrik Levkowetz e471386cf2 Merged in [15715] from rcross@amsl.com:
Fix issue where new non-session location field value isn't saved.  Fixes #2625.
 - Legacy-Id: 15756
Note: SVN reference [15715] has been migrated to Git commit a47fba833f
2018-11-12 13:35:17 +00:00
Henrik Levkowetz 0735021d2b Added a test for various name part combinations, and fixed a couple of issues with utils.name.name_parts().
- Legacy-Id: 15753
2018-11-11 15:12:20 +00:00
Henrik Levkowetz e024cca8c4 Merged in [15708] and [15730] from housley@vigilsec.com:
Added a mail alias for all review team secretaries. Fixes #2139.  Use
the alias if there is one for email about review request changes.  Added
a test for the review team secretary alias functionality.
 - Legacy-Id: 15747
Note: SVN reference [15708] has been migrated to Git commit 6067dcbfef

Note: SVN reference [15730] has been migrated to Git commit d525ade0b0
2018-11-10 20:32:42 +00:00
Ryan Cross a47fba833f Fix issue where new non-session location field value isn't saved. Fixes #2625. Commit ready for merge.
- Legacy-Id: 15715
2018-11-05 04:25:20 +00:00
Henrik Levkowetz ef3cbf4684 Merged in [15595] from rjsparks@nostrum.com:
Converted stats, submit, sync. and utils/test to use factories. Excluded utils/test_data from coverage checks.
 - Legacy-Id: 15599
Note: SVN reference [15595] has been migrated to Git commit 39e76a836996ec9ed8ea449c92bdfae91c3dbaae
2018-10-19 12:54:58 +00:00
Henrik Levkowetz d0b73e9b67 Tweaked the mail sending routines' handling of the copy flag.
- Legacy-Id: 15586
2018-10-16 15:05:17 +00:00
Henrik Levkowetz e20e8b7df5 Changed the mail sending routines so it's possible to not copy the debug archive on some messages.
- Legacy-Id: 15585
2018-10-16 14:32:10 +00:00
Henrik Levkowetz 0e52b30e5b Added the message-id to log lines for sent messages
- Legacy-Id: 15582
2018-10-15 18:46:45 +00:00
Henrik Levkowetz caeb6fcb87 Added a '--reminder' switch to the GDPR request mailer, for repeat notifications.
- Legacy-Id: 15530
2018-10-07 22:05:26 +00:00
Henrik Levkowetz 43dd7d5e90 Tweaked the management command to send GDPR consent request emails to correctly perform exclusion based on the time of previous consent request emails.
- Legacy-Id: 15504
2018-09-29 20:08:09 +00:00
Henrik Levkowetz 1dd47e18f5 Whitespace changes only.
- Legacy-Id: 15499
2018-09-29 12:46:12 +00:00
Henrik Levkowetz 7ea8a3f431 Tweaked the gdpr reminder command to avoid bounces to RT and copies to gmail.
- Legacy-Id: 15480
2018-09-24 18:37:45 +00:00
Henrik Levkowetz 30f456f83a Changed the default minimum response time used by the delete_data_lacking_consent management command from 14 to 7 days.
- Legacy-Id: 15466
2018-09-18 13:35:27 +00:00
Henrik Levkowetz 4fba531e9e Merged in the latest GDPR changes. This refines the handling of the consent checkbox on the account page; refines the Consent Needed warning given on login if consent is needed; tweaks several models to set the on_deletion fields for FK to User and Person appropriately; adds a Person.needs_consent() method to capture the logic of which fields require consent; refines the Person.plain_name() method and the user.log.log() function; and adds 2 management commands to send out consent requests and delete non-consent information, respectively.
- Legacy-Id: 15464
2018-09-16 23:12:43 +00:00
Henrik Levkowetz a389e24fb6 Refined utils.log.log() to flush stdout before logging if we're writing log messages to stderr.
- Legacy-Id: 15463
2018-09-16 14:14:26 +00:00
Henrik Levkowetz b85e1c46d9 Added a delete_data_lacking_consent management command that deletes person records and person information for which we need consent according to GDPR, but have not received it.
- Legacy-Id: 15461
2018-09-16 13:54:32 +00:00
Henrik Levkowetz 3f9dbeada8 Expanded the send_gdpr_consent_request command to add support for deletion dates, email reminder interval, email sending rate limiting, and sending to a list of users.
- Legacy-Id: 15460
2018-09-16 13:52:29 +00:00
Henrik Levkowetz b2bfc7858a Generalized the log.assertion() function a bit to be able to provide more information on assertion failure.
- Legacy-Id: 15426
2018-08-10 15:31:12 +00:00
Henrik Levkowetz e53318084d Added a tiny utility function unwrap() to unwrap wrapped text for matching expected strings in tests.
- Legacy-Id: 15396
2018-07-19 15:53:05 +00:00
Henrik Levkowetz 2daef52bea This commit replaces the code defined group features with features held
in a database table:

- Added a GroupFeatures model to the group models, and removed the old 
  features.py
- Added a agenda type for future use in showing different group types on
  different agendas.
- Renamed the group feature has_materials to has_nonsession_materials.
- Added API resources and admin support for the new tables.
- Added a Directorate (with reviews) group type as complement to
  Directorate, to distinguish between directorates with and without reviews.
- Adjusted tests as needed.
- Updated the fixtures, and fixed the generate_fixtures script to include
  the new AgendaTypeName objects.

There still exists about 70 instances of code comparing the group type
with a list of types; most of these should probably be replaced with new
features, instead, to make it possible to add new group types through the
database table, rather than having to edit the code.  That was the purpose
of this refactoring from the start, but the presence of this large number
of comparisons of group type against lists of types defeats the goal until
we add appropriate features and replace the group type list comparisons.
 - Legacy-Id: 15316
2018-07-12 10:51:48 +00:00
Henrik Levkowetz e87442f33a Merged in [15282] from housley@vigilsec.com:
Set reply-to for I-D Announcements that include a CC to a mail list.  Fixes #2530.
 - Legacy-Id: 15289
Note: SVN reference [15282] has been migrated to Git commit 733c304e3d
2018-07-10 17:16:47 +00:00
Henrik Levkowetz dbfdb94c34 Merged in [15267] from rcross@amsl.com:
Fix issue with decorator on utils.html.remove_tags().
 - Legacy-Id: 15270
Note: SVN reference [15267] has been migrated to Git commit 0d255f7d0874f01163f292568e76fa9d830a54e2
2018-06-19 21:23:35 +00:00
Henrik Levkowetz 97db5f71b5 Added a management command to send out gdpr consent requests.
- Legacy-Id: 15269
2018-06-19 19:23:46 +00:00
Robert Sparks b86bd84278 Improvements to tests after converting to factories suggested by studying coverage_changes. Commit ready to merge.
- Legacy-Id: 15248
2018-06-13 22:09:01 +00:00
Henrik Levkowetz 4db4dd16b0 Merged in ^/personal/henrik/6.79.1-gdpr@15180 containing GDPR-adaptations.
- Legacy-Id: 15181
2018-05-28 11:26:24 +00:00
Henrik Levkowetz 2522082979 Changed the email origin field during test to hold user.username in order to exercise more of the code. Changed the EmailFactory to also use user.username as origin.
- Legacy-Id: 15172
2018-05-26 08:32:20 +00:00
Robert Sparks 25e9893a13 Changed how the mail utilities handle lists in their override and extra arguments. Added a test for send_mail_preformatted exercising override and extra. Commit ready for merge.
- Legacy-Id: 15153
2018-05-22 15:38:04 +00:00
Henrik Levkowetz df7df69a59 Updated comment text
- Legacy-Id: 15131
2018-05-05 12:49:39 +00:00
Henrik Levkowetz 5f37a71889 Added origin information to all places where we create email address entries.
- Legacy-Id: 15127
2018-05-05 12:40:30 +00:00
Henrik Levkowetz 61a16856ad Updated the admin interface to use the simple-history admin support for Person history.
- Legacy-Id: 15097
2018-05-01 11:07:12 +00:00
Henrik Levkowetz 53c4ac36db Removed the Person.address field, which is not being used. This was a legacy from the 2001 perl-based datatracker tables. Fixes issue #2504.
- Legacy-Id: 15095
2018-04-27 14:00:33 +00:00
Henrik Levkowetz 198c003b6a Merged in [15084] from rjsparks@nostrum.com:
Corrected the implementation of clear_ballot. Added a test for it. Restored functionality to ballot popups that was missing because a template variable was not passed in.
 - Legacy-Id: 15092
Note: SVN reference [15084] has been migrated to Git commit ee346edaba
2018-04-26 13:41:22 +00:00
Henrik Levkowetz fac57c24a7 Tweaked the wording of coverage_changes output.
- Legacy-Id: 15048
2018-04-17 11:35:36 +00:00
Henrik Levkowetz ab8e73aca5 Included django.utils.html.escape() in the linkify() code, and removed it from various templates, to make sure escape is consistently applied before linkify. Fixes issue #2492.
- Legacy-Id: 15035
2018-04-09 16:36:07 +00:00
Henrik Levkowetz a96c6c4c5b Tweaked the test runner to more consistently force the test coverage tests to be run last.
- Legacy-Id: 15030
2018-04-08 12:42:03 +00:00
Henrik Levkowetz 675c652052 Additional changes to speed up the IESG agenda docs page: Changed telechat_page_count() to accept a list of documents if that's already been generated, to avoid double work. Changed the reviewed_by_teams list to provide acronyms directly, to avoid group lookups during template rendering. Provided page counts directly to the template instead of repeated (costly) filtering through telechat_page_count, with new document lookups. Removed the telechat_page_count template filter, due to its cost. Tweaked some lookups in fill_in_document_table_attributes() . Added to the select_related() list for documents in IESG agenda_documents().
- Legacy-Id: 14988
2018-04-01 18:51:48 +00:00
Henrik Levkowetz ec6424c9c4 Merged in [14882] from rjsparks@nostrum.com:
Replaced line that had mysteriously been deleted before commit.  Sorry for the noise.
 - Legacy-Id: 14905
Note: SVN reference [14882] has been migrated to Git commit 4a7cdf26d3
2018-03-24 13:39:49 +00:00
Henrik Levkowetz a485c74314 Merged in [14880] from rjsparks@nostrum.com:
Added a Draft test suite.
 - Legacy-Id: 14901
Note: SVN reference [14880] has been migrated to Git commit e09a28cad2
2018-03-22 16:34:10 +00:00
Robert Sparks 4a7cdf26d3 Replaced line that had mysteriously been deleted before commit. Commit ready for merge. Sorry for the noise.
- Legacy-Id: 14882
2018-03-20 21:14:46 +00:00
Robert Sparks e09a28cad2 Failed to add a file in a previous commit. Commit ready for merge.
- Legacy-Id: 14880
2018-03-20 18:21:37 +00:00
Henrik Levkowetz 75deb35e10 Merged in [14851] from housley@vigilsec.com:
Improve parser for references in Internet-Drafts. Fixes #2360
 - Legacy-Id: 14867
Note: SVN reference [14851] has been migrated to Git commit 565b10e00e
2018-03-19 22:37:03 +00:00
Russ Housley 565b10e00e Improve parser for references in Internet-Drafts. Fixes #2360
- Legacy-Id: 14851
2018-03-17 18:25:31 +00:00
Henrik Levkowetz 63c97ad439 Fixed a bug in a RegexValidator error message string interpolation.
- Legacy-Id: 14841
2018-03-17 13:09:58 +00:00
Henrik Levkowetz 9341f96832 Tweaked the document sanitizer to insert a charset meta tag after sanitization.
- Legacy-Id: 14832
2018-03-16 11:13:03 +00:00
Henrik Levkowetz 428c451692 Added a missing tag to the sanitizer whitelist (telling lxml's Cleaner to not clean style with style=False is apparently not always enough). Fixes issue #2470.
- Legacy-Id: 14794
2018-03-14 18:52:11 +00:00
Henrik Levkowetz 02105889b6 Tweaked the mime type validator so it can be called also when there's no required mime types.
- Legacy-Id: 14779
2018-03-13 13:54:04 +00:00
Henrik Levkowetz 2b52919c5e Added sanitize_document() and replaced sanitize_html() with sanitize_fragment() in utils.html
- Legacy-Id: 14776
2018-03-13 13:21:41 +00:00
Henrik Levkowetz f3f77dce48 Changed order of arguments in a test assert for better error legibility.
- Legacy-Id: 14774
2018-03-13 12:38:53 +00:00
Henrik Levkowetz 724f1ceccc Added xmpp as an acceptable protocol in links when sanitizing.
- Legacy-Id: 14766
2018-03-12 13:17:41 +00:00
Henrik Levkowetz 3b7a191feb Removed a debug print statement
- Legacy-Id: 14753
2018-03-09 15:49:25 +00:00
Henrik Levkowetz 2c0348cc7a Found that django's urlize() does not deal well with adjacent parantheses. Replaced it with a filter based on bleach.linkify, which does better.
- Legacy-Id: 14752
2018-03-09 15:07:04 +00:00
Henrik Levkowetz 2fd344f810 Tweaks to handle text types better and make set operation clearer.
- Legacy-Id: 14745
2018-03-07 21:10:47 +00:00
Henrik Levkowetz 802f201d81 Modified the sanitizer and upload handler to strip also the content of some tags, and to produce valid files (if the content is otherwise valid).
- Legacy-Id: 14744
2018-03-07 19:00:24 +00:00
Henrik Levkowetz 5964cdd880 Removed unused data.
- Legacy-Id: 14741
2018-03-07 08:24:43 +00:00
Henrik Levkowetz 2828683cee Replaced html sanitization code that called html5lib directly with calls to bleach, and upgraded the requirements to let us use the latest html5lib and bleach.
- Legacy-Id: 14739
2018-03-06 18:35:34 +00:00
Henrik Levkowetz b92ad2f992 Added sanitization of uploaded html content for session agendas and minutes, and did some refactoring of the upload form classes.
- Legacy-Id: 14738
2018-03-06 15:55:30 +00:00
Henrik Levkowetz 0800304b67 Added TeX escaping utility functions and template filters. Removed
html escaping and added TeX escaping for relevant parts of the bibtext
template.  Fixes issue #2459.
 - Legacy-Id: 14711
2018-02-27 18:15:21 +00:00
Henrik Levkowetz 9ffe1e425a Reverted unintentional commit
- Legacy-Id: 14709
2018-02-27 17:58:25 +00:00
Henrik Levkowetz a5db4d00de Updated PLAN
- Legacy-Id: 14708
2018-02-27 17:55:43 +00:00
Henrik Levkowetz 1ed8e967e7 Merged in ^/personal/henrik/6.72.1-django-1.11@14676: Upgrade to Django 1.11
- Legacy-Id: 14695
2018-02-25 19:55:16 +00:00
Henrik Levkowetz 71a9ffafc5 Changed allow_lazy to the @keep_lazy decorator.
- Legacy-Id: 14674
2018-02-22 00:13:32 +00:00
Henrik Levkowetz 36b5e2ef8e Changed name from .rel to .remote_field for remote accessors.
- Legacy-Id: 14665
2018-02-21 23:56:36 +00:00
Henrik Levkowetz 06362ec046 Got rid of old migrations and created new initial migrations.
- Legacy-Id: 14662
2018-02-21 23:46:22 +00:00
Henrik Levkowetz 5638cf3da3 Changed all usage of ForeignKey and OneToOneFiled in model.py files to the compatibility versions from ietf.utils.models.
- Legacy-Id: 14661
2018-02-20 15:36:05 +00:00
Henrik Levkowetz c760792339 Added proxy ForeignKey and OneToOneField classes which set the now required on_delete parameter on the fields to the legacy value.
- Legacy-Id: 14660
2018-02-20 15:33:20 +00:00
Henrik Levkowetz dd7853c7a3 Check line lenght before assuming there's a first character.
- Legacy-Id: 14619
2018-02-06 15:18:11 +00:00
Henrik Levkowetz a4cc4bba2e Merged in ^/personal/henrik/6.68.4-ipr@14609 from henrik@levkowetz.com:
Changes to the IPR disclosure pages, requested during WG Chairs Lunch where the changes to IPR handling introduced by RFC8179 were presented.
 - Legacy-Id: 14610
2018-02-01 15:41:34 +00:00
Henrik Levkowetz 2687c8d839 Added a workaround for libmagic mislabelling plain text content with a line beginning with 'virtual' as text/x-c++.
- Legacy-Id: 14586
2018-01-30 08:29:21 +00:00
Henrik Levkowetz a3d1d5595d Added an option to list the slugs that can be used with the generate_apache_perms management command.
- Legacy-Id: 14576
2018-01-26 22:40:48 +00:00
Henrik Levkowetz fd01a612b7 Added a management command to generate apache 'Require user ...' stanzas from group:type:role specifications.
- Legacy-Id: 14574
2018-01-26 20:43:40 +00:00
Henrik Levkowetz d6e4404d63 Only do coverage set/reset operations if we're in test mode. Fixes a 500 error during draft submission yang checking.
- Legacy-Id: 14569
2018-01-25 11:31:38 +00:00
Henrik Levkowetz 717868cae2 Rewrote text_to_dict() and dict_to_text() to support unicode without RFC2822 encoding issues. Added initial values in IPR update forms, from the original disclosure, in order to make updates easier. Addresses issue #2413.
- Legacy-Id: 14531
2018-01-17 00:21:34 +00:00
Henrik Levkowetz 008eaf7fda Modified the run_yang_model_checks management command to accept document aliases on the command line.
- Legacy-Id: 14443
2017-12-19 14:09:40 +00:00
Henrik Levkowetz 6567e707ce Merged in ^/personal/henrik/6.64.2-ballotapi@14426. This provides personal API keys and a ballot position API at /api/iesg/position. Also added an endpoint description at /api/.
- Legacy-Id: 14430
2017-12-17 23:43:44 +00:00
Henrik Levkowetz a08c8dc76f Added an API endpoint to support automation of IESG ballot position posting, at /api/iesg/position. Added tests for the API endpoint, and updated the apikey validation decorator tests. Tweaked the decorator to handle a weakness found during testing.
- Legacy-Id: 14429
2017-12-17 21:55:27 +00:00
Henrik Levkowetz b0863c8963 Modified some names.
- Legacy-Id: 14428
2017-12-16 18:43:08 +00:00
Henrik Levkowetz ec4e238185 Added some missing files.
- Legacy-Id: 14427
2017-12-16 18:39:26 +00:00
Henrik Levkowetz e7209c6e50 Added a new personal event table to keep track of personal API key logins, and a management command to send out reports about activity to users with API keys. Added a weekly cronjob script to trigger weekly reports, and a monthly script for future use. Added a @require_api_key decorator to validate API keys for API key views and log in the API key owner. Modified the API key management urls to use create and disable rather than add and delete. Updated the API key list view. Added an API placeholder view function for ballot position setting, for test purposes. Added tests for the decorator and management command.
- Legacy-Id: 14426
2017-12-16 18:37:52 +00:00
Henrik Levkowetz 152261a869 Added new infrastructure for personal API keys, to generate, view, and delete them.
- Legacy-Id: 14423
2017-12-14 14:30:59 +00:00
Henrik Levkowetz 660c81c272 Tweaked the file content read refactoring in [14406] to try latin-1 conversion if unicode doesn't work.
- Legacy-Id: 14410
Note: SVN reference [14406] has been migrated to Git commit 967ece7e7d
2017-12-10 17:48:09 +00:00
Henrik Levkowetz 967ece7e7d Started refactoring of reading text from document files (drafts, charters, etc.) in order to normalise on one way of doing this, and making that return unicode rather than undecoded bytes. This is the first step of two, in order to gauge the possible issues and report on discrepancies.
- Legacy-Id: 14406
2017-12-08 21:51:11 +00:00
Henrik Levkowetz 3cd3c3526f Tweaked the coverage_changes management comand output.
- Legacy-Id: 14402
2017-12-06 13:31:33 +00:00
Henrik Levkowetz 57c18ef3fe Changed settings and handling of adhoc wiki paths to be more testing-friendly.
- Legacy-Id: 14390
2017-12-04 15:09:03 +00:00
Henrik Levkowetz 47a0b9d3ce Changed handling of yang modeules from drafts with errors, to not place them in separate directories, due to possible follow-on effects. Added use of IANA-held modules when validating, and regular rsyncing from IANA.
- Legacy-Id: 14389
2017-12-04 14:37:15 +00:00
Henrik Levkowetz 98d234cd86 Refactored the handling of BallotDocEvent in tests, only creating them as needed, in preparation for stricter checks on state transitions to avoid mismatches between state and ballot.
- Legacy-Id: 14380
2017-11-30 16:02:31 +00:00
Henrik Levkowetz b29e096f92 Added a test for the IESG past documents page, and tweaked some test documents to have titles.
- Legacy-Id: 14303
2017-11-04 16:15:39 +00:00
Henrik Levkowetz 260bac9346 Updated some tests to match api views refactoring.
- Legacy-Id: 14295
2017-11-03 16:42:09 +00:00
Henrik Levkowetz 3269a696fe Variable naming fix
- Legacy-Id: 14286
2017-11-03 10:36:50 +00:00
Henrik Levkowetz 23a3056260 Fixed a typo.
- Legacy-Id: 14270
2017-11-01 14:51:08 +00:00
Henrik Levkowetz 6e23f99019 Improved the help message for the run_yang_model_checks management command.
- Legacy-Id: 14267
2017-11-01 14:39:53 +00:00
Henrik Levkowetz 48fe02d58c Permit tildes in romanization of draft author names when looking for draft authors. Can be used in romanization of arabic names.
- Legacy-Id: 14256
2017-11-01 11:51:24 +00:00
Henrik Levkowetz af64f0e064 Fixed a call parameter list discrepancy.
- Legacy-Id: 14252
2017-11-01 05:07:35 +00:00
Henrik Levkowetz 09d8ad1e1b Fixed some pyflakes issues.
- Legacy-Id: 14238
2017-10-30 11:47:05 +00:00
Henrik Levkowetz 1a7e9b619a Added support for ad-hoc trac instances, with arbitrary names and filesystem paths, but still bound to a particular group's roles for management of trac admin rights.
- Legacy-Id: 14232
2017-10-30 11:10:15 +00:00
Henrik Levkowetz 9be7d57a03 Tweaked the test-runner random state handling to display the correct random state filename and make sure to set the same random state.
- Legacy-Id: 14209
2017-10-14 12:05:01 +00:00
Henrik Levkowetz 0e00adc5ee Another tweak to the draft author extraction code, to handle some name transliterations using multiple leading grave accents.
- Legacy-Id: 14149
2017-09-21 09:28:18 +00:00
Henrik Levkowetz 2c1438c240 Moved unidecode_name from utils.text to person.name.
Modified UserFactory to use a new locale for each new user, instead of the
same locale for a whole test run.  This (almost) ensures the exercise of
code to deal with non-ascii names, something which would not happen if a
locale with ascii names was chosen at the start of a run.

Modified name.initials() to not use non-word characters as initials.

Modified unidecode_name() to do more normalization, to conform to the
conventions used in internet-drafts.

Added saving of the factory-boy random state in order to be able to re-run
a test suite with the same pseudo-random sequence as in a previous failed
run.

Fixed an issue with email formatting in test_api_submit_ok().

Modified the draft author extraction code to deal better with names with
embedded apostrophes.
 - Legacy-Id: 14141
2017-09-20 15:36:30 +00:00
Henrik Levkowetz 1530a6c394 Added a new TestCase assertion: assertMailboxContains(), to be able to better express some test cases.
- Legacy-Id: 14139
2017-09-19 19:47:48 +00:00
Henrik Levkowetz 33b275b04f Added ietf.utils.text.unidecode_name() and replaced various uses of unidecode() with it, in order to normalize the generation of ascii versions of names, to avoid different practices in space stripping and space normalization in different parts of the code.
- Legacy-Id: 14128
2017-09-17 15:12:18 +00:00
Henrik Levkowetz 9f785c0dce Tweaked the test util function create_person() to create ascii-only .ascii fields, which it did not do before.
- Legacy-Id: 14123
2017-09-16 09:21:45 +00:00
Henrik Levkowetz aafd6290a6 Added an option to ietf.utils.draft.Draft to pull document name from the source file name.
- Legacy-Id: 14089
2017-08-31 14:48:43 +00:00
Henrik Levkowetz aaccbba6e8 Fixed a bug in coverage_changes.
- Legacy-Id: 14037
2017-08-10 10:53:59 +00:00
Henrik Levkowetz 1eeaf49c5c Added an error exit when the test runner is invoked with incompatible switches (--parallel and --html-report)
- Legacy-Id: 14024
2017-08-02 13:00:16 +00:00
Henrik Levkowetz 4dad8cdf9a Added a test to check for missing admin models, and added the admin models that were currently missing.
- Legacy-Id: 14021
2017-08-01 14:00:29 +00:00
Henrik Levkowetz 138f3d7382 Fixed a variable initialization issue.
- Legacy-Id: 14009
2017-07-30 14:33:34 +00:00
Henrik Levkowetz 2070a4efcf Fixed a pyflakes issue.
- Legacy-Id: 14008
2017-07-30 13:55:30 +00:00
Henrik Levkowetz 8a53653682 Added detailed line-by line changes in code coverage to the coverage_changes management command. Use verbosity 2 or higher.
- Legacy-Id: 14007
2017-07-30 13:50:06 +00:00
Henrik Levkowetz d00921b94b Added a new release-coverage format which whould make it easier to show line-by-line code test coverage changes in the future.
- Legacy-Id: 13967
2017-07-24 17:38:00 +00:00
Henrik Levkowetz 2de6855ccd Fixed a time arithmetics issue in meeting cutoff date methods
- Legacy-Id: 13966
2017-07-23 14:47:46 +00:00
Henrik Levkowetz 15f8b4504e Change the test telechat dates so that the first active telechat doesn't change to lie in the past if the tests are run across midnight.
- Legacy-Id: 13889
2017-07-15 10:01:35 +00:00
Henrik Levkowetz 86b252c90e Some refactoring of the agenda and minutes validation work.
- Legacy-Id: 13850
2017-07-13 10:42:48 +00:00
Henrik Levkowetz 46fc7b77fd Added more validation of extension, mime type, etc. for uploaded meeting agendas and minutes. Added '.md' (markdown) as an accepted file type. Html with frames is now rejected. Factored out validation code into separate functions.
- Legacy-Id: 13849
2017-07-12 22:45:57 +00:00
Henrik Levkowetz d2395f9521 Merged in [13832] from rjsparks@nostrum.com:
Treat Area Groups similar to Working Groups and Research Groups where it makes sense to do so.
 - Legacy-Id: 13836
Note: SVN reference [13832] has been migrated to Git commit 2c92aa213d
2017-07-11 21:57:01 +00:00
Robert Sparks 2c92aa213d Treat Area Groups similar to Working Groups and Research Groups where it makes sense to do so. Commit ready for merge.
- Legacy-Id: 13832
2017-07-11 19:12:21 +00:00
Henrik Levkowetz 95ddc35eae Added wiki creation for groups in state 'bof'
- Legacy-Id: 13829
2017-07-11 16:06:00 +00:00
Henrik Levkowetz 2c39de61ab Replaced the local copy of the draft/rfc htmlization code with rfc2html as installed modeule.
- Legacy-Id: 13827
2017-07-11 13:52:08 +00:00
Henrik Levkowetz eceed7476d Added the html_report switch setting to the test runner object.
- Legacy-Id: 13822
2017-07-09 18:10:51 +00:00
Henrik Levkowetz ca921fe45d Added a switch to control generation of the code coverage html report, which now defaults to false. The report can be generated post-testing with "coverage html -d static/coverage/".
- Legacy-Id: 13821
2017-07-09 16:50:38 +00:00
Henrik Levkowetz d26c7085c6 Added code coverage settings and configuration to ignore debug-related lines of code.
- Legacy-Id: 13817
2017-07-09 15:00:22 +00:00
Henrik Levkowetz 42b1a41227 Added the new format '4' to the list of recognized formats in the coverage_changes management command.
- Legacy-Id: 13766
2017-07-03 21:18:46 +00:00
Henrik Levkowetz 97a5ed1f6c Added a test to verify that we can load the 500 page template
- Legacy-Id: 13765
2017-07-03 21:17:45 +00:00
Henrik Levkowetz fb88596c59 Fixed an issue with the scope of the logfunc variable in production.
- Legacy-Id: 13761
2017-07-03 14:03:53 +00:00
Henrik Levkowetz db96292229 Added a utility function to show just the text content of the test client response.
- Legacy-Id: 13737
2017-06-30 11:45:24 +00:00
Henrik Levkowetz 2070bbfed6 Merged in [13722] and [13712] from rcross@amsl.com:
Remove all use of request.session from secretariat apps.  Add tests for affected views.  Fixes #1455.
 - Legacy-Id: 13724
Note: SVN reference [13712] has been migrated to Git commit 915b445bcffcd4687fb1677230b98282d22dd9a0

Note: SVN reference [13722] has been migrated to Git commit 3a2cb9d3d846168f75d9796b4e5383bdf10b1f60
2017-06-28 14:55:10 +00:00
Henrik Levkowetz f6f0278859 Changed the url coverage code to handle deeper chains of url includes, and to handle url includes through url lists in addition to url modules. Added information in the release coverage dictionaries about the view functions coupled to the urls.
- Legacy-Id: 13706
2017-06-22 15:07:06 +00:00
Henrik Levkowetz 5e80e00aea Corrected the settings names used in the yang extraction command.
- Legacy-Id: 13695
2017-06-20 14:06:42 +00:00
Henrik Levkowetz b42f1cbeb5 Replaced the use of unaccent.asciify(), which has similar functionality to unidecode.unidecode(). Changed the draft parser to work exclusively with unicode text, which both makes the removal of unaccent easier, and takes us closer to Py35 compatibility. Adjusted callers of the draft parser to send in unicode.
- Legacy-Id: 13673
2017-06-18 18:23:18 +00:00
Henrik Levkowetz be80e68b47 Corrected the spelling of norwegian words in unaccent.py
- Legacy-Id: 13671
2017-06-18 09:33:51 +00:00
Henrik Levkowetz a18cf20868 Removed code which has been marked as unreachable since February/March 2017 without triggering any usage indications.
- Legacy-Id: 13667
2017-06-17 17:54:40 +00:00
Henrik Levkowetz e6c8b6621c Changed mentions of people with roles on the various group pages to link to both their profile page and their email address.
- Legacy-Id: 13664
2017-06-17 17:05:38 +00:00
Henrik Levkowetz 34a2352288 Make sure wordwrap() and friend works as intended if they are used as template filters and given string arguments.
- Legacy-Id: 13653
2017-06-16 13:15:02 +00:00
Henrik Levkowetz 8d420d4d2f Added initial data to the versioninfo migration.
- Legacy-Id: 13638
2017-06-16 08:55:31 +00:00
Henrik Levkowetz c4b64e51af Updated the VersionInfo migration.
- Legacy-Id: 13637
2017-06-16 08:45:11 +00:00
Henrik Levkowetz d98054c103 Added a new yang checker, 'yanglint', to the existing Yang checker class, in
addition to the existing 'pyang' checker.

Added modal overlay displays showing the yang check results every place the
yin/yang symbol is shown (red or green) to indicate the presencee and result
of yang checks.  Added a Yang Validation: line in the document
meta-information section on the document's page in the datatracker.

Added the result of the xym extaction to the yang check results, to make
extration failures visible.

Added the version of the used xym, pyang, and yanglint commands to the check
results.

Added an action to move successfully extracted and validated modules to the
module library directories immediately on submission.

Added the xym and pyang repositories as svn:external components, rather than
listing them in requirements.txt, as there has been delays of many months
between essential features in the repositories, and an actual release.  We may
get occasional buildbot failures if broken code is pulled in from the
repository, but better that than the functionality failure of severely
outdated componets.

Added a new management command to re-run yang validation for active drafts for
which yang modules were found at submission time, in order to pick up imported
models which may have arrived in the model libraries after the draft's
submission.  Run daily from bin/daily.

Added a table to hold version information for external commands.  The yang
checker output should include the version information of the used checkers,
but seems unnecessary to run each command with its --version switch every
time we check a module...

Added a new management command to collect version information for external
commands on demand.  To be run daily from bin/daily.

Added tests to verify that xym, pyang and yanglint information is available
on the submission confirmation page, and updated the yang module contained in
the test document to validate under both pyang and yanglint.

Updated admin.py and resource.py files as needed.
 - Legacy-Id: 13630
2017-06-15 16:09:28 +00:00
Henrik Levkowetz 9bd64ba6f2 Merged in [13559] from rcross@amsl.com:
When changing a TelechatDate in the admin interface, retain all document associations.  Fixes #1349.
 - Legacy-Id: 13577
Note: SVN reference [13559] has been migrated to Git commit 77f4bf21ff
2017-06-10 05:33:36 +00:00
Henrik Levkowetz 4dcdda08d2 Added a test to the test suite which checks if there are interleaved schema and data migrations that have not been released yet. Having split migrations, first all schema and then all data, will permit doing 2 sequential releases; but if the data and schema migrations are interleaved (beyond this) it is problematic to arrange for release without ending up with a prolonged period when running code and table structure is out of sync (while the normally more time-consuming data migrations run).
- Legacy-Id: 13576
2017-06-09 21:37:14 +00:00
Henrik Levkowetz 7385a8be51 Changed name and signature for a log helper.
- Legacy-Id: 13571
2017-06-09 19:38:48 +00:00
Henrik Levkowetz 023a32715d Merged in ^/branch/iola/author-stats-r13145 from olau@iola.com, with additional features from ^/personal/henrik/6.52.1-authorstats.
- Legacy-Id: 13550
2017-06-06 18:36:59 +00:00
Henrik Levkowetz a34d078428 Commented out again a function that was commented in by mistake in the committed code.
- Legacy-Id: 13543
2017-06-06 16:30:53 +00:00
Henrik Levkowetz 5ca4309691 Fixed a bug in wordwrap() where an URL (or any word) longer than width could prevent line breaking in following text.
- Legacy-Id: 13541
2017-06-06 14:03:02 +00:00
Henrik Levkowetz fb93a8c293 Added dates to unreachable() calls, to make it easier to decide when to remove marked code.
- Legacy-Id: 13538
2017-06-06 07:27:07 +00:00
Henrik Levkowetz b2da91de1b Added a middleware exception handler for the case of people submitting unicode outside the Basic Multilingual Plane, which cannot currently be saved to the database.
- Legacy-Id: 13508
2017-06-03 21:47:51 +00:00
Henrik Levkowetz 5b2087f910 Eliminated several variations on word wrapping, keeping only what used to be wrap_text(), but renamed as ietf.utils.text.wordwrap(). This performs better than django.utils.text.wrap() when there are indented text parts. Replaced django's default wordwrap filter with one calling ietf.utils.text.wordwrap in templates. Changed to triggered wrapping in some cases, with the maybewordwrap filter, which triggers on lines longer than 100 characters. This fixes the issue with undesired wrapping of reviews.
- Legacy-Id: 13505
2017-06-02 23:13:22 +00:00
Henrik Levkowetz 16d129cacf Added examples of how our different text wrapper functions work to ietf.utils.text. Run 'python ietf/utils/text.py | less' to see the results.
- Legacy-Id: 13498
2017-06-02 17:59:26 +00:00
Henrik Levkowetz 76628be3fd Merged in ^/branch/iola/author-stats-r13145 from olau@iola.dk, and fixed some tests in code which moved after the latest merge with trunk. The test suite passes, but the migrations are _not_ ready to run, because of numbering conflicts (again due to code changes on trunk since the latest sync).
- Legacy-Id: 13479
2017-05-31 20:59:26 +00:00
Henrik Levkowetz fe30afdfe7 Updated the check_referential_integrity management command to show progress bars when not in verbose mode.
- Legacy-Id: 13369
2017-05-16 15:07:56 +00:00
Henrik Levkowetz b5319c576a Updated the utils.DumpInfo model to match the database.
- Legacy-Id: 13330
2017-05-12 15:27:23 +00:00
Henrik Levkowetz 38bfdb4095 Fixed a bug in the earlier author extraction bugfix.
- Legacy-Id: 13295
2017-05-10 12:21:17 +00:00
Henrik Levkowetz fb70e9a4ff Fixed an issue with the author extraction code.
- Legacy-Id: 13288
2017-05-09 19:19:55 +00:00
Henrik Levkowetz b1c4b75fbe Fixed a pyflakes issue.
- Legacy-Id: 13235
2017-04-18 20:55:43 +00:00
Henrik Levkowetz 0b730c9554 Updated the referential integrity checking comand.
- Legacy-Id: 13234
2017-04-18 20:21:58 +00:00
Henrik Levkowetz d6b9392f72 Fixed an if-statement which incorrectly disabled coverage testing when the parallel testing count was set to 1.
- Legacy-Id: 13231
2017-04-18 16:47:28 +00:00
Henrik Levkowetz 978e8f72f5 Changed the signature of ietf.utils.aliases.dump_sublist() to include a list of the alias domains, in order to be able to generate IRTF group aliases @irtf.org. Updated ietf/bin/generate-*-aliases accordingly.
- Legacy-Id: 13206
2017-04-12 18:51:38 +00:00
Henrik Levkowetz d19179ba44 Merged in [13181] from housley@vigilsec.com:
This completes a good chunk of the downref registry work requested in ticket
#2069. The registry has been imported to the database and can be shown, and
entries can be added to the registry.  Addresses issue #2069.
 - Legacy-Id: 13190
Note: SVN reference [13181] has been migrated to Git commit f0125634f8
2017-04-10 22:47:01 +00:00
Russ Housley f0125634f8 Add downref registry to the datatracker at the request of the IESG. Still need to integrate with Last Call message generation
- Legacy-Id: 13181
2017-04-03 17:36:27 +00:00
Ole Laursen d9585b4b73 Add some extra test data to exercise the statistics better
- Legacy-Id: 13167
2017-03-29 09:06:12 +00:00
Ole Laursen a1ba9dfb0a Remove accidentally committed debug
- Legacy-Id: 13160
2017-03-28 14:56:06 +00:00
Ole Laursen 5b677dc6ba Merge author stats branch into new branch from trunk
- Legacy-Id: 13159
2017-03-28 14:36:40 +00:00
Henrik Levkowetz 14c748df98 Changed the creation of temporary test dirs to use names which include the test class name, in order to make it possible to run test cases in parallel. Disabled coverage testing when running in parallel (those classes cannot be pickled and distributed to the test runner threads, and also won't pass coverage data back). Tweaked the TestCase.tempdir() method.
- Legacy-Id: 13157
2017-03-27 21:41:18 +00:00
Henrik Levkowetz 605fa26ec3 Made relative paths work for the create_group_wikis commnand. Added wiki and svn dir patterns to docker/settings_local.py.
- Legacy-Id: 13150
2017-03-27 17:07:48 +00:00
Ole Laursen 5608ad0061 Move the assert on new X in the test coverage up before the percentage
to try get the list of URLs out before the percentage
 - Legacy-Id: 13144
2017-03-27 08:45:20 +00:00
Ole Laursen ef4d55f0c9 Apply patch from Henrik Levkowetz to fix some problems of author parse
errors where the affiliation is mistakenly thought to be an extra
author (some of these still remain)
 - Legacy-Id: 13142
2017-03-27 08:33:49 +00:00
Henrik Levkowetz d2c2d80341 Merged in [13112] from lars@netapp.com:
Fix various \'RemovedInDjango20Warning\' warnings.
 - Legacy-Id: 13141
Note: SVN reference [13112] has been migrated to Git commit 8a70e9773d
2017-03-26 22:06:47 +00:00
Lars Eggert 8a70e9773d Fix various "RemovedInDjango20Warning" warnings. Commit ready for merge.
- Legacy-Id: 13112
2017-03-25 19:16:45 +00:00
Henrik Levkowetz 3f92aee755 Added the rfcmarkup module. This will be changed to a library dependency when the module is published.
- Legacy-Id: 13038
2017-03-20 13:48:13 +00:00
Henrik Levkowetz 53481eebd8 Added a utility function to create a test-class specific temporary directory.
- Legacy-Id: 13026
2017-03-19 19:25:26 +00:00
Henrik Levkowetz bb5096da8a Added more email validation for draft submission author emails, and blocked some baths that could lead to bad email addresses ('none') being set as document author email addresses.
- Legacy-Id: 13010
2017-03-14 14:07:21 +00:00
Henrik Levkowetz 7b3ce6e795 Improved the log.assertion() and unreachable() functions to send a deeper traceback when reporting by email, rather than just the last stack frame.
- Legacy-Id: 12999
2017-03-13 05:37:02 +00:00
Henrik Levkowetz fe8cd06a76 Back out the htmlization code for the moment.
- Legacy-Id: 12997
2017-03-12 12:12:44 +00:00
Henrik Levkowetz 319a4850d6 Added rfcmarkup code. Will probably be migrated to a separate lib in time.
- Legacy-Id: 12994
2017-03-11 14:02:16 +00:00
Henrik Levkowetz 58dae78a43 Modified the REST API to provide group descriptions taken from the group charters if group.description is blank.
- Legacy-Id: 12993
2017-03-11 13:46:19 +00:00
Henrik Levkowetz 4daf66d11c Added DocEvent rev= parameter througout the code.
- Legacy-Id: 12978
2017-03-06 15:08:21 +00:00
Henrik Levkowetz ff31644ccf Tweaked assertion() and unreachable() to raise an exception instead of sending email (to the test mailbox ...) when running the test suite.
- Legacy-Id: 12977
2017-03-06 14:31:45 +00:00
Henrik Levkowetz 0bcb4ae30c Fixed a problem with assertion() and unreachable(), which could break if certain names (like 'args') were set in the calling frame.
- Legacy-Id: 12975
2017-03-06 13:51:08 +00:00
Henrik Levkowetz 3c6f00869f Dead code removal: meeting models and test_utils.
- Legacy-Id: 12965
2017-03-04 05:19:39 +00:00
Henrik Levkowetz 49ba774ae3 Updated unreachable() marking.
- Legacy-Id: 12962
2017-03-04 05:14:51 +00:00
Henrik Levkowetz 3a21e03f8d Marked a function as unrachable. Added support in send_mail_message for the Message.msgid field.
- Legacy-Id: 12954
2017-03-03 15:43:43 +00:00
Henrik Levkowetz ea70381f29 Merged in [12950] from rcross@amsl.com:
Fixes #2219.  Area Group chairs are denied material management access.
 - Legacy-Id: 12951
Note: SVN reference [12950] has been migrated to Git commit c6d1a59aa4
2017-03-02 21:21:07 +00:00
Henrik Levkowetz 5bb5ca727b Created a special function log.unreachable() to mark code unreachable, and changed a function name from log.affirm() to log.assertion().
- Legacy-Id: 12933
2017-02-28 20:16:52 +00:00
Henrik Levkowetz 5928bd9faa Added svn symlink check and creation to test setup. Forced test settings to use an empty INTERNAL_IPS list for consistency in code coverage figures.
- Legacy-Id: 12929
2017-02-28 18:02:30 +00:00
Henrik Levkowetz 4f12513b1f Tweaked the affirm() failure message.
- Legacy-Id: 12925
2017-02-27 16:42:23 +00:00
Henrik Levkowetz 5f053ad21a Cleaned up the remaining explicit url names, using dotted-paths to view
functions instead.  In all almost 700 changes.
 - Legacy-Id: 12923
2017-02-26 23:21:49 +00:00
Henrik Levkowetz 9fb1a8e05f Tweaked an import to make Python 2to3 able to handle it.
- Legacy-Id: 12922
2017-02-26 10:54:33 +00:00
Henrik Levkowetz 84527c9df6 Added some code to utils.urls.url() to output remaining explicit url names together with the matching dotted-path to the view function.
- Legacy-Id: 12921
2017-02-25 20:32:20 +00:00
Henrik Levkowetz 49af9b4e1a Added an affirm(statement_string) function, which acts like an assert, and lets us trigger an email message with traceback when the statement to affirm is False. Use instead of assert if we still want the server to produce a response, even if something is seriously wrong.
- Legacy-Id: 12901
2017-02-23 11:48:55 +00:00
Henrik Levkowetz 988a2b808f Added explicit charset setting and consistent encoding to several email paths through the datatracker. After introducing support for non-ascii names in email addresses, using role.formatted_email() to insert names into email content doesn't work any more, since uncode names will now be rfc2047-encoded in formatted_email(). Added another method role.name_and_email() for this case. Replaced another case of '%s <%s>' name-and-email formatting with formataddr(). Fixed some tests which did not send unicode email bodies to the email functions. Fixes issue #2017.
- Legacy-Id: 12896
2017-02-21 19:17:10 +00:00
Henrik Levkowetz c889856dfa Added a validator to Document.title to prevent control chars in the title (complementing a recent cleanup of more than 120 instances of document titles containing vertical tabs)
- Legacy-Id: 12893
2017-02-20 20:47:37 +00:00
Henrik Levkowetz 858d855eff Changed the new account creation to require a proper name at the same time as the account password is set, before actually creating the account. Also tweaked the password strength and confirmation code.
- Legacy-Id: 12892
2017-02-20 16:50:39 +00:00
Henrik Levkowetz 8930d29a8e Merged in Django-1.10 upgrade work from ^/personal/henrik/6.43.1-django-1.10
- Legacy-Id: 12881
2017-02-19 18:18:00 +00:00
Henrik Levkowetz cf4a4b02a7 Reworked the email address handling in order to be able to support non-ascii names as part of email address fields. Reworked the generation of user names in the test suite to generate names from multiple non-ascii locales. Fixes issue #2080.
- Legacy-Id: 12872
2017-02-18 21:50:18 +00:00
Henrik Levkowetz c344a18bdf Fixed an issue with the test-crawler which could cause false positives for urls containing apostrophe.
- Legacy-Id: 12851
2017-02-16 09:58:34 +00:00
Ole Laursen d2e85a3aa3 Apply draft parser patch from Henrik to improve the patch on trunk to
combine paragraphs across page splits - this makes the country part of
the parser find more countries
 - Legacy-Id: 12848
2017-02-15 19:10:59 +00:00
Ole Laursen b2ff10b0f2 Add support for extracting the country line from the author addresses
to the draft parser (incorporating patch from trunk), store the
extracted country instead of trying to turn it into an ISO country
code, add country and continent name models and add initial data for
those, add helper function for cleaning the countries, add author
country and continent charts, move the affiliation models to
stats/models.py, fix a bunch of bugs.
 - Legacy-Id: 12846
2017-02-15 18:43:57 +00:00
Ole Laursen 882579bab3 Don't check url reverses on the form admin:blahblah for the time being
- Legacy-Id: 12845
2017-02-15 18:39:35 +00:00
Henrik Levkowetz 44ad914fba Tweaked the company name extraction code in class Draft.
- Legacy-Id: 12842
2017-02-15 14:09:54 +00:00
Henrik Levkowetz bb5e5b97ba Another tweak to handle page break paragraph joins better in class Draft.
- Legacy-Id: 12840
2017-02-14 17:41:30 +00:00
Henrik Levkowetz 6158221fa8 Tweaked the author extraction to recognize short lines as paragraph ends, not only lines ending in '.' or ':'
- Legacy-Id: 12837
2017-02-14 14:23:15 +00:00
Henrik Levkowetz 7f607c51d2 Changed from old-style to new-style middleware.
- Legacy-Id: 12824
2017-02-12 17:07:48 +00:00
Henrik Levkowetz a1606e8f7d Added a 'htmlfilters' tag library containing 'removetags'
- Legacy-Id: 12823
2017-02-12 17:05:55 +00:00
Henrik Levkowetz aa5e61d958 Updated all urlpatterns to use ietf.utils.urls.url() instead of django's,
in order to autogenerate dotted path url pattern names.  Updated a number
of url reverses to use dotted path, and removed explicit url pattern names
as needed.

Changed some imports to prevent import of ietf.urls before django
initialization was complete.


Changed 3 cases of form classes being curried to functions; django 1.10
didn't accept that.

Started converting old-style middleware classes to new-style middleware
functions (incomplete).

Tweaked a nomcom decorator to preserve function names and attributes, like
a good decorator should.

Replaced the removed django templatetag 'removetags' with our own version
which uses bleach, and does sanitizing in addition to removing explicitly
mentionied html tags.

Rewrote the filename argument handling in a management command which had
broken with the upgrade.
 - Legacy-Id: 12818
2017-02-11 14:43:01 +00:00
Henrik Levkowetz 19d585255a Django 1.10 does not permit the view argument to url() to be a string any more. Changed all cases view arguments to url() which were strings to instead reference the actual imported function.
- Legacy-Id: 12815
2017-02-10 14:49:16 +00:00
Henrik Levkowetz 49a899fe66 Added support for management of trac wikis for explicitly named groups, such as the IESG and IAOC. Tweaked the dummy-run handling of the create_group_wikis management command.
- Legacy-Id: 12789
2017-02-08 16:37:31 +00:00
Henrik Levkowetz de3976bec0 Merged in Django 1.9 upgrade work from ^/personal/henrik/6.41.1-django-1.9
- Legacy-Id: 12764
2017-01-31 15:13:01 +00:00
Henrik Levkowetz 55b430b525 Merged in [12748] from rcross@amsl.com:
Fixes #2172.  Add Requested Resources and People who must be there, to email notifications of session requests.
 - Legacy-Id: 12757
Note: SVN reference [12748] has been migrated to Git commit eeb35e997d
2017-01-30 23:22:45 +00:00
Henrik Levkowetz 40e8380751 Merged django 1.9 work forward to copy of trunk @6.41.0
- Legacy-Id: 12756
2017-01-30 23:13:02 +00:00
Ryan Cross eeb35e997d Fixes #2172. Add Requested Resources and People who must be there, to email notifications of session requests. Commit ready for merge.
- Legacy-Id: 12748
2017-01-27 23:32:52 +00:00
Ole Laursen 90051a1575 Add support for displaying and editing author affiliation and country
when submitting a draft, replace the Submission.authors line-based
text field with a JSON field
 - Legacy-Id: 12745
2017-01-27 16:10:31 +00:00
Ole Laursen 4426e3386f Add assertNoFormPostErrors for use instead of testing the status code
of form post responses to make it easier to see what goes wrong when
the form in a test suddently doesn't validate
 - Legacy-Id: 12744
2017-01-27 13:27:19 +00:00
Ole Laursen 9308948195 Add person, affiliation and country (through django-countries) to
DocumentAuthor, rename author field to email and make it optional (for
modeling old email-less submissions), remove the authors many to many
referencing field from Document as it is not really pointing the right
place.

Update the Secretariat tools to show affiliation and country.

Add migration for getting rid of the fake email addresses that the
migration script created some years ago (just set the author email
field to null).
 - Legacy-Id: 12739
2017-01-26 17:10:08 +00:00
Henrik Levkowetz 79c44d25ea Changed settings_sqlitetest so that local developer additions to context processors and middleware classes are avoided.
- Legacy-Id: 12723
2017-01-24 20:42:51 +00:00
Henrik Levkowetz 6f5c0fdd23 Added a trac-admin role name, for help in assigning trac-admin rights to trac instances. Extended the create_group_wikis management command to create and update wikis for groups of type 'team','ag' and 'dir', in addition to 'wg','rg' and 'area'; and also add people with role trac-admin in the group or in the secretariat to those given TRAC_ADMIN permissions in a Trac instance.
- Legacy-Id: 12721
2017-01-24 17:21:11 +00:00
Henrik Levkowetz a5d1aca525 Save database object before using it.
- Legacy-Id: 12716
2017-01-23 19:42:03 +00:00
Henrik Levkowetz 613f0c8233 Pyflakes fixes.
- Legacy-Id: 12714
2017-01-23 18:35:58 +00:00
Henrik Levkowetz bdbddc8efd Rewrote a test to provide on-disk files instead of StringIO files to argparse (it seems optparse could handle this, but argparse can't).
- Legacy-Id: 12713
2017-01-23 18:23:05 +00:00
Henrik Levkowetz 450c39af23 Changed the failure leeway of the code coverage test percentage from 0.005% back to 0.02%.
- Legacy-Id: 12707
2017-01-19 20:01:19 +00:00
Ole Laursen aebfe44f9e Add simple detection of formal languages used in draft, partially
based on the code in getauthors by Jari Arkko
 - Legacy-Id: 12657
2017-01-16 16:08:56 +00:00
Ole Laursen 34a9f36534 Add helper for getting word count from draft
- Legacy-Id: 12655
2017-01-16 11:35:48 +00:00
Henrik Levkowetz 6eec4c2648 Converted all management commands which set up additional command-line switches to use the argparse approach instead of the deprecated optparse approach.
- Legacy-Id: 12645
2017-01-11 18:42:38 +00:00
Henrik Levkowetz 7b95411f58 The context_instance parameter to render_to_response() and render_to_string() is now deprecated. Fixed this, and generally changed the idiom render_to_response('template.html', {}, RequestContext(request)) to the newer and simpler render(request, 'template.html', {}). About 200 instances.
- Legacy-Id: 12637
2017-01-09 21:26:30 +00:00
Henrik Levkowetz 9c095365dd Commented out debug output from ietf.utils.urls.url(). Tweaked the url() handling of django views.
- Legacy-Id: 12634
2017-01-09 16:32:38 +00:00
Henrik Levkowetz eb5aa9a691 Django 1.9: Refined the autonaming url() replacement.
- Legacy-Id: 12622
2016-12-22 18:08:21 +00:00
Henrik Levkowetz d5460b8f7f Django 1.9: Merged work forward to copy of latest release.
- Legacy-Id: 12621
2016-12-22 17:30:50 +00:00
Henrik Levkowetz bfc44dea82 Django 1.9: Added an alternative implementation of urls() which derives an url pattern name from the callable, in order to maintain DRY for url patterns.
- Legacy-Id: 12617
2016-12-22 17:20:29 +00:00
Henrik Levkowetz 3944ab95cc Django 1.9: Supplied a required argument to TemplateDoesNotExist()
- Legacy-Id: 12616
2016-12-22 17:14:51 +00:00
Henrik Levkowetz 471bbde4d5 Django 1.9: Adjusted import path
- Legacy-Id: 12615
2016-12-22 17:13:26 +00:00
Henrik Levkowetz ec0d6a7e4f Merged in [12593] from rjsparks@nostrum.com:
Send mail to team secretaries when a review is explicitly requested. Change the From and Subject for assignment emails to reviewers to make it clearer what the review is for and who assigned it. Fixes #2053 and #2114.
 - Legacy-Id: 12602
Note: SVN reference [12593] has been migrated to Git commit 10f4892eae
2016-12-21 18:01:00 +00:00
Henrik Levkowetz 7e6baf15b7 Merged in [12588] from rjsparks@nostrum.com:
Change how we display review text to follow the tecnique used with other large blocks of pasted or imported text (shepherds writeups for instance). Fixes #2104.
 - Legacy-Id: 12600
Note: SVN reference [12588] has been migrated to Git commit 7c182375af
2016-12-21 17:26:03 +00:00
Henrik Levkowetz 8e9858379b Fixed some pyflakes and form initialization issues with the upgrade to DurationField.
- Legacy-Id: 12599
2016-12-21 17:03:39 +00:00
Henrik Levkowetz 67b2fc1107 Extended the form fields used for duration to accept additional formats, including formats the secretariat are currently using, in order to avoid trouble with the TimedeltaField --> DurationField transition.
- Legacy-Id: 12598
2016-12-21 15:26:32 +00:00
Robert Sparks 10f4892eae Send mail to team secretaries when a review is explicitly requested. Change the From and Subject for assignment emails to reviewers to make it clearer what the review is for and who assigned it. Fixes #2053 and #2114. Commit ready for merge.
- Legacy-Id: 12593
2016-12-19 22:32:03 +00:00
Robert Sparks 7c182375af Change how we display review text to follow the tecnique used with other large blocks of pasted or imported text (shepherds writeups for instance). Fixes #2104. Commit ready for merge.
- Legacy-Id: 12588
2016-12-19 20:38:51 +00:00
Henrik Levkowetz 481a96c935 Changed the handling of skip tests and messages in order to show which tests will be skipped, and why, at the start of a test run.
- Legacy-Id: 12568
2016-12-17 21:49:53 +00:00
Henrik Levkowetz b14e981e66 Merged in [12520] from rjsparks@nostrum.com:
Slight refactor of the review models to capture review team settings. Allows configuring review teams to get automatic suggestions for reviews or not. Provides a better admin for creating/managing review teams. Fixes #2048 and #2072.
 - Legacy-Id: 12544
Note: SVN reference [12520] has been migrated to Git commit b24bdb5bc3
2016-12-15 20:55:40 +00:00
Henrik Levkowetz 65481b5df7 Moved warnings settings from manage.py and test_runner.py to one single place, settings.py. Decreased the verbosity level when invoking 'loaddata' from the test runner.
- Legacy-Id: 12535
2016-12-15 13:25:20 +00:00
Henrik Levkowetz 8e56935cfd Undid commit 12527, which contained much too much.
- Legacy-Id: 12528
2016-12-15 10:38:54 +00:00
Henrik Levkowetz e3c6e0ac6b Changed the state shown in all_id.txt for drafts which have entered IESG processing, but then been replaced, to show as 'Replaced' instead of 'In IESG Processing'. This is consistent with how they are shown in all_id2.txt.
- Legacy-Id: 12527
2016-12-15 10:36:15 +00:00
Robert Sparks b24bdb5bc3 Slight refactor of the review models to capture review team settings. Allows configuring review teams to get automatic suggestions for reviews or not. Provides a better admin for creating/managing review teams. Fixes #2048 and #2072. Commit ready for merge.
- Legacy-Id: 12520
2016-12-14 23:26:32 +00:00
Henrik Levkowetz 03d90163a5 Merged django-1.8 work forward to copy of 6.39.1
- Legacy-Id: 12504
2016-12-10 19:32:46 +00:00
Henrik Levkowetz 2978798f39 Made serialization work for RegexStringValidator
- Legacy-Id: 12492
2016-12-09 12:40:21 +00:00
Henrik Levkowetz 57eb5844e5 Merged in [12489] from rjsparks@nostrum.com:
Save a new reviewrequest for each team requested in the request form. Fixes #2094.
 - Legacy-Id: 12491
Note: SVN reference [12489] has been migrated to Git commit 4fff1ab101e4f7cd27dc8ded889e09905866076f
2016-12-09 12:01:48 +00:00
Henrik Levkowetz 2c27d5c611 Moved optional text wrapping before html escaping in markup_unicode(), used by get_unicode_document_content(). Fixes a problem with lines being wrapped when they should not be.
- Legacy-Id: 12480
2016-12-08 16:27:05 +00:00
Henrik Levkowetz 217b31f109 Made the RegexStringValidator deconstructible so we can generate migrations.
- Legacy-Id: 12476
2016-12-07 19:28:01 +00:00
Henrik Levkowetz 6304bfda69 Fixed template_coverage_collection initialisation. Tweaked a message. Updated coverage numbers.
- Legacy-Id: 12471
2016-12-07 11:33:33 +00:00
Henrik Levkowetz db7e554c5e Changed to using TestCase from ietf.utils, and turned off coverage checking when loading templates in TemplateChecksTestCase.
- Legacy-Id: 12470
2016-12-07 11:31:31 +00:00
Henrik Levkowetz b628a0844b Changed to using TestCase from ietf.utils in some places, for consistent test-suite failure output.
- Legacy-Id: 12467
2016-12-07 11:03:32 +00:00
Henrik Levkowetz 7dfbe51913 Don't count the template loading done in TemplateChecksTestCase towards template coverage numbers.
- Legacy-Id: 12464
2016-12-06 15:55:39 +00:00
Henrik Levkowetz e4ce339235 Merged in [12461] from rjsparks@nostrum.com:
Added migration to fetch text from reviews in the mail archives and populate the review documents. Fixes #2064.  Will patch into production.
 - Legacy-Id: 12463
Note: SVN reference [12461] has been migrated to Git commit 63a9599bafdcacf49f1bb374b2156d8473da93d1
2016-12-05 21:03:49 +00:00
Henrik Levkowetz 8f6e51839f Fixed some additional places to use the new settings.TEMPLATES structure.
- Legacy-Id: 12459
2016-12-05 19:04:27 +00:00
Henrik Levkowetz 14776bae70 Merged django18 work forward to r12447
- Legacy-Id: 12455
2016-12-05 15:03:40 +00:00
Henrik Levkowetz 0bb7854591 Merged django18 work forward to 6.39
- Legacy-Id: 12449
2016-12-05 14:01:25 +00:00
Henrik Levkowetz b897519fff Changed the test runner to use the verbosity set on the command line when loading fixtures.
- Legacy-Id: 12446
2016-12-05 12:56:53 +00:00
Henrik Levkowetz 5959b3a46a Added a validator for the reviewer settings' Filter Regexp field, to make sure that we don't later hit exceptions when trying to compile and use the regexp entered.
- Legacy-Id: 12439
2016-12-01 21:30:39 +00:00
Henrik Levkowetz b914f46313 Merged in ^/branch/iola/review-tracker-r12128@12397, bringing in the review tool functionality described in RFC7735. This adds the ability to set up review management pages for review teams such as genart, secdir, opsdir, etc.; letting the review team secretaries manage requested and completed reviews; letting the reviewers keep track of and document their reviews, and more. See the RFC for full specification, and the branch commit log for a full commit history.
- Legacy-Id: 12419
2016-11-29 14:54:19 +00:00
Henrik Levkowetz cec0f33817 Removed unused stuff.
- Legacy-Id: 12409
2016-11-28 21:20:28 +00:00
Henrik Levkowetz 53d68af9cd Adapted the utils template tests to the new template handling in Django 1.8.
- Legacy-Id: 12407
2016-11-28 21:05:17 +00:00
Henrik Levkowetz 554ec2d78c Removed an unused import
- Legacy-Id: 12406
2016-11-28 21:04:15 +00:00
Henrik Levkowetz 623ce0e1bd Merged in [12378] from rjsparks@nostrum.com:
Change the lines saying that a test has failed to make it easier to cut-paste to run the failed test again.
 - Legacy-Id: 12386
Note: SVN reference [12378] has been migrated to Git commit be51bc2e78
2016-11-18 10:45:40 +00:00
Robert Sparks be51bc2e78 Change the lines saying that a test has failed to make it easier to cut-paste to run the failed test again. Commit ready for merge.
- Legacy-Id: 12378
2016-11-17 02:52:45 +00:00
Henrik Levkowetz 964d34282f Added common patterns for template file names to ignore when testing template parsing and template test coverage.
- Legacy-Id: 12354
2016-11-16 02:14:32 +00:00
Henrik Levkowetz bb7d3b476f Tweaked the create_group_wiki management command and its test to show more information on failures.
- Legacy-Id: 12273
2016-11-07 17:13:09 +00:00
Henrik Levkowetz 888bd44a10 Stash the test results summary in ./.testresults instead of ./testresults, to avoid visual littering.
- Legacy-Id: 12272
2016-11-07 17:11:48 +00:00