Commit graph

129 commits

Author SHA1 Message Date
Henrik Levkowetz bdc73e771a Python2/3 compatibility: replaced six.ensure_text() with either six.text_type or django's force_text(), depending on the case, and fixed a variable scope issue.
- Legacy-Id: 16461
2019-07-16 13:20:05 +00:00
Henrik Levkowetz 8c6eb3a30a Python2/3 compatibility: Changed the use of open() and StringIO to io.open() etc.
- Legacy-Id: 16458
2019-07-15 19:14:04 +00:00
Henrik Levkowetz 48bab56be9 Removed a print() call
- Legacy-Id: 16420
2019-07-07 13:04:20 +00:00
Henrik Levkowetz 1225f8af6b Refactored submission code to be clearer and only do mimetype extraction in one place, made the point where files are saved less obscure, fixed bytes/str issues for file read and write, fixed regex strings, fixed variable name visibility due to scope changes in py3.
- Legacy-Id: 16375
2019-07-04 15:49:42 +00:00
Henrik Levkowetz 97a555936d Changed a large number of test assertions to use assertContains() and removed separate, now unnecessary,status_code checks. Also changed many assertTrue(... in ...) to use assertIn().
- Legacy-Id: 16346
2019-06-30 21:03:34 +00:00
Henrik Levkowetz 15e74e89cb Converted instances of assertTrue(text in response.content) to the recommended assertContains(response, text), and similar for assertFalse(... in ...)
- Legacy-Id: 16336
2019-06-29 13:43:31 +00:00
Henrik Levkowetz d7f5c84182 Initial 2to3 patch with added copyright statement updates.
- Legacy-Id: 16309
2019-06-27 14:40:54 +00:00
Henrik Levkowetz 13705976e2 Merged in [16287] from rjsparks@nostrum.com:
Send mail to a list when a new IETF WG -00 is submitted. Fixes #2730.
 - Legacy-Id: 16289
Note: SVN reference [16287] has been migrated to Git commit 9ace7db719
2019-06-19 14:32:25 +00:00
Henrik Levkowetz db12f2b948 Fixed some name versus pk issues lingering after the Document/DocAlias primary key refactoring. Fixes issue #2729.
- Legacy-Id: 16288
2019-06-19 12:57:52 +00:00
Robert Sparks 9ace7db719 Send mail to a list when a new IETF WG -00 is submitted. Fixes #2730. Commit ready for merge.
- Legacy-Id: 16287
2019-06-18 21:24:49 +00:00
Henrik Levkowetz 426870b766 Merged in the Document and DocAlias primary key change and m2m work in ^/personal/henrik/6.96.1-docalias.
- Legacy-Id: 16261
2019-06-15 12:00:48 +00:00
Henrik Levkowetz a2cafded25 Changed DocAlias to be many-to-many, in order to be able to handle STD aliases referring to multiple RFCs.
- Legacy-Id: 16249
2019-06-14 20:38:14 +00:00
Henrik Levkowetz 815602351f This is a series of 50 migrations that changes the Document and DocAlias
primary keys from character strings to integers, and makes corresponding code
changes.

This was prompted by database limitations discovered when trying to make
DocAlias use a m2m document field; with 255 long strings as primary keys for
Document and DocAlias this violated the MySQL database limitations.

Changing the primary keys to integers should also improve efficiency.  

Due to the data migrations which create the new integer primary keys and adds
corresponding integer foreign keys matching the previous string foreign keys
in all tables having foreign keys to Document and DocAlias, some of these
migrations take a long time.  The total set of migrations are expected to have
a runtime on the order of 2 hours.
 - Legacy-Id: 16237
2019-06-10 11:32:46 +00:00
Robert Sparks 21a673490b Incremental improvements to the submit_existing tests. Commit ready for merge.
- Legacy-Id: 15833
2018-12-18 23:22:40 +00:00
Robert Sparks 3d6c1f991b Make state/substate changes on new submissions sensitive to stream. Fixes #2642. Commit ready for merge.
- Legacy-Id: 15829
2018-12-17 22:38:22 +00:00
Henrik Levkowetz ef3cbf4684 Merged in [15595] from rjsparks@nostrum.com:
Converted stats, submit, sync. and utils/test to use factories. Excluded utils/test_data from coverage checks.
 - Legacy-Id: 15599
Note: SVN reference [15595] has been migrated to Git commit 39e76a836996ec9ed8ea449c92bdfae91c3dbaae
2018-10-19 12:54:58 +00:00
Henrik Levkowetz 232774a5cc Adjusted the test_submit_nonascii_name() to match the output from idnits 2.16.0, which emits warnings rather than errors for nonascii content.
- Legacy-Id: 15525
2018-10-07 14:18:00 +00:00
Henrik Levkowetz e87442f33a Merged in [15282] from housley@vigilsec.com:
Set reply-to for I-D Announcements that include a CC to a mail list.  Fixes #2530.
 - Legacy-Id: 15289
Note: SVN reference [15282] has been migrated to Git commit 733c304e3d
2018-07-10 17:16:47 +00:00
Henrik Levkowetz 4db4dd16b0 Merged in ^/personal/henrik/6.79.1-gdpr@15180 containing GDPR-adaptations.
- Legacy-Id: 15181
2018-05-28 11:26:24 +00:00
Henrik Levkowetz f0c0753e28 Added email origin information to some function calls that needed it.
- Legacy-Id: 15130
2018-05-05 12:49:10 +00:00
Henrik Levkowetz d076a1728c Added a 'recent drafts' page, showing drafs submitted the last 7 days.
- Legacy-Id: 15099
2018-05-01 17:55:22 +00:00
Henrik Levkowetz fec59ad937 Fixed a PyQuery property access that should have been a method call.
- Legacy-Id: 15087
2018-04-26 11:51:39 +00:00
Henrik Levkowetz a9fbea113b Changed incorrect use of 404 status code in api to 400. Fixes issue #2498.
- Legacy-Id: 15044
2018-04-13 11:11:48 +00:00
Henrik Levkowetz 9bd18df83b Use .assertIn(...) in tests instead of .assertTrue(x in y) as the failure messages are much more informative
- Legacy-Id: 15026
2018-04-07 10:09:07 +00:00
Henrik Levkowetz b75ee89865 Added a test case for reference extraction.
- Legacy-Id: 14869
2018-03-20 11:04:12 +00:00
Henrik Levkowetz 47a0b9d3ce Changed handling of yang modeules from drafts with errors, to not place them in separate directories, due to possible follow-on effects. Added use of IANA-held modules when validating, and regular rsyncing from IANA.
- Legacy-Id: 14389
2017-12-04 14:37:15 +00:00
Henrik Levkowetz 98d234cd86 Refactored the handling of BallotDocEvent in tests, only creating them as needed, in preparation for stricter checks on state transitions to avoid mismatches between state and ballot.
- Legacy-Id: 14380
2017-11-30 16:02:31 +00:00
Henrik Levkowetz 7c9364b61f Added some more debugging output for occasional author extraction failures during test. See also [14226].
- Legacy-Id: 14233
Note: SVN reference [14226] has been migrated to Git commit 215008211a
2017-10-30 11:22:45 +00:00
Henrik Levkowetz 215008211a Added some debugging code to help identify random test failures
- Legacy-Id: 14226
2017-10-27 05:53:23 +00:00
Henrik Levkowetz 7b2d921a5a Tweaked the submission search form to accept draft names which include revision numbers. Fixes issue #2380.
- Legacy-Id: 14171
2017-09-27 15:08:40 +00:00
Henrik Levkowetz 2d817427fc Added a cancel button to the submission confirmation page. Fixes issue #2379.
- Legacy-Id: 14170
2017-09-27 14:18:12 +00:00
Henrik Levkowetz 2c1438c240 Moved unidecode_name from utils.text to person.name.
Modified UserFactory to use a new locale for each new user, instead of the
same locale for a whole test run.  This (almost) ensures the exercise of
code to deal with non-ascii names, something which would not happen if a
locale with ascii names was chosen at the start of a run.

Modified name.initials() to not use non-word characters as initials.

Modified unidecode_name() to do more normalization, to conform to the
conventions used in internet-drafts.

Added saving of the factory-boy random state in order to be able to re-run
a test suite with the same pseudo-random sequence as in a previous failed
run.

Fixed an issue with email formatting in test_api_submit_ok().

Modified the draft author extraction code to deal better with names with
embedded apostrophes.
 - Legacy-Id: 14141
2017-09-20 15:36:30 +00:00
Henrik Levkowetz 6aa2cfca89 Updated tests to match the submission API info page.
- Legacy-Id: 14127
2017-09-17 14:02:52 +00:00
Henrik Levkowetz 3af2554b2f Added an API for draft submission, at /api/submit. Added an urls.py file under api/ to hold api urls, and moved those from ietf/urls.py. Refactored out many parts of the regular submission forms and functions in submit/forms.py and submit/views.py in order to re-use the appropriate parts for the submission API. Moved support functions to submit/utils.py. Added a new validation errors for missing docName in xml-based submissions. Updated the submission test document templates to use insert additional values. Added failure and success test cases for automated API submissions, and refactored some test utility functions.
- Legacy-Id: 14125
2017-09-16 09:35:42 +00:00
Henrik Levkowetz b42f1cbeb5 Replaced the use of unaccent.asciify(), which has similar functionality to unidecode.unidecode(). Changed the draft parser to work exclusively with unicode text, which both makes the removal of unaccent easier, and takes us closer to Py35 compatibility. Adjusted callers of the draft parser to send in unicode.
- Legacy-Id: 13673
2017-06-18 18:23:18 +00:00
Henrik Levkowetz 98dc0d0092 Only run yanglint and yanglint tests if we have it configured.
- Legacy-Id: 13636
2017-06-15 22:13:43 +00:00
Henrik Levkowetz 4fc2c6f960 Merged in ^/personal/henrik/6.54.2-yang@13630:
Added a new yang checker, 'yanglint', to the existing Yang checker class, in
addition to the existing 'pyang' checker.

Added modal overlay displays showing the yang check results every place the
yin/yang symbol is shown (red or green) to indicate the presencee and result
of yang checks.  Added a Yang Validation: line in the document
meta-information section on the document's page in the datatracker.

Added the result of the xym extaction to the yang check results, to make
extration failures visible.

Added the version of the used xym, pyang, and yanglint commands to the check
results.

Added an action to move successfully extracted and validated modules to the
module library directories immediately on submission.

Added the xym and pyang repositories as svn:external components, rather than
listing them in requirements.txt, as there has been delays of many months
between essential features in the repositories, and an actual release.  We may
get occasional buildbot failures if broken code is pulled in from the
repository, but better that than the functionality failure of severely
outdated componets.

Added a new management command to re-run yang validation for active drafts for
which yang modules were found at submission time, in order to pick up imported
models which may have arrived in the model libraries after the draft's
submission.  Run daily from bin/daily.

Added a table to hold version information for external commands.  The yang
checker output should include the version information of the used checkers,
but seems unnecessary to run each command with its --version switch every
time we check a module...

Added a new management command to collect version information for external
commands on demand.  To be run daily from bin/daily.

Added tests to verify that xym, pyang and yanglint information is available
on the submission confirmation page, and updated the yang module contained in
the test document to validate under both pyang and yanglint.

Updated admin.py and resource.py files as needed.
 - Legacy-Id: 13634
2017-06-15 20:16:48 +00:00
Henrik Levkowetz 207ea51723 Added a test case for invalid yang in a draft submission.
- Legacy-Id: 13633
2017-06-15 19:22:36 +00:00
Henrik Levkowetz d98054c103 Added a new yang checker, 'yanglint', to the existing Yang checker class, in
addition to the existing 'pyang' checker.

Added modal overlay displays showing the yang check results every place the
yin/yang symbol is shown (red or green) to indicate the presencee and result
of yang checks.  Added a Yang Validation: line in the document
meta-information section on the document's page in the datatracker.

Added the result of the xym extaction to the yang check results, to make
extration failures visible.

Added the version of the used xym, pyang, and yanglint commands to the check
results.

Added an action to move successfully extracted and validated modules to the
module library directories immediately on submission.

Added the xym and pyang repositories as svn:external components, rather than
listing them in requirements.txt, as there has been delays of many months
between essential features in the repositories, and an actual release.  We may
get occasional buildbot failures if broken code is pulled in from the
repository, but better that than the functionality failure of severely
outdated componets.

Added a new management command to re-run yang validation for active drafts for
which yang modules were found at submission time, in order to pick up imported
models which may have arrived in the model libraries after the draft's
submission.  Run daily from bin/daily.

Added a table to hold version information for external commands.  The yang
checker output should include the version information of the used checkers,
but seems unnecessary to run each command with its --version switch every
time we check a module...

Added a new management command to collect version information for external
commands on demand.  To be run daily from bin/daily.

Added tests to verify that xym, pyang and yanglint information is available
on the submission confirmation page, and updated the yang module contained in
the test document to validate under both pyang and yanglint.

Updated admin.py and resource.py files as needed.
 - Legacy-Id: 13630
2017-06-15 16:09:28 +00:00
Henrik Levkowetz 55d36e284d Changed the '%s <%s>'%(name,email) idiom in several places to use email.utils.formataddr() instead, to avoid future problems with names that would need quotes. Added comments in 2 places where this change would not be appropriate.
- Legacy-Id: 13536
2017-06-05 22:40:11 +00:00
Henrik Levkowetz 76628be3fd Merged in ^/branch/iola/author-stats-r13145 from olau@iola.dk, and fixed some tests in code which moved after the latest merge with trunk. The test suite passes, but the migrations are _not_ ready to run, because of numbering conflicts (again due to code changes on trunk since the latest sync).
- Legacy-Id: 13479
2017-05-31 20:59:26 +00:00
Ole Laursen 5b677dc6ba Merge author stats branch into new branch from trunk
- Legacy-Id: 13159
2017-03-28 14:36:40 +00:00
Henrik Levkowetz 14c748df98 Changed the creation of temporary test dirs to use names which include the test class name, in order to make it possible to run test cases in parallel. Disabled coverage testing when running in parallel (those classes cannot be pickled and distributed to the test runner threads, and also won't pass coverage data back). Tweaked the TestCase.tempdir() method.
- Legacy-Id: 13157
2017-03-27 21:41:18 +00:00
Lars Eggert 8a70e9773d Fix various "RemovedInDjango20Warning" warnings. Commit ready for merge.
- Legacy-Id: 13112
2017-03-25 19:16:45 +00:00
Henrik Levkowetz 4daf66d11c Added DocEvent rev= parameter througout the code.
- Legacy-Id: 12978
2017-03-06 15:08:21 +00:00
Henrik Levkowetz cf4a4b02a7 Reworked the email address handling in order to be able to support non-ascii names as part of email address fields. Reworked the generation of user names in the test suite to generate names from multiple non-ascii locales. Fixes issue #2080.
- Legacy-Id: 12872
2017-02-18 21:50:18 +00:00
Ole Laursen 90051a1575 Add support for displaying and editing author affiliation and country
when submitting a draft, replace the Submission.authors line-based
text field with a JSON field
 - Legacy-Id: 12745
2017-01-27 16:10:31 +00:00
Ole Laursen 9308948195 Add person, affiliation and country (through django-countries) to
DocumentAuthor, rename author field to email and make it optional (for
modeling old email-less submissions), remove the authors many to many
referencing field from Document as it is not really pointing the right
place.

Update the Secretariat tools to show affiliation and country.

Add migration for getting rid of the fake email addresses that the
migration script created some years ago (just set the author email
field to null).
 - Legacy-Id: 12739
2017-01-26 17:10:08 +00:00
Ole Laursen 76bcab6efc Show formal languages used on new submissions and make the field
editable, also add it to the test with a simple JSON example
 - Legacy-Id: 12662
2017-01-17 13:52:16 +00:00
Ole Laursen 1850a26b23 Add words and formal languages to Submission, set them in the submit
view, clean up submit view a bit to avoid duplicated attribute setting
code
 - Legacy-Id: 12660
2017-01-16 17:37:23 +00:00