Commit graph

623 commits

Author SHA1 Message Date
Henrik Levkowetz 138f3d7382 Fixed a variable initialization issue.
- Legacy-Id: 14009
2017-07-30 14:33:34 +00:00
Henrik Levkowetz 2070a4efcf Fixed a pyflakes issue.
- Legacy-Id: 14008
2017-07-30 13:55:30 +00:00
Henrik Levkowetz 8a53653682 Added detailed line-by line changes in code coverage to the coverage_changes management command. Use verbosity 2 or higher.
- Legacy-Id: 14007
2017-07-30 13:50:06 +00:00
Henrik Levkowetz d00921b94b Added a new release-coverage format which whould make it easier to show line-by-line code test coverage changes in the future.
- Legacy-Id: 13967
2017-07-24 17:38:00 +00:00
Henrik Levkowetz 2de6855ccd Fixed a time arithmetics issue in meeting cutoff date methods
- Legacy-Id: 13966
2017-07-23 14:47:46 +00:00
Henrik Levkowetz 15f8b4504e Change the test telechat dates so that the first active telechat doesn't change to lie in the past if the tests are run across midnight.
- Legacy-Id: 13889
2017-07-15 10:01:35 +00:00
Henrik Levkowetz 86b252c90e Some refactoring of the agenda and minutes validation work.
- Legacy-Id: 13850
2017-07-13 10:42:48 +00:00
Henrik Levkowetz 46fc7b77fd Added more validation of extension, mime type, etc. for uploaded meeting agendas and minutes. Added '.md' (markdown) as an accepted file type. Html with frames is now rejected. Factored out validation code into separate functions.
- Legacy-Id: 13849
2017-07-12 22:45:57 +00:00
Henrik Levkowetz d2395f9521 Merged in [13832] from rjsparks@nostrum.com:
Treat Area Groups similar to Working Groups and Research Groups where it makes sense to do so.
 - Legacy-Id: 13836
Note: SVN reference [13832] has been migrated to Git commit 2c92aa213d
2017-07-11 21:57:01 +00:00
Robert Sparks 2c92aa213d Treat Area Groups similar to Working Groups and Research Groups where it makes sense to do so. Commit ready for merge.
- Legacy-Id: 13832
2017-07-11 19:12:21 +00:00
Henrik Levkowetz 95ddc35eae Added wiki creation for groups in state 'bof'
- Legacy-Id: 13829
2017-07-11 16:06:00 +00:00
Henrik Levkowetz 2c39de61ab Replaced the local copy of the draft/rfc htmlization code with rfc2html as installed modeule.
- Legacy-Id: 13827
2017-07-11 13:52:08 +00:00
Henrik Levkowetz eceed7476d Added the html_report switch setting to the test runner object.
- Legacy-Id: 13822
2017-07-09 18:10:51 +00:00
Henrik Levkowetz ca921fe45d Added a switch to control generation of the code coverage html report, which now defaults to false. The report can be generated post-testing with "coverage html -d static/coverage/".
- Legacy-Id: 13821
2017-07-09 16:50:38 +00:00
Henrik Levkowetz d26c7085c6 Added code coverage settings and configuration to ignore debug-related lines of code.
- Legacy-Id: 13817
2017-07-09 15:00:22 +00:00
Henrik Levkowetz 42b1a41227 Added the new format '4' to the list of recognized formats in the coverage_changes management command.
- Legacy-Id: 13766
2017-07-03 21:18:46 +00:00
Henrik Levkowetz 97a5ed1f6c Added a test to verify that we can load the 500 page template
- Legacy-Id: 13765
2017-07-03 21:17:45 +00:00
Henrik Levkowetz fb88596c59 Fixed an issue with the scope of the logfunc variable in production.
- Legacy-Id: 13761
2017-07-03 14:03:53 +00:00
Henrik Levkowetz db96292229 Added a utility function to show just the text content of the test client response.
- Legacy-Id: 13737
2017-06-30 11:45:24 +00:00
Henrik Levkowetz 2070bbfed6 Merged in [13722] and [13712] from rcross@amsl.com:
Remove all use of request.session from secretariat apps.  Add tests for affected views.  Fixes #1455.
 - Legacy-Id: 13724
Note: SVN reference [13712] has been migrated to Git commit 915b445bcffcd4687fb1677230b98282d22dd9a0

Note: SVN reference [13722] has been migrated to Git commit 3a2cb9d3d846168f75d9796b4e5383bdf10b1f60
2017-06-28 14:55:10 +00:00
Henrik Levkowetz f6f0278859 Changed the url coverage code to handle deeper chains of url includes, and to handle url includes through url lists in addition to url modules. Added information in the release coverage dictionaries about the view functions coupled to the urls.
- Legacy-Id: 13706
2017-06-22 15:07:06 +00:00
Henrik Levkowetz 5e80e00aea Corrected the settings names used in the yang extraction command.
- Legacy-Id: 13695
2017-06-20 14:06:42 +00:00
Henrik Levkowetz b42f1cbeb5 Replaced the use of unaccent.asciify(), which has similar functionality to unidecode.unidecode(). Changed the draft parser to work exclusively with unicode text, which both makes the removal of unaccent easier, and takes us closer to Py35 compatibility. Adjusted callers of the draft parser to send in unicode.
- Legacy-Id: 13673
2017-06-18 18:23:18 +00:00
Henrik Levkowetz be80e68b47 Corrected the spelling of norwegian words in unaccent.py
- Legacy-Id: 13671
2017-06-18 09:33:51 +00:00
Henrik Levkowetz a18cf20868 Removed code which has been marked as unreachable since February/March 2017 without triggering any usage indications.
- Legacy-Id: 13667
2017-06-17 17:54:40 +00:00
Henrik Levkowetz e6c8b6621c Changed mentions of people with roles on the various group pages to link to both their profile page and their email address.
- Legacy-Id: 13664
2017-06-17 17:05:38 +00:00
Henrik Levkowetz 34a2352288 Make sure wordwrap() and friend works as intended if they are used as template filters and given string arguments.
- Legacy-Id: 13653
2017-06-16 13:15:02 +00:00
Henrik Levkowetz 8d420d4d2f Added initial data to the versioninfo migration.
- Legacy-Id: 13638
2017-06-16 08:55:31 +00:00
Henrik Levkowetz c4b64e51af Updated the VersionInfo migration.
- Legacy-Id: 13637
2017-06-16 08:45:11 +00:00
Henrik Levkowetz d98054c103 Added a new yang checker, 'yanglint', to the existing Yang checker class, in
addition to the existing 'pyang' checker.

Added modal overlay displays showing the yang check results every place the
yin/yang symbol is shown (red or green) to indicate the presencee and result
of yang checks.  Added a Yang Validation: line in the document
meta-information section on the document's page in the datatracker.

Added the result of the xym extaction to the yang check results, to make
extration failures visible.

Added the version of the used xym, pyang, and yanglint commands to the check
results.

Added an action to move successfully extracted and validated modules to the
module library directories immediately on submission.

Added the xym and pyang repositories as svn:external components, rather than
listing them in requirements.txt, as there has been delays of many months
between essential features in the repositories, and an actual release.  We may
get occasional buildbot failures if broken code is pulled in from the
repository, but better that than the functionality failure of severely
outdated componets.

Added a new management command to re-run yang validation for active drafts for
which yang modules were found at submission time, in order to pick up imported
models which may have arrived in the model libraries after the draft's
submission.  Run daily from bin/daily.

Added a table to hold version information for external commands.  The yang
checker output should include the version information of the used checkers,
but seems unnecessary to run each command with its --version switch every
time we check a module...

Added a new management command to collect version information for external
commands on demand.  To be run daily from bin/daily.

Added tests to verify that xym, pyang and yanglint information is available
on the submission confirmation page, and updated the yang module contained in
the test document to validate under both pyang and yanglint.

Updated admin.py and resource.py files as needed.
 - Legacy-Id: 13630
2017-06-15 16:09:28 +00:00
Henrik Levkowetz 9bd64ba6f2 Merged in [13559] from rcross@amsl.com:
When changing a TelechatDate in the admin interface, retain all document associations.  Fixes #1349.
 - Legacy-Id: 13577
Note: SVN reference [13559] has been migrated to Git commit 77f4bf21ff
2017-06-10 05:33:36 +00:00
Henrik Levkowetz 4dcdda08d2 Added a test to the test suite which checks if there are interleaved schema and data migrations that have not been released yet. Having split migrations, first all schema and then all data, will permit doing 2 sequential releases; but if the data and schema migrations are interleaved (beyond this) it is problematic to arrange for release without ending up with a prolonged period when running code and table structure is out of sync (while the normally more time-consuming data migrations run).
- Legacy-Id: 13576
2017-06-09 21:37:14 +00:00
Henrik Levkowetz 7385a8be51 Changed name and signature for a log helper.
- Legacy-Id: 13571
2017-06-09 19:38:48 +00:00
Henrik Levkowetz 023a32715d Merged in ^/branch/iola/author-stats-r13145 from olau@iola.com, with additional features from ^/personal/henrik/6.52.1-authorstats.
- Legacy-Id: 13550
2017-06-06 18:36:59 +00:00
Henrik Levkowetz a34d078428 Commented out again a function that was commented in by mistake in the committed code.
- Legacy-Id: 13543
2017-06-06 16:30:53 +00:00
Henrik Levkowetz 5ca4309691 Fixed a bug in wordwrap() where an URL (or any word) longer than width could prevent line breaking in following text.
- Legacy-Id: 13541
2017-06-06 14:03:02 +00:00
Henrik Levkowetz fb93a8c293 Added dates to unreachable() calls, to make it easier to decide when to remove marked code.
- Legacy-Id: 13538
2017-06-06 07:27:07 +00:00
Henrik Levkowetz b2da91de1b Added a middleware exception handler for the case of people submitting unicode outside the Basic Multilingual Plane, which cannot currently be saved to the database.
- Legacy-Id: 13508
2017-06-03 21:47:51 +00:00
Henrik Levkowetz 5b2087f910 Eliminated several variations on word wrapping, keeping only what used to be wrap_text(), but renamed as ietf.utils.text.wordwrap(). This performs better than django.utils.text.wrap() when there are indented text parts. Replaced django's default wordwrap filter with one calling ietf.utils.text.wordwrap in templates. Changed to triggered wrapping in some cases, with the maybewordwrap filter, which triggers on lines longer than 100 characters. This fixes the issue with undesired wrapping of reviews.
- Legacy-Id: 13505
2017-06-02 23:13:22 +00:00
Henrik Levkowetz 16d129cacf Added examples of how our different text wrapper functions work to ietf.utils.text. Run 'python ietf/utils/text.py | less' to see the results.
- Legacy-Id: 13498
2017-06-02 17:59:26 +00:00
Henrik Levkowetz 76628be3fd Merged in ^/branch/iola/author-stats-r13145 from olau@iola.dk, and fixed some tests in code which moved after the latest merge with trunk. The test suite passes, but the migrations are _not_ ready to run, because of numbering conflicts (again due to code changes on trunk since the latest sync).
- Legacy-Id: 13479
2017-05-31 20:59:26 +00:00
Henrik Levkowetz fe30afdfe7 Updated the check_referential_integrity management command to show progress bars when not in verbose mode.
- Legacy-Id: 13369
2017-05-16 15:07:56 +00:00
Henrik Levkowetz b5319c576a Updated the utils.DumpInfo model to match the database.
- Legacy-Id: 13330
2017-05-12 15:27:23 +00:00
Henrik Levkowetz 38bfdb4095 Fixed a bug in the earlier author extraction bugfix.
- Legacy-Id: 13295
2017-05-10 12:21:17 +00:00
Henrik Levkowetz fb70e9a4ff Fixed an issue with the author extraction code.
- Legacy-Id: 13288
2017-05-09 19:19:55 +00:00
Henrik Levkowetz b1c4b75fbe Fixed a pyflakes issue.
- Legacy-Id: 13235
2017-04-18 20:55:43 +00:00
Henrik Levkowetz 0b730c9554 Updated the referential integrity checking comand.
- Legacy-Id: 13234
2017-04-18 20:21:58 +00:00
Henrik Levkowetz d6b9392f72 Fixed an if-statement which incorrectly disabled coverage testing when the parallel testing count was set to 1.
- Legacy-Id: 13231
2017-04-18 16:47:28 +00:00
Henrik Levkowetz 978e8f72f5 Changed the signature of ietf.utils.aliases.dump_sublist() to include a list of the alias domains, in order to be able to generate IRTF group aliases @irtf.org. Updated ietf/bin/generate-*-aliases accordingly.
- Legacy-Id: 13206
2017-04-12 18:51:38 +00:00
Henrik Levkowetz d19179ba44 Merged in [13181] from housley@vigilsec.com:
This completes a good chunk of the downref registry work requested in ticket
#2069. The registry has been imported to the database and can be shown, and
entries can be added to the registry.  Addresses issue #2069.
 - Legacy-Id: 13190
Note: SVN reference [13181] has been migrated to Git commit f0125634f8
2017-04-10 22:47:01 +00:00
Russ Housley f0125634f8 Add downref registry to the datatracker at the request of the IESG. Still need to integrate with Last Call message generation
- Legacy-Id: 13181
2017-04-03 17:36:27 +00:00
Ole Laursen d9585b4b73 Add some extra test data to exercise the statistics better
- Legacy-Id: 13167
2017-03-29 09:06:12 +00:00
Ole Laursen a1ba9dfb0a Remove accidentally committed debug
- Legacy-Id: 13160
2017-03-28 14:56:06 +00:00
Ole Laursen 5b677dc6ba Merge author stats branch into new branch from trunk
- Legacy-Id: 13159
2017-03-28 14:36:40 +00:00
Henrik Levkowetz 14c748df98 Changed the creation of temporary test dirs to use names which include the test class name, in order to make it possible to run test cases in parallel. Disabled coverage testing when running in parallel (those classes cannot be pickled and distributed to the test runner threads, and also won't pass coverage data back). Tweaked the TestCase.tempdir() method.
- Legacy-Id: 13157
2017-03-27 21:41:18 +00:00
Henrik Levkowetz 605fa26ec3 Made relative paths work for the create_group_wikis commnand. Added wiki and svn dir patterns to docker/settings_local.py.
- Legacy-Id: 13150
2017-03-27 17:07:48 +00:00
Ole Laursen 5608ad0061 Move the assert on new X in the test coverage up before the percentage
to try get the list of URLs out before the percentage
 - Legacy-Id: 13144
2017-03-27 08:45:20 +00:00
Ole Laursen ef4d55f0c9 Apply patch from Henrik Levkowetz to fix some problems of author parse
errors where the affiliation is mistakenly thought to be an extra
author (some of these still remain)
 - Legacy-Id: 13142
2017-03-27 08:33:49 +00:00
Henrik Levkowetz d2c2d80341 Merged in [13112] from lars@netapp.com:
Fix various \'RemovedInDjango20Warning\' warnings.
 - Legacy-Id: 13141
Note: SVN reference [13112] has been migrated to Git commit 8a70e9773d
2017-03-26 22:06:47 +00:00
Lars Eggert 8a70e9773d Fix various "RemovedInDjango20Warning" warnings. Commit ready for merge.
- Legacy-Id: 13112
2017-03-25 19:16:45 +00:00
Henrik Levkowetz 3f92aee755 Added the rfcmarkup module. This will be changed to a library dependency when the module is published.
- Legacy-Id: 13038
2017-03-20 13:48:13 +00:00
Henrik Levkowetz 53481eebd8 Added a utility function to create a test-class specific temporary directory.
- Legacy-Id: 13026
2017-03-19 19:25:26 +00:00
Henrik Levkowetz bb5096da8a Added more email validation for draft submission author emails, and blocked some baths that could lead to bad email addresses ('none') being set as document author email addresses.
- Legacy-Id: 13010
2017-03-14 14:07:21 +00:00
Henrik Levkowetz 7b3ce6e795 Improved the log.assertion() and unreachable() functions to send a deeper traceback when reporting by email, rather than just the last stack frame.
- Legacy-Id: 12999
2017-03-13 05:37:02 +00:00
Henrik Levkowetz fe8cd06a76 Back out the htmlization code for the moment.
- Legacy-Id: 12997
2017-03-12 12:12:44 +00:00
Henrik Levkowetz 319a4850d6 Added rfcmarkup code. Will probably be migrated to a separate lib in time.
- Legacy-Id: 12994
2017-03-11 14:02:16 +00:00
Henrik Levkowetz 58dae78a43 Modified the REST API to provide group descriptions taken from the group charters if group.description is blank.
- Legacy-Id: 12993
2017-03-11 13:46:19 +00:00
Henrik Levkowetz 4daf66d11c Added DocEvent rev= parameter througout the code.
- Legacy-Id: 12978
2017-03-06 15:08:21 +00:00
Henrik Levkowetz ff31644ccf Tweaked assertion() and unreachable() to raise an exception instead of sending email (to the test mailbox ...) when running the test suite.
- Legacy-Id: 12977
2017-03-06 14:31:45 +00:00
Henrik Levkowetz 0bcb4ae30c Fixed a problem with assertion() and unreachable(), which could break if certain names (like 'args') were set in the calling frame.
- Legacy-Id: 12975
2017-03-06 13:51:08 +00:00
Henrik Levkowetz 3c6f00869f Dead code removal: meeting models and test_utils.
- Legacy-Id: 12965
2017-03-04 05:19:39 +00:00
Henrik Levkowetz 49ba774ae3 Updated unreachable() marking.
- Legacy-Id: 12962
2017-03-04 05:14:51 +00:00
Henrik Levkowetz 3a21e03f8d Marked a function as unrachable. Added support in send_mail_message for the Message.msgid field.
- Legacy-Id: 12954
2017-03-03 15:43:43 +00:00
Henrik Levkowetz ea70381f29 Merged in [12950] from rcross@amsl.com:
Fixes #2219.  Area Group chairs are denied material management access.
 - Legacy-Id: 12951
Note: SVN reference [12950] has been migrated to Git commit c6d1a59aa4
2017-03-02 21:21:07 +00:00
Henrik Levkowetz 5bb5ca727b Created a special function log.unreachable() to mark code unreachable, and changed a function name from log.affirm() to log.assertion().
- Legacy-Id: 12933
2017-02-28 20:16:52 +00:00
Henrik Levkowetz 5928bd9faa Added svn symlink check and creation to test setup. Forced test settings to use an empty INTERNAL_IPS list for consistency in code coverage figures.
- Legacy-Id: 12929
2017-02-28 18:02:30 +00:00
Henrik Levkowetz 4f12513b1f Tweaked the affirm() failure message.
- Legacy-Id: 12925
2017-02-27 16:42:23 +00:00
Henrik Levkowetz 5f053ad21a Cleaned up the remaining explicit url names, using dotted-paths to view
functions instead.  In all almost 700 changes.
 - Legacy-Id: 12923
2017-02-26 23:21:49 +00:00
Henrik Levkowetz 9fb1a8e05f Tweaked an import to make Python 2to3 able to handle it.
- Legacy-Id: 12922
2017-02-26 10:54:33 +00:00
Henrik Levkowetz 84527c9df6 Added some code to utils.urls.url() to output remaining explicit url names together with the matching dotted-path to the view function.
- Legacy-Id: 12921
2017-02-25 20:32:20 +00:00
Henrik Levkowetz 49af9b4e1a Added an affirm(statement_string) function, which acts like an assert, and lets us trigger an email message with traceback when the statement to affirm is False. Use instead of assert if we still want the server to produce a response, even if something is seriously wrong.
- Legacy-Id: 12901
2017-02-23 11:48:55 +00:00
Henrik Levkowetz 988a2b808f Added explicit charset setting and consistent encoding to several email paths through the datatracker. After introducing support for non-ascii names in email addresses, using role.formatted_email() to insert names into email content doesn't work any more, since uncode names will now be rfc2047-encoded in formatted_email(). Added another method role.name_and_email() for this case. Replaced another case of '%s <%s>' name-and-email formatting with formataddr(). Fixed some tests which did not send unicode email bodies to the email functions. Fixes issue #2017.
- Legacy-Id: 12896
2017-02-21 19:17:10 +00:00
Henrik Levkowetz c889856dfa Added a validator to Document.title to prevent control chars in the title (complementing a recent cleanup of more than 120 instances of document titles containing vertical tabs)
- Legacy-Id: 12893
2017-02-20 20:47:37 +00:00
Henrik Levkowetz 858d855eff Changed the new account creation to require a proper name at the same time as the account password is set, before actually creating the account. Also tweaked the password strength and confirmation code.
- Legacy-Id: 12892
2017-02-20 16:50:39 +00:00
Henrik Levkowetz 8930d29a8e Merged in Django-1.10 upgrade work from ^/personal/henrik/6.43.1-django-1.10
- Legacy-Id: 12881
2017-02-19 18:18:00 +00:00
Henrik Levkowetz cf4a4b02a7 Reworked the email address handling in order to be able to support non-ascii names as part of email address fields. Reworked the generation of user names in the test suite to generate names from multiple non-ascii locales. Fixes issue #2080.
- Legacy-Id: 12872
2017-02-18 21:50:18 +00:00
Henrik Levkowetz c344a18bdf Fixed an issue with the test-crawler which could cause false positives for urls containing apostrophe.
- Legacy-Id: 12851
2017-02-16 09:58:34 +00:00
Ole Laursen d2e85a3aa3 Apply draft parser patch from Henrik to improve the patch on trunk to
combine paragraphs across page splits - this makes the country part of
the parser find more countries
 - Legacy-Id: 12848
2017-02-15 19:10:59 +00:00
Ole Laursen b2ff10b0f2 Add support for extracting the country line from the author addresses
to the draft parser (incorporating patch from trunk), store the
extracted country instead of trying to turn it into an ISO country
code, add country and continent name models and add initial data for
those, add helper function for cleaning the countries, add author
country and continent charts, move the affiliation models to
stats/models.py, fix a bunch of bugs.
 - Legacy-Id: 12846
2017-02-15 18:43:57 +00:00
Ole Laursen 882579bab3 Don't check url reverses on the form admin:blahblah for the time being
- Legacy-Id: 12845
2017-02-15 18:39:35 +00:00
Henrik Levkowetz 44ad914fba Tweaked the company name extraction code in class Draft.
- Legacy-Id: 12842
2017-02-15 14:09:54 +00:00
Henrik Levkowetz bb5e5b97ba Another tweak to handle page break paragraph joins better in class Draft.
- Legacy-Id: 12840
2017-02-14 17:41:30 +00:00
Henrik Levkowetz 6158221fa8 Tweaked the author extraction to recognize short lines as paragraph ends, not only lines ending in '.' or ':'
- Legacy-Id: 12837
2017-02-14 14:23:15 +00:00
Henrik Levkowetz 7f607c51d2 Changed from old-style to new-style middleware.
- Legacy-Id: 12824
2017-02-12 17:07:48 +00:00
Henrik Levkowetz a1606e8f7d Added a 'htmlfilters' tag library containing 'removetags'
- Legacy-Id: 12823
2017-02-12 17:05:55 +00:00
Henrik Levkowetz aa5e61d958 Updated all urlpatterns to use ietf.utils.urls.url() instead of django's,
in order to autogenerate dotted path url pattern names.  Updated a number
of url reverses to use dotted path, and removed explicit url pattern names
as needed.

Changed some imports to prevent import of ietf.urls before django
initialization was complete.


Changed 3 cases of form classes being curried to functions; django 1.10
didn't accept that.

Started converting old-style middleware classes to new-style middleware
functions (incomplete).

Tweaked a nomcom decorator to preserve function names and attributes, like
a good decorator should.

Replaced the removed django templatetag 'removetags' with our own version
which uses bleach, and does sanitizing in addition to removing explicitly
mentionied html tags.

Rewrote the filename argument handling in a management command which had
broken with the upgrade.
 - Legacy-Id: 12818
2017-02-11 14:43:01 +00:00
Henrik Levkowetz 19d585255a Django 1.10 does not permit the view argument to url() to be a string any more. Changed all cases view arguments to url() which were strings to instead reference the actual imported function.
- Legacy-Id: 12815
2017-02-10 14:49:16 +00:00
Henrik Levkowetz 49a899fe66 Added support for management of trac wikis for explicitly named groups, such as the IESG and IAOC. Tweaked the dummy-run handling of the create_group_wikis management command.
- Legacy-Id: 12789
2017-02-08 16:37:31 +00:00
Henrik Levkowetz de3976bec0 Merged in Django 1.9 upgrade work from ^/personal/henrik/6.41.1-django-1.9
- Legacy-Id: 12764
2017-01-31 15:13:01 +00:00
Henrik Levkowetz 55b430b525 Merged in [12748] from rcross@amsl.com:
Fixes #2172.  Add Requested Resources and People who must be there, to email notifications of session requests.
 - Legacy-Id: 12757
Note: SVN reference [12748] has been migrated to Git commit eeb35e997d
2017-01-30 23:22:45 +00:00