ietfdb (6.80.1) ietf; urgency=medium * Merged in [15153] from rjsparks@nostrum.com: Changed how the mail utilities handle lists in their override and extra arguments. Added a test for send_mail_preformatted exercising override and extra. * Merged in [15152] from rjsparks@nostrum.com: Send email when the responsible AD for a conflict review is changed. Include the stream manager and stream steering group when the responsible AD changes or when the telechat date changes. Fixes #2497. * Merged in [15151] from rjsparks@nostrum.com: Provide an email group button for WG and RG documents. Fixes #2512. * Merged in [15145] from rjsparks@nostrum.com: Secretariat requested tweak to the request for minutes template. * Updated the non-WG mailing list page to include advertised lists for closed WGs, on request from the secretariat. -- Henrik Levkowetz 23 May 2018 12:55:09 +0000 ietfdb (6.80.0) ietf; urgency=medium This release fixes a number of bugs, and adds a view that makes it easier for the secretariat to generate the meeting minutes request emails. * Merged in [15125] from rjsparks@nostrum.com: Added a view that generates the 'send minutes' email for the secretariat. Fixes #2493. * Merged in [15124] from rjsparks@nostrum.com: Provide a __unicode__ member for ImportantDate. * Merged in [15123] from rjsparks@nostrum.com: Use importantdates for material submisssion cutoff and revision if they exist. Fixes #2514. * Merged in [15119] and [15120] from rjsparks@nostrum.com: Change stream state to replaced when an ISE, IAB, or IRTF document is replaced. Fixes #1963. * Merged in [15114] from rjsparks@nostrum.com: Restored a test that was unintentionally removed. Removed unused templates. Related to #2513. * Merged in [15113] from rjsparks@nostrum.com: Removed the already disabled 'Make RFC' functionality from /secr/drafts. Fixes #2513. Merged in [15112] from rjsparks@nostrum.com: Removed the 'announce' functionality from /secr/drafts. Addresses part of #2513. * Merged in [15111] from rjsparks@nostrum.com: Removed the \'replaces\' functionality from /secr/drafts. Addresses part of #2513. * Merged in [15110] from rjsparks@nostrum.com: Made settings.URL_REGEXPS.name non-greedy. Added a test to exercise the bug at #2429. Fixes #2429. * Fixed an incorrect file name in a log message. -- Henrik Levkowetz 08 May 2018 06:37:38 +0000 ietfdb (6.79.1) ietf; urgency=medium This bugfix release fixes problems with IESG ballot clearing, marks dead code as unreachable for later removal, adds new tests to increase test coverage, and fixes some issues with the test suite. It also tweaks the roles which are permitted to request document publication, loosens the need for trailing slashes for some URLs, and adds a 'Recent drafts' page. * Merged in [15084] from rjsparks@nostrum.com: Corrected the implementation of clear_ballot. Added a test for it. Restored functionality to ballot popups that was missing because a template variable was not passed in. * Added a 'recent drafts' page, showing drafs submitted the last 7 days. * Merged in [15083] from rjsparks@nostrum.com: Removed memoize from latest_event. * Merged in [15080] from rjsparks@nostrum.com: Marked a couple of functions as unreachable. * Reverted a change to UserFactory to avoid nepalese names. The issue is bigger, and needs a different resulution. Changed the url pattern that used \w to match names to instead use a more inclusive regex. * Fixed a PyQuery property access that should have been a method call. * Merged in [15076] from rjsparks@nostrum.com: Added a test for meeting/requests. * Tweaked the IPR factory to be less likely to produce invalid patent info. * Merged in [15075] from rjsparks@nostrum.com: Improved testing for docs_for_ad. * Merged in [15074] from rjsparks@nostrum.com: Simplified docs_in_iesg_process. Added a test for it. * Merged in [15073] from rjsparks@nostrum.com: Rearranged ietf/doc/urls.py to allow matching some special pages under /doc without a trailing slash. Adjusted several of those special URLs to work without the trailing slash. * Added permission for a document shepherd to request publication. Fixes issue #2509. -- Henrik Levkowetz 02 May 2018 10:42:26 +0000 ietfdb (6.79.0) ietf; urgency=medium **IPR patent info field fixes, IESG position API enhancement, and more.** This release enhances the IESG position API to send notification emails on changes, widens some IPR disclosure form fields to accept more text, and fixes a patent application number validation issue for IPR forms. It adds some useful links and a new filtering option to the past meetings page, updates document stream status indications so as to be less prone to misinterpretation, addresses an issue with session request forms, and fixes an issue with I-D expirations and another related to ordering of documents on IESG agendas. From the commit log: * Set file upload permissions to 0644 to avoid problems with photo access. * Increased the length of the patent title in IPR forms * Fixed a typo. Fixes issue #2500 which caused different document ordering on the IESG agenda compared with the IESG agenda documents page. * Increased the size of the patent_notes form field, and tweaked the validate_patent_number validator to accept US application numbers containing a series number and forward slash. * Merged in [15055] from rcross@amsl.com: Fixed an issue where user can make two session request submissions, corrupting data. Also, ignore deleted session requests in edit view. * Merged in [15047] from rcross@amsl.com: Reduced the get_meeting buffer to handle earlier shceduling requests date. * Added origin indication to a group of stats templates to make it easier to find the origin of stats configurations. * Added group links to the past meetings page, and also important dates links to IETF meetings with important-date pages. * Added IETF to past meetings filter pane, in order to be able to filter out interims. * Fixed a problem with the expire-ids code, and added logging to catch future problems. * Modified the IESG position setting API to send email messages about positions set. * Tweaked the wording of coverage_changes output. * Fixed an issue where simplified sorting of DocEvents resulted in some ballot icons not being shown on the IESG agenda documents page. * Changed incorrect use of 404 status code in api to 400. Fixes issue #2498. * Removed what could be a misleading 'RFC' from the stream indication in document search results. -- Henrik Levkowetz 23 Apr 2018 12:28:57 +0000 ietfdb (6.78.0) ietf; urgency=medium **Document search on IRTF state, performance improvements** This release brings document search support for filtering on IRTF states, courtesy of Mat Ford, and further performance improvements to the IESG agenda document page and document search pages. There are also improvements to the testing infrastructure, HTML cleanup, some added API documentation, and a number of bugfixes, as follows: * Merged in [15012] from ford@isoc.org: Added IRTF State option to Additional Search Criteria pulldown. * Included django.utils.html.escape() in the linkify() code, and removed it from various templates, to make sure escape is consistently applied before linkify. Fixes issue #2492. * Tweaked the error return from Document.get_document_content() to be more helpful during development. * Fixed various html issues. * Added more information about personal API keys to the API help page. * Tweaked the test runner to more consistently force the test coverage tests to be run last. * Changed calls to .related_that*() and .relations_that*() to use tuples rather than lists (a slight optimization). * Silenced another deprecation warning from a library. * Reduced the max number of results returned by document searches from 500 to 200, for a better user experience. A faster return and a search term tweak to pinpoint the desired search result is better than a long wait and then maybe having to tweak the search terms anyway. * Use .assertIn(...) in tests instead of .assertTrue(x in y) as the failure messages are much more informative * Changed the signature of some document methods to not accept lists (but still accept tuples) in order to make caching possible. Added @memoize to some methods. * Added another prefetch to search doc query results. * Imposed sort order (most recent docs first) on document search results, in order to retain the most recent docs in case we have to chop the result list due too many hits. * Tweaked the django_cprofile_middleware patch to use a query arg of psort instead of sort, to avoid conflict with app query args 'sort'. * Added back the on-agenda calendar links to document lists, now in an implementation with better performance characteristics. * Updated minimum requirements for bootstrap3, to work with django 1.11 * Fixed an issue where the change of slide naming between IETF 96 and 97 caused the datatracker to not provide correct link to pre-IETF-97 materials. Thanks to tjw.ietf@gmail.com for the heads-up. * Merged in [14970] from rjsparks@nostrum.com: Factories for the ipr app. Coverage of ietf.ipr.views.ajax_search, and a factory-driven test of ietf.ipr.views.edit. * Enhanced debug.py: made the mark() function indicate file and line number it was called from. Added if debug: conditions around some other function actions. * Another round of iesg page optimizations. Reduced additional instances of individual lookups to a single lookup followed by annotation of document objects. Reduced several instances of repeating the same work multiple times (queries for telechat documents, queries for telechat dates). * Fixed a number of tests to deal correctly with model method caching. * Changed the ballot_icon template tag to use an already found ballot if available, rather than doing lookups to find it (twice!). * Added a patch for debug mode, to add a filtering option to the django-cprofile-middleware. * Be more specific with bootstrap's layout classes for small devices in the base template, to avoid the bottom links and footer overlaying the bottom of the content area. Fixes a problem with entering login information on small devices. -- Henrik Levkowetz 10 Apr 2018 16:33:46 +0000 ietfdb (6.77.1) ietf; urgency=medium **Performance improvements** This release brings performance improvements, focused in particular on the IESG agenda documents page. Some of the changes will also provide performance benefits for other pages, in particular pages that lists documents, such as search result pages and WG/RG document lists. All in all, the changes speed up the rendering of /iesg/agenda/documents with a factor of about 2. The page generation time is still roughly proportional to the number of documents on the page, so a large number of documents on future IESG telechats will still give a slow page. More substantial refactoring will probably be needed for further speedups. From the commit log: * Added an index for (doc, type) on DocEvent to speed up latest_event() queries. * Changed the reviewed_by_teams list to provide acronyms directly, to avoid group lookups during template rendering. * Changed telechat_page_count() to accept a list of documents if such a list has already been generated, to avoid double work. * Provided telechat page counts to the template from the view, instead of repeated (costly) filtering through the telechat_page_count template filter in the template, which caused new, redundant document lookups every time. Removed the telechat_page_count template filter altogether, due to its cost. * Tweaked some lookups in fill_in_document_table_attributes(). * Added terms to the select_related() list for documents in IESG agenda_documents(). * Added a get_active_ads() utility function that uses the cache to reduce database access, and replaced database queries in various places with calls to get_active_ads(). * Moved some utility functions so as to reduce the likelihood of import * loops. * Changed some instances of .slug to _id, to avoid unnecessary database lookups. * Added a profiling middleware to development settings, if installed. -- Henrik Levkowetz 02 Apr 2018 11:55:07 +0000 ietfdb (6.77.0) ietf; urgency=medium **Automation API endpoint: /api/meeting/session/video/url** This release provides provides a new API endpoint to let video urls be associated with sessions, intended for use by Meetecho to update the datatracker with sesssion recording links as soon as a recording has been uploaded to the YouTube (or equivalent). It also introduces a new 'Recording Manager' role, used to authorize the API access. A short description of the API is available at https://datatracker.ietf.org/api#session-video-url-api -- Henrik Levkowetz 30 Mar 2018 19:11:56 +0000 ietfdb (6.76.2) ietf; urgency=medium This release fixes a number of blemishes and cleans the slate for merging in a meetecho API branch. * Fixed an issue which could result in server 500 errors for paths like /meeting/99/session/plenary. * Removed superfluous stats pages headers which would be repeated with different layout on the next line, as part of the graphs. * Added a test for IPR edit. * Filled in the IPR edit form, section V, with patent information derived from the patent_info field, the same way we do for the IPR update form. * Added a pip version restriction to requirements.txt, as there seems to be issues with pip 9.0.2 and 9.0.3; they seem to leave dross when down- and upgrading packages. * Don't show links to individual meeting stats pages when there's no content (meeting 71 and earlier currently has no attendance records). * Added meeting dates to the meeting stats overview graph. * Changed the name of a loop variable so as not to obscure a previous local variable. * Don't show future meetings in the meeting stats overview. * Changed settings so that cache keys used by memcached includes the release version, in order to avoid stale and incorrect cache content on new release deployment. Made it easier to pick out cache key composition by normalizing the variable name. Adjusted cache prefix and the composition of some keys to have a unique and consistent cache key prefix. * Use the actual defaults from settings.py when showing the defaults on /accounts/settings/. -- Henrik Levkowetz 28 Mar 2018 20:54:53 +0000 ietfdb (6.76.1) ietf; urgency=medium **IETF 101 code sprint, release 2** This release provides improved meeting attendance statistics (under /stats/meeting/overview/), improved test coverage, additional menu entries, html template fixes, total page count on the Last Call page, the ability for ADs to clear ballots themselves without going through the secretariat, and more. It also changes the default cookie setting for showing or hiding the duplicate left-hand menu column. (If you don't like the new default, use the 'User>Preferences' menu item to revert to the old look.) Again, many thanks to all the contributors! From the commit log: * Fixed some issues with agenda and proceedings code for meetings numbered 64 and below, actualized by the introduction of meeting records for such meetings for stats purposes. * The template filter 'keep_spacing' can interact badly with 'wordwrap'. Removed it from several places, and made text document display more consistent. Fixes issue #2481. * Merged in [14902] from rjsparks@nostrum.com: Test coverage improvements for ietf/secr/telechat/views. * Merged in [14900] from rjsparks@nostrum.com: Improvements to test coverage for ietf/secr/drafts/views.py. * Merged in [14899] from rjsparks@nostrum.com: Cleaner charter factory simplifies test code. * Added submenu to the Statistics menu entry, pointing at the individual stats pages, and entries for NomComs and Releases. * Merged in [14896] from rjsparks@nostrum.com: Improved test coverage for ietf/doc/views_charter.py. * Fixed the base template to have id='content' for the content also when not showing the left-hand menu. * Updated the cookies tests to match the new left_menu default. * Changed the default setting for the datatracker left-hand menu to 'off'. The top menu is always present, and works well. * Tweaked the document list iCalendar links to not show for presentations on days in the past. * Merged in [14884] from adam@nostrum.com: Adding page count to last call page. * Merged in [14883] from adam@nostrum.com: Adding link to documents in last call to the AD dashboard side menu. * Merged in [14849] from ben@nostrum.com: Give ADs the red Clear Ballot button * Merged in [14880] from rjsparks@nostrum.com: Added a Draft test suite. * Merged in [14879] from kivinen@iki.fi: Backfilled IETF meetings starting from 1 and backfilled the attendee numbers for them. Modified the overview attendee statistics page to use different colors for different venue continents. * Tweaked the meeting registratoin import to deal more gracefully with too long names and email addresses, and to fill in the total attendees number for the meeting. * Tweaked mergeready to accept an additional hold-for-merge file outside the repository. -- Henrik Levkowetz 26 Mar 2018 13:38:04 +0000 ietfdb (6.76.0) ietf; urgency=medium **IETF 101 code sprint, release 1** This release contains datatracker bugfixes and enhancements from the IETF-101 Code Sprint. It brings new views to merge duplicate person records, fixes an annoying bug in downref identification and a bunch of links that didn't point to the right document, corrects the mime-type for JSON responses, and brings substantial improvments to test coverage. Many thanks to everyone who contributed! * Merged in [14864] from rjsparks@nostrum.com: Improved test coverage for ietf.meeting.views. * Updated Note Well. Fixes issue #2474. * Tweaked person factory ascii name generation to not include periods inside names. * Tweaked a factory to make duplicate mailing list names less likely in tests * Test data template tweak. * Merged in [14863] from rjsparks@nostrum.com: Improved test coverage for ietf.utils.draft. * Merged in [14862] from rcross@amsl.com: Add view for merge person records. * Merged in [14855] from rjsparks@nostrum.com: Improved test coverage for ietf.meeting.views. * Added a test case for reference extraction. * Merged in [14854] from rjsparks@nostrum.com: Improved test coverage over meeting.views. * Merged in [14851] from housley@vigilsec.com: Improve parser for references in Internet-Drafts. Fixes #2360 * Merged in [14850] from rjsparks@nostrum.com: Improvements to test coverage in ietf.meeting.views * Removed premature save of upload document state (doing it after verifying that the save to file worked instead). Tweaked materials documents revision extraction regexes. Added a test on no change of materials rev after failed upload. Corrected the naming of materials in make_meeting_test_data. Refined the test crawl of meeting materials pages. * Added a couple of meeting material crawls to check that internal material links aren't broken. * Check attribute isn't None before accessing sub-attributes. * Merged in [14845] from rcross@amsl.com: Remove unused roles/forms.py. * Merged in [14844] from adam@nostrum.com: Fixing 'href' field for https://datatracker.ietf.org/meeting//json. * Fixed some materials docname-with-rev regexes that were too narrow. * Merged in [14843] from adam@nostrum.com: Updated (deprecated, unregistered) 'text/json' with 'application/json' everwhere. * Merged in [14842] from rcross@amsl.com: Add check for deleted session when scheduling. Fixes #2450. * Fixed a bug in a RegexValidator error message string interpolation. -- Henrik Levkowetz 21 Mar 2018 10:58:52 +0000 ietfdb (6.75.2) ietf; urgency=medium **Session iCalendar links on document lists** This release adds session ical links to group document lists and document search results, fixes some issues related to upload sanitization and url resolution, improves performance for some pages, and adds and improves some tests in the test suite. From the commit log: * Merged in [14830] from rjsparks@nostrum.com: Simple coverage tests for ietf/secr/drafts/reports.py. * Tweaked the document sanitizer to insert a charset meta tag after sanitization. * Fixed a mistaken change in session urls. * Tweaked a test case to make it easier to get at failures. * Made the materials_document() view function more robust in handling various materials document names. * Gave the 404 message a class which will make it easier to pick out the message during testing. * Updated the meeting materials document view to handle urls with revisions. * Updated the meeting document href patterns in settings.py to reflect that we now have versioned agendas, minutes, and slides. * Added session ical links to document lists (WG docs, search results, etc.) for documents on upcoming meeting agendas. Reduced the query and rendering times of document lists some more through additional prefetch_related(). * Added result caching for the Person.email() method. * Added result caching for a couple of Document methods. * Added a missing tag to the sanitizer whitelist (telling lxml's Cleaner to not clean style with style=False is apparently not always enough). Fixes issue #2470. * Added a debug import. * Updated a django patch to add origin information to query sets. -- Henrik Levkowetz 17 Mar 2018 10:42:11 +0000 ietfdb (6.75.1) ietf; urgency=medium **Modified HTML upload sanitization** Feedback from wgchairs@ietf.org indicated that stripping out all styling from uploaded files is too harsh. This release modifies the sanitization to permit