ietfdb (6.126.0) ietf; urgency=medium **Groundwork for upcoming automatic scheduling assistance** * Merged in ^/branch/dash/automatic-scheduler@17395 from sasha@dashcare.nl, which adds groundwork for upcoming automatic scheduling assistance: ~ Added a management command to create a dummy IETF 999 meeting. ~ Added display of new constraints and joint sessions to agenda builder interface. ~ The new timerange, time_relation and wg_adjacent constraints, along with the joint_with_groups option, are now reflected in the special requests field. This allows them to be taken into account while scheduling sessions. ~ Clarified the wording in the session request form regarding conflicts with BOFs. ~ Added support for structured entry and storage of joint sessions in meetings: - Also adds additional tests for the SessionForm - Fixes a javascript error in session requests for non-WG groups, that could cause incorrect form behaviour. - Expands the tests added in [17289] a bit. ~ Added support for the timerange, wg_adjacent and time_relation constraints. This adds three new constraints to the database and relevant UIs: - timerange: "This WG can't meet during these timeframes" - wg_adjacent: "Schedule adjacent to another WG (directly following, no breaks, same room)" - time_relation: schedule the two sessions of one WG on subsequent days or with at least one day seperation -- Henrik Levkowetz 09 Apr 2020 16:25:23 +0000 ietfdb (6.125.0) ietf; urgency=medium **Various meeting-related fixes and improvements** * Merged in [17590] from rcross@amsl.com: Added support for variable length meetings to secr/meetings app. * Changed the handling of some exceptions during draft submission to give user feedback rather than server 500 responses, in order to deal better with severely malformed drafts. * Added a workaround for the current libmagic on OpenSUSE, which quite easily can mischaracterise text/plain documents as text/x-Algol68. Fixes issues #2941 and #2956. * Added validation of the session duration in interim meeting requests, with added values in settings.py for min and max duration. * Clarified the standalone XML draft submission requirements, and mentioned the xml2rfc switch to use for v3 sources. * Changes to accept a wider range of URLs when displaying call-in links from the Session agenda_note and remote_instructions fields. * Changed some fields to raw_id_fields in the MessageAdmin, for improved admin page load times. * Added 'Remote instructions' at the top of interim sesssion pages, and made the 'Meeting Details' button available to the group chairs, not only secretariat. -- Henrik Levkowetz 08 Apr 2020 16:51:46 +0000 ietfdb (6.124.0) ietf; urgency=medium **Enhanced 'Upcoming Meetings' page, and more** * Added links to agenda/materials pop-up, materials download, etherpad, jabber room, and webex call-in session for interims on the upcoming meetings page. With the earlier changes from [17555], this fixes issue #2937. * Changed the IPR patent number regex to permit space between country code and serial number, and expanded on the help text for the IPR patent number field. * Added verification of response data to IphoneAppJsonTests * Merged in [17564] from pusateri@bangj.com: Added interim meetings to agenda.json API. Fixes #2946. * Merged in [17562] from jennifer@painless-security.com: Add tooltips with doc name to 'updates' and 'obsoletes' links. Fixes #2866; * Fixed a Py2/3 issue in the djangobwr's bower_install command * Added a check for availability of 'bower' in bin/mkrelease. * Cleaned up the contributors list in bin/mkrelease a bit. * Merged in [17557] from fenton@bluepopcorn.net: Provide more consistent links to people pages. Fixes #2918. * Changed an obsolete document.href() to document.get_href(). Fixes issue #2945. * Tweaked the upcoming calendar and calendar entries slightly, to render with times first and on two lines on narrow screens. * Merged in [17555] from rjsparks@nostrum.com: Remove the not-quite-working customization widgets from /meeting/upcoming and /meeting/past. Simplify those views. Correct the list of sessions on those pages when one interim has more than one session. Fixes #2938. Partially addresses #2937. * Prevent an exception on missing author.email.person when listing author emails. -- Henrik Levkowetz 03 Apr 2020 10:04:43 +0000 ietfdb (6.123.1) ietf; urgency=medium **Fixes for meeting-related issues** * Merged in [17543] from rjsparks@nostrum.com: Repaired construction of group_hierarchy used for the customisation controls at /meeting/upcoming. Fixes #2940. * Merged in [17542] from rjsparks@nostrum.com: Show calendar entries on /meeting/upcoming in utc. Show end times. Partially addresses #2936. * Don't show agenda buttons for Meetecho recordings (after the session concludes) if there isn't a Meetecho UrlResource. Fixes issue #2934 * Merged in [17538] from rjsparks@nostrum.com: Allow an out-of-area AD assigned as the AD for a WG to approve interim requests for that WG. Fixes #2930. * Changed the object factory instances of nomcom private key and cert to be byte objects (matching the production settings), and fixed the issue with nomcom key handling under Py3 found by fenton@bluepopcorn.net. Did some renaming in nomcom/tests.py to better match setup/teardown function names to functionality. * Merged in [17521] from housley@vigilsec.com: Improve performance of log.assertion() and log.unreachable() * Merged in [17505] from housley@vigilsec.com: Improve performance of many document list pages -- Henrik Levkowetz 27 Mar 2020 14:27:24 +0000 ietfdb (6.123.0) ietf; urgency=medium **IETF 107 code sprint** This release contains datatracker bug fixes and enhancements from the IETF-107 Code Sprint, our first virtual code sprint. It was a different experience, and enjoyable despite not getting to sit down with a beer together afterwards. A lot of good contributions were made; Thanks to everyone who contributed! * Merged in [17496] from rjsparks@nostrum.com: Remove the rest of the log.assertions checking that iesg_state existed in places we expected it to. Removed unnecessary imports. * Changed the page for upcoming meetings to show the current IETF meeting for 7 days from its start date, while interims are shown for today and forward. Also changed the upcoming.ics calendar to show future sessions, even if the meeting to which they belong started in the past. This improves on [17518]. * Changed the starting point of display of upcoming meetings to be 7 days before today, rather than today, to let meetings linger a bit in the listing and iCalendar file after the meeting has started. Triggered by an observation from resnick@episteme.net about IETF 107 sessions disappearing from 'upcoming.ics' on meeting week Monday. * Merged in [17495] from rjsparks@nostrum.com: Removes a log.assertion() that was checking that we covered the edges when we changed documents to always have an iesg state. * Merged in [17494] from rjsparks@nostrum.com: Use current email addresses when we have them when listing document authors. Fixes #1902. * Merged in [17493] from mahoney@nostrum.com: Changed awkward IESG/IAB Nominating Committee names to just NomCom, updated a ref. Fixes #2860. * Merged in [17492] from rcross@amsl.com: On session request form, made the Special Requests field smaller and display 200 character limit. Fixes #2875. * Merged in [17491] from rcross@amsl.com: Prevent use of capital letters in group acronym. Fixes #2709. * Merged in [17490] from rjsparks@nostrum.com: Basic regex validation on community rule entry form. Fixes #2928. * Merged in [17489] from rcross@amsl.com: Removed redundant URL secr/groups/search because search page is available here secr/groups. Resolves issue with Add link. Fixes #2708. * Merged in [17488] from rcross@amsl.com: Removed the drafts secretariat tool because this functionality is now provided by the core Datatracker. Moved ID reports to proceedings tool. Fixes #1655. * Merged in [17487] from rjsparks@nostrum.com: Let chairs know what to do after material submission uploads have been cut off. Fixes #2887. * Merged in [17486] from valery@smyslov.net: Added docker/run modifications to support Cygwin. * Merged in [17484] from valery@smyslov.net: When requesting a new WG session, and retrieving information about the previous session, look back to the previous time the group met, instead of simply checking the previous IETF meeting and maybe not finding any information to retrieve. * Merged in [17483] from peter@akayla.com: Changed things so that only WGs/RGs can be closed, per RJS. Fixes #1578. * Merged in [17466] from rcross@amsl.com: Added a migration to cancel 107 sessions * Added a check to see if any files matching the submitted draft name and revision already exists on disk in the active drafts or archived drafts directories, and if so reject the submission. Fixes issue #2908 * Made sure to strip possible mail header field values of whitespace before applying email.utils.unquite(). Resolution by kivinen@iki.fi, Fixes issue #2899. * Merged in [17480] from rjsparks@nostrum.com: Show UTC times in interim announcements if the interim has a non-UTC timzone. Fixes #2922. -- Henrik Levkowetz 24 Mar 2020 17:53:45 +0000 ietfdb (6.122.0) ietf; urgency=medium **Added agenda webex URL support, and meeting-related tweaks and bugfixes** * Added webex URL to agenda.ics if Room.webex_url is non-empty. Fixes issue #2926. * Added another check to the check_draft_event_revision_integrity management command, and refined it somewhat. * Added a utility function to convert objects to dictionaries (for comparisons, for instance). * Added a --dry-run option to bin/mkdevbranch, and added some exception handling. * Help tablesorter see 'Month Year' dates as dates with a hidden day digit. Fixes issue #2921. * Refactored and extended check_draft_event_revision_integrity a bit. * Tweaked bin/mkpatch some for -c handling * Merged in [17442] from rjsparks@nostrum.com: Allow area groups to request interim meetings. Fixed #2919. * Additional tweaks to bin/mkpatch; removing buggy -r option. * Added automatic naming to bin/mkpatch when changeset or revision range is given. * Added WebEx room resource name, query method and template logic to show WebEx room resources. * Removed a debug statement * Made links from agenda room names to floorplans conditional on the room having a floor plan set. -- Henrik Levkowetz 20 Mar 2020 19:50:12 +0000 ietfdb (6.121.0) ietf; urgency=medium **Tweaks for wholly virtual meeting, for IETF-107** * Added code to show a webex call-in button on the agenda page if the session agenda-note contains and IETF webex URL. * Merged in [17425] from rjsparks@nostrum.com: Make required AD approval of virtual interims configurable. Fixes #2912. * Added a management command to check draft event revision numbers. To be extended for other checks. * Merged in [17419] from rjsparks@nostrum.com: Don't warn about idcutoff when the cutoff is after the meeting starts. Fixes #2907. * Merged in [17418] from rjsparks@nostrum.com: Correctly represent cancelled sessions in ics files. Fixes #2905. * Merged in [17396] from rjsparks@nostrum.com: Move charters for replaced groups to a new replaced state. Close any outstanding ballots on them. Fixes #2889, #2873, and #1286. * Avoid trying to open meeting documents with empty .uploaded_filename. * Added a progress bar for verbosity=1 of the community list index update command. * Merged back fixes from production * Corrected the extent of a try/except block, moving more code inside the block. Fixes a submission exception that should just be a document error reported back to the user. * Added a guard against accessing attributes of None. -- Henrik Levkowetz 13 Mar 2020 14:48:11 +0000 ietfdb (6.120.0) ietf; urgency=medium **Submission API changes, Py2/3 transition fixes** * Added the ability to use the submission API with active secondary account email addresses. Fixes issue #2639. * Tweaked the ReviewAssignmentAdmin, adding a raw_id_field. * Replaced most cases of using of urlopen(), instead using the higher-level 'requests' module where it simplifies the code. * Added a data migration to fix up incorrect external URLs to mailarchive. * Fixed a Py2/3 issue with review.mailarchive.construct_query_url(). * Renamed a migration to conform to migration naming conventions, using underscores instead of dashes in the name. * Py2/3 compatibility tweaks for pyflakes. * Changed some cases of urlopen() to use requests.get() * Python3 is more ticklish about comparing strings to None than Py2. Fixed an issue with this in generate_sort_key() for document searches. * Fixed a Py2/3 issue in the pyflakes management command, and tweaked the verbose output format. * Merged back production changes to two scripts indirectly called by /a/www/www6s/scripts/run-ietf-report, through /a/www/www6s/scripts/run-report. * Changed the release script to not pick up other email addresses than those of contributors from the release notes. * Tweaked the check_referential_integrity management command verbose output. -- Henrik Levkowetz 07 Mar 2020 22:55:58 +0000 ietfdb (6.119.1) ietf; urgency=medium **Py2/3 fixes, Change to use the "requests" lib instead of urlopen()** * Py2/3 compatibility tweaks for pyflakes. * Changed some cases of urlopen() to use requests.get() * Python3 is more ticklish about comparing strings to None than Py2. Fixed an issue with this in generate_sort_key() for document searches. * Fixed a Py2/3 issue in the pyflakes management command, and tweaked the verbose output format. * Merged back production changes to two scripts indirectly called by /a/www/www6s/scripts/run-ietf-report, through /a/www/www6s/scripts/run-report. * Changed the release script to not pick up other email addresses than those of contributors from the release notes. -- Henrik Levkowetz 03 Mar 2020 11:23:46 +0000 ietfdb (6.119.0) ietf; urgency=medium **Improved email handling, and roundup of Py2/3 conversion issues** This release provides a reworked email sending infrastructure so that all emails sent by the datatracker, except for logging and failure emails sent to ADMINS, are captured as a Message objects. Together with two new management commands, this makes it possible to recover from email delivery failures by re-sending any emails from the datatracker that would otherwise have been lost. This release also includes backporting Python 2/3 tranistion issues that were initially patched in the production code during the recent server transition, and a number of additional unrelated bugfixes. From the commit log: * Merged in [17336] from rcross@amsl.com: Add rfp-announce@ietf.org to Announcement app To: address list. Changed name of secr dashboard from WG Chair Dahsboard to IETF Dashboard. * Merged in [17325] from rjsparks@nostrum.com: Allow review team secretaries and the secretariat to reset the next reviewer in queue for review teams using the RotateAlphabetically policy. Partially addresses #2879. * Fixed an issue with some tests that mocked urlopen() responses, and tweaked error messages on bad results in sync/rfceditor.py * Added charset decoding of data read from an urlopen() object in a few places, and removed some use-once functions that were now dead code. * Moved code from a couple of simple urlopen() wrappers used only once into the calling scripts. Removed a case of Py2 virtualenv activation. * Added management commands to show saved messages (unsent and otherwise) and to send or re-send saved messages as needed: - show_messages: Show outgoing messages that have been saved as Message objects. By default all messages from the last 2 weeks are shown. Selection can be made based on date and sent/unsent state. With the --pk option, only a list of primary keys are shown, otherwise, creation and send date, message-id, sender and primary recipients, and subject line is shown. - send_messages: Send (or re-send) messages saved as Message objects as outgoing emails. To show existing Message objects, use the show_messages management command. Messages to send can be indicateb by date ranges, a list of primary keys, or a list of Message-IDs. Unless the --resend switch is given, the inclusion of already sent messages in the date range or message lists will result in an error exit, in order to prevent inadvertent re-sending of message. Alternatively, the --unsent switch can be used to send only messages marked as not already sent from a date range or message list. * Added a migration to set an explicit content-type for Message objects where missing. * Added a migration to correct Message fields containing strings that were repr() of list instances instead of comma-separated email addresses. * Added saving of outgoing emails as Message instances, with accompanying test. * Did a slight refactoring of infer_message() and send_scheduled_message_from_send_queue(), and added setting of a Message.sent date when sent. * Fixed an issue where the lookup of recognized country names during draft submission grabbed the unicode name instead of the ascii name for non-ascii country strings in XML submissions. * Updated the list of utility python files that should be ignored by code coverage measurements. * Fixed log.log() to hand on the right type (str) to underlying functions, irrespective of Python version. * Fixed a test to be more specific when looking for a generated Message object, to make sure to get the right one. * Added html unescape for email addresses from form data that might use html entities for angle brackets. * Added a 'sent' field to Message, to be able to track sent status for captured outgoing messages (which don't have a SendQueue instance). * Fixed a number of bad Message.content_type settings when creating Message * objects. * Fixed a string vs list issue for a Message from form case. * Added missing .as_strings() to some gather_address_lists() calls, in order to assign strings rather than lists to Message CharField instances. Fixed a Message content_type setting. * More Py2/3 adaptations fetched from production. * Converted send-scheduled-mail to py3 * Added a cron script to be called every 15 minutes, to have a place where send-scheduled-mail can live and be invoked with the right environment. * Moved patch.py into our project tree -- it's not being maintained by the external maintainers any more. * Merged in Python 2/3 changes pulled from the production server after the switch to python 3. * Merged in cronjob script changes pulled from the production server after the switch to python 3. * Added cleaning of the session request form's 'comments' field, to convert any html entered to text. Related to [17322]. * Removed an instance of 'autoescape off' for submitted meeting session comments in a template, since it is not needed in practice; in order to avoid an XSS injection opportunity. * Updated the PLAN file, removing some completed actions. * Changed some remaining cases of open() to io.open(), for Py2/Py3 consistency. * Tweaked settings.py to make django_extension commands available if installed. -- Henrik Levkowetz 27 Feb 2020 19:56:12 +0000 ietfdb (6.118.0) ietf; urgency=medium **RG and NomCom support refinements, new data merge management commands, bug fixes** * Merged in [17288] from rjsparks@nostrum.com: Made internal review email about an RG use the IRSG instead of the IESG in text and addresses. Fixes #2881. * Merged in [17287] from rjsparks@nostrum.com: Set the default for use_milestone_dates to True. Allow changing use_milestone_dates from the edit milestone dates form for charters if it is the initial chartering effort. Fixes #2884. * Wrapped more code in draft submission in an exception block, in order not to continue processing after an XML parse exception. Fixes issue #2885. * Fixed a bug in the resource description for SchedulingEvent that prevented it from being accessed under /api/v1/meeting/schedulingevent/. * Added search fields to the admin index for SchedulingEvent. * Tweaked send_mail_mime() to recognize settings.SERVER_MODE='repair', in order to avoid mail floods when running the 'mergedata' management command. It is assumed that the appropriate mails went out when the data was entered into the system the first time. * Added a management command to dump data based on table names and primary keys derived from a list of SQL 'INSERT INTO' commands representing the items to be dumped. The output formats are the same as for dumpdata, and are suitable for later use with the loaddata or mergedata management commands. * Added a management command to merge data from one of two divergent django databases to the other, taking care to insert or update records as appropriate, and update foreign keys and many-to-many keys appropriately. Accepts the dump formats generated by the dumpdata command. * Updated the check_referential_integrity command, adding a --delete command to remove dangling references to removed records, and also adding colorized success/fail indications for each FK and m2m key inspected. * Changed the xml parser settings for legacy xml draft submissions to remove comments before converting to .txt. Fixes and issue with comments in text. * jsonfield 3.0 and higher is not compatible with Django 1.11. Changed the version requirements accordingly. * Added missing test directory creation and removal in setUp() and tearDown() in MeetingTests. * Merged in [17286] from rjsparks@nostrum.com: Added Reply-To to the default group review text. Fixes #2883. * Merged in [17284] from jennifer@painless-security.com: Create DocEvent when related IprDisclosure is posted or removed. Fixes #2811. * Merged in [17275] from jennifer@painless-security.com: Added links to user profiles from nominee list and feedback page. Fixes #2839. * Merged in [17273] from rcross@amsl.com: Updated link to session request tool instructions. -- Henrik Levkowetz 18 Feb 2020 22:59:34 +0000 ietfdb (6.117.0) ietf; urgency=medium **Python 2/3 transition fixes, sumbmission and chartering improvements** * Merged in [17268] from rjsparks@nostrum.com: Set a meaningful state in a test fixture. Tests surprisingly were not broken by this though. * Merged in [17266] from rjsparks@nostrum.com: Improve the tests for session draft bundles to exercise the code that extracts draftnames from agendas. * Removed exec of activate_this.py from 2 more scripts that are in activbe use. * Ran scripts invoked from bin/hourly and bin/daily through pyflakes3 and trial invocation, and tweaked the code (with an added comment) to consistently require virtualenv to be set up before script invocation. * Added some raw_id fields to avoid long page load times for admin pages. * Fixed a py2/py3 issue in sync.rfceditor.post_approved_draft(), and enhanced tests to execute the failing code (with Mock functions) instead of skipping the code. * Hardcoded an rsync target path, and added a bash trap for errors in hourly and daily cron scripts. * Fixed some py2/py3 issues in ID index generation modules. * Fixed a py2/py3 issue in building meeting materials bundles. * Added a setting for a coming yang-catalog submission checker url. * Did some re-ordering of PLAN items after a priority discussion. * Fixed a py2/py3 issue in post_approved_draft() * Changed the location of the file cache for htmlized documents so it will be handled by backup rsync of the /a/ file system. * Added a newline to test run command output, to separate dot progress indication from teardown messages. * Added a test case for draft submission errors due to periods in document names. * Added a suggestion to the error message issued for invalid characters in document names found in draft submission XML files, when the invalid character is a period. In this particular case, the problem most likely is inclusion of the filename extension in the document name. Fixes issue #2874. * Changed draft submission field validation taking place in clean() to associate the errors with the field in question, rather than raising them as general form errors. * Fixed a Py3 issue trying run .decode() on (already decoded) string instances. * Merged in [17246] from rjsparks@nostrum.com: Only show groups in reasonable states when showing current chartering efforts (otherwise crufty charter states can would lead to abandoned or replaced groups showing here). Fixes #2872. -- Henrik Levkowetz 05 Feb 2020 22:28:38 +0000 ietfdb (6.116.1) ietf; urgency=medium * Refined the links from document stats pages to author pages so as to not link to non-existent pages. Fixes 404 errors for a number of stats page links. * Merged in [17234] from rjsparks@nostrum.com: Restrict the ability to change whether a group uses milestone dates to the ADs and the secretariat. Fixes #2869. * Merged in [17233] from rjsparks@nostrum.com: Be clear about who changed a group's personnel in the resulting email message. Fixes #2868. * Merged in [17228] from rjsparks@nostrum.com: Improved admin handling of DocumentURL. Fixes #2865. * Merged in [17227] from rjsparks@nostrum.com: Allow document Additional URLs to become empty. Fixes #2864. * Added (currently inactive) code to trigger exceptions on failure to resolve template variables during tests, and added fixes for a couple of places that triggered such exceptions. * Merged in [17219] from rjsparks@nostrum.com: Update some uses of pos.ad in templates to pos.balloter. Fixes #2863. * Merged in [17217] from olau@iola.dk: Tighten validation of session conflicts in the session request tool, preventing conflicts where source = target. Make sure the 'use previous session' button doesn't carry forward conflicts that are no longer valid. Also fix a bug with the group list in the dropdowns being computed statically at module import time instead of being queried dynamically upon each page request. * Merged in [17211] from rjsparks@nostrum.com: Upgrade to the newer js-cookie library that replaced jquery.cookie. Fixes #2832. * Fixed a Py2/3 compatibility issue with the verified errata code. * Merged in [17201] from rjsparks@nostrum.com: Make it easier for the nomcom chair to manage generic IESG requirements. Fixes #2794. * Added a check for uncomitted changes to bin/mergedevbranch. * Added fetch and use of the RFC-Editor's errata.json file in order to set a new 'verified-errata' document tag, and use that to show the link to inline-errata documents only when there is verified errata. Fixes issue #2861. -- Henrik Levkowetz 17 Jan 2020 18:20:00 +0000 ietfdb (6.116.0) ietf; urgency=medium **Dateless group milestones, improved author stats, and bugfixes** * Merged in [17196] from rcross@amsl.com: Removed agenda@ietf.org from audio import notification To: list. * Merged in [17195] from rcross@amsl.com: Updated admin permissions to grant secretariat access to LiaisonStatementGroupContacts. * Merged in [17185] from rjsparks@nostrum.com: Added support for dateless milestones. Fixes issue #2799. * Specified explicit UTF-8 decoding of submission checker output, in order to avoid issues when reporting errors due to UTF-8 content. * Merged in [17181] from olau@iola.dk: Fixed a problem with assignments to the same session causing a crash. * Merged in [17179] from housley@vigilsec.com: Added references and referened-by buttons to each RFC listed on the person profile page. In addition, changed the buttons on the document references and document referenced-by pages to match the buttons on the main document page (the buttons with the long arrows). Fixes #2854, although it does not include the number of references in the button. * Merged in [17178] from housley@vigilsec.com: Turned text names into links on the stats pages. Fixes issue #2854. * Refactored some document methods to align better with Django's get_absolute_url(), and added Person.get_absolute_uri() * Merged in [17177] from rjsparks@nostrum.com: Improve can_ballot. Fixes #2856. * Limited the permitted versions of mypy and related django-slugs, to avoid issues related to using Django-1.11. We should update to latest versions when we've moved to Django 2.2.x * Changed ReviewerSettings history change-reason field to be a TextField, in order to accomodate longer change descriptions. * Merged in [17173] from rjsparks@nostrum.com: Worked around a problem with displaying IESG evaluation information for non-approve ballot types. Partially addresses #2851. -- Henrik Levkowetz 09 Jan 2020 21:58:16 +0000 ietfdb (6.115.0) ietf; urgency=medium **IRSG document ballot support** * Merged in ^/branch/akayla/irsg/6.113.1.dev0 from peter@akayla.com: This provides support for IRSG ballots, similar to the IESG ballots support which has been in use for quite some time. The IRSG ballots differ from IESG ballots in a number of ways, described in detail in the RFP and SoW for this work and implemented here. * Changed print() statements into syslog logging in a cron-invoked script, to remove non-error output. Fixes issue #2849. -- Henrik Levkowetz 20 Dec 2019 15:44:39 +0000 ietfdb (6.114.0) ietf; urgency=medium **Refactoring and renaming of some meetings models and fields** * Merged in [17148] from rjsparks@nostrum.com: Only gather addresses instead of name_addrs for the group_secretaries Mailtrigger Recipient. Fixes #2847. * Merged in [17147] from rjsparks@nostrum.com: Show diff controls for a bis -00 document. Fixes #2846. * Merged /branch/iola/meeting-improvement-r16992@17152 from olau@iola.dk: - Renamed Meeting.agenda to Meeting.schedule together with a bunch of related internal things, - Moved Session.status, .requested, and .requested_by to a new SchedulingEvent. - Turned sessions into regular sessions and non-sessions into misc. sessions in the UI and code to avoid ambiguity. This doesn't change the data in the DB except for uses of TimeSlotTypeName where 'session' is now 'regular'. * Added a requirement limitation for the coverage module. * Added handling for a decoding exception during submission processing. * Merged in [17120] from rjsparks@nostrum.com: Add links to the main website and the mailarchive to the search page. * Added an OPTIONS settings to the DATABASES setting in docker/settings_local.py, to make some migration operations work under docker. -- Henrik Levkowetz 18 Dec 2019 20:38:33 +0000 ietfdb (6.113.0) ietf; urgency=medium **Review tool queue management with pluggable policies** This release introcudes the ability for review group secretaries to choose between different management policies for the review queue, as it turns out that both the needs and preferences of the groups and group secretaries vary in this regard. With this release it is possible to choose between two policies: a Rotate Alphabetically policy, and a Least Recently Used policy. Both policies provides for various exceptions. There are also some unrelated minor bugfixes. From the commit log: * Merged in work from sasha@dashcare.nl on Review Queue Managemnt: This abstracts queue management, making it possible to implement different policies for each team. It provides two concrete policies: RotateAlphabeticallyReviewerQueuePolicy, which rotates an alphabetically ordered reviewer list with consideration for skip indications, and is the default policy; and LeastRecentlyUsedReviewerQueuePolicy, a simple least-recently-used policy. Also see issue #2721, and #2656, which documents various flaws in the old queue code. * Added fallback handling from utf-8 to latin1 when reading charter text files. * Added normalization of draft title extracted from submitted XML. * Removed code under docker/ from test coverage calculation. * Fixed a backwards compatibiltiy issue for the factoryboy lib and updated a dependency requirement. -- Henrik Levkowetz 07 Dec 2019 19:28:38 +0000 ietfdb (6.112.0) ietf; urgency=medium **Review tool tickets roundup, part 6; and other fixes** This provides more review tool fixes, and clears the slate for an upcoming branch merge. From the commit log: * Merged in [17085] from sasha@dashcare.nl: Notify secretary when a review request is closed. The assigned reviewer was already notified, but secretaries were always excluded. Fixes issue #2278 * When group secretary email addresses occurred in some mailtrigger To: entries, they were given with both name and address parts, while when occurring in Cc: entries, no name parts were present. Fixed one case of this related to current review tool changes, and adjusted a test case for consistency. Also added a general utility function to exclude selected email addresses from email address lists. * Added support for bibxml3 entries using the traditional reference entry name (using 'reference.I-D....'), instead of just the plain draft name. * Merged in [17070] from sasha@dashcare.nl: Improved 'complete review' workflow for secretaries. When a secretary completes a review, 'link to a review message' is automatically selected, and the first non-reply mail is used to fill in the review details. The secretary can still modify all details. The order of fields for secretaries is also modified to fit this workflow. All cases where 'link to review message' is used, by reviewers or secretaries, now attempt to fill in the 'reviewed version' if found in the email subject. Fixes issue #2061. * Merged in [17069] from sasha@dashcare.nl: Added history for assignments, requests and unavailability. Fixes issues #2074 and #2358. * Added a test for bibtex page content-type. * Fixed an incorrect content_type. * Merged in [17097] from rjsparks@nostrum.com: Removed unreachable assertion from slide reordering utility function. * Merged in [17090] from rjsparks@nostrum.com: Added pages to assignment email. Fixes #2838. * Merged in [17083] from kivinen@iki.fi: Moved hard coded defaults for max items to show and days to show to the settings.py. * Merged in [17071] from rjsparks@nostrum.com: Use doc.name explicitly now that name is no lonker the pk. Pointed out by Sasha in #2050. -- Henrik Levkowetz 27 Nov 2019 23:04:12 +0000 ietfdb (6.111.0) ietf; urgency=medium **IETF 106 code sprint** This release contains datatracker bug fixes and enhancements from the IETF-106 Code Sprint. This sprint brought in new contributors, and saw substantial work from previous contributors. Thanks to everyone who contributed! * Merged in [17062] from rjsparks@nostrum.com: Change the default template for assigned review emails to name the assigned reviewer. Fixes #2830. * Merged in [17061] and [17024] from rjsparks@nostrum.com: Allow slides drag-drop between sessions for groups meeting with multiple sessions. Fixes #2700. * Merged in [17044] from valery@smyslov.net: Make it clearer that a chair can't revise materials for a meeting once the proceedings deadline has passed. Fixes issue #2612. * Merged in [17037] from kivinen@iki.fi: Added padding to the column in the unavailability period list of the reviewers page. * Merged in [17036] from housley@vigilsec.com: Drop old 'slides' section from agenda.json * Merged in [17035] from housley@vigilsec.com: Improved performance for some agenda-related pages * Merged in [17034] from kivinen@iki.fi: Added two new configuration settings for the review team secretary, one to set how many days to include in the reviewers list, and another one to limit the number of completed items in the list for each person. This version replaces the one I did earlier, and includes much more test cases to test different limits on the reviewers page. * Added an instance of force_text() to ensure non_ascii mail subjects are unicode when used for string interpolation, rather than bytes (which will trigger ascii decode()). * Merged in [17033] from kivinen@iki.fi: Reviewers page template changes. Changed draft names so they never wrap. Changed the Unavailability period table so it is much narrower. * Merged in [17031] from fenton@bluepopcorn.net: Add edit button for Secreatriat on concluded groups. Fixes #2781. * RFCs with inline errata now have a non-beta URL. Updated the setting. * Merged in [17028] from fenton@bluepopcorn.net: Added a Closing note field to state specific WG edit. Fixes #2779. * Merged in [17027] from rcross@amsl.com: Fix issue where draft resurrect feature does not move most recent draft file from the archive directory back to the current draft directory. Fixes #2646. * Tweaked docker-init.sh to create a needed GID entry before using it when creating a new user. Fixes an issue when the user GID in the host OS is missing in the docker image. * Added fullcalendar to the bower.json file. * Changed the call arguments for submission tool conversion from XML to v3 text to match those of the standalone tool. Fixes an issue where would have line ends stripped when converted by the datatracker. * Tweaked a test's person name form to match the usage in the tested code. * Fixed naming of staged xml draft submission files and tweaked the arguments for the xml2rfc parser invocation for xml draft submissions. Added some more logging. -- Henrik Levkowetz 21 Nov 2019 06:50:19 +0000 ietfdb (6.110.1) ietf; urgency=medium This release provides various fixes, and clears the slate for the upcoming IETF 106 Sprint work. * Merged in [16981] from rjsparks@nostrum.com: Look at the tree of replaces-relationships when determining if some reviewer has reviewed this (or perhaps an ancestor) of this document. Fixes #2821. * Merged in [16966] from rjsparks@nostrum.com: Use the recently released shepherd writeup templates. Fixes #2820. * Fixed a wrong link to field help for liaison forms. * Merged in [16939] from sasha@dashcare.nl: Allow adding review wishes from document and search pages. On the main page of a document and in document search results, a new button allows review team members to add a review wish for that document. For reviewers that are only on one team, this essentially works identical to tracking a document. Reviewers that are on multiple teams are lead through an intermediate step to select a review team, and then returned to their search or document page. Fixes issue #2050. * Merged in [16564] from pusateri@bangj.com and added tests: Convert markdown to html if Accept header prioritizes text/html over text/markdown. Fixes #1926. * Merged in [16977] from rjsparks@nostrum.com: Disregard unassigned reviews when building the sets to compare when checking for race-conditions on assigning reviews. Partially addresses #2821. * Moved the ietf/names/generate_fixtures.py command to a management command "generate_name_fixture", and updated it to include a necessary dbtemplate object in the fixture. -- Henrik Levkowetz 11 Nov 2019 21:35:46 +0000 ietfdb (6.110.0) ietf; urgency=medium **Trial bibxml3/ pages for drafts, and review ticket fixes** * Added a trial version of datatracker draft bibxml pages, located under /doc/bibxml3/; for example: https://datatracker.ietf.org/doc/bibxml3/draft-iab-fiftyyears-01.xml * Added a management command to generate draft bibxml3 files. * Fixed some type: ignore annotations * Tweaked the submission tool to be more liberal with v3 xml submissions. * Rewrote the ietf/bin/fix-ambiguous-timestamps command as a management command (fix_ambiguous_timestamps) and made it look at all instances of DateTimeField by means of model introspection. * Merged in [16934] from sasha@dashcare.nl: Sort unassigned review requests by doc name by default. Fixes issue #2088. * Merged in [16933] from sasha@dashcare.nl: Allow specifying review type for suggested reviews in LC and telechat. If a review is suggested on the 'manage unassigned reviews' page, and the document is in both last call and telechat, the assign form now asks for the type of review that should be assigned. This commit also fixes two bugs in this process: - Comparisons in some cases between strings and integers (group/views.py:1485/1487) - Rejections when assigning suggested reviews, as they could be considered a newly opened request due to not having a pk (group/views.py:1508) Fixes issue #2119 * Merged in [16927] from sasha@dashcare.nl: Small cleanup from changeset [16924], see issue #2217. -- Henrik Levkowetz 05 Nov 2019 23:29:35 +0000 ietfdb (6.109.0) ietf; urgency=medium **Review tool tickets roundup, part 5; and other fixes** * Merged in [16926] from rjsparks@nostrum.com: Flipped a sense to put the grace period on the right side of 'today' when sending review overdue reminders to team secretaries. Fixes #2816. * Merged in [16937] from rjsparks@nostrum.com: Only show a blocked document once in the set of blocked documents on an ADs document page. Fixes #2818. * Added mising GET parameters to the fragment cache in the agenda.ics template. Fixes issue #2817. * Merged in [16924] from sasha@dashcare.nl: Allow submission of unsolicited reviews by secretaries. - For team secretaries, a button 'Submit unsolicited review' will now appear next to 'Request review' on the document's main page. - If the secretary is a secretary for multiple teams, they are taken through an intermediate page to select for which team they are submitting their review. - The form is similar (and using the same code) as the usual review completion, with a few extra fields for the review type and reviewer, which would usually already be known. - When submitting the review, a ReviewRequest and ReviewAssignment are automatically created. The assignment is then immediately closed in the usual way. Other workflows are unchanged. The issues with the review form in #2061 are slightly worse for the unsolicited review scenario, but that will be improved when #2061 is fixed. Fixes issue #2217. * Merged in [16923] from sasha@dashcare.nl: Show doc authors and WG chairs in unassigned reviews views. Fixes issue #2087. * Merged in [16922] from sasha@dashcare.nl: Added additional content validation for uploaded texts. Permitted MIME types are now text/plain, text/markdown and text/x-rst. This applies to all usages of get_cleaned_text_file_content(), including reviews, but also other similar places where text can either be written either into a textarea or uploaded. Fixes issue #2584. * Merged in [16921] from sasha@dashcare.nl: Use a DBTemplate when a review assignment is mailed. The message sent to a reviewer when a new review is assigned to them, is now taken from a DBTemplate, allowing it to be customised per team. This includes a migration to create a default template, which looks the same as before. Fixes issue #2123. -- Henrik Levkowetz 30 Oct 2019 13:34:17 +0000 ietfdb (6.108.0) ietf; urgency=medium **Review tool tickets roundup, part 4; and other fixes** * Merged in [16904] from sasha@dashcare.nl: Always list all last call drafts in assignment summary. If a draft is scheduled for a telechat, but also assigned for LC review, it will now be listed both in the telechat section and under 'last calls'. Fixes #2118. * Merged in [16902] from sasha@dashcare.nl: Fixed a test failure in [16895]. This also makes a small text clarification, which was also the initial reason for the test failure. Related to #2526. * Merged in [16895] from sasha@dashcare.nl: List previous reviews in assignments email. This includes a migration to update the database templates. Note that not all templates currently included them (secdir at least), and previous reviews are only listed in templates that included */** before. As before, previous reviews are only included if they are done by the same reviewer(s) as the current assignment. This also fixes a bug in database template 182 (/group/defaults/email/open_assignments.txt) which referred to r.review_request..doc.rev and r.review_request..requested_rev in the template, and updates the test to use the current version of template 182. Fixes issue #2526. * Merged in [16891] from sasha@dashcare.nl: Set default start date for UnavailabilityPeriod to today. Fixes #2198. * Merged in [16883] from sasha@dashcare.nl: Do not allow reviewers to reject overdue reviews. If a review request is past the deadline, reviewers will no longer be able to reject the assignment. Fixes issue #2277. * Merged in [16907] from rjsparks@nostrum.com: Correct construction for a reverse URL in an edge case. Fixes #2815. * Merged in [16875] from rjsparks@nostrum.com: Adds docs with blocking positions to the ADs my-documents page. Fixes #2800. * Merged in [16860] from rjsparks@nostrum.com: Be resilient to statistics queries that provide a state but not a substate. Fixes #2813. * Fixed a couple of places where document alias.document was used instead of alias.docs.all(). * Merged in [16854] from rjsparks@nostrum.com: Remove some pre-ReviewAssignment refactor logic, and simplify what remains while still allowing a group secretary to not assign a row on the unassigned requests form after touching the control to assign a reviewer. Fixes #2812. * Merged in [16853] from rjsparks@nostrum.com: Add submitted and posted dates to several ipr notification email messages. Fixes #2807. * Merged in [16852] from sasha@dashcare.nl: Show stacked bar graph for in time and late reviews. On the stats page, a single stacked bar graph is now shown which combines in time and late reviews, replacing the separate graphs for these statistics. Tests for the charts are also expanded to validate the actual graph content for both stacked and non-stacked charts. Fixes issue #2331. * Merged in [16850] from sasha@dashcare.nl: Close review assignments when reviewer is removed from team. If a reviewer is removed from a team, any accepted or assigned review assignments are set to rejected if the request deadline has not passed, or set to no-response when the request deadline has passed. Per [16819], if this scenario happens, and there are no other active or completed review assignments, the request is shown as an unassigned review. Fixes #2185. * Merged in [16833], [16834], [16835], [16836] from rjsparks@nostrum.com: Changes to displaying the ballot_icon to account for IRSG ballots. Updates Grunt generated css and map files based on updated ietf.less to add styles for IRSG ballot positions. Updated copyrights in grunt generated bootstrap.js. -- Henrik Levkowetz 23 Oct 2019 15:57:50 +0000 ietfdb (6.107.0) ietf; urgency=medium **Use for last-call announcements.** * Merged in [16824] and [16825] from rjsparks@nostrum.com: Introduced the use of for last-call announcements, changing various email message templates and recipients to match. * Merged in [16819] from sasha@dashcare.nl: Return review req to 'requested' status if no review assignments are open. If a review assignment is rejected, withdrawn, marked no response, etc., and this leaves a review request without any assigned/accepted/completed review assignments, return the request state to 'requested', which means it will be shown as an unassigned review in all interfaces. Fixes issue #2186. * Merged in [16818] from rjsparks@nostrum.com: Make hybrid pdfs when converting ppt materials to pdf. Fixes #2769. * Send output of the review reminders script to log, rather than stdout (since this is run from cron, stdout and stderr should be quiet if nothing unexpected happens) * Added a guard against missing conflrev state in IESG agenda code. -- Henrik Levkowetz 21 Oct 2019 15:02:14 +0000 ietfdb (6.106.0) ietf; urgency=medium * Merged in [16824] and [16825] from rjsparks@nostrum.com: Introduced the use of for last-call announcements, changing various email message templates and recipients to match. * Merged in [16819] from sasha@dashcare.nl: Return review req to 'requested' status if no review assignments are open. If a review assignment is rejected, withdrawn, marked no response, etc., and this leaves a review request without any assigned/accepted/completed review assignments, return the request state to 'requested', which means it will be shown as an unassigned review in all interfaces. Fixes issue #2186. * Merged in [16818] from rjsparks@nostrum.com: Make hybrid pdfs when converting ppt materials to pdf. Fixes #2769. * Send output of the review reminders script to log, rather than stdout (since this is run from cron, stdout and stderr should be quiet if nothing unexpected happens) * Added a guard against missing conflrev state in IESG agenda code. -- Henrik Levkowetz 21 Oct 2019 13:58:55 +0000 ietfdb (6.106.0) ietf; urgency=medium **Review tool tickets roundup, part 3, slide upload fixes, and more.** * Added some type checking annotation tweaks and mypy_test() output tweaks. * Moved the code that generates .txt and .html draft files from .xml to a place where potential errors can be caught and displayed better. Related to ticket #2814. * Updated PLAN: Moved reworked email sending sooner to the head of the list. * Merged in [16815] from rjsparks@nostrum.com: Link to the about pages for groups even if they don't have a charter (some have other forms of descriptive text, and all have more information on the group pages than they did a few years ago). Fixes #2744. * Merged in [16813] from rjsparks@nostrum.com: Don't offer to propose slides when the next view will not allow you to propose slides. Fixes #2763. * Merged in [16792] from sasha@dashcare.nl: Send opt-in reminders for unconfirmed review assignments. If enabled for a team, reminders will be sent every X days to reviewers for review assignments they have not accepted or rejected. Fixes issue #2475. * Merged in [16785] from rjsparks@nostrum.com: Improved alerts when looking at non-current versions of a charter document. Fixes #2774. * Turned the ReviewAssignmentAdmin.review field into a raw_id_field, to avoid a form that lists all documents in the database in a drop-down. Fixes issue #2810. * Merged in [16770] from rjsparks@nostrum.com: Populate completed_on when rejecting a ReviewAssignment. Migrate past rejected assignments accordingly. Fixes #2782. * Merged in [16765] from sasha@dashcare.nl: Send daily reminders of overdue reviews to secretaries. See issue #2277 * Merged in [16763] from rjsparks@nostrum.com: Allowed the responsible AD to be None for documents in iesg states idexists and dead. Fixes #2803 and #993. * Merged in [16762] from rjsparks@nostrum.com: Use form validation to prevent interim slide titles that end with -nn until we change interims to use the modern HREF schemes. Fixes #2797. -- Henrik Levkowetz 16 Oct 2019 14:14:16 +0000 ietfdb (6.105.0) ietf; urgency=medium **Review tool tickets roundup, part 2** * Merged in [16759] from rjsparks@nostrum.com: Restrict comments total on nomcom private index to comments on positions. Fixes #2792. * Merged in [16705] from sasha@dashcare.nl: Send periodic reminders of open reviews every X days (opt-in). Fixes issue #2337 * Merged in [16704] from sasha@dashcare.nl: Send reminders before long UnavailabilityPeriods expire A notification is sent to the team secretary and reviewer, if an unavailability period lasting 30 days or more will end in 3 days. Fixes issue #2231. * Merged in [16703] from sasha@dashcare.nl: Fix send-review-reminders and add it to daily cron. This fixes a syntax error and a Python 3 incompatibility, and adds send-review-reminders to the daily cron script. Related to issue #2231. * Fixed an issue with the htmlization of links in release notes. * Merged in [16607] from rcross@amsl.com: Fixed an issue where third session requests, which rerequire AD approval, get left with approved status, instead of scheduled, after Secretraiat sends out notifications of official schedule. Fixes #2765. * Merged in [16701] from rcross@amsl.com: Added a migration to remove any break / registration sessions that have already been created for future meetings. Goes with changes for #2713. * Added log.unreachable() for the code that creates unknown-email-* addresses. Related to issue #2696. * Fixed a bug in gather_submission_submitter(). * Fixed an incorrect version number for mypy in requirements3.txt * Removed some test assertions which are incorrect after the previous commit. * From rcross@amsl.com: Don't copy breaks etc, when creating new Empty Schedule for meetings. Fixes issue #2713. * Added a type hint stub file for ietf/api/__init__.py, to capture module variables created during module initialization. -- Henrik Levkowetz 07 Oct 2019 15:06:27 +0000 ietfdb (6.104.0) ietf; urgency=medium **Review tool tickets roundup, part 1** This release constitutes the first batch of fixes from the review tool tickets work package. It also includes a few unrelated bugfixes. From the commit log: * Merged in [16688] from sasha@dashcare.nl: When completing a review, show which addresses it is posted to. Fixes issue #2478. * Merged in [16680] from sasha@dashcare.nl: Made the review_completed mailtrigger configurable per team and review type. This includes a migration to change mailtrigger slugs to be up to 64 characters instead of 32, as some slugs would not fit and require clunky abbreviations. A data migration creates triggers for existing teams, and they are also created on the fly if a trigger does not exist yet, providing a safe fallback for new review teams. The review_completed mailtrigger serves as the template for new triggers. This commit also includes tests for gather_address_lists(), as none existed. Fixes issue #2354. * Merged in [16672] from sasha@dashcare.nl: Use mailtriggers to find destinations in review app As the review app has several conditionals that don't fit entirely well within mailtriggers, the templates use a bit of extra context to figure out who exactly to include. This also extends the tests for review, to check for correct recipients. It also adds a tiny feature to mailtrigger to entirely exclude certain addresses, as required by the review-generated mails. Fixes issue #2328. * Merged in [16670] from sasha@dashcare.nl: Allow secretary to adjust date when completing a review. This also fixes other issues identified in #2590, around the modification of historical document events. The behaviour is now: - When the assigned reviewer posts a review, a single event is created, set to current date/time. - When the secretary records a review in the datatracker, they may set a different completion date, which is autofilled if an email is selected. One event is generated for the original completion date, and one for the secretary's action. - Each revision generates a new event, rather than updating previous existing events. Fixes issue #2590 * Improved the wrapping of long lines in emails sent by the datatracker: In wordwrap(), consider lines consisting entirely of some non-alphanumeric characters like ---- or === to be block (paragraph) separators. Fixes issue #2806. * Merged in [16668] from sasha@dashcare.nl: Sort names alphabetically in review group history notes when changing the list of members. Fixes issue #2560 * Removed debug-related code from group edit clean_acronym(). Fixes issue #2804. * Merged in [16667] from sasha@dashcare.nl: Added an optional comment field for closing review requests. Fixes issue #2071 * Updated the docker files to build and run an image based on devuan:ascii instead of debian:jessie. -- Henrik Levkowetz 01 Oct 2019 16:15:46 +0000 ietfdb (6.103.1) ietf; urgency=medium **Test improvement: Starting to use Python type hints and mypy type checking** This release addresses two issues with the previous release, and as a result of a test case failing to discover one of the issues, it also introduces the use of Python type hints according to PEP 484. Type hints for common Python library modules are provided by mypy, and type hints for Django are provided by the django-stubs module. The problematic function that was used with the wrong call signature has received a type annotation that will reveal similar errors in the future. This does not provide type annotation generally in the datatracker code; that will require a lot of additional work (some of which can be automated), but it gives us static type checking for Django and Python library calls, and makes type hints useful when we provide them in datatracker code. From the commit log: * Fixed up mypy issues or added type:ignore comments as needed for a clean mypy run. * Added a mypy .ini file * Added mypy and django-stubs to Py3 requirements, and added a mypy test case. * Changed the call signature of docevent_from_submission() to take an optional Person object, rather than person name. Added a type hint for the signature. * Tweaked the condition for waiving submission email confirmation slightly. -- Henrik Levkowetz 30 Sep 2019 16:55:32 +0000 ietfdb (6.103.0) ietf; urgency=medium **Draft submission tool changes** This release contains two changes to the submission tool: During draft submission, the email verification is skipped for logged-in users, and author email addresses are now required for all authors, in order to avoid author notification problems during IETF and RFC-Editor processing. Commit log details: * Changed draft submission to not require an email round-trip when the submitter is logged in. Added corresponding test cases. * Added code to detect conflicting information for draft name, revision, and title when both XML and text versions of a document are submitted. * Changed the draft submission form and views so that an email address is required for each author in order to complete self-service draft submission. It is possible to add these as metadata updates, but this will lead to submission through the secretariat (and the email addresses will still be required). Also updated related templates to make the handling of form section headings more consistent. * Added some caching to /release/ pages. With now more than 350 releases, the page response times had increased to typically 4s. Caching in view and template brought this down to ~160ms. -- Henrik Levkowetz 26 Sep 2019 18:42:18 +0000 ietfdb (6.102.0) ietf; urgency=medium **Links to RFC text with in-lined errata. HTML for v3 XML draft submissions** * Added conversion of version 3 XML draft submissions to v3 HTML, in addition to text. Added a link to HTML versions (when present) to the document status page, and renamed the link to the htmlized version to 'htmlized'. Converted the test-file for XML submission tests to v3. Added test code to check that HTML is generated for v3 XML submissions. * Added class 'bg-danger' to submission error messages on the submission upload result page. The big 'Meta-Data errors found' pane with red background is so noticeable that class 'text-danger' (which gives bold text, but no change of background) is insufficient to stand out clearly. Adding 'bg-danger' gives field error messages red background, too. * Added a link from the datatracker RFC pages to the new RFC renderings with inline errata on the RFC-Editor web page (where available). * Updated rfc-editor sync code to recognise the new Tooling Issue queue state. * Updated ietf.sync.rfceditor.parse_index() to use the new rfc-index schema (changed placement of the page-count element). * Added code to catch a form error regularly caused by form spam and return a failure instead of triggering a server 500. -- Henrik Levkowetz 20 Sep 2019 16:18:30 +0000 ietfdb (6.101.0) ietf; urgency=medium **IANA expert review tracking, new constraints labelling** This release adds IANA expert review tracking for drafts, changes the session request constraints labels, improves the handling of session slide proposal uploads, and adds a number of smaller enhancements and issue fixes. From the commit log: * Merged in [16656] from rjsparks@nostrum.com: Added IANA expert review tracking. * Merged in [16650], [16655] from rjsparks@nostrum.com: Removed the bootstrap project todo page. #2778 points to the old content so we can make new tickets for anything that is still relevant. * Merged in [16649] from rjsparks@nostrum.com: Guard against attempts to use the review request view with a ReviewRequest id that does not match the document for the ReviewRequest. This would have exposed #2776 much earlier. * Merged in [16647] from rjsparks@nostrum.com: Corrected a view that was looking at review assignment ids when it should have been looking at request review ids. Fixes #2776. * Added a 'domain' variable to context supplied to the agenda_package.txt template. Fixes issue #2793. * Merged in [16646] from rjsparks@nostrum.com: Don't manually linewrap when looking at feedback - let the pasted styling do that. Fixes #2761. * Merged in [16630] from rjsparks@nostrum.com: Rework how we are labeling constraints to provide a quick-fix for session requests at IETF106. Turned off several javascript functions that added/enforced old (possibly no longer correct) semantics for the session request forms. Fixes #2754. * Merged in [16629] from rjsparks@nostrum.com: Removing unused template. I think it was an artifact of the Django 1.2 to 1.6 conversion from 6 years ago. * Merged in [16615] from peter@akayla.com: Spelling correction in a comment. No open issue. * Merged in [16614] from peter@akayla.com: Improved so that new draft diffs (from the working group drafts page) work correctly for both -00 drafts and later revisions. Fixes issue #2697. * Merged in [16604] from rjsparks@nostrum.com: Look at all scheduled sessions for a group when deciding to display a cancelled banner. Fixes #2764. * The logic for displaying 'Email requested to be sent for earlier discuss' on IESG positions was wrong; it would count later discusses and discusses from other ADs as an 'earlier discuss'. Moved any_email_sent() from ballot to position, and refined the criteria. * Made sure Person.active_drafts() and .expired_drafts() don't return duplicate entries. Fixes issue #2784. * Changed requirements to permit pyang 2.0 and greater. * Added a migration to fix some m2m table columns incorrectly left as varchar(8) when we changed the size of name slugs to varchar(32). * Merged in [16601] from rjsparks@nostrum.com: A management command to correct the state of several slide presentations uploaded during IETF105, and perform some missed powerpoint to pdf conversions. * Merged in [16600] from rjsparks@nostrum.com: Fixed a subtle bug in the code that converts slides to PDF that was writing back into settings. * Merged in [16599] from rjsparks@nostrum.com: Correct the uploaded_filename for slides that go through the approve-proposed-slides workflow. * Merged in [16598] from rjsparks@nostrum.com: Add the ppt* to pdf conversion to the approve-suggested-slides workflow. Fixes #2762. * Added textual information about patent application formats according to WIPO's Handbook on Industrial Property Information and Documentation, as input to future extension of the regex set. * Added some new formats to the regex validator for patent numbers and patent application numbers. This is still not comprehensive. * Adjusted migrations for Python2/3 compatibility. * Updated database options settings to match our production database and dump files. * Fixed an issue where the text file charset information for an upload would be overwritten on upload of multiple files for a draft. Fixes issue #2768. * Added 30 minutes caching to agenda.ics -- Henrik Levkowetz 08 Sep 2019 15:28:29 +0000 ietfdb (6.100.0) ietf; urgency=medium **IETF 105 code sprint** This release contains datatracker bug fixes and enhancements from the IETF-105 Code Sprint. This sprint saw several participants diving deeper into the datatracker than before, accomplishing substantial tasks. Thanks to everyone who contributed! Notable improvements: The document list pages produced by searches, group document pages, and similar has been speeded up by a factor of more than 2 by doing detailed timing measurements and one by one addressing the biggest time sinks. A number of places where links to session slides were versioned have been changed so as to always point to the latest version. A calendar display component has been added, making it straightforward to display for instance calendar pages for upcoming meetings. Ability to handle markdown uploads and on-the-fly conversion for agendas and minutes has been added, as well as the ability to provide WG closing notes, sent out in announcement and saved for posterity on the group history page. The recently introduced Slide proposal feature has been made more solid, as has the Python 2 / Python 3 compatibility. Additional information, excerpted from the commit log: * Merged in [16593] from rjsparks@nostrum.com: Added sorting of the documents in the agenda, minutes, bluesheets section of the session_details view. Fixes #2759. * Merged in [16592] from rjsparks@nostrum.com: Changed how the session details page links to documents to not link to a specific version. Fixes #2758. See also #2756. * Merged in [16590] from rjsparks@nostrum.com: Example of using webtest. The full testcase was not converted because the add_rule form is modified in-flight by javascript when a control value is changed. * Merged in [16583] from rjsparks@nostrum.com: Restored the ability to remove documents while managing a personal list. Required updating code to match the recent change to Document's primary key. Fixes #2757. * Merged in [16569] from rjsparks@nostrum.com: Added quotes around document name in the mailarchive search link on a document's main page. Fixes #2659. * Merged in [16589] from rjsparks@nostrum.com: Added a calendar to the upcoming meetings page. Fixes #2719. * Prevent login name case issues by generating apache permissions file entries for both uppercase and lowercase logins. * Added a Py3 version of requirements.txt, excluding Trac which is still only available under Py2.7 * Did some preliminary work on converting bower management of web resources to yarn. New management command, replacing bower_install will be needed. * Merged in [16562] from rcross@amsl.com: Added encoding='utf-8' parameter to io.open operations in tests that starting failing since adding unicode_literals. * Merged in [16561] from rjsparks@nostrum.com: Added a prompt for login for approving slide suggestions instead of just presenting an error. Fixes #2750. * Merged in [16559] from fenton@bluepopcorn.net: Added Closing Notes to history and about pages for groups. Fixes issue #2725. * Merged in [16558] from rjsparks@nostrum.com: Made removed IPR disclosures visually distinctive on search results. Partially addresses #2687. * Merged in [16557] from pusateri@bangj.com: Added the ability to upload agenda and minutes in markdown format. Partially addresses issue #1926. We're still displaying markdown as text. Fixes #2704. * Fixed a number of additional Py2/3 compatibility issues. * Added LookupError to the exceptions caught during text upload decoding, leading to upload form validation error. * Merged in [16554] from rjsparks@nostrum.com: Rework how SlideSuggestion objects name their files. Improve the UI when there are multiple pending suggestions for updating the same presentation. * Moved djangobwr into the local repository, due to consistent difficulties pulling it from github as an svn:external * Enabled display of group pages also for groups of type adhoc and ise. * Updated the PLAN to adjust priority on modeling BCPs correctly * Added another check before building submitter email for mailtrigger entries, to ensure valid email addresses. -- Henrik Levkowetz 05 Aug 2019 16:43:03 +0000 ietfdb (6.99.1) ietf; urgency=medium This bugfix release contains a number of fixes for issues with the big Python 3 conversion release, many of which has already been patched in production. Additionally, there are some fixes related to new slide upload functionality and the test suite. From the commit log: * Fixed some issues with the Py2/3 changes to the populate_yang_model_dirs management command. * Added a check for submitter found before trying to get submitter name and email when generating email notification address lists. * Shortened generated document titles a tad, to avoid line wrapping in the IESG agenda package (which sometimes breaks the relevant tests). * Merged in [16515] from rjsparks@nostrum.com: Save files correctly when updates to an existing set of slides are proposed. Fixes #2748. * Added information about versions (Datatracker, Python, and Django) at the beginning of test suite runs, to be able to clearly see which Python and Django versions were used during a test run. * Replaced six.ensure_str with django's force_str, in order to be able to use earlier versions of six which more frequently are available on the development and production target platforms. * Changed PDF page-counting code to read PDF files as binary. * Replaced the email-sending logger.error() on bad Submission.submitter email address with syslog logging. * Added an email gather handler for submission_submitter that tries an Alias name lookuup if the submitter doesn't look as if it contains an email address. This deals better with submission records that only has a name in the submitter field. * Changed import path for settings_local.py to work with absolute_import under Py2, and also under Py3. * Changed agenda file reading to read files as binary when looking for draft names listed in the agenda, in order to avoid issues with older uploads using latin-1 or other encodings where the lower 127 character match ASCII. -- Henrik Levkowetz 18 Jul 2019 20:34:35 +0000 ietfdb (6.99.0) ietf; urgency=medium **Python 3 compatible code** This is a major release which contains conversion of the datatracker Python code to run under Python-3.7 where it previously was specific to Python-2.7. The code was first converted to Python-3.7, and then additional work was done to make it compatible with both Python 2 and 3. The changes updated 458 files, and touched close to 4000 lines. Making the code compatible with both Python 2 and 3 lets us continue to use the converted codebase until we're ready to move the primary webserver's mod_wsgi completely to Python 3. Otherwise, we would have to merge all new bugfixes and sprint contributions to both trunk (Python 2.7) and the new Python 3 branch, with doubled merge work until we were ready to move the server to Python 3. (Testing, though, will still need to be done under both 2 and 3 to ensure we don't loose compatibility.) -- Henrik Levkowetz 16 Jul 2019 22:21:56 +0000 ietfdb (6.98.4) ietf; urgency=medium This is a small bugfix release, to clear the slate before merging in the Python 2/3 conversion. From the commit log: * Added a validation step for SearchablePersonField, to avoid later server 500 errors on bad input. * Merged in [16404] from rjsparks@nostrum.com: Only look for ietf/datatracker-env images when extracting the most recent build to tag as latest. * Merged in [16359] from rcross@amsl.com: Fix registration import, use user.person if it exists. * Merged in [16096] from rcross@amsl.com: Update admin permissions. Grant secretariat change permissions on dbtemplate so they can update proceedings pages. * Turned off html autoescape in IPR email templates. This fixes inappropriate html escapes that occurred in various IPR-related emails. * Added missing code to skip coverage measurement for skippable tests. -- Henrik Levkowetz 16 Jul 2019 14:14:12 +0000 ietfdb (6.98.3) ietf; urgency=medium * Increased the length of the list Subscribed email field from 64 to 128, updated the import_mailman_listinfo management command, and added a migration for the model change. -- Henrik Levkowetz 04 Jul 2019 16:07:14 +0000 ietfdb (6.98.2) ietf; urgency=medium This is a minor release that fixes some bugs and tweaks some settings. From the commit log: * Changed some tests to match production group type changes for some special groups. * Made the bin/daily cron script quiteter. * Various changes to make yang-related management commands quieter * Added a fix for an issue in get_meeting_registration_data() which could happen if we tried to create a Person record for with a user of an existing Person. * Changed the permitted length of patent_title from 127 to 255. * Added additional guards against duplicate m2m entries. * Added a utility to check copyright statements in specified files. * Updated some functions and views in secr/srec/ to use GroupFeatures instead of hardcoded lists of group types. * Made session requests also work for ad-hoc groups, which may not have a parent area. -- Henrik Levkowetz 03 Jul 2019 20:27:54 +0000 ietfdb (6.98.1) ietf; urgency=medium This is a bugfix release that cleans up some remaining issues from the Document/DocAlias primary key refactoring, and also fixes some other unrelated issues. From the commit log: * Fixed bad logic for the display of previous related IPR information in the IPR details template. * Tweaked some test failure messages for better debugging. * Merged in [16287] from rjsparks@nostrum.com: Send mail to a list when a new IETF WG -00 is submitted. Fixes #2730. * Fixed some name versus pk issues lingering after the Document/DocAlias primary key refactoring. Fixes issue #2729. * Added the option to order API resource lists by primary key, for stable ordering when paging through a list. * Added generation of Meta.ordering for API ModelResources in the * mkresources management command. * The Document and DocAlias primary key refactoring caused the Tastypie API under /api/v1/ to change -- reverting that to use the old form, with name ask API key. * Added a better error message for too long draft names during submission. * Adjusted the test_runner to generate migration coverage data also when permitting mixed migrations. * Added a guard against duplicate insertions in SearchRule.name_contains_index * Fixed a docalias problem with generate-draft-aliases * Merged in [16248] from rjsparks@nostrum.com: Removed the IETF chair as a recipient of new IPR disclosures if there is no group or active document to send them to. Fixes #2727. * Added a management command to update the index table used by some community list search rules. Running this fixes the missing listings of drafts related to a group on group document pages and in searches. * Improved the support for the mixed-release switch in the bin/mkrelease script. -- Henrik Levkowetz 20 Jun 2019 13:16:54 +0000 ietfdb (6.98.0) ietf; urgency=medium **Refactoring of the primary index type for Document and DocAlias objects** This release contains a series of 216 migration steps that changes the Document and DocAlias primary keys from character strings to integers, and makes corresponding code changes. This was prompted by database limitations discovered when trying to make DocAlias use a m2m document field; with 255 long strings as primary keys for Document and DocAlias this violated the MySQL database limitations. Changing the primary keys to integers should also improve efficiency. Additionally, in order to make it possible to have DocAlias objects referring to multiple documents, in order to reflect STD and BCP names that refer to multiple RFCs, DocAlias has been refactored to use many-to-many tables instead of foreign keys do toduments. Due to the data migrations which create the new integer primary keys and adds corresponding integer foreign keys matching the previous string foreign keys in all tables having foreign keys to Document and DocAlias, some of these migrations take a long time. The total set of migrations are expected to have a runtime on the order of 2 hours. This release is scheduled to be deployed on 16 Jun 2019 at 16:00 UTC, with an expected datatracker downtime, for safety backup and application of the migrations to run, of about 3 hours. -- Henrik Levkowetz 16 Jun 2019 14:16:12 +0000 ietfdb (6.97.1) ietf; urgency=medium This is a bugfix release which clears the way for the upcoming major release. From the commit log: * Added validation of draft names extracted from XML submission files. * Added static resources for favicon.ico and robots.txt at the top of the ietf static tree. * Updated references to RFC 5226, to RFC 8126, which obsoletes 5226. This fixes issue #2726. * Fixed an issue where the 'Request closing group' button incorrectly was showing for group chairs. Fixes issue #2724. * Moved some explicit URLs from code and templates to settings (related to the production server www6/ directory cleanup). * Refined the handling of draft xml version 3 versus version 2. This fixes a problem where v3 documents were rendered with no section titles in the Table of Contents. -- Henrik Levkowetz 14 Jun 2019 13:30:43 +0000 ietfdb (6.97.0) ietf; urgency=medium **Additional meeting-related links** This is a small feature release which provides a new email notification to the RFC Editor, new meeting-related menu links and iCalendar links, and restores some review management page information inadvertently lost in release 6.96.0. From the commit log: * Added a link to the meeting registration page to the meetings menu. * Added a link to the registration page from the meeting 'Important Dates' page. * Removed additional instances of links to unposted IPR disclosures that would result in 404 pages. * After feedback from the RFC-Editor staff, changed the URL POST notification of Group name changes to an Email notification. * Added .ics links for future meetings to the group meetings tab. * Added a signal receiver for notifications to the RFC editor about changed Group names. * Added sample notification JWT signature verification code to /api/index.html. * Updated settings with EC/P-256 api keys (JWT does not support RS2048, which is what we had). Added a variable for RFC Editor group name notifications, with a placeholder value. * Fixed template and view to restore previous review display on the Manage Unassigned Reviews page. Fixes issue #2718. * Tweaked the review assignment presentation in review/request_info.html. * Added review assignment information to the review-document view. Addresses issue #2717. -- Henrik Levkowetz 23 May 2019 15:09:45 +0000 ietfdb (6.96.0) ietf; urgency=medium **Refactored review request handling, introducing review assingnments** This release splits out document review assignments as individual objects, instead of being part of review requests. This makes handling of review assignments more flexible and intuitive. The datatracker menu URL for past meeting proceedings has been corrected, as well as the URL for BOF requests, and old, now invalid URLs to MHonArc mailing list repositories have been removed. The Wiki maintenance scripts now support a generic NomCom wiki, managed by the chairs and adivisors of currently active NomComs. Additional details from the commit log: * Merged in a refactoring of document review code from rjsparks@nostrum.com. This introduces ReviewAssignments, letting us do multiple assingments for a given RevieRequest, without conflating the assignments and their results with the request itself. * Updated the ReviewAssignment api resource to match the current model. * Added a no-follow option to the test crawler, in order to be able to easily test a specific list of URLs. * Modified a template to handle old reviews with incomplete information appropriately, instead of giving 500 errors. * Fixed a problem in 3 views that were accessible through URLs without meeting numbers (e.g., .../meeting/proceedings/overview/) but gave 500 errors when accessed. * Corrected an old URL for past meeting proceedings. * Fixed some migrations issues that prevented the review assignment refactoring migrations from being reversed and re-applied. * Changed a ReviewAssignmentDocEvent type from 'closed_review_request' to 'closed_review_assignment', for consistency and readability. * After having test_duplicate_person_name() fail for a non-ascii name, changed the test to always supply a non-ascii name. * Removed links to the old MHonArc mail archives, and related view. * Changed the BOF request procedures page URL. * Tweaked the greate_group_wikis management command to accept trailing fileglob wildcards on adhoc wiki group acronyms. Added a generic NomCom wiki to the list of adhoc wikis to be created and maintained, with admins from active nomcom* groups. * Fixed bad html in a proceedings template. -- Henrik Levkowetz 06 May 2019 15:34:13 +0000 ietfdb (6.95.1) ietf; urgency=medium This is a bugfix release, addressing the following issues: * Modified the display of updates to a rejected IPR to avoid links that result in 404 pages. * Combined [16154] from rjsparks@nostrum.com with previous work on ticket #2633 in [16124]. This adds bluesheet links to the meeting materials page. Fixes issue #2633. * Merged in [16152] from rjsparks@nostrum.com: Show minutes on the popup for the meeting materials button on the agenda page when those minutes are available. Fixes #2711. * Updated the role handling to use GroupFeatures.groupman_roles consistently for group management access. Fixes a IRTF RG delegate permissions issue. * Fixed some program group feature settings. * Added a missing test directory cleanup action. * Merged in [16145] from housley@vigilsec.com: Updated the out-of-date links in the footer of each datatracker page. Fixes #2710. * Merged in [16144] from housley@vigilsec.com: Removed some clutter from I-D upload page. The information was moved to the Instructions tab, but a higligh to the Note Well and BCP 78 was added. Fixes #2057. * Added some HTTP header settings for better security. Brings results at https://securityheaders.com/ up to an 'A' grade. -- Henrik Levkowetz 26 Apr 2019 11:23:57 +0000 ietfdb (6.95.0) ietf; urgency=medium **IETF 104 code sprint** This release contains datatracker bug fixes and enhancements from the IETF-104 Code Sprint. We have several first-time sprint contributors, and a substantial set of improvements. Thanks to everyone who contributed! Notable improvements: The datatracker now supports reminders of and direct addition of downref entries at the time of document approval. Document review management has been improved with table sort options. There are a number of improvements to the agenda and materials pages, and there is now support for meeting session presenters to upload their own slides, for later approval by chairs. Additional comments, excerpted from the commit log: * Merged in [16126] from housley@vigilsec.com: Allow Secretariat to handle downrefs when they approve a document * Merged in [16124] from rcross@amsl.com: Add No Bluesheets indicator to materials page. * Merged in [16121] from rjsparks@nostrum.com: Provides menubar offset for wg anchors on the /wg page. Fixes #1481. * Merged in [16110] from rjsparks@nostrum.com: Include the note itself in email sent when the RFC editor note changes after document approval. Also changed the flow to redirect back to the writeups pages after editing the RFC editor note. Fixes #2672. * Merged in [16109] from rjsparks@nostrum.com: Protect drafts in certain IRTF states from expiring. Fixes #2669. * Merged in [16108] from fenton@bluepopcorn.net: Moved old nomcom announcements to database and removed references to legacy website. Fixes #2686. * Merged in [16107] from rjsparks@nostrum.com: Show the agenda note, if it exits, on a sessions SUMMARY line in the ical view of an agenda. Fixes #2676. * Merged in [16106] from rjsparks@nostrum.com: Don't allow users to edit the 'active' value for unknown-email-* addresses. Related to #2696. * Merged in [16103] from rjsparks@nostrum.com: Modified the admin model for SlideSubmission to use a couple of raw_id_field widgets. * Merged in [16102] from rjsparks@nostrum.com: Allow logged in users to propose slides for meeting sessions. Fixes #2547 and #2403. * Merged in [16095] from peter@akayla.com: Fixed bad calls to rfcdiff for -00 drafts and replaced drafts. Fixes issue #2697. * Merged in [16094] from magnus.westerlund@ericsson.com: Added sortable columns to Review Team Reviewers page. Partially addresses issue #2656 * Merged in [16092] from rcross@amsl.com: Remove ability to cancel a break session. Fixes #2703. * Changed the etherpad url schema to https. * Temorary workaround to upload xml validation. The tree.validate() of xml2rfc should be updated to check vocabulary version and validate against the appropriate schema, to fix this properly. * Added dbtemplate validation to the DBTemplate class, to make sure one cannot enter invalid DBtemplates through the admin. * Merged in [16089] from mahoney@nostrum.com: added instructions for entering the draft name. Fixes #1375. * Added creation of a group with the user's GID inside the docker container, if it's missing. * Merged in [16088] from rcross@amsl.com: Fixed multiple issues with secretariat non-session view, including an indicator of cancelled sessions. Fixes #2701. * Merged in [16087] from housley@vigilsec.com: Add downward references to telechat moderator package and Secretariat telechat pages. * Merged in [16085] from rcross@amsl.com: Fixed an issue where cancelling one break session would cause other breaks to get cancelled. Fixes #2699. * Added filtering by meeting for the UrlResource admin. * Changed docker/updatedb to use https://, as cloudflare doesn't do the right thing for our http:// database dump URL any more. * Added a skipIf() decorator for a test which should not be run unless the ghostscript binary is available in the test environment. -- Henrik Levkowetz 09 Apr 2019 11:55:21 +0000 ietfdb (6.94.1) ietf; urgency=medium This is a minor bugfix release, to clear the slate for the IETF 104 sprint. * Put a length limit to generated cache keys in stats/views.py, to avoid tickling memcached's limit of 250. * Changed sending of PubReq messages to explicitly use on-behalf-of, without Reply-To. Reply-To is picked up as requestor by RT. Fixes issue #2691 anew. * Changed etherpad server address from etherpad.tools.ietf.org to etherpad.ietf.org. -- Henrik Levkowetz 19 Mar 2019 18:01:07 +0000 ietfdb (6.94.0) ietf; urgency=medium **Added review and AD emails 'Reply-To:' fields** This release contains another change to email sending, adding a Reply-To header field for mail sent on behalf of a user, and also contains a number of other changes and fixes, as follows: * Merged in [16022] from rcross@amsl.com: Modified IPR Patent number regex to allow a new Italian patent number format, IT[15 digits]. Fixes #2694. * Reverted earlier application of on_behalf_of() at mail sending call points, and instead did all on_behalf_of processing in condition_message(). Added insertion of Reply-To header fields in this case. Changed all use of the mail-sending 'extra' parameters to use value lists. Updated a bunch of tests accordingly. * Removed a statement about IETF being an activity of the Internet Society from a template file, and tweaked the remaining text. * Converted a test to use .assertIn() instead of .assertTrue(... in ...) for better failure reports. * Tweaked the group_features_group_filter() function. * Refactored some templates to reduce materials page permission lookups from ~2*#sessions to just a few. * Rewrote a function to be more readable and actually do the right thing. This moved some filtering from database into code, but it seems acceptable here. * Transitioned group/views.py to unicode strings * Improved a somewhat misleading UI for adding account email addresses. Fixes issue #2692. * Changed the on-behalf-of name template from 'On behalf of ...' to '... via Datatracker' to make the originator's name come first. * Changed ietf/utils/mail.py to use unicode strings, and changed a call in ietf/review/utils.py to use unicode for mail Subject, to fix a server 500 issue. -- Henrik Levkowetz 13 Mar 2019 15:09:42 +0000 ietfdb (6.93.0) ietf; urgency=medium **Changed review and AD emails to use 'On behalf of ...' sender** This release changes the email sender for emails sent through the datatracker by ADs and reviewers to say "On behalf of" and the username, with a datatracker email address, instead of using the user's from address, since that has increasingly (and rightly) been running into problems with SPF and DMARC processing. The release also provides session short names in the meeting/agenda.json file, for use by Meetecho scripts. From the commit log: * Added on_behalf_of() calls for from addresses needing it throughout the code. * Added html escaping of initial text-area content in a view function to avoid mangling. * Updated the tests to always use @ietf.org addresses where appropriate, and not do so when not appropriate. All test user login addresses have been changed away from @ietf.org. * Added two new functions to replace email From: of users with On-behalf-of addresses, with associated values in settings. Added a catchall stanza in the mail sending pipeline to catch anything not already changed to On-behalf-of addresses, which should have been changed. * Merged in [15982] from rjsparks@nostrum.com: Add session.short to the output from meeting/agenda.json when it is available. -- Henrik Levkowetz 05 Mar 2019 17:52:55 +0000 ietfdb (6.92.1) ietf; urgency=medium This release contains one database schema migration delayed from the previous release in order to deploy new code before a fields used by old code was removed, and one migration to extend the width of a field used to hold URLs. From the commit log: * Increased the DocumentURL.url field length to the legal URL maximum of 2083. * Removed the GroupFeatures.has_dependencies field. * Added a note to the draft/RFC stats pages about missing data. -- Henrik Levkowetz 26 Feb 2019 12:27:22 +0000 ietfdb (6.92.0) ietf; urgency=medium **Generalized the submission approval protection for special groups** Before this rewrite, code explicitly looked for draft names like draf-iab-, draft-iana=, etc., but did not derive the list of draft name prefixes needing approval from the group types in the database. The new code extends this protection to any new group types in the database (and a number of old ones that were not protected before such as draft-iesg- ). There are also some bugfixes. From the commit log: * Increased the document count for group document lists from the default(200) to 500, as MPLS now exceeds 200. * Rewrote the submission deduce_group() to consider all groups with acronyms that are also GroupTypeNames. Added GroupTypeName for iesg,ise,adhoc groups, and appropriate GroupFeatures for each. * Fixed a problem with the secretariat IESG agenda handling, where removal of a document from the telechat agenda could lead to a server 500 in a different view. Insetead, redirect to the removed document (which will show as not on the agenda any more). * Tweaked mkdevbranch to take an optional extra message paragraph. * Modified a view function to fill in missing default CommunityLists for groups with features.has_documents == True, to avoid a possible server error. * Removed the has_dependencies GroupFeature, and used has_documents instead in the one place it was used. * Fixed a submission failure for xml files missing abstract sections. -- Henrik Levkowetz 25 Feb 2019 13:08:34 +0000 ietfdb (6.91.0) ietf; urgency=low **Expanded group property table entries** This release contains a refactoring of a large number of instances in the code where group types (wg, area, rg, etc.) have been hardcoded in the past in order to reflect different properties for different groups. Instead, group types have now aquired a list of new 'feature' settings, for instance is_schedulable, show_on_agenda, has_session_materials, and more. This will make it easier to add new group types without having to change the code, by indicating for each new group type which properties it should have. Additionally, some minor unrelated fixes has been applied. From the commit log: * Merged in GroupFeatures refactoring and expansion from ^/personal/henrik/6.89.4.dev0@15911. * Reverted r15071 with respects to showing the submit for publication button to shepherds. Fixes issue #2673. * Updated code and templates to use the new docman and groupman group features. * Updated fixture with new group features and more. * Added a tiny management command to list group types per feature. * Removed origin information from the liaison_title.html template, as it's used in a context where html comments will be rendered. Fixes issue #2674. -- Henrik Levkowetz 04 Feb 2019 11:57:59 +0000 ietfdb (6.90.0) ietf; urgency=low * Modified the rolodex code to make sure we capture information about the origin of newly added email addresses. This requires the secretariat to be clear about where new email addresses come from, in order to be able to remove them if we later receive a GDPR personal data deletion request. * In the draft metadata extraction code, moved the __future__ imports down so as not to obscure the module docstring. Fixes the earlier inability to run '$ ietf/utils/draft.py -h'. * Applied a series of patches for python 3 compatibility from dkg@fifthhorseman.net to the draft metadata extraction code: - Use a list of dictionary keys. In python3, dict.keys() produces a dict_keys object, not a list. Since this code treats it as a list, we'll just be explicit about that. - Fix regex manipulation for word characters. In python 3.7, re.sub() started treating unknown escape sequences in as errors. Fix this by sending an escaped \ where we mean to pass it through raw. - Use // for explicit integer division. Without this fix, in modern versions of python, the changed line produces: 'TypeError: 'float' object cannot be interpreted as an integer'. - Use modern form of exception handling. - Use the print function instead of print statement. * Added more name normalization for test factory person names. -- Henrik Levkowetz 21 Jan 2019 11:49:20 +0000 ietfdb (6.89.4) ietf; urgency=low * Moved __future__ imports down in ietf/utils/draft.py so as not to obscure the module docstring. Fixes inability to run '$ ietf/utils/draft.py -h'. * Applied 5 patches from dkg@fifthhorseman.net, for py3 compatibility in the text draft metainfo extractor ietf/utils/draft.py: - Use a list of dictionary keys. In python3, dict.keys() produces a dict_keys object, not a list. Since this code treats it as a list, we'll just be explicit about that. - Fix regex manipulation for word characters. In python 3.7, re.sub() started treating unknown escape sequences in as errors. Fix this by sending an escaped \ where we mean to pass it through raw. See https://docs.python.org/3/library/re.html#re.sub - Use // for explicit integer division. Without this fix, in modern versions of python, the changed line produces: TypeError: 'float' object cannot be interpreted as an integer - Use modern form of exception handling - Use print() function instead of print statement * Added a bit of name normalization for test factory person names. -- Henrik Levkowetz 14 Jan 2019 22:29:06 +0000 ietfdb (6.89.3) ietf; urgency=low * Updated the timezone information to match the latest timezone database release (2018i) * Merged in [15865] from rjsparks@nostrum.com: Cleanup: removed the infrastructure for importing reviews from the old review tool. * Merged in [15864] from rjsparks@nostrum.com: Refactored DocumentInfo to address overloading the external_url field with strings that are not URLs. * Merged in [15846] from rjsparks@nostrum.com: Simple tests of some log utilities. * Merged in [15845] from rjsparks@nostrum.com: simple test of BowerStorageFinder. * Merged in [15833] from rjsparks@nostrum.com: Incremental improvements to the submit_existing tests. * Merged in [15829] from rjsparks@nostrum.com: Make state/substate changes on new submissions sensitive to stream. Fixes #2642. * Merged in [15828] from rjsparks@nostrum.com: Show more group types on the concluded groups page. Fixes #2643. * Additional tweaks to submission yang testing and the test runner * Updated PLAN -- Henrik Levkowetz 14 Jan 2019 15:25:32 +0000 ietfdb (6.89.2) ietf; urgency=low * Added a migration to capture changes to HistoricalRecords triggered by changes in django-simple-history * Bumped the minimum version requirement on django-simple-history; HistoricalChanges which we now use was introduced with 2.3.0. * Changed the construction of element IDs that could trigger ad-blockers in a NomCom page to avoid the problem. * Removed an assertion for iesg state for documents that can be in any stream. * Merged in [15824] from rjsparks@nostrum.com: Make it more obvious that a group has been replaced when looking at its pages. Fixes #2661. * Merged in [15818] from rjsparks@nostrum.com: Whitespace only change to make the conditional structure followable. * Merged in [15822] and [15809] from rjsparks@nostrum.com: Added an explicit ID-Exists state for the IESG state machine. Reworked code so that the IESG state machine always has a state. Added the ability to release a document from a working group, research group, or the independent stream. Releasing a document removes all stream state, and sets the document to have no stream. * Fixed a bug in the RFC Index parsing that caused extraction of only one paragraph of an abstract, even if there were multiple. * Merged in [15816] from rjsparks@nostrum.com: Provide a correct link to jabber logs for plenaries in past agendas. Fixes #2641. * Merged in [15810] from rcross@amsl.com: Grant secretariat admin permissions to mailtrigger models. Fixes #2637. * Merged in [15808] from rjsparks@nostrum.com: Allow notify_ad_when to be blank. * Merged in [15807] from rjsparks@nostrum.com: Repaired another instance of a literal path with a reverse lookup. Related to #2657. * Added a description of the /doc//doc.json API to the /api/ index page, and also added a table-of-contents column. * Merged in [15806] from rjsparks@nostrum.com: Changed an inappropriate literal path to a reverse url lookup. Fixes #2657. * Merged in [15804] from rjsparks@nostrum.com: Add history to ReviewerSettings. Fixes #2147. -- Henrik Levkowetz 19 Dec 2018 08:27:48 +0000 ietfdb (6.89.1) ietf; urgency=low * Merged in [15787] from rcross@amsl.com: Rework page flow and forms of secr/meetings app. Fixes #2609. * Added code to catch a possible attribute error on event notification. * Added two new management commands to make it easier to load back selected objects that have been removed by mistake (providing they are available in a full database dump or backup that can be loaded and worked with): dumprelated and loadrelated. * Increased the document limit from 200 to 500 for the AD documents page. -- Henrik Levkowetz 03 Dec 2018 18:09:35 +0000 ietfdb (6.89.0) ietf; urgency=low This release provides improved export of statistics charts and data, improved and more consistent test coverage, and fixes a few bugs: * Fixed an issue with the To: and Cc: addresses of extra review email to responsible AD. * Changed the liaison tool From: address to . * Always show the liaison submitted date on liaison pages * Added testing of unidecode_name() and name_parts(), in order to not rely on random name generation to excercise all paths. * Updated highcharts from v4.2.7 to v6.2.0, and added support for CSV export from charts. Also tweaked the look of the highcharts context menu separators. * Added handling for an EOF exception that can occur when using the django file-based cache. -- Henrik Levkowetz 20 Nov 2018 16:30:42 +0000 ietfdb (6.88.0) ietf; urgency=low **IETF 103 code sprint** This release contains datatracker bug fixes and enhancements from the IETF-103 Code Sprint. Again, this looks to be a sprint with an extremely large number of contributions. Thanks to everyone who contributed! Notable improvements: Reviewer statistics are now more readable. Reasons for reviewer unavailability is now only shown to authorised persons. It is now possible to set a mail alias for review group secretaries, instead of having individual mails sent on updates. AD notifications about reviews can now be controlled better and the default settings are better tuned. Import of review text from mailing lists should now handle zero search results better. Logging in is now possible using all active email addresses registered for an account, not only the primary login name. Inaccurate, misspelled or misleading text has been fixed for a number of web pages and email messages. Additionally, a large number of individual bugs has been squashed: * Merged in [15722] from rcross@amsl.com: Added a link to meeting/../agendas/list. Fixes #2627. * Merged in [15720] from rjsparks@nostrum.com: Invalidate the get_state_slug cache when the get_state cache is invalidated. Fixes #2630. * Merged in [15719] from rcross@amsl.com: Removed resources field from secr/meetings/../rooms. Tweaked CSS for this page to eliminate horizontal scrolling. Includes new base templates to facilitate transition to bootstrap. Fixes #2628. * Merged in [15717] from rcross@amsl.com: Reworked the buttons on secr/meetings. Fixes #2629. * Merged in [15715] from rcross@amsl.com: Fixed the issue where new non-session location field value wasn't saved. Fixes #2625. * Merged in [15709] from rjsparks@nostrum.com: Show counts on the active groups page and on the active WGs page. Fixes #2060. * Updated the script to fix ambiguous time stamps so that it will fix ambiguous time fields for additional types of objects. * Added a test for various name part combinations, and fixed a couple of issues with utils.name.name_parts(), making the test coverage percentages more stable. * Added exclusion of cancelled session when doing audio file import. This fixes an server 500 error when a cancelled session is shown in the same room and time slot as an actual session. * Corrected the rendering of non-ASCII email names in nomcom templates. * Merged in [15708] and [15730] from housley@vigilsec.com: Added a mail alias for all review team secretaries. Fixes #2139. Use the alias if there is one for email about review request changes. Added a test for the review team secretary alias functionality. * Added a test that checks the mailarch search failure return, and tweaks the no-result return handling. * Merged in [15706] from kivinen@iki.fi: Added easier to detect error message when search returns 0 results. This currently parses HTML result from the mail archive system, but when we get proper API to mail archive system it should be changed to do that instead. Fixes #2126. * Merged in [15705] from johnl@taugh.com: Trivial fix for trivial issue #1907, warn user that added addresses will be confirmed. * Merged in [15702] from johnl@taugh.com: Allow email as well as username when logging in. * Merged in [15701] from kivinen@iki.fi: Changed format of the reviewer statistics in lists. Fixes #2131. * Merged in [15683] and [15699] from housley@vigilsec.com: Do not show reasons for review team member unavailability outside the team. Fixes #2614. * Added normalisation of the email addresses returned by ietf.nomcom.utils.parse_email, in order to not treat feedback from the same person as coming from different sources due to email name+address variations. * Merged in [15698] from rjsparks@nostrum.com: Notify the RFC Editor when an RFC Editor note is changed after a draft has been approved. Fixes #2440. * Merged in [15696] from peter@akayla.com: Minor text fixes. Fixes issue #2621. * Merged in [15694] from rjsparks@nostrum.com: Clearly label cancelled sessions on the week-view on the agenda page. Fixes #2619. * Merged in [15693] from mahoney@nostrum.com: Fixed link to webex request page. Fixes issue #2564. * Added a guard against getting the attribute of None in api_set_session_video_url(). Fixes a server 500 for some hackathon sessions. * Provided a tool-tip with full time information for the document history dates. * Merged in [15689] from mahoney@nostrum.com: It need not be an error to find duplicate person records, such as for instance for 'Matt Larson'. Took out the statement in the profile template that this is an error. Fixes issue #2616. * Factored out some parts of the meeting/agenda.html into separate files, and added session buttons for 'other' session (e.g., hotrfc) * Merged in [15688] from mahoney@nostrum.com: Fixed bad variable in title of requests.html. Fixes issue #2608. * Don't show slides in main IETF meeting agenda for sessions with their own session agenda. * Merged in [15684] from rjsparks@nostrum.com: Removed the field and widget that made editing document states easier on the document admin page since they do not survive validation failure of other fields on the form. Excluded states from the model form since the default states widget is almost impossible to use without producing bad data. If states need to be edited through the admin, it will, for now, need to be done gruelingly through the admin form for State. Fixes #2524. * Merged in [15681] from rjsparks@nostrum.com: Only do state-parent cross-validation on the group edit form when both fields are used on the form. (Many invocations of the form suppress all but one field.) Fixes #2597. * Merged in [15679] from mahoney@nostrum.com: Verb tense used in Ballot Issued emails now matches whether the LC end date is in the past or future. Fixes issue #2482. * Merged in [15677] from rjsparks@nostrum.com: Tune the recipients when a teams settings or a reviewer indicate ADs should be notified about a review. * Merged in [15676] from rjsparks@nostrum.com: Let a reviewer indicate that an AD should give this review early attention. Fixes #2553. * Merged in [15675] from rjsparks@nostrum.com: Let review teams opt in to poking a responsible AD when unhappy reviews are submitted. Fixes #2544. -- Henrik Levkowetz 13 Nov 2018 09:47:39 +0000 ietfdb (6.87.3) ietf; urgency=low This release contains one single fix, to give stable test results for the test_edit_timeslot_type() test. * Merged in [15648] from rjsparks@nostrum.com: Use only ietf meetings for testing editing timeslots. -- Henrik Levkowetz 02 Nov 2018 02:51:32 +0000 ietfdb (6.87.2) ietf; urgency=medium This is a small bugfix release to clean the slate before the code sprint. From the commit log: * Merged in [15613] from rjsparks@nostrum.com: Make having a trailing slash optional for doc/html/docname. Fixes #2247. * Merged in [15603] from rjsparks@nostrum.com: Changed some text and button labels to make the workflow for commiting group schedules more self-documenting. * Clarified that the document search author field accepts also author affiliation as a search term. * Removed microscopic photos from nomcom feedback page buttons. Few browsers render them correctly with badges, and they are too small to provide meaningful information. * Changed the generation of the 1id-index.txt file to not list individual drafts in states Candidate for WG Adoption and Call for WG Adoption Issued as group drafts. -- Henrik Levkowetz 31 Oct 2018 16:13:17 +0000 ietfdb (6.87.1) ietf; urgency=medium This is a bugfix release, whith the following changes: * Merged in [15601] from rjsparks@nostrum.com: Corrected some DocTypeName entries' prefix value. Made the search code more robust against DocTypeName objects that have an empty prefix value. Fixes #2599. * Merged in [15596] from rjsparks@nostrum.com: Improved coverage for several tests. * Merged in [15595] from rjsparks@nostrum.com: Converted stats, submit, sync. and utils/test to use factories. Excluded utils/test_data from coverage checks. * Fixed some incorrect topic audience id slugs, to make the NomCom topic availability filtering work correctly. * Updated the nomcom reference from 3777 to 7437 on the Person profile edit page. * Merged in [15580] from rjsparks@nostrum.com: Better constrain a liaison test object to match the preconditions for the test. * Tweaked the mail sending routines' handling of the copy flag. * Changed the mail sending routines so it's possible to not copy the debug archive on some messages. * Added 'os.chdir()' to virtualenv_activate to place us correctly to pick up patch files. * Added Message-ID to log lines for sent email messages. -- Henrik Levkowetz 22 Oct 2018 16:17:25 +0000 ietfdb (6.87.0) ietf; urgency=medium **Nomcom, agenda, and test improvements, new API for secretariat use** This is a small feature release, with the following changes: * Updated the nomcom admin pages for Feedback to make it possible to see and sort on the Nominees field. * Merged in [15551] from rjsparks@nostrum.com: Used factories instead of make_test_data for malinglists, mailtrigger, message, person, secr/announcement, and secr/areas tests. * Merged in [15550] from rjsparks@nostrum.com: Added meeting.agenda_info_note. * Merged in [15549] from rjsparks@nostrum.com: Renamed meeting.agenda_note to meeting.agenda_warning_note. * Changed the 'View' button on the NomCom email classification page to render the email body within 
, and to not offer classification 
    choices, as this encourages a slow workflow, and also seems to have been 
    broken for some time.

  * Bleach 3.0.0 has a bug that can result in exceptions for some inputs.  
    Bug report filed.  Disallowed that version in requirements.txt

  * Turned the nomcom chair dropdown menu item 'Classify pending feedback' 
    into a tab with a mail count badge.

  * Merged in [15263], [15264], [15265], [15287] from rcross@amsl.com:
    Added new custom API endpoints for export of person data, restricted
    to secretariat use.  Added a test for the new custom API.

 -- Henrik Levkowetz   15 Oct 2018 05:46:07 +0000


ietfdb (6.86.0) ietf; urgency=medium

  **NomCom tool improvements**

  This is a feature release with only small tweaks, mostly related to the
  NomCom tools, but there are also some IESG page, test suite, account, and
  utility script improvements, 

  * Added pagination to the nomcom pending-feedback page, to reduce load 
    time.

  * Changed the default sort for nomcom.models.Nominees to be 
    'person__name'.

  * Merged in [15523] from rjsparks@nostrum.com:
    Converted the liaisons tests to use factories.  

  * Added a '--reminder' switch to the GDPR request mailer, for repeat 
    notifications.

  * Merged in [15522] from rcross@amsl.com:
    Made email address read-only in rolodex edit form.  Fixes #2602.   

  * Prevent issues for logins without Person records by not letting 
    Person-less users through login.

  * Added an send_email field to BallotPostitionDocEvent to reflect whether
    the 'Send email' button was used to save the ballot position.  Added icons
    on documents' IESG evaluation record to reflect this.

  * Adjusted the test_submit_nonascii_name() to match the output from 
    idnits 2.16.0, which emits warnings rather than errors for nonascii content.

  * Built out test_edit_nomcom_view() to also test adding and removing 
    reminder dates.

  * Added a form clean() for the reminder dates of the nomcom edit form, to 
    make something appropriate out of edits that remove reminder dates, rather 
    than clicking 'delete'.  This fixes a potential exception, when a blank 
    date cannot be saved, and also does what the user expected (deletes a 
    blanked date).

  * The nomcom send_reminders command was not being run daily.  Fixed that.

  * Fixed a typo in the GDPR consent request email template.

  * Merged in [15492] from rjsparks@nostrum.com:
    Use factories for ipr tests.  

 -- Henrik Levkowetz   08 Oct 2018 13:23:53 +0000


ietfdb (6.85.1) ietf; urgency=medium

  **Email address managment and NomCom tool improvements**

  This release brings no new user features, but provides a script that
  will read email delivery-status messages and disable addresses with
  permanent delivery failures.  There are also some other tweaks:

  * Added a management command to deactivate email addresses found in 
    delivery-status emails or given on the command-line.

  * Added column sorting to the private nomcom nominee table.

  * Tweaked the management command to send GDPR consent request emails, to 
    correctly perform exclusion based on the time of previous consent request 
    emails.

  * Added a PersonEvent type for email address deactivation.

  * Tweaked PersonAdmin to show name_from_draft after the name field.

  * Added a clarification to the GDPR consent request email text after
    receiving about 100 reply emails in response to the first email round that
    contained what seems to be automatic responses indicating 'Yes, I
    confirm'.  All those were from gmail addresses.  The consent request
    now clearly says that people need to log in to their account in order
    to indicate consent.

 -- Henrik Levkowetz   30 Sep 2018 14:47:48 +0000


ietfdb (6.85.0) ietf; urgency=medium

  **Improvements in NomCom pages and personal information handling**

  This is a combined feature and bugfix release, with fixes related
  to User and Person record handling and merging, discovered when people
  responded to the GDPR consent request emails, and enhancements in
  the NomCom tools:

  * Added username validation to the change username form, to avoid 
    database key collision exceptions.  Instead give a message to contact the 
    secretariat to resolve the situation.

  * Fixed a problem where an exception could be raised when reporting an 
    exception during draft XML processing to the user.

  * Added 'consent' to the fields handled when merging person records.

  * Merged in [15486] from rjsparks@nostrum.com:
    Allow a nomcom to show nominees that have accepted nomination before 
    feedback is open. Fixes #2598.  

  * Tweaked the consent request email text to point out an alternative to 
    follow email links.

  * Merged in [15450] from rcross@amsl.com:
    Change content-type of ajax_get_liaison_info to application/json.   

  * Tweaked the text of the personal information help page to remove some 
    ambiguity.

  * A string.Template with a unicode template must be given unicode 
    dictionary values if the values contain unicode codepoints.  Providing 
    values that are objects with __str__() methods, or utf-8 encoded strings 
    doesn't work.  Fixes an issue with nomcom template interpolation for 
    nominees with non-ascii names.

  * Tweaked the GDPR reminder command to avoid bounces to RT and copies to 
    gmail.

  * Changed the sort order of nominees on the private nomcom index page to be
    according to name (instead of email address).

  * Fixed an issue where a user record without matching person record will 
    cause an exception.

  * Removed the 2.5 hour session length choice for IETF 103, since there 
    will be no Friday sessions that meeting.

  * Updated the personal information notice email text based on advice from 
    councel (Tom Zych).

 -- Henrik Levkowetz   28 Sep 2018 23:50:01 +0000


ietfdb (6.84.0) ietf; urgency=medium

  **GDPR updates part 4**

  This is the last in a series of GDPR-related releases.  It introduces two
  management commands; one to email out reminders to all users that still have
  not provided consent for storage of the personal information for which the
  GDPR requires consent, and the other to delete all information for which
  we don't have required consent.

  The two new management commands are:

    - ietf/manage.py send_gdpr_consent_request
    - ietf/manage.py delete_data_lacking_consent

  Both of these provide descriptions of the commands and switches when given
  a --help switch, to assist the secretariat in using these.

  There are also a few bugfixes and features not related to the GDPR, see
  below:

  * Merged in the latest GDPR changes.  This refines the handling of the 
    consent checkbox on the account page; refines the Consent Needed warning 
    given on login if consent is needed; tweaks several models to set the 
    on_deletion fields for FK to User and Person appropriately; adds a 
    Person.needs_consent() method to capture the logic of which fields require 
    consent; refines the Person.plain_name() method and the user.log.log() 
    function; and adds 2 management commands to send out consent requests and 
    delete non-consent information, respectively.

  * Refined utils.log.log() to flush stdout before logging if we're writing 
    log messages to stderr.

  * Moved document methods rfc_number() and rfcnum() to the abstract parent 
    class to have them available also for historical document records.  Should 
    fix the recent server 500 on /doc/draft-ietf-aaa-diameter/17/.

  * Merged in [15448] from rjsparks@nostrum.com:
    Add CORS headers for the api and views that serve json. Fixes #2546, 
    #2551, and #2552.  

  * Added a bit of javascript and styling to make it possible to use URL 
    fragment identifiers to link to position-specific tabs on the nomocom 
    requirements and questionnaire pages.

  * Merged in [15446] from rjsparks@nostrum.com:
    Use unversioned links in ics agendas. Partially fixes #2542.  

  * Merged in [15444] from rjsparks@nostrum.com:
    Review assignment email includes a link to the review request that 
    ensures the reviewer is logged in. Fixes #2543.  

  * Added a 'Mailing list discussion' link to the 'Additional URLs' section 
    of the status page for drafts with doc.group and doc.group.list_archive set.

  * Merged in [15441] from peter@akayla.com:
    Fixed 2 trivial spelling errors in NomCom tools.  Fixes issue #2569.   

  * Fixed a problem with fetching the mysql release signing key from a gpg 
    keyserver.

  * Added more hint information when patch actions in ietf/checks.py fail.

  * Rewrote docker/run to work around a missing switch in BSD 'xargs'.

  * Merged in [15435] from rjsparks@nostrum.com:
    Include area groups in the request for minutes. Do not include sessions 
    that have been canceled.  Fixes #2568. 

 -- Henrik Levkowetz   18 Sep 2018 14:07:27 +0000


ietfdb (6.83.1) ietf; urgency=medium

  This is a small bugfix and cleanup release:

  * Replaced some cases of using person.plain_name when linking to a person's
    profile with person.name, as the link using plain_name can fail.

  * Generalized the log.assertion() function a bit to be able to provide 
    more information on assertion failure.

  * Merged in [15423] from rcross@amsl.com:
    Removed WG milestone management from secretariat groups app.   

  * Removed youtube video url import, which is not needed any more given the
    /api/meeting/session/video/url API endpoint.  This also prevents duplicate
    video URLs.

  * Changed the menu listing of review dirs and their display on the 
    Directorates page back to what it was before the 'reviews' group type was 
    split out.  Added a 'Reviews' column to the Directoratges page.

  * Added a guard against comparing email origin with username of None.

  * Added a None guard for the person merge last_login date comparison.  
    Fixes issue #2562

 -- Henrik Levkowetz   14 Aug 2018 12:31:32 +0000


ietfdb (6.83.0) ietf; urgency=medium

  **IETF 102 code sprint**

  This release contains datatracker bugfixes and enhancements from the
  IETF-102 Code Sprint.  While we've had many more code sprint participants
  before, I don't believe we've ever had this number of contributions from an
  IETF week :-)

  The release contains a number of different enhancements to the document
  review pages (thanks Tero, Peter); refactoring of a number of tests to use
  test object factories instead of indiscriminate creation of a large number
  of often unused test objects, thus speeding up the tests (thanks to Robert);
  fixes to template language and anchors (thanks to Jean); a fix for annoying
  issue with a liaison form (thanks, Ben); a number of updates to secretariat
  pages and proceedings (thanks, Ryan); a fix for a long-standing issue with
  the document search forms behaviour when no search string has been given
  (thanks, Jim); and a test to expose bugs in downref listing in last call
  announcements (thanks, Russ).

  From the commit log:

  * Show links to personal profile pages from group photo pages also when no 
    photo is available -- the profile pages now contains more than just the 
    photo.

  * Merged in [15399] from rjsparks@nostrum.com:
    Use factories for ietf.ietfauth.tests.  

  * Merged in [15395] from rjsparks@nostrum.com:
    Allow the IRTF stream to set shepherds and shepherd writeups. Fixes #1993.  

  * Merged in [15387] from rjsparks@nostrum.com:
    Use factories for ietf.iesg.tests.

  * Merged in [15380] from rjsparks@nostrum.com:
    Use factories for ietf.idindex.tests.

  * Merged in [15370] from rjsparks@nostrum.com:
    Use factories for ietf.group.tests_review.

  * Merged in [15358] from kivinen@iki.fi:
    Fixed the review tablesorter issue by adding display:none character in the
    front of the date. Tablesorter looks first characters and picks the parser
    based on that, meaning that if the text starts with number, it will try to
    parse everything in column as numbers, and if there is anything that is
    not numebr it will fail, and only sorts the column once. This is bit ugly
    but I do not think we want to add yet another jquery meta tool. Fixes
    #2089, and #2151.  

  * Added a missing migration (for review unavailable-period reason)

  * Merged in [15357] from mahoney@nostrum.com:
    Added anchors for working group rows. Still needs some work since the 
    anchor ends up being under the menu bar. Partially addresses issue #1481.  

  * Merged in [15356] from ben@nostrum.com:
    Changed newlines to commas in to contacts for new liaison email 
    notification. Fixes issue 2525.  

  * Merged in [15355] from rjsparks@nostrum.com:
    Used factories for ietf.group.tests_info.  

  * Tweaked a test to prevent failures when looking for a specific string 
    in line-wrapped text.

  * Added a tiny utility function unwrap() to unwrap wrapped text for 
    matching expected strings in tests.

  * Merged in [15354] from kivinen@iki.fi:
    Added default to review completion draft version number. If the reviewer
    is filling in the review form himself the default will be the latest
    version of the draft. If it is someone else filling in the review
    then the version number is the latest version at the point when review
    was assigned (i.e., it assumes reviewer did review on version he was
    requested to and then document authors might have updated the document).
    Also mark the document versions that are earlier than the review request
    date with red background and those which are newer with green background.
    Fixes #2125.  

  * Merged in [15353] from peter@akayla.com:
    Add a sidebar to display feedback section names and simplify navigation.
    Could use some visual improvements but otherwise works.  Fixes #2431.  

  * Merged in [15351] from rcross@amsl.com:
    Update secr/drafts/tests_views to align with view logic.  Fixes #2480.  

  * Merged in [15349] from kivinen@iki.fi:
    Removed the requirement for review_content field to be mandatory when
    using \'link\' method. This partially addresses #2520. 

  * Merged in [15347] from kivinen@iki.fi:
    Add review request date to the top. Add tooltips for drafts versions
    providing the date they were sumbmitted. Fixes #2218. 

  * Limited the document name for uploaded slides to use 128 slugified 
    characters from the slideset title as part of the name.  Fixes a possible 
    server 500 issue.

  * Merged in [15345] from rcross@amsl.com:
    Remove duplicate functionality in secr/drafts.  Fixes #2540.   

  * Merged in [15344] from mahoney@nostrum.com:
    email confirming interim meeting request now says 'the Area Director of 
    the ' instead of ' Area Director'. Fixes issue #2522.  

  * Added additional checks to the meeting_session_video_url api test.

  * Fixed api_set_session_video_url() to update the external video url if 
    given a new value.

  * Merged in [15342] from peter@akayla.com:
    Added optional reviewer unavailability reason support.  Fixes issue #2146.   

  * Merged in [15335] from rcross@amsl.com:
    Fixed the ordering of meeting select in secr/meetings.  Fixes #2539.

  * Adjusted some timezones in test code to use permitted time zones after 
    the previous changes in listed timezones.  Added a migration for the 
    changed timezone list.

  * Fixed display of xml2rfc exceptions when processing draft submission in 
    xml form.

  * Modified the list of timezones available when setting a meeting 
    timezone so as to not show timezones that are links.  (Also edited the 102 
    meeting timezone in the database to avoid this).  Fixes issues #2531 and 
    #2534.

  * Merged in [15338] from fenton@bluepopcorn.net:
    Adds _required_ parameter to search widget. Fixes #1832.

  * Merged in [15331] from housley@vigilsec.com:
    Add test for downref in Last Call announcement 

  * Merged in [15329] from rcross@amsl.com:
    Add important dates link to proceedings.  Fixes #2538.   

  * Merged in [15328] from rcross@amsl.com:
    Give secretariat the ability to cancel non-working group sessions. 
    Fixes #2537.   

  * Updated a test to give predictable results (no future meeting dates) 
    for commit [15361].

  * Fixed an issue where /meeting/ showed the materials page of an earlier 
    meeting, rather than the current.

  * Merged in [15327] from mahoney@nostrum.com:
    Clarified description in the search form, removing the etc since you 
    can't search on author names. Fixes issue #1159.  

  * Merged in [15321] from rjsparks@nostrum.com:
    Converted ietf.group.tests to factories.  

  * Merged in [15320] from rjsparks@nostrum.com:
    The cookie tests were all calling make_test_data, but not using anything
    it created. Removed the calls.  

  * Merged in [15314] from rjsparks@nostrum.com:
    Converted the community app tests to use factories.  

 -- Henrik Levkowetz   25 Jul 2018 19:08:41 +0000


ietfdb (6.82.0) ietf; urgency=medium

  **Group features moved from code to database**

  This release replaces the code defined group features with features held
  in a database table:

  - Added a GroupFeatures model to the group models, and removed the old
    features.py
  - Added a agenda type for future use in showing different group types on
    different agendas.
  - Renamed the group feature has_materials to has_nonsession_materials.
  - Added API resources and admin support for the new tables.
  - Added a Directorate (with reviews) group type as complement to
    Directorate, to distinguish between directorates with and without
    reviews.
  - Adjusted tests as needed.
  - Updated the fixtures, and fixed the generate_fixtures script to include
    the new AgendaTypeName objects.

  There still exists about 70 instances of code comparing the group type
  with a list of types; most of these should probably be replaced with new
  features, instead, to make it possible to add new group types through the
  database table, rather than having to edit the code.  That was the purpose
  of this refactoring from the start, but the presence of this large number
  of comparisons of group type against lists of types defeats the goal until
  we add appropriate features and replace the group type list comparisons.

  Additional changes:

  * Added exception logging for ietf/bin/rfc-editor-queue-updates.  Changed 
    the logging in the ietf/bin/rfc-editor-* scripts to use 
    ietf.utils.log.log().  Reordered some imports.

  * Replaced the use of a deprecated Django field attribute with the 
    sanctioned replacement.

  * Merged in [15282] from housley@vigilsec.com:
    Set reply-to for I-D Announcements that include a CC to a mail list.
    Fixes #2530. 

  * Added support for patent 'kind codes' as part of patent numbers.

  * Tweaked the rsync arguments for IANA YANG models.

  * Upped the permitted number of search results for the /stream/ise/ page 
    (and friends) from 200 to 1000.  Fixes issue #2527.

 -- Henrik Levkowetz   14 Jul 2018 16:47:58 +0000


ietfdb (6.81.3) ietf; urgency=medium

  This is a bugfix release to clear the board for the final GDPR release.
  From the commit log:

  * Merged in [15266] from rcross@amsl.com:
    Updated the blue sheet template.  Replaced start time / end time with date.  

  * Merged in [15267] from rcross@amsl.com:
    Fixed an issue with a decorator on utils.html.remove_tags().   

  * Tweaked the 'Related documents' part of the Document admin page to more 
    intuitively show the relationships.

  * Tweaked unidecode_name() to avoid double-quotes in the ascii string.

  * Merged in [15248] from rjsparks@nostrum.com:
    Improvements to tests after converting to factories suggested by studying
    coverage_changes.  

  * Merged in [15247] from rjsparks@nostrum.com:
    Tweaked the color palette for the agenda editor.  

  * Removed two instances of log.assertion(); we have enough data on these
    now.

  * Changed an attribute guard to work with a reverse relationship attribute
    (using hasattr()).

  * Fixed a broken mailto link on the account edit page.

  * Merged in [15238] from rjsparks@nostrum.com:
    Use factories for ietf.doc.tests_status_change. Repair previous commit
    errors in ietf.doc.tests_material.  

  * Merged in [15228] from rjsparks@nostrum.com:
    Use factories for ietf.doc.tests_downref.  

  * Merged in [15227] from rjsparks@nostrum.com:
    Use factories for ietf.doc.tests_conflict_review.  

  * Merged in [15226] from rjsparks@nostrum.com:
    Use factories for ietf.doc.tests_charter.  

  * Tweaked unidecode_name() to not produce single-letter ascii surnames 
    from non-ascii codepoints.  The unidecode transliteration is in any case 
    somewhat arbitrary, and in most cases a real person will tweak the ascii 
    name of his account.  When running tests, however, this tweak avoids some 
    false test failures.  And no, it's not simple to fix the draft 
    author-extraction heuristics to deal well with single-letter surnames.

  * Added a guard against trying to access attributes of a None object.

  * Merged in [15225] from rjsparks@nostrum.com:
    Use factories for ietf.doc.tests_draft.  

 -- Henrik Levkowetz   20 Jun 2018 21:47:42 +0000


ietfdb (6.81.2) ietf; urgency=medium

  **GDPR updates part 3**

  This is the third in a small series of releases to address compliance with
  the European GDPR (General Data Protection Regulation).  It provides
  serialized (machine-readable) access to information associated with a user's
  account, adds photo and some links to the user account page, and improves
  the annotation of which information is consent-based on the account page.
  It adds a warning on login if consent is missing for some personal
  information, and requests the user to give consent.  It also brings in
  refactoring of a number of test cases to use test object factories, giving
  improved test performance.

  * Merged in [15219] from rjsparks@nostrum.com:
    Use factories for ietf.doc.tests_ballots. Fixed a long-hidden bug in
    test_edit_verify_permissions where non-existing usernames were used and
    passed the tests accidently. Added list_email to the GroupFactory.  

  * Merged in [15217], [15205], [15204], [15203], [15202], and [15201] from
    rjsparks@nostrum.com:
    Converted ietf.doc.tests to use object Factories. Created draft type
    specific factories.  

  * Fixed the logic of gathering which personal information needs consent 
    confirmation in the code that requests consent on login.

  * Added a request to give consent for non-LI information on user login.  
    Addresses issue #2505.

  * Fixed a bug in a form clean method.

  * Added a missing related object to the person information export.  
    Related to issue #2501.

  * Tweaked the language on the user account page.  Related to issue #2501.

  * Added a recursive object to JSON serializer and a view which will let 
    any logged-in user download a JSON serialized copy of the datatracker 
    information related to his person record.  Added information about this, 
    and a link, to the account page.  Related to issue #2501.

  * Added a requirement of oauth2client, needed by google-api-python-client 
    but not always automatically pulled in.

  * Added a link to the overall IETF Statement concerning personal data to 
    the datatracker personal-information page.

  * Added some text about BOF requests to the important-dates page for the 
    session request opens dates, on request from the IESG.

 -- Henrik Levkowetz   08 Jun 2018 16:05:15 +0000


ietfdb (6.81.1) ietf; urgency=medium

  **GDPR updates part 2**

  This is the second in a small series of releases to address compliance
  with the European GDPR (General Data Protection Regulation).

  This release provides two data migrations to polulate email address origin
  and other trace information, to make it possible to differentiate between
  that personal information which is to be treated as Consent-based, that
  which falls under Legitimate Interest.

 -- Henrik Levkowetz   29 May 2018 14:38:09 +0000


ietfdb (6.81.0) ietf; urgency=medium

  **GDPR updates part 1**

  This is the first in a small series of releases to address compliance with
  with the European GDPR (General Data Protection Regulation).  

  This release provides necessary GUI and internal changes, but lacks some
  admin utilities, and also data migration routines which takes some time to
  run, and therefore will be packaged in a separate release and applied
  separately.

  From the commit log:

  * Updated the personal information page with reviewed text from legal 
    counsel.  Fixes issue #2503.

  * Removed the Person.address field, which is not being used.  This was a 
    legacy from the 2001 perl-based datatracker tables.  Fixes issue #2504.

  * Added django-simple-history and replaced the old (and unused) 
    PersonHistory class with a history=HistoricalRecords() field on Person.  
    Added the needed migrations and changes to admin, resources, and settings.  
    Related to issues #2505 and #2507.

  * Added a new field name_from_draft to Person, to hold the name field 
    equivalent as captured from drafts, in case name has been modified by the 
    user and we're asked to remove that info under GDPR.  Added history for 
    Email, and also an origin field to capture from where we got an email 
    address (draft name, username, meeting registration, etc.)  Added a 
    log.assertion() to Email.save() in order to ensure we don't create any 
    email without setting origin.

  * Added origin information to all places where we create email address 
    entries.

  * Removed all references to the removed Person.affiliation field.

  * Added email origin information to some function calls that needed it.

  * Overwrite earlier email origin when we've picked up the address from a 
    submission.

  * Added a consent field to the Person model.

  * Disallow profile changes without consent given.  Together with previous 
    commits this fixes issues #2505 and #2507.

  * Added another category of personal information to the 
    personal-information page, after review of personal information in the 
    code.  Completes issue #2501.

  * Added a dagger at the end of some fields in the account data forms to 
    signify consent-based fields, and made the consent field required.

  * Simplified the email.origin assignment code for outgoing liaisons.

  * Fixed a long-standing bug in the liaison.name() code.

  * Added assingment of the person.name_from_draft field on draft 
    submission.


  * Updated the edit_profile template with information about consent-based 
    fields.  Fixes issue #2502.

 -- Henrik Levkowetz   29 May 2018 11:10:28 +0000

ietfdb (6.80.1) ietf; urgency=medium

  This is a bugfix release which also clears the slate for the upcoming
  GDPR-related release.  From the commit log:

  * Added a meeting helper function update_important_dates(), and a management
    command update_important_dates to update the important dates for a meeting
    if the offsets in the MeetingDateName table has been changed.

  * Added some text to the important dates page for 'opensched', on request
    from the IESG via Stephanie.

  * Merged in [15153] from rjsparks@nostrum.com:
    Changed how the mail utilities handle lists in their override and extra
    arguments. Added a test for send_mail_preformatted exercising override and
    extra.  

  * Merged in [15152] from rjsparks@nostrum.com:
    Send email when the responsible AD for a conflict review is changed.
    Include the stream manager and stream steering group when the responsible
    AD changes or when the telechat date changes. Fixes #2497.  

  * Merged in [15151] from rjsparks@nostrum.com:
    Provide an email group button for WG and RG documents. Fixes #2512.  

  * Merged in [15145] from rjsparks@nostrum.com:
    Secretariat requested tweak to the request for minutes template.  

  * Updated the non-WG mailing list page to include advertised lists for
    closed WGs, on request from the secretariat.

 -- Henrik Levkowetz   24 May 2018 15:53:08 +0000


ietfdb (6.80.0) ietf; urgency=medium

  This release fixes a number of bugs, and adds a view that makes it easier
  for the secretariat to generate the meeting minutes request emails.

  * Merged in [15125] from rjsparks@nostrum.com:
    Added a view that generates the 'send minutes' email for the secretariat.
    Fixes #2493.  

  * Merged in [15124] from rjsparks@nostrum.com:
    Provide a __unicode__ member for ImportantDate.  

  * Merged in [15123] from rjsparks@nostrum.com:
    Use importantdates for material submisssion cutoff and revision if they
    exist. Fixes #2514.  

  * Merged in [15119] and [15120] from rjsparks@nostrum.com:
    Change stream state to replaced when an ISE, IAB, or IRTF document is
    replaced. Fixes #1963.

  * Merged in [15114] from rjsparks@nostrum.com:
    Restored a test that was unintentionally removed. Removed unused
    templates.  Related to #2513.

  * Merged in [15113] from rjsparks@nostrum.com:
    Removed the already disabled 'Make RFC' functionality from /secr/drafts.
    Fixes #2513.
    
    Merged in [15112] from rjsparks@nostrum.com:
    Removed the 'announce' functionality from /secr/drafts. Addresses part of
    #2513.

  * Merged in [15111] from rjsparks@nostrum.com:
    Removed the \'replaces\' functionality from /secr/drafts. Addresses 
    part of #2513.  

  * Merged in [15110] from rjsparks@nostrum.com:
    Made settings.URL_REGEXPS.name non-greedy. Added a test to exercise the 
    bug at #2429. Fixes #2429.  

  * Fixed an incorrect file name in a log message.

 -- Henrik Levkowetz   08 May 2018 06:37:38 +0000


ietfdb (6.79.1) ietf; urgency=medium

  This bugfix release fixes problems with IESG ballot clearing, marks dead
  code as unreachable for later removal, adds new tests to increase test
  coverage, and fixes some issues with the test suite.  It also tweaks the
  roles which are permitted to request document publication, loosens the
  need for trailing slashes for some URLs, and adds a 'Recent drafts' page.

  * Merged in [15084] from rjsparks@nostrum.com:
    Corrected the implementation of clear_ballot. Added a test for it.
    Restored functionality to ballot popups that was missing because a
    template variable was not passed in.  

  * Added a 'recent drafts' page, showing drafs submitted the last 7 days.

  * Merged in [15083] from rjsparks@nostrum.com:
    Removed memoize from latest_event.  

  * Merged in [15080] from rjsparks@nostrum.com:
    Marked a couple of functions as unreachable.  

  * Reverted a change to UserFactory to avoid nepalese names.  The issue is 
    bigger, and needs a different resulution.  Changed the url pattern that 
    used \w to match names to instead use a more inclusive regex.

  * Fixed a PyQuery property access that should have been a method call.

  * Merged in [15076] from rjsparks@nostrum.com:
    Added a test for meeting/requests.

  * Tweaked the IPR factory to be less likely to produce invalid patent 
    info.

  * Merged in [15075] from rjsparks@nostrum.com:
    Improved testing for docs_for_ad. 

  * Merged in [15074] from rjsparks@nostrum.com:
    Simplified docs_in_iesg_process. Added a test for it. 

  * Merged in [15073] from rjsparks@nostrum.com:
    Rearranged ietf/doc/urls.py to allow matching some special pages under 
    /doc without a trailing slash. Adjusted several of those special URLs to 
    work without the trailing slash. 

  * Added permission for a document shepherd to request publication.  Fixes 
    issue #2509.

 -- Henrik Levkowetz   02 May 2018 10:42:26 +0000


ietfdb (6.79.0) ietf; urgency=medium

  **IPR patent info field fixes, IESG position API enhancement, and more.**

  This release enhances the IESG position API to send notification emails on
  changes, widens some IPR disclosure form fields to accept more text, and
  fixes a patent application number validation issue for IPR forms.  It adds
  some useful links and a new filtering option to the past meetings page,
  updates document stream status indications so as to be less prone to
  misinterpretation, addresses an issue with session request forms, and fixes
  an issue with I-D expirations and another related to ordering of documents
  on IESG agendas.

  From the commit log:

  * Set file upload permissions to 0644 to avoid problems with photo access.

  * Increased the length of the patent title in IPR forms

  * Fixed a typo.  Fixes issue #2500 which caused different document ordering
    on the IESG agenda compared with the IESG agenda documents page.

  * Increased the size of the patent_notes form field, and tweaked the
    validate_patent_number validator to accept US application numbers
    containing a series number and forward slash.

  * Merged in [15055] from rcross@amsl.com:
    Fixed an issue where user can make two session request submissions,
    corrupting data.  Also, ignore deleted session requests in edit view.  

  * Merged in [15047] from rcross@amsl.com:
    Reduced the get_meeting buffer to handle earlier shceduling requests date.   

  * Added origin indication to a group of stats templates to make it easier 
    to find the origin of stats configurations.

  * Added group links to the past meetings page, and also important dates 
    links to IETF meetings with important-date pages.

  * Added IETF to past meetings filter pane, in order to be able to filter 
    out interims.

  * Fixed a problem with the expire-ids code, and added logging to catch 
    future problems.

  * Modified the IESG position setting API to send email messages about 
    positions set.

  * Tweaked the wording of coverage_changes output.

  * Fixed an issue where simplified sorting of DocEvents resulted in some 
    ballot icons not being shown on the IESG agenda documents page.

  * Changed incorrect use of 404 status code in api to 400.  Fixes issue 
    #2498.

  * Removed what could be a misleading 'RFC' from  the stream indication in 
    document search results.

 -- Henrik Levkowetz   23 Apr 2018 12:28:57 +0000


ietfdb (6.78.0) ietf; urgency=medium

  **Document search on IRTF state, performance improvements**

  This release brings document search support for filtering on IRTF states,
  courtesy of Mat Ford, and further performance improvements to the IESG
  agenda document page and document search pages.  There are also improvements
  to the testing infrastructure, HTML cleanup, some added API documentation,
  and a number of bugfixes, as follows:

  * Merged in [15012] from ford@isoc.org:
    Added IRTF State option to Additional Search Criteria pulldown.  

  * Included django.utils.html.escape() in the linkify() code, and removed 
    it from various templates, to make sure escape is consistently applied 
    before linkify.  Fixes issue #2492.

  * Tweaked the error return from Document.get_document_content() to be 
    more helpful during development.

  * Fixed various html issues.

  * Added more information about personal API keys to the API help page.

  * Tweaked the test runner to more consistently force the test coverage 
    tests to be run last.

  * Changed calls to .related_that*() and .relations_that*() to use tuples 
    rather than lists (a slight optimization).

  * Silenced another deprecation warning from a library.

  * Reduced the max number of results returned by document searches from 500
    to 200, for a better user experience.  A faster return and a search term
    tweak to pinpoint the desired search result is better than a long wait and
    then maybe having to tweak the search terms anyway.

  * Use .assertIn(...) in tests instead of .assertTrue(x in y) as the 
    failure messages are much more informative

  * Changed the signature of some document methods to not accept lists (but
    still accept tuples) in order to make caching possible.  Added @memoize to
    some methods.

  * Added another prefetch to search doc query results.

  * Imposed sort order (most recent docs first) on document search results, 
    in order to retain the most recent docs in case we have to chop the result
    list due too many hits.

  * Tweaked the django_cprofile_middleware patch to use a query arg of 
    psort instead of sort, to avoid conflict with app query args 'sort'.

  * Added back the on-agenda calendar links to document lists, now in an 
    implementation with better performance characteristics.

  * Updated minimum requirements for bootstrap3, to work with django 1.11

  * Fixed an issue where the change of slide naming between IETF 96 and 97 
    caused the datatracker to not provide correct link to pre-IETF-97 
    materials.  Thanks to tjw.ietf@gmail.com for the heads-up.

  * Merged in [14970] from rjsparks@nostrum.com:
    Factories for the ipr app. Coverage of ietf.ipr.views.ajax_search, and 
    a factory-driven test of ietf.ipr.views.edit.  

  * Enhanced debug.py: made the mark() function indicate file and line 
    number it was called from.  Added if debug: conditions around some other 
    function actions.

  * Another round of iesg page optimizations.  Reduced additional instances 
    of individual lookups to a single lookup followed by annotation of document 
    objects.  Reduced several instances of repeating the same work multiple 
    times (queries for telechat documents, queries for telechat dates).

  * Fixed a number of tests to deal correctly with model method caching.

  * Changed the ballot_icon template tag to use an already found ballot if 
    available, rather than doing lookups to find it (twice!).

  * Added a patch for debug mode, to add a filtering option to the 
    django-cprofile-middleware.

  * Be more specific with bootstrap's layout classes for small devices in 
    the base template, to avoid the bottom links and footer overlaying the 
    bottom of the content area.  Fixes a problem with entering login 
    information on small devices.

 -- Henrik Levkowetz   10 Apr 2018 16:33:46 +0000


ietfdb (6.77.1) ietf; urgency=medium

  **Performance improvements**

  This release brings performance improvements, focused in particular on the
  IESG agenda documents page.  Some of the changes will also provide
  performance benefits for other pages, in particular pages that lists
  documents, such as search result pages and WG/RG document lists.  All in
  all, the changes speed up the rendering of /iesg/agenda/documents with a
  factor of about 2.  The page generation time is still roughly proportional
  to the number of documents on the page, so a large number of documents on
  future IESG telechats will still give a slow page.  More substantial
  refactoring will probably be needed for further speedups.

  From the commit log:

  * Added an index for (doc, type) on DocEvent to speed up latest_event()
    queries.

  * Changed the reviewed_by_teams list to provide acronyms directly, to avoid
    group lookups during template rendering.

  * Changed telechat_page_count() to accept a list of documents if such a list
    has already been generated, to avoid double work.

  * Provided telechat page counts to the template from the view, instead of
    repeated (costly) filtering through the telechat_page_count template
    filter in the template, which caused new, redundant document lookups every
    time.  Removed the telechat_page_count template filter altogether, due to
    its cost. 

  * Tweaked some lookups in fill_in_document_table_attributes().

  * Added terms to the select_related() list for documents in IESG
    agenda_documents().

  * Added a get_active_ads() utility function that uses the cache to reduce
    database access, and replaced database queries in various places with
    calls to get_active_ads().

  * Moved some utility functions so as to reduce the likelihood of import
  * loops.

  * Changed some instances of .slug to _id, to avoid unnecessary database
    lookups.

  * Added a profiling middleware to development settings, if installed.

 -- Henrik Levkowetz   02 Apr 2018 11:55:07 +0000


ietfdb (6.77.0) ietf; urgency=medium

  **Automation API endpoint: /api/meeting/session/video/url**

  This release provides provides a new API endpoint to let video urls be
  associated with sessions, intended for use by Meetecho to update the
  datatracker with sesssion recording links as soon as a recording has been
  uploaded to the YouTube (or equivalent).  It also introduces a new
  'Recording Manager' role, used to authorize the API access.  A short
  description of the API is available at
  https://datatracker.ietf.org/api#session-video-url-api

 -- Henrik Levkowetz   30 Mar 2018 19:11:56 +0000


ietfdb (6.76.2) ietf; urgency=medium

  This release fixes a number of blemishes and cleans the slate for merging
  in a meetecho API branch.

  * Fixed an issue which could result in server 500 errors for paths like 
    /meeting/99/session/plenary.

  * Removed superfluous stats pages headers which would be repeated with 
    different layout on the next line, as part of the graphs.

  * Added a test for IPR edit.

  * Filled in the IPR edit form, section V, with patent information derived 
    from the patent_info field, the same way we do for the IPR update form.

  * Added a pip version restriction to requirements.txt, as there seems to be
    issues with pip 9.0.2 and 9.0.3; they seem to leave dross when down- and
    upgrading packages.

  * Don't show links to individual meeting stats pages when there's no 
    content (meeting 71 and earlier currently has no attendance records).

  * Added meeting dates to the meeting stats overview graph.

  * Changed the name of a loop variable so as not to obscure a previous 
    local variable.

  * Don't show future meetings in the meeting stats overview.

  * Changed settings so that cache keys used by memcached includes the 
    release version, in order to avoid stale and incorrect cache content on new 
    release deployment.  Made it easier to pick out cache key composition by 
    normalizing the variable name.  Adjusted cache prefix and the composition 
    of some keys to have a unique and consistent cache key prefix.

  * Use the actual defaults from settings.py when showing the defaults on 
    /accounts/settings/.

 -- Henrik Levkowetz   28 Mar 2018 20:54:53 +0000


ietfdb (6.76.1) ietf; urgency=medium

  **IETF 101 code sprint, release 2**

  This release provides improved meeting attendance statistics (under
  /stats/meeting/overview/), improved test coverage, additional menu entries,
  html template fixes, total page count on the Last Call page, the ability
  for ADs to clear ballots themselves without going through the secretariat,
  and more.  It also changes the default cookie setting for showing or hiding
  the duplicate left-hand menu column.  (If you don't like the new default, use
  the 'User>Preferences' menu item to revert to the old look.)

  Again, many thanks to all the contributors!

  From the commit log:

  * Fixed some issues with agenda and proceedings code for meetings 
    numbered 64 and below, actualized by the introduction of meeting records 
    for such meetings for stats purposes.

  * The template filter 'keep_spacing' can interact badly with 'wordwrap'.
    Removed it from several places, and made text document display more
    consistent.  Fixes issue #2481.

  * Merged in [14902] from rjsparks@nostrum.com:
    Test coverage improvements for ietf/secr/telechat/views.  

  * Merged in [14900] from rjsparks@nostrum.com:
    Improvements to test coverage for ietf/secr/drafts/views.py.  

  * Merged in [14899] from rjsparks@nostrum.com:
    Cleaner charter factory simplifies test code.  

  * Added submenu to the Statistics menu entry, pointing at the individual 
    stats pages, and entries for NomComs and Releases.

  * Merged in [14896] from rjsparks@nostrum.com:
    Improved test coverage for ietf/doc/views_charter.py.  

  * Fixed the base template to have id='content' for the content also when 
    not showing the left-hand menu.

  * Updated the cookies tests to match the new left_menu default.

  * Changed the default setting for the datatracker left-hand menu to 
    'off'.  The top menu is always present, and works well.

  * Tweaked the document list iCalendar links to not show for presentations 
    on days in the past.

  * Merged in [14884] from adam@nostrum.com:
    Adding page count to last call page.  

  * Merged in [14883] from adam@nostrum.com:
    Adding link to documents in last call to the AD dashboard side menu.  

  * Merged in [14849] from ben@nostrum.com:
    Give ADs the red Clear Ballot button 

  * Merged in [14880] from rjsparks@nostrum.com:
    Added a Draft test suite.

  * Merged in [14879] from kivinen@iki.fi:
    Backfilled IETF meetings starting from 1 and backfilled the attendee 
    numbers for them. Modified the overview attendee statistics page to use 
    different colors for different venue continents.  

  * Tweaked the meeting registratoin import to deal more gracefully with 
    too long names and email addresses, and to fill in the total attendees 
    number for the meeting.

  * Tweaked mergeready to accept an additional hold-for-merge file outside 
    the repository.

 -- Henrik Levkowetz   26 Mar 2018 13:38:04 +0000


ietfdb (6.76.0) ietf; urgency=medium

  **IETF 101 code sprint, release 1**

  This release contains datatracker bugfixes and enhancements from the IETF-101
  Code Sprint.  It brings new views to merge duplicate person records, fixes
  an annoying bug in downref identification and a bunch of links that didn't
  point to the right document, corrects the mime-type for JSON responses, and
  brings substantial improvments to test coverage.

  Many thanks to everyone who contributed!

  * Merged in [14864] from rjsparks@nostrum.com:
    Improved test coverage for ietf.meeting.views.  

  * Updated Note Well.  Fixes issue #2474.

  * Tweaked person factory ascii name generation to not include periods 
    inside names.

  * Tweaked a factory to make duplicate mailing list names less likely in tests

  * Test data template tweak.

  * Merged in [14863] from rjsparks@nostrum.com:
    Improved test coverage for ietf.utils.draft.  

  * Merged in [14862] from rcross@amsl.com:
    Add view for merge person records.   

  * Merged in [14855] from rjsparks@nostrum.com:
    Improved test coverage for ietf.meeting.views.  

  * Added a test case for reference extraction.

  * Merged in [14854] from rjsparks@nostrum.com:
    Improved test coverage over meeting.views.  

  * Merged in [14851] from housley@vigilsec.com:
    Improve parser for references in Internet-Drafts. Fixes #2360 

  * Merged in [14850] from rjsparks@nostrum.com:
    Improvements to test coverage in ietf.meeting.views 

  * Removed premature save of upload document state (doing it after 
    verifying that the save to file worked instead).  Tweaked materials 
    documents revision extraction regexes.  Added a test on no change of 
    materials rev after failed upload.  Corrected the naming of materials in 
    make_meeting_test_data.  Refined the test crawl of meeting materials pages.

  * Added a couple of meeting material crawls to check that internal 
    material links aren't broken.

  * Check attribute isn't None before accessing sub-attributes.

  * Merged in [14845] from rcross@amsl.com:
    Remove unused roles/forms.py.   

  * Merged in [14844] from adam@nostrum.com:
    Fixing 'href' field for
    https://datatracker.ietf.org/meeting//json.  

  * Fixed some materials docname-with-rev regexes that were too narrow.

  * Merged in [14843] from adam@nostrum.com:
    Updated (deprecated, unregistered) 'text/json' with 'application/json' 
    everwhere.  

  * Merged in [14842] from rcross@amsl.com:
    Add check for deleted session when scheduling.  Fixes #2450.   

  * Fixed a bug in a RegexValidator error message string interpolation.

 -- Henrik Levkowetz   21 Mar 2018 10:58:52 +0000


ietfdb (6.75.2) ietf; urgency=medium

  **Session iCalendar links on document lists**

  This release adds session ical links to group document lists and document
  search results, fixes some issues related to upload sanitization and url
  resolution, improves performance for some pages, and adds and improves some
  tests in the test suite.  From the commit log:

  * Merged in [14830] from rjsparks@nostrum.com:
    Simple coverage tests for ietf/secr/drafts/reports.py.

  * Tweaked the document sanitizer to insert a charset meta tag after 
    sanitization.

  * Fixed a mistaken change in session urls.

  * Tweaked a test case to make it easier to get at failures.

  * Made the materials_document() view function more robust in handling 
    various materials document names.

  * Gave the 404 message a class which will make it easier to pick out the 
    message during testing.

  * Updated the meeting materials document view to handle urls with 
    revisions.

  * Updated the meeting document href patterns in settings.py to reflect 
    that we now have versioned agendas, minutes, and slides.

  * Added session ical links to document lists (WG docs, search results, 
    etc.) for documents on upcoming meeting agendas.  Reduced the query and 
    rendering times of document lists some more through additional 
    prefetch_related().

  * Added result caching for the Person.email() method.

  * Added result caching for a couple of Document methods.

  * Added a missing tag to the sanitizer whitelist (telling lxml's Cleaner 
    to not clean style with style=False is apparently not always enough).  
    Fixes issue #2470.

  * Added a debug import.

  * Updated a django patch to add origin information to query sets.

 -- Henrik Levkowetz   17 Mar 2018 10:42:11 +0000


ietfdb (6.75.1) ietf; urgency=medium

  **Modified HTML upload sanitization**

  Feedback from wgchairs@ietf.org indicated that stripping out all styling
  from uploaded files is too harsh.  This release modifies the sanitization to
  permit