Commit graph

216 commits

Author SHA1 Message Date
Henrik Levkowetz db7adf1fb5 Modified the code which adds non-session entries to a schedule so as to bind a Session object also to Break and Registration timeslots, through the Schedule/Timeslot/Session m2m table. This makes the agenda templates work as expected. There's also a migration and a standalone script that fills in this information for existing schedules.
- Legacy-Id: 7975
2014-06-30 17:05:40 +00:00
Henrik Levkowetz 00bf73a4df Tweaked test-crawl to show the complete chain of referrers for a failing URL.
- Legacy-Id: 7974
2014-06-29 20:58:39 +00:00
Henrik Levkowetz 9ff2b9ff9a Merged in support for RG pages, from branch/iola/rg-support.
- Legacy-Id: 7866
2014-06-09 20:52:14 +00:00
Henrik Levkowetz 618e01f870 Tweaked the test-crawler output.
- Legacy-Id: 7645
2014-04-25 15:11:14 +00:00
Henrik Levkowetz da4f9dc258 Print a message at the end of a test crawl also when there's no FAILs.
- Legacy-Id: 7626
2014-04-22 21:31:53 +00:00
Henrik Levkowetz b8cd069070 Merged [7557] from rjsparks@nostrum.com: Update reference relationships (asyncronously) for new rfcs found when syncing with the rfc index. Fixes ticket #1347.
- Legacy-Id: 7572
Note: SVN reference [7557] has been migrated to Git commit 9b43e159623ed1fb53438541b25a1605922943dd
2014-04-06 15:01:12 +00:00
Ole Laursen f52e676793 Move chartering groups to /group/chartering/ (with a redirect from the old location), list RGs there too, and amend the previous fix to the chartering process for non-WGs so they have internal and external review states instead of just approved. Also move the code in wginfo/ to group/ as it is no longer just about info for WGs.
- Legacy-Id: 7556
2014-04-01 11:59:08 +00:00
Henrik Levkowetz 5a424fe2da Merged in [7153] from rjsparks@nostrum.com:
Fixed a typo in code that looks for the IRTF chair email address while adding replaces relationships.
Made that code slightly more robust.
Fixes ticket #1253
 - Legacy-Id: 7531
Note: SVN reference [7153] has been migrated to Git commit 99c01fa54a
2014-03-21 11:55:51 +00:00
Henrik Levkowetz e1c543fd35 Modified the test crawler to save a log file and to provide more console output to help predict remaining runtime.
- Legacy-Id: 7522
2014-03-19 20:28:07 +00:00
Henrik Levkowetz 9ea9ba1d66 Merged in [7138] from rjsparks@nostrum.com:
Reworked SMTP Exception handling, adding sending a ticket to the secretariat when there are errors handing messages off for delivery.
Added SMTP exception handling to send-scheduled-mail.
This is related to ticket #1208
 - Legacy-Id: 7256
Note: SVN reference [7138] has been migrated to Git commit 64727c1c33
2014-02-17 18:55:37 +00:00
Henrik Levkowetz 6e97c2c8da Merged in the port from Django 1.2 to Django 1.6 from olau@iola.dk: branch/iola/djangoport@7121, and fixed some merge issues.
- Legacy-Id: 7152
2014-01-18 21:02:40 +00:00
Robert Sparks 64727c1c33 Reworked SMTP Exception handling, adding sending a ticket to the secretariat when there are errors handing messages off for delivery.
Added SMTP exception handling to send-scheduled-mail.

This is related to ticket #1208
 - Legacy-Id: 7138
2014-01-15 21:29:27 +00:00
Ole Laursen 591e90a7a0 Put a proper command-line interface on the test crawler, spit out number of errors (if any), also add wrapper script for running the test-crawler and idindex generation scripts, in the future other (safe, non-mutating) scripts operating on real data can be added too
- Legacy-Id: 7107
2014-01-12 16:55:42 +00:00
Ole Laursen 5dcd140a63 Improve test-crawler regexp so it can catch and visit linked feed URLs
- Legacy-Id: 7104
2014-01-10 17:34:33 +00:00
Robert Sparks e3d9fbabb1 Moved the target date for ending the ietf-announce header changes forward since we missed the start date.
Commit ready for merge.
Ryan - could you apply this to the current production branch as a patch and start a cron job that runs this file daily until the 24th please?
 - Legacy-Id: 7085
2014-01-08 18:56:25 +00:00
Ole Laursen e54bd17499 Port from django.core.management.setup_environ to DJANGO_SETTINGS_MODULE
- Legacy-Id: 6985
2013-12-18 15:51:11 +00:00
Henrik Levkowetz ecb5cd6526 Merged in branch/iola/shimfree@6856. This pretty much completes the shim removal. Next step is upgrading Django to 1.6, and then doing some final polishing.
- Legacy-Id: 6966
2013-12-17 21:29:05 +00:00
Henrik Levkowetz fa1075427a Merged [6857] from rjsparks@nostrum.com:
Script to run from cron to send daily messages to IETF Announce showing the email header field change.
This should be run from Jan 6 to Jan 20, after which point we need to deploy analagous changes to the code that constructs last call announcements.

This is related to bug #811

Commit ready for merge.
 - Legacy-Id: 6952
Note: SVN reference [6857] has been migrated to Git commit c500184eee
2013-12-16 22:25:16 +00:00
Henrik Levkowetz bf05a61fd9 Import SendQueue from the right place.
- Legacy-Id: 6946
2013-12-16 21:58:33 +00:00
Robert Sparks c500184eee Script to run from cron to send daily messages to IETF Announce showing the email header field change.
This should be run from Jan 6 to Jan 20, after which point we need to deploy analagous changes to the code that constructs last call announcements.

This is related to bug #811

Commit ready for merge.
 - Legacy-Id: 6857
2013-12-09 19:01:44 +00:00
Ole Laursen f75339a3e3 Remove shim layer from bin/dump-draft-info
- Legacy-Id: 6820
2013-12-06 17:58:05 +00:00
Ole Laursen 78d9bfff24 Move send_scheduled_announcement functionality to messages/utils.py
- Legacy-Id: 6768
2013-11-28 13:05:51 +00:00
Ole Laursen 35939cc20c Remove references to old announcement models and remove the model definitions
- Legacy-Id: 6767
2013-11-27 16:45:05 +00:00
Ole Laursen 9457669310 Merge in changes from the old shimfree branch
- Legacy-Id: 6764
2013-11-27 12:55:50 +00:00
Ole Laursen 9f1f7790e2 Add bin/expire-submissions script for making sure old submissions are canceled automatically
- Legacy-Id: 6718
2013-11-15 16:22:29 +00:00
Ole Laursen ecf68dbb05 Revamp and clean up submit models:
- Rename IdSubmissionDetail to Submission
- Rename various submission fields to correspond to the conventions in
  the new schema
- Use a name model for the states instead of IdSubmissionStatus
- Drop the TempIdAuthor model which is based on splitting up author
  names
- Add a simple textual SubmissionEvent for tracking events in the
  lifetime of a submission
- Delete a bunch of obsolete fields
- Make sure all submission have an access key so we can depend on it
- Add state for when approval is needed from previous authors

A couple of migrations take care of transforming the
IdSubmissionDetail and moving data over/cleaning it up.

Also revamp the submit view code:

- Make form code do validation/cleaning only so there's a clear
  separation of concerns
- Reduce uses of inheritance that made the code hard to follow -
  forms now don't inherit from each other, views don't call each other
  but instead reuse common utilities, templates share CSS/utilities
  instead of relying on inheritance
- Move email rendering/sending to separate file
- Drop the in-grown terminology use (auto post vs. manual posts)
- Make the status page explain who is emailed for what purpose
- Add history table with recorded events
- Make the status page handle its post actions by itself instead of
  duplicating most of the setup logic in a number of simple views
- Fix a couple of minor bugs and handle some edge cases better
- Expand tests with a couple of more cases

Possibly the submit tool could still use more help text added to
explain the process, ideally what's explained in the tool instructions
page should be inlined or self-evident.
 - Legacy-Id: 6714
2013-11-15 13:30:32 +00:00
Henrik Levkowetz 239fa9762c Tweaked the utility script to find missing submission confirmation emails.
- Legacy-Id: 6492
2013-10-26 18:10:30 +00:00
Ole Laursen b552ff31aa Replace the announcement pages with a new page collecting all IESG
review decisions. Add migration to split up iesg_approve/disapprove
events of the form "IESG has approved ... and state has been changed
..." into the approve/disapprove event and a synthesized state change
event. Also regularize the descriptions a bit. This simplifies the
code in the new page.
 - Legacy-Id: 6340
2013-10-02 15:37:44 +00:00
Ole Laursen a3f39aef3c Include referrer in test crawler crash reports, format the result like
a 500
 - Legacy-Id: 6063
2013-08-15 09:58:27 +00:00
Ole Laursen 0b1aabc64d Add extra exception trap to test crawler to output URL under test for None responses
- Legacy-Id: 6056
2013-08-14 16:15:42 +00:00
Ole Laursen 742dbace24 Move views and templatetags from idrfc/ to doc, the I-D specific views
go into views_draft.py and tests in tests_draft.py; what's left in
idrfc is wrapper code, templates and the ballot_icon templatetag
 - Legacy-Id: 6051
2013-08-14 10:32:25 +00:00
Henrik Levkowetz 8675711c27 Merged [5902],[5904] from markus.stenberg@iki.fi:
Added support and scripts for generation of wg- and draft-aliases.  Fixes issue #713.
 - Legacy-Id: 5937
Note: SVN reference [5902] has been migrated to Git commit e5b551f4ce

Note: SVN reference [5904] has been migrated to Git commit e68e51cc55
2013-08-01 12:44:49 +00:00
Henrik Levkowetz cbfe489ff5 Merged in more shim-layer removals from olau@iola.dk
(-r5465:HEAD from branch/iola/shimfree).  Copying relevant
commit messages here:

 - Deleted dead code in many places.

 - Renamed id-something to draft-something, make the "is
   this eligible for expiration" logic clearer

 - Added a name for IPR search URL

 - Revamped the ballot popup view

 - URL reversed the IPR search link instead of hardcoding it

 - Cleaned up search views and remove dead code, port them to the new DB
   schema, hack related views in iesg/ and wginfo/ to use the new search
   interfaces, avoid camelCase in search GET parameters (with
   backwards-compat fallback), add some simple search unit tests, remove
   caching from views_search.py index pages as they're now pretty fast to
   generate, rewrite ballot popup JS, regularize some CSS classes to use
   hyphen-notation rather than camelCase, move some of the search
   templates to doc/.

   idrfc/ now mostly contains some wrapper code still in use by other
   subdirs, some ported code not yet moved, and dead code.

 - Fixed output bug in test crawler and print referrer upon errors so it's
   easier to figure out where a link came from

 - Added /doc/in-last-call/ to crawler, report original page as referrer in
   a redirect chain rather than intermediate URL

 - Ported idindex to new schema, speed them up, add tests, refactor index
   page in views_search to share code with the text index file, get rid
   of some special-case idindex filters from ietf_filters, move
   "/drafts/" redirects to a file in /doc/

 - Ported /idtracker/status/ and /idtracker/status/last-call/ overview of
   drafts in IESG process to new schema in /doc/iesg/ and
   /doc/iesg/last-call/

 - Added redirects for all of /idtracker/*, removed all view code and other
   dead code from idtracker/

 - Removed the idtracker sitemap indexing drafts - in its current form,
   it adds nothing of value to the HTML-based /doc/all/ view, and it's
   pretty slow

 - Ported idtracker feeds to new schema, move them to doc/, cleaned up
   idtracker/ - only templatetags/ietf_filters and proxy code is left
 - Legacy-Id: 5836
2013-07-17 20:52:39 +00:00
Henrik Levkowetz e678659b56 Merged in a lot of shim-layer removals from olau@iola.dk
(-r5194:5465 from branch/iola/shimfree).  Copying relevant
commit messages here:

  - Removed .related many to many relationship, it's not really useful
    since we always have to restrict on the relationship type anyway,
    instead add two helpers for doing the necessary queries (in both
    directions)

  - Added migration for transforming the .desc on the new_revision events
    into something more akin to what is actually shown in the history page

  - Added migration for blanking IESG notes that just consist of "RFC
    XXXX", these have been superfluous for some time

  - Grant stream chairs access to changing the stream on a draft

  - Hacked the format_history_text filter to be less weird, using the same
    formatting for snippets and full text, also link up legacy ballot set
    events

  - Moved the decoraters + utilities to new ietfauth/utils.py file

  - Added simple helper to Email to identify invalid email addresses (from
    legacy author entries)

  - Used new new_revision .desc format for when drafts are submitted

  - Improved the looks of the button class by adding extra contrast and a
    linear gradient. Currently the gradient is only visible in fairly
    recent browsers.

  - Rewrote draft and RFC tabs in terms of the new schema, porting
    write-up and history tabs as well

  - Fixed two bugs in RFC Editor syncing: make sure documents we don't know
    beforehand get a "draft" type and make sure individually submitted
    drafts get the type="individ" group instead of NULL

  - Made the CSS-styled button feel a bit nicer to use by flattening the
    active state, also introduce some temporary styles until browsers
    catch up with the standard syntax

  - Added migrations for fixing 1) a dummy RFC entry, 2) three stand-alone
    RFCs that didn't get their doc.type set, 3) a big bunch of historic
    stand-alone RFCs that have doc.group=None - set these to the
    individual submission "none" group for the time being so the view code
    doesn't have to deal with a special case.

    In some cases this is wrong since there actually was a WG associated
    but unfortunately fixing them properly requires detective work
    (probably parsing the RFCs) and in at least some cases recreating
    historic WGs. In case someone ends up doing this, the documents to
    check can still be found with

    Document.objects.filter(name__startswith="rfc", group__type="individ")
    since there are almost no new RFCs that didn't went through the I-D
    process.

  - Merged the I-D and RFC views by showing I-D information on RFCs too.
    I-Ds that have been published as RFCs redirect to the RFC URL. Also
    support alias URLs so e.g. /doc/bcpXXXX redirects to /doc/rfcXXXX.

  - Fixed revision augmentation so events after RFC publication gets a "RFC"
    designation

  - Fixed a bug with tabs not using provided name but rather doc.name

  - Displaying draft-iesg state rather than doc.friendly_state as IESG state,
    also show a notice that the IESG state refers to post-RFC processing
    if it does, like the old separate RFC page did

  - Fixed the RFC number doc.note migration to catch combined "RFC XXX; BCP
    XXX" notes too, use the opportunity to remove inserted HTML tags from
    notes and rely on linebreaksbr filter instead (the other thing was a
    left-over from the Perl days), update the various uses of the note to
    reflect that

  - Refactored slightly to make views_doc.py independent of other idrfc code

  - Moveed idrfc/views_doc.py to doc/ with associated templates, replace the
    somewhat fragile simple URL tests for views_doc.py with ordinary unit
    tests. The new tests are still fairly basic but at least test more
    than the URL tests did.

  - Made sure RFC's (and BCP/STD/FYI) are stored as RFC123 instead of
    RFC0123 in the alias table with a new migration and a change to the
    RFC Editor sync, this in turn makes /doc/std1/ do the right thing

  - Now /doc/std1/ works, we can actually do a local link in
    urlize_ietf_docs rather than linking to the tools.ietf.org server

  - Fixed history text formatter: sanitize HTML before adding linebreaks and
    non-breaking spaces, this cuts the time to render a history page with
    long comments in half

  - Added a test crawler that walks through the crawlable part of the site,
    reporting errors and slow pages

  - Got rid of initial "No record" positions when showing old positions,
    it's just noise

  - Added a .select_related() to the document main tab to reduce the number
    of DB queries, unfortunately it seems it doesn't really help with
    Django 1.2.x due to a bug (Document inherits from DocumentInfo which
    makes things a bit more complicated)

  - Introduced a simple cache in doc.get_state so repeated reads don't
    cause a DB query

  - Cleaned up the search code in preparation for removal of the shim-layer;
    use a static button and don't send extraneous GET parameters

  - Removed dead code in several places
 - Legacy-Id: 5830
2013-07-15 20:55:24 +00:00
Henrik Levkowetz bb7fbd8cab Merged [5794] to [5800] from olau@iola.dk:
* Skip IANA - Review Needed and Version Changed - Review Needed changes from
    IANA as it turns out that the Datatracker is the authoritative source on
    these states. Also improve logging so that the raw JSON from IANA is
    dumped, the parsed JSON on imported changes is dumped and we write to
    syslog before starting a sync script in the notification view.

  * Only add charter state change link when chartering, it doesn't make sense
    for an approved charter. Fixes #861.

  * Don't display group.comments on the charter document page. Apparently
    group.comments have been used by the Secretariat in the past for
    chartering comments, but it's not possible to edit the field and the whole
    thing doesn't make sense from a modelling perspective - a feature like
    this should probably use the note field on the charter.  Fixes issue
    #1048.

  * Don't crash if an attachment has been created already, just reuse it -
    this case shouldn't actually happen, but apparently sometimes does,
    probably due to a concurrency issue

  * Add link to charter pages from the milestones editing page (besides the
    cancel button), fixes #1044.

  * Special-case proposed working groups with respect to milestones, they get
    the charter milestones on the WG charter page rather than the current set

  * Show approved milestones on /doc/charter-xyz/ page if the charter is
    approved instead of only showing proposed milestones for proposed charters
 - Legacy-Id: 5812
Note: SVN reference [5794] has been migrated to Git commit a603b8e056

Note: SVN reference [5800] has been migrated to Git commit 0f69f87a56
2013-07-14 14:00:39 +00:00
Ole Laursen 0f69f87a56 Skip IANA - Review Needed and Version Changed - Review Needed changes
from IANA as it turns out that the Datatracker is the authoritative
source on these states. Also improve logging so that the raw JSON from
IANA is dumped, the parsed JSON on imported changes is dumped and we
write to syslog before starting a sync script in the notification view.
 - Legacy-Id: 5800
2013-06-21 16:48:05 +00:00
Henrik Levkowetz b6098566a9 Fixed missing name change when moving settings from rfceditor module to settings.py.
- Legacy-Id: 5787
2013-06-11 10:45:20 +00:00
Henrik Levkowetz bd0c07290a Merged in branch/iola/milestones, which provides datatracker support for management of WG milestones.
- Legacy-Id: 5739
2013-05-15 22:07:23 +00:00
Henrik Levkowetz f2c4ed9f24 Changed additional places to use IANA sync settings from settings.py
- Legacy-Id: 5681
2013-04-21 15:57:56 +00:00
Henrik Levkowetz e4fd951f2c Use IANA URL settings from settings.py in some more places.
- Legacy-Id: 5676
2013-04-21 14:38:20 +00:00
Ole Laursen 4f7de7b4f4 Add /doc/in-last-call/ to crawler, report original page as referrer in
a redirect chain rather than intermediate URL
 - Legacy-Id: 5633
2013-04-10 11:28:56 +00:00
Ole Laursen 920c0cd9fb Fix output bug in test crawler and print referrer upon errors so it's
easier to figure out where a link came from
 - Legacy-Id: 5612
2013-04-04 14:47:53 +00:00
Ole Laursen d70fb9b4b1 Delete dead code, rename id-something to draft-something, make the "is
this eligible for expiration" logic clearer
 - Legacy-Id: 5606
2013-04-04 13:22:08 +00:00
Henrik Levkowetz 594ca1f256 Changed the logging target of auxiliary IANA and RFC-Ed sync scripts from LOG_LOCAL0 to LOG_USER, matching what we've already used elsewhere for datatracker logging.
- Legacy-Id: 5453
2013-02-24 20:14:20 +00:00
Ole Laursen db2c2a418c Add a test crawler that walks through the crawlable part of the site,
reporting errors and slow pages
 - Legacy-Id: 5311
2013-01-21 10:33:12 +00:00
Ole Laursen 6e89f3b606 Merge changes from old milestones branch
- Legacy-Id: 5198
2013-01-15 16:45:19 +00:00
Henrik Levkowetz 1fbcd783f3 Merged branch/iola/statesync up to @5118 to trunk.
- Legacy-Id: 5147
2012-12-20 20:32:03 +00:00
Ole Laursen b19f6d8060 Merge in milestones branch
- Legacy-Id: 5091
2012-12-03 14:36:30 +00:00
Henrik Levkowetz 09c891efba Added an utility script to extract information about the outgoing confirmation request email related to a draft submission.
- Legacy-Id: 4955
2012-10-25 21:27:24 +00:00
Ole Laursen 9c6fd95eab Add date option to RFC Editor index mirroring script so one can
control what the publication cut off date is, also fix crash problem
with some old RFCs not having a page count
 - Legacy-Id: 4907
2012-10-10 09:22:26 +00:00
Ole Laursen e569bf3f90 Improve the syslog output for the IANA review email script, also
handle error cases more graceful
 - Legacy-Id: 4880
2012-09-24 10:08:41 +00:00
Ole Laursen 915908c203 Add HTTP auth to IANA sync, password is supposed to be saved in settings_local.py
- Legacy-Id: 4875
2012-09-21 09:15:35 +00:00
Henrik Levkowetz c5e364dbfc Added a script to create NewRevisionDocEvent entries for charter files.
- Legacy-Id: 4859
2012-09-19 19:50:37 +00:00
Ole Laursen 85d2cdddc2 Add POST interface for IANA/RFC Editor for triggering updates, add
discrepancies report for seeing differences between the
Datatracker/RFC Editor/IANA, add tests of IANA/RFC Editor integration,
add script for weekly discrepancies emails
 - Legacy-Id: 4851
2012-09-17 15:57:48 +00:00
Ole Laursen 5282bd1d07 Add sync scripts for reading from IANA changes API, reading from the
protocols page (to see when references to newly published RFCs have
been updated) and parsing IANA review emails to be included as
comments
 - Legacy-Id: 4850
2012-09-17 15:54:22 +00:00
Ole Laursen 09e6203f18 Revamp sync from RFC Editor to include more info, fix a couple of bugs
and email the Secretariet when a draft sent to the RFC Editor ends up
in the queue, split it up so it's easier to test; also moved the
location of the binaries to bin/
 - Legacy-Id: 4848
2012-09-17 15:45:27 +00:00
Ole Laursen 488add5004 Add script and code (and tests) to send milestone reminders to ADs
(for reviews) and group chairs.
 - Legacy-Id: 4557
2012-06-30 16:46:31 +00:00
Ole Laursen 03bed41630 Merged in changes from the conversion branch
- Legacy-Id: 3187
2011-06-28 18:20:20 +00:00
Henrik Levkowetz eafcdccd3d Merged from log:branch/iesg-tracker@2571: IOLA's port of cron-scripts.
- Legacy-Id: 2578
2010-10-26 14:49:12 +00:00
Henrik Levkowetz 266b7820d0 Merged from log:branch/2.00@2363: Current release branch head to trunk.
- Legacy-Id: 2365
2010-07-21 12:48:05 +00:00
Henrik Levkowetz fa77ac30d5 Merged r840 from branch/2.00 to trunk, and updated version information in ietf/__init__.py
- Legacy-Id: 841
2007-07-04 21:17:57 +00:00
Bill Fenner 306bece87c Import users from htpasswd file, searching IESGLogin, WgPassword,
LiaisonUser, and EmailAddresses for a mapping to a PersonOrOrgInfo.
It doesn't yet import all groups or permissions; that'll be a
separate (repeatable) process.

The goal is to eventually eliminate completely the WgPassword and
LiaisonUser tables.  IESGLogin could go too eventually, but that
requires schema changes, not just changes in the authentication
mechanisms.
 - Legacy-Id: 616
2007-06-22 16:07:23 +00:00
Bill Fenner c5f65ea060 "pretty-print" with sed instead of xmllint, because of a limitation
in python's xml deserializer (whitespace is significant when
recognizing <None/> nodes, so a general pretty-printer will cause
them to not be recognized.)

This changes empty elements to the more verbose <foo></foo> format,
but that is handled properly by the xml parser.
 - Legacy-Id: 382
2007-06-14 12:18:40 +00:00
Bill Fenner 4e25407e25 Use python module instead of file path to settings file.
Switch back to using a temp file, since unflatten/gvpr don't
like taking input straight from the while loop.
 - Legacy-Id: 226
2007-06-04 13:54:29 +00:00
Bill Fenner 15a3474aed Annotate output graph with a parallelogram node with the model name.
- Legacy-Id: 183
2007-05-30 14:24:05 +00:00
Henrik Levkowetz 5872696834 Moving ietf to trunk/ietf
[[Split portion of a mixed commit.]]
 - Legacy-Id: 96.1
2007-05-04 12:37:28 +00:00