Commit graph

162 commits

Author SHA1 Message Date
Ole Laursen de7ae60985 Move markup_txt.py from idrfc/ to utils/
- Legacy-Id: 6046
2013-08-13 14:33:22 +00:00
Henrik Levkowetz 6baea429aa Merged in the nomcom app from ejimenez@yaco.es and esanchez@yaco.es, and accompanying dbtemplate app and the (standard) docutils module. Added some tweaks to get all tests to pass and migrations to run.
- Legacy-Id: 5948
2013-08-06 10:09:29 +00:00
Henrik Levkowetz 8675711c27 Merged [5902],[5904] from markus.stenberg@iki.fi:
Added support and scripts for generation of wg- and draft-aliases.  Fixes issue #713.
 - Legacy-Id: 5937
Note: SVN reference [5902] has been migrated to Git commit e5b551f4ce

Note: SVN reference [5904] has been migrated to Git commit e68e51cc55
2013-08-01 12:44:49 +00:00
Henrik Levkowetz 4ed0337cae Made the code which uses the 'magic' module to determine file type and encoding work with both the old and new interface to python-magic.
- Legacy-Id: 5876
2013-07-25 14:45:01 +00:00
Henrik Levkowetz cbfe489ff5 Merged in more shim-layer removals from olau@iola.dk
(-r5465:HEAD from branch/iola/shimfree).  Copying relevant
commit messages here:

 - Deleted dead code in many places.

 - Renamed id-something to draft-something, make the "is
   this eligible for expiration" logic clearer

 - Added a name for IPR search URL

 - Revamped the ballot popup view

 - URL reversed the IPR search link instead of hardcoding it

 - Cleaned up search views and remove dead code, port them to the new DB
   schema, hack related views in iesg/ and wginfo/ to use the new search
   interfaces, avoid camelCase in search GET parameters (with
   backwards-compat fallback), add some simple search unit tests, remove
   caching from views_search.py index pages as they're now pretty fast to
   generate, rewrite ballot popup JS, regularize some CSS classes to use
   hyphen-notation rather than camelCase, move some of the search
   templates to doc/.

   idrfc/ now mostly contains some wrapper code still in use by other
   subdirs, some ported code not yet moved, and dead code.

 - Fixed output bug in test crawler and print referrer upon errors so it's
   easier to figure out where a link came from

 - Added /doc/in-last-call/ to crawler, report original page as referrer in
   a redirect chain rather than intermediate URL

 - Ported idindex to new schema, speed them up, add tests, refactor index
   page in views_search to share code with the text index file, get rid
   of some special-case idindex filters from ietf_filters, move
   "/drafts/" redirects to a file in /doc/

 - Ported /idtracker/status/ and /idtracker/status/last-call/ overview of
   drafts in IESG process to new schema in /doc/iesg/ and
   /doc/iesg/last-call/

 - Added redirects for all of /idtracker/*, removed all view code and other
   dead code from idtracker/

 - Removed the idtracker sitemap indexing drafts - in its current form,
   it adds nothing of value to the HTML-based /doc/all/ view, and it's
   pretty slow

 - Ported idtracker feeds to new schema, move them to doc/, cleaned up
   idtracker/ - only templatetags/ietf_filters and proxy code is left
 - Legacy-Id: 5836
2013-07-17 20:52:39 +00:00
Henrik Levkowetz e678659b56 Merged in a lot of shim-layer removals from olau@iola.dk
(-r5194:5465 from branch/iola/shimfree).  Copying relevant
commit messages here:

  - Removed .related many to many relationship, it's not really useful
    since we always have to restrict on the relationship type anyway,
    instead add two helpers for doing the necessary queries (in both
    directions)

  - Added migration for transforming the .desc on the new_revision events
    into something more akin to what is actually shown in the history page

  - Added migration for blanking IESG notes that just consist of "RFC
    XXXX", these have been superfluous for some time

  - Grant stream chairs access to changing the stream on a draft

  - Hacked the format_history_text filter to be less weird, using the same
    formatting for snippets and full text, also link up legacy ballot set
    events

  - Moved the decoraters + utilities to new ietfauth/utils.py file

  - Added simple helper to Email to identify invalid email addresses (from
    legacy author entries)

  - Used new new_revision .desc format for when drafts are submitted

  - Improved the looks of the button class by adding extra contrast and a
    linear gradient. Currently the gradient is only visible in fairly
    recent browsers.

  - Rewrote draft and RFC tabs in terms of the new schema, porting
    write-up and history tabs as well

  - Fixed two bugs in RFC Editor syncing: make sure documents we don't know
    beforehand get a "draft" type and make sure individually submitted
    drafts get the type="individ" group instead of NULL

  - Made the CSS-styled button feel a bit nicer to use by flattening the
    active state, also introduce some temporary styles until browsers
    catch up with the standard syntax

  - Added migrations for fixing 1) a dummy RFC entry, 2) three stand-alone
    RFCs that didn't get their doc.type set, 3) a big bunch of historic
    stand-alone RFCs that have doc.group=None - set these to the
    individual submission "none" group for the time being so the view code
    doesn't have to deal with a special case.

    In some cases this is wrong since there actually was a WG associated
    but unfortunately fixing them properly requires detective work
    (probably parsing the RFCs) and in at least some cases recreating
    historic WGs. In case someone ends up doing this, the documents to
    check can still be found with

    Document.objects.filter(name__startswith="rfc", group__type="individ")
    since there are almost no new RFCs that didn't went through the I-D
    process.

  - Merged the I-D and RFC views by showing I-D information on RFCs too.
    I-Ds that have been published as RFCs redirect to the RFC URL. Also
    support alias URLs so e.g. /doc/bcpXXXX redirects to /doc/rfcXXXX.

  - Fixed revision augmentation so events after RFC publication gets a "RFC"
    designation

  - Fixed a bug with tabs not using provided name but rather doc.name

  - Displaying draft-iesg state rather than doc.friendly_state as IESG state,
    also show a notice that the IESG state refers to post-RFC processing
    if it does, like the old separate RFC page did

  - Fixed the RFC number doc.note migration to catch combined "RFC XXX; BCP
    XXX" notes too, use the opportunity to remove inserted HTML tags from
    notes and rely on linebreaksbr filter instead (the other thing was a
    left-over from the Perl days), update the various uses of the note to
    reflect that

  - Refactored slightly to make views_doc.py independent of other idrfc code

  - Moveed idrfc/views_doc.py to doc/ with associated templates, replace the
    somewhat fragile simple URL tests for views_doc.py with ordinary unit
    tests. The new tests are still fairly basic but at least test more
    than the URL tests did.

  - Made sure RFC's (and BCP/STD/FYI) are stored as RFC123 instead of
    RFC0123 in the alias table with a new migration and a change to the
    RFC Editor sync, this in turn makes /doc/std1/ do the right thing

  - Now /doc/std1/ works, we can actually do a local link in
    urlize_ietf_docs rather than linking to the tools.ietf.org server

  - Fixed history text formatter: sanitize HTML before adding linebreaks and
    non-breaking spaces, this cuts the time to render a history page with
    long comments in half

  - Added a test crawler that walks through the crawlable part of the site,
    reporting errors and slow pages

  - Got rid of initial "No record" positions when showing old positions,
    it's just noise

  - Added a .select_related() to the document main tab to reduce the number
    of DB queries, unfortunately it seems it doesn't really help with
    Django 1.2.x due to a bug (Document inherits from DocumentInfo which
    makes things a bit more complicated)

  - Introduced a simple cache in doc.get_state so repeated reads don't
    cause a DB query

  - Cleaned up the search code in preparation for removal of the shim-layer;
    use a static button and don't send extraneous GET parameters

  - Removed dead code in several places
 - Legacy-Id: 5830
2013-07-15 20:55:24 +00:00
Henrik Levkowetz 3105101cc5 Changed test running and test data generation to work with database rollback
(for instance with InnoDB).

Added settings.DATABASE_TEST_OPTIONS which can be used to force for instance
InnoDB use during testing.

Using this has the potential of reducing the time it takes to run the test
suite substantially; a few test runs indicate a reduction in test run time by
50% on a run-of-the-mill linux server.
 - Legacy-Id: 5823
2013-07-15 07:21:34 +00:00
Robert Sparks a741642717 Added editing last call messages, requesting, issuing and tracking IETF LCs to status-change documents
Added a Cancel button to the form that allows editing the relations for status-change documents
Added instructions to the agenda section 3.3

This adds states to status-change- documents and has a migration that must be applied.

This fixes bug #1039
 - Legacy-Id: 5770
2013-06-01 21:01:45 +00:00
Henrik Levkowetz 5bee7ac538 Provide the necessary message information to the user confirmation messaging. Fixes issue #1049.
- Legacy-Id: 5767
2013-05-31 16:32:25 +00:00
Henrik Levkowetz bd0c07290a Merged in branch/iola/milestones, which provides datatracker support for management of WG milestones.
- Legacy-Id: 5739
2013-05-15 22:07:23 +00:00
Henrik Levkowetz 927efe2b7c Merged [5600] and [5602] from rjsparks@nostrum.com:
Changed the header to last call messages to IETF Announce and to IANA to make
replying to the announcement less likely to create an unnecessary ticket.
 - Legacy-Id: 5652
Note: SVN reference [5602] has been migrated to Git commit 19dcf10316
2013-04-15 22:24:22 +00:00
Ole Laursen 3d1eb07afe Port idindex to new schema, speed them up, add tests, refactor index
page in views_search to share code with the text index file, get rid
of some special-case idindex filters from ietf_filters, move
"/drafts/" redirects to a file in /doc/
 - Legacy-Id: 5634
2013-04-10 11:48:07 +00:00
Henrik Levkowetz 7a98fff79c Merged [5476] from rjsparks@nostrum.com:
* Added a document type for changing the publication status of other documents, and the UI for manipulating them.
* Added charters, conflict reviews, and status changes to the AD's \'My Documents\' page
* Improved the presentation of all document types on the Agenda, Future Telechats, Scribe and Moderator package pages
* Made the reverse name for document relationships explicit rather than deriving them from forward names
* Added a way to get to the underlying Document (when it exists) from an RfcWrapper
* Unified several common forms used by different document types
 - Legacy-Id: 5629
Note: SVN reference [5476] has been migrated to Git commit 10b4f7a4dc
2013-04-07 19:11:18 +00:00
Robert Sparks 19dcf10316 Changed the header to last call messages to IETF Announce and to IANA to make replying to the announcement less likely to create an unnecessary ticket
- Legacy-Id: 5602
2013-03-27 19:04:50 +00:00
Henrik Levkowetz cc60a1aa5d Restored fix to utils/mail.py.
- Legacy-Id: 5521
2013-03-09 20:06:11 +00:00
Henrik Levkowetz e1a7bf92ff Added logging for mail sending problems.
- Legacy-Id: 5489
2013-03-06 19:36:06 +00:00
Henrik Levkowetz 2f3476690f In order that document states (which may be used by old documents) should not need to be removed when state machines change and states go out of use, all document states have a 'used' field. The code which accesses the state fields have not been requiring that used=True for the states returned, which is an error. Fixed this by adding 'used=True' to all State.objects.get() and State.objects.filter() call where it's relevant.
- Legacy-Id: 5486
2013-03-05 23:24:41 +00:00
Robert Sparks 10b4f7a4dc * Added a document type for changing the publication status of other documents, and the UI for manipulating them.
* Added charters, conflict reviews, and status changes to the AD's "My Documents" page
* Improved the presentation of all document types on the Agenda, Future Telechats, Scribe and Moderator package pages
* Made the reverse name for document relationships explicit rather than deriving them from forward names
* Added a way to get to the underlying Document (when it exists) from an RfcWrapper
* Unified several common forms used by different document types
 - Legacy-Id: 5476
2013-03-01 15:47:51 +00:00
Henrik Levkowetz c4015a302b Added variations on the recognized date formats during submitted draft parsing, such that comma need not be followed by whitespace in the formats using comma as a separator between some of the fields. Added extraction of drafts referenced by a document, in addition to RFCs referenced.
- Legacy-Id: 5456
2013-02-24 20:17:22 +00:00
Robert Sparks 589ff698ee Merge forward complete
- Legacy-Id: 5328
2013-01-24 21:28:27 +00:00
Ole Laursen accf6d4470 Move idrfc/views_doc.py to doc/ with associated templates, replace the
somewhat fragile simple URL tests for views_doc.py with ordinary unit
tests. The new tests are still fairly basic but at least test more
than the URL tests did.
 - Legacy-Id: 5295
2013-01-18 13:17:00 +00:00
Ole Laursen 6e89f3b606 Merge changes from old milestones branch
- Legacy-Id: 5198
2013-01-15 16:45:19 +00:00
Henrik Levkowetz 1fbcd783f3 Merged branch/iola/statesync up to @5118 to trunk.
- Legacy-Id: 5147
2012-12-20 20:32:03 +00:00
Henrik Levkowetz 9d9e2851e0 Test the argument list, not the tuple of argument tuple.
- Legacy-Id: 5134
2012-12-16 18:30:25 +00:00
Henrik Levkowetz 97a7ab30db Added simple logging of test runs, to make it easier to be sure that tests has been run successfully on the latest code.
- Legacy-Id: 5131
2012-12-15 22:47:11 +00:00
Ole Laursen b19f6d8060 Merge in milestones branch
- Legacy-Id: 5091
2012-12-03 14:36:30 +00:00
Henrik Levkowetz 4946a3f694 Updated draft submission author extraction module to handle dash-separated double given names.
- Legacy-Id: 5088
2012-12-03 13:17:33 +00:00
Emilio Jiménez 65afd10d4a Refactor mange group views:
* Create a new models NomComGroup with public_key field. Default type is nomcom
 * News urls and views to separate members and chair forms
 * Add rolodex url when person not found.
 * Views access is only for secretariat and chair roles
See #904
 - Legacy-Id: 5069
2012-11-22 10:46:39 +00:00
Emilio Jiménez 230f6cfae7 view only accessible by the secretariat that allows to assign the email of the members and the chair of the NomCom
See #904
 - Legacy-Id: 5066
2012-11-20 00:40:57 +00:00
Henrik Levkowetz 0e67b2c758 Tweaked the handling of the syslog module not being available on windows.
- Legacy-Id: 5058
2012-11-07 21:41:02 +00:00
Ole Laursen 45cf853524 Dump redacted events in DeletedEvent table
- Legacy-Id: 5025
2012-11-07 14:59:17 +00:00
Ole Laursen daa5d49adb Fix RFC Editor group acronym to be rfceditor in test data
- Legacy-Id: 5024
2012-11-07 14:58:49 +00:00
Henrik Levkowetz cfcce2f03e Merged [5000] from rjsparks@nostrum.com:
Made it more obvious when an intended status or shepherding AD is not set
Removed an unused template
Show reset approval text immediately.

Fixes bug #900
 - Legacy-Id: 5015
Note: SVN reference [5000] has been migrated to Git commit 2470ad77dd
2012-11-07 13:01:34 +00:00
Henrik Levkowetz cfe466eb37 Merged [4991] from adam@nostrum.com:
Fixing time column wrapping problem that exhibits in (at least) Safari.
 - Legacy-Id: 5008
Note: SVN reference [4991] has been migrated to Git commit 5e384591fd
2012-11-06 15:23:17 +00:00
Henrik Levkowetz 8aa4922978 Remove dependency on syslog module -- it won't be available under Python 2.7 under Windows.
- Legacy-Id: 4974
2012-10-30 22:17:24 +00:00
Henrik Levkowetz 45585957ef Added support for reverse-order (i.e., Japanese, Chinese, and other) names with uppercase family name in the draft submission author extraction.
- Legacy-Id: 4949
2012-10-23 12:33:21 +00:00
Henrik Levkowetz d0214d3b25 Merged [4806] from rjsparks@nostrum.com:
Better handling for BoFs, particularly those that won't create WGs
 - Legacy-Id: 4939
2012-10-21 14:40:02 +00:00
Henrik Levkowetz c5e364dbfc Added a script to create NewRevisionDocEvent entries for charter files.
- Legacy-Id: 4859
2012-09-19 19:50:37 +00:00
Ole Laursen 6062e16c2d Email IANA and RFC Editor when a draft is pulled from the queue at the
Datatracker, add page for editing IANA states, add more info to the
/doc/draft-XXXXX/doc.json dump for the RFC Editor, add page for
editing consensus, add page for requesting publication at the RFC
Editor for alternate streams (this will email the RFC Editor and set
the draft in the appropriate state), make it possible for alternate
streams to change the intended RFC status of a draft in the stream,
refactor how IANA copies are handled slightly so it's less code, put
drafts automatically in IANA Review "Need Review" state upon last
call, fix a bug in ballot issuing, remove a bit of dead code
 - Legacy-Id: 4857
2012-09-17 16:15:45 +00:00
Ole Laursen 85d2cdddc2 Add POST interface for IANA/RFC Editor for triggering updates, add
discrepancies report for seeing differences between the
Datatracker/RFC Editor/IANA, add tests of IANA/RFC Editor integration,
add script for weekly discrepancies emails
 - Legacy-Id: 4851
2012-09-17 15:57:48 +00:00
Ole Laursen 9fdcbc38ab Add time zone helpers for converting between local IETF db time and UTC
- Legacy-Id: 4849
2012-09-17 15:51:33 +00:00
Henrik Levkowetz 7467fa48a5 Tweaked the author extraction code to handle company names in the author list on the first page, when the company names contain a comma, such as for instance 'Foo Bar, Inc'.
- Legacy-Id: 4781
2012-08-22 12:52:32 +00:00
Henrik Levkowetz eb28ac8177 Removed the ValueError exceptions introduced in the previous revision of the draft author extraction code. Fixes issue #858.
- Legacy-Id: 4753
2012-08-06 15:16:53 +00:00
Henrik Levkowetz 798f7697af Merged [4600] from rjsparks@nostrum.com:
* Adds a new document type for conflict reviews, with a ballot for the IESG 5742 response to a review request
* Integrated the new document type into the iESG agenda views (including RSS feeds)* Removed the Edit and Add buttons from the document main view.
* Replaced Add with actions appropriate for the document type, such as "Begin IESG Processing" or "Begin IETF Conflict Review", and made most data directly editable on the document's main page, depending on access permissions.
* Removed a manual editing step that the secretariat had to perform when sending conflict review messages. The view now composes the message correctly given the stream.
* Added a pencil icon motif to differentiate fields that are editable.
* Generalized several views and helper functions to use Document instead of (e.g.) IdWrapper
* Generalized reading documents from the repository
* Added a way to get from IdWrapper  to the underlying Document to facilitate migrating way from the Wrapper classes* Added many helpers to Document to assist with migrating off IdWrapper
* Minor fixes and other changes
  * Fixes to document main view to avoid (silent) template failures. 
  * Began removing some of the code that is no longer reachable post-migration  * Corrected the behavior of the undefer code and added test cases for it
  * Improved initial population of notification lists and added the ability to regenerate the initial list
* Made the test code that scans for template coverage more robust
* Deployment notes:
  * new setting: CONFLICT_REVIEW_PATH. The associated directory will need to be created

This branch fixes bugs #805, #744 and #812
 - Legacy-Id: 4652
Note: SVN reference [4600] has been migrated to Git commit 798e7a50e7
2012-07-28 16:29:28 +00:00
Henrik Levkowetz 0c49999fc9 Updated utils/draft.py and modified the submit app code accordingly.
New features (keep in mind that utils/draft.py can be run standalone
to do extraction of draft author data, too):

  * The handling of author info formatted in columns causes problems
    in the face of an author named for instance A. Author with the
    company 'Al Author and Associates', causing breakage of email
    addresses longer than 'Al Author and'.  Tweaked the recognition
    of column data to require multiple (not only one) space around
    'and'.

  * Added support for extraction of author affiliation.

  * Tweaked the meaning of -t, --timestamp and added --notimestamp; and
    made the default be to emit leading timestamps based ont the draft
    file time.

  * Added support for running author extraction on RFCs, by not bailing
    out on not finding a draft name when RFC information is available.

  * Added support for additional date formats and author name formats.

  * Improved creation date extraction -- previously, the first supported
    date format which was recognized on the first page of the draft would
    be used, rather than the first date in a supported format.  This could
    cause errors if the Status of Memo section or Abstract contained a
    date occurring at the start of a line.

  * Tweaked the honorific regex to make things work better for the case
    when the full name in the author's address section includes a first
    name which isn't part of the first-page abbreviated name.  Fixes
    problems with draft-chiappa-lisp-introduction and similar.

  * Added a special case for people who provide their email address as
    'foo&cisco.com' instead of 'foo@cisco.com'.  Bah.

  * Added an alternative, more human-readable key-value-pair attribute
    output mode with a '-a' switch.

  * Tweaded the first-name regex to capture cases where the first name
    is indicated with an alternate first letter: 'Y(J) Stein'.  Fixes
    problems with draft-anavi-tdmoip and similar.
 - Legacy-Id: 4612
2012-07-11 12:51:33 +00:00
Robert Sparks f3fd4db35c Merge forward complete - checkpointing before fixing popup position dialogs
- Legacy-Id: 4595
2012-07-04 19:35:46 +00:00
Ole Laursen 6050cbf76a Give test groups a list_email so we can email them
- Legacy-Id: 4592
2012-07-04 18:29:53 +00:00
Henrik Levkowetz 1bc2f7daa2 Tweak test setup message for site_id change.
- Legacy-Id: 4567
2012-07-03 13:42:36 +00:00
Henrik Levkowetz aa00b6e35d Remove old duplicate debug.py
- Legacy-Id: 4565
2012-07-03 11:31:30 +00:00
Henrik Levkowetz 350914da15 Minor cleanup of test_runner.py. Added guards against settings which can make tests inexplicably fail. Updated check to avoid running tests on production server.
- Legacy-Id: 4564
2012-07-03 11:31:11 +00:00