Commit graph

291 commits

Author SHA1 Message Date
Henrik Levkowetz 437418ed36 Added a management command to deactivate email addresses found in delivery-status emails or given on the command-line.
- Legacy-Id: 15501
2018-09-29 19:42:34 +00:00
Henrik Levkowetz bf3b1f7a45 Added 'consent' to the fields handled when merging person records.
- Legacy-Id: 15488
2018-09-27 12:31:29 +00:00
Henrik Levkowetz 4fba531e9e Merged in the latest GDPR changes. This refines the handling of the consent checkbox on the account page; refines the Consent Needed warning given on login if consent is needed; tweaks several models to set the on_deletion fields for FK to User and Person appropriately; adds a Person.needs_consent() method to capture the logic of which fields require consent; refines the Person.plain_name() method and the user.log.log() function; and adds 2 management commands to send out consent requests and delete non-consent information, respectively.
- Legacy-Id: 15464
2018-09-16 23:12:43 +00:00
Henrik Levkowetz 75f703578c Tweaked the Person.needs_consent() method to take historic roles into account and to handle records without associated usesrs.
- Legacy-Id: 15459
2018-09-16 13:49:25 +00:00
Henrik Levkowetz 44d4d76092 Added a migration to capture a change in on_delete for the Person.user field.
- Legacy-Id: 15457
2018-09-16 13:46:28 +00:00
Henrik Levkowetz 74359e9538 Refined Person.plain_name() to avoid leading or trailing spaces when only one name component is present.
- Legacy-Id: 15456
2018-09-16 13:44:27 +00:00
Henrik Levkowetz cf82379503 Undid unintentional commit
- Legacy-Id: 15455
2018-09-14 20:31:17 +00:00
Henrik Levkowetz 89b8fe6c1b Moved document methods rfc_number() and rfcnum() to the abstract parent class to have them available also for historical document records. Should fix the recent server 500 on /doc/draft-ietf-aaa-diameter/17/.
- Legacy-Id: 15454
2018-09-14 20:29:07 +00:00
Henrik Levkowetz 6dacd694bc More name normalization
- Legacy-Id: 15452
2018-09-13 18:58:30 +00:00
Henrik Levkowetz 0ca9ec769c Added a None guard for the person merge last_login date comparison. Fixes issue #2562
- Legacy-Id: 15418
2018-07-30 16:57:59 +00:00
Henrik Levkowetz d8005ab0f1 Fixed an issue in Person.needs_consent(): avoid multiple mentions of 'email address'.
- Legacy-Id: 15268
2018-06-19 19:19:58 +00:00
Henrik Levkowetz 31352b0064 Added Person.needs_consent() which returns a list of descriptions of fields for which consent is needed, or [].
- Legacy-Id: 15258
2018-06-14 18:56:01 +00:00
Henrik Levkowetz 35e16ef7d5 Changed User ForeignKeys to not delete objects pointing at a user when the user is deleted, instead setting the ForeignKey field to None.
- Legacy-Id: 15257
2018-06-14 18:53:48 +00:00
Henrik Levkowetz 0df2c0dee8 Tweaked unidecode_name() to avoid double-quotes in the ascii string.
- Legacy-Id: 15256
2018-06-14 15:06:08 +00:00
Henrik Levkowetz dbe9211963 Tweaked unidecode_name() to not produce single-letter ascii surnames from non-ascii codepoints. The unidecode transliteration is in any case somewhat arbitrary, and in most cases a real person will tweak the ascii name of his account. When running tests, however, this tweak avoids some false test failures. And no, it's not simple to fix the draft author-extraction heuristics to deal well with single-letter surnames.
- Legacy-Id: 15239
2018-06-10 14:48:13 +00:00
Henrik Levkowetz a4d6d3c01e Data migrations to populate email origin and Person.name_from_draft.
- Legacy-Id: 15189
2018-05-29 13:10:20 +00:00
Henrik Levkowetz 2522082979 Changed the email origin field during test to hold user.username in order to exercise more of the code. Changed the EmailFactory to also use user.username as origin.
- Legacy-Id: 15172
2018-05-26 08:32:20 +00:00
Henrik Levkowetz 6ec050d807 Removed a long-running data migration (it will return in the following release).
- Legacy-Id: 15171
2018-05-24 16:11:05 +00:00
Henrik Levkowetz 619b20d2e7 Data migration to assign email origin based on existing records (author, role, and more).
- Legacy-Id: 15148
2018-05-17 16:50:49 +00:00
Henrik Levkowetz f47e1ff2ff Updated email admin to show origin in lists.
- Legacy-Id: 15147
2018-05-17 16:48:49 +00:00
Henrik Levkowetz 70ed611472 Changed the field type for the Person.consent field.
- Legacy-Id: 15146
2018-05-17 16:45:21 +00:00
Henrik Levkowetz 874aad0322 Added a consent field to the Person model.
- Legacy-Id: 15142
2018-05-08 16:24:26 +00:00
Henrik Levkowetz a66639299d Removed all references to the removed Person.affiliation field.
- Legacy-Id: 15129
2018-05-05 12:47:55 +00:00
Henrik Levkowetz 9fe73b5736 Updated admin and resources with email history entries.
- Legacy-Id: 15128
2018-05-05 12:41:47 +00:00
Henrik Levkowetz 5f37a71889 Added origin information to all places where we create email address entries.
- Legacy-Id: 15127
2018-05-05 12:40:30 +00:00
Henrik Levkowetz 37f0d141e9 Added a new field name_from_draft to Person, to hold the name field equivalent as captured from drafts, in case name has been modified by the user and we're asked to remove that info under GDPR. Added history for Email, and also an origin field to capture from where we got an email address (draft name, username, meeting registration, etc.) Added a log.assertion() to Email.save() in order to ensure we don't create any email without setting origin.
- Legacy-Id: 15126
2018-05-05 12:37:15 +00:00
Henrik Levkowetz 61a16856ad Updated the admin interface to use the simple-history admin support for Person history.
- Legacy-Id: 15097
2018-05-01 11:07:12 +00:00
Henrik Levkowetz c7d31b44c3 Added django-simple-history and replaced the old (and unused) PersonHistory class with a history=HistoricalRecords() field on Person. Added the needed migrations and changes to admin, resources, and settings. Related to issues #2505 and #2507.
- Legacy-Id: 15096
2018-04-27 17:36:20 +00:00
Henrik Levkowetz 53c4ac36db Removed the Person.address field, which is not being used. This was a legacy from the 2001 perl-based datatracker tables. Fixes issue #2504.
- Legacy-Id: 15095
2018-04-27 14:00:33 +00:00
Henrik Levkowetz 4da044e2db Pyflakes fixes.
- Legacy-Id: 14989
2018-04-01 19:06:02 +00:00
Henrik Levkowetz 8e97669dff Added a get_active_ads() utility function that uses the cache to reduce database access. Moved some utility functions so as to reduce import loops.
- Legacy-Id: 14986
2018-04-01 18:36:36 +00:00
Henrik Levkowetz 934c884f4d Migration to match the new endpoint choice avilable in the PersonalApiKey model.
- Legacy-Id: 14973
2018-03-30 15:11:20 +00:00
Henrik Levkowetz e8f999dc07 Added an API endpoint to let Meetech associate recording urls with sessions.
- Legacy-Id: 14967
2018-03-29 11:04:47 +00:00
Henrik Levkowetz 2354cc74d4 Tweaked person factory ascii name generation to not include periods inside names.
- Legacy-Id: 14875
2018-03-20 15:24:20 +00:00
Henrik Levkowetz bccb7fb849 Merged in [14862] from rcross@amsl.com:
Add view for merge person records.
 - Legacy-Id: 14871
Note: SVN reference [14862] has been migrated to Git commit 85f4861f9f
2018-03-20 12:31:34 +00:00
Ryan Cross 85f4861f9f Add view for merge person records. Commit ready for merge.
- Legacy-Id: 14862
2018-03-18 18:01:56 +00:00
Henrik Levkowetz b9e34b3eec Merged in [14843] from adam@nostrum.com:
Updated (deprecated, unregistered) 'text/json' with 'application/json' everwhere.
 - Legacy-Id: 14853
Note: SVN reference [14843] has been migrated to Git commit 3c4c0f2584
2018-03-17 18:48:01 +00:00
Ryan Cross f0a4ff213f Backout premature commit of person/tests.py. Commit ready for merge
- Legacy-Id: 14846
2018-03-17 15:51:28 +00:00
Ryan Cross 01bfbff170 Remove unused roles/forms.py. Commit ready for merge
- Legacy-Id: 14845
2018-03-17 15:49:02 +00:00
Adam Roach 3c4c0f2584 Updated (deprecated, unregistered) 'text/json' with 'application/json' everwhere. Commit ready for merge.
- Legacy-Id: 14843
2018-03-17 15:14:57 +00:00
Henrik Levkowetz 98f64dfea4 Added result caching for the Person.email() method.
- Legacy-Id: 14797
2018-03-14 19:14:16 +00:00
Henrik Levkowetz 06362ec046 Got rid of old migrations and created new initial migrations.
- Legacy-Id: 14662
2018-02-21 23:46:22 +00:00
Henrik Levkowetz 5638cf3da3 Changed all usage of ForeignKey and OneToOneFiled in model.py files to the compatibility versions from ietf.utils.models.
- Legacy-Id: 14661
2018-02-20 15:36:05 +00:00
Henrik Levkowetz 87f4997bf5 Fixed unbalanced parentheses.
- Legacy-Id: 14520
2018-01-14 19:20:24 +00:00
Henrik Levkowetz 4431c3eac3 Exluded another locale from author name generation during testing, due to problematic romanization.
- Legacy-Id: 14519
2018-01-14 18:55:22 +00:00
Henrik Levkowetz b48996f383 Exclude arabic names from PersonFactory due to difficulties with the romanization of the names in connection with draft author extraction.
- Legacy-Id: 14500
2018-01-10 15:20:16 +00:00
Henrik Levkowetz f28eacdd47 Tastypie cannot serialize binary fields. Exclude from PersonalApiKey resource.
- Legacy-Id: 14434
2017-12-18 17:57:48 +00:00
Henrik Levkowetz 789b88d25d Adjusted the personalapikey migration.
- Legacy-Id: 14433
2017-12-18 17:56:46 +00:00
Henrik Levkowetz 6567e707ce Merged in ^/personal/henrik/6.64.2-ballotapi@14426. This provides personal API keys and a ballot position API at /api/iesg/position. Also added an endpoint description at /api/.
- Legacy-Id: 14430
2017-12-17 23:43:44 +00:00
Henrik Levkowetz b0863c8963 Modified some names.
- Legacy-Id: 14428
2017-12-16 18:43:08 +00:00
Henrik Levkowetz e7209c6e50 Added a new personal event table to keep track of personal API key logins, and a management command to send out reports about activity to users with API keys. Added a weekly cronjob script to trigger weekly reports, and a monthly script for future use. Added a @require_api_key decorator to validate API keys for API key views and log in the API key owner. Modified the API key management urls to use create and disable rather than add and delete. Updated the API key list view. Added an API placeholder view function for ballot position setting, for test purposes. Added tests for the decorator and management command.
- Legacy-Id: 14426
2017-12-16 18:37:52 +00:00
Henrik Levkowetz 152261a869 Added new infrastructure for personal API keys, to generate, view, and delete them.
- Legacy-Id: 14423
2017-12-14 14:30:59 +00:00
Henrik Levkowetz f2f21c4ef3 Improved cleaning of unidecode results in ascii name conversion.
- Legacy-Id: 14405
2017-12-08 12:36:23 +00:00
Henrik Levkowetz 999cd502e2 Tweaked ascii name normalization.
- Legacy-Id: 14381
2017-11-30 16:09:06 +00:00
Henrik Levkowetz 2c1438c240 Moved unidecode_name from utils.text to person.name.
Modified UserFactory to use a new locale for each new user, instead of the
same locale for a whole test run.  This (almost) ensures the exercise of
code to deal with non-ascii names, something which would not happen if a
locale with ascii names was chosen at the start of a run.

Modified name.initials() to not use non-word characters as initials.

Modified unidecode_name() to do more normalization, to conform to the
conventions used in internet-drafts.

Added saving of the factory-boy random state in order to be able to re-run
a test suite with the same pseudo-random sequence as in a previous failed
run.

Fixed an issue with email formatting in test_api_submit_ok().

Modified the draft author extraction code to deal better with names with
embedded apostrophes.
 - Legacy-Id: 14141
2017-09-20 15:36:30 +00:00
Henrik Levkowetz 33b275b04f Added ietf.utils.text.unidecode_name() and replaced various uses of unidecode() with it, in order to normalize the generation of ascii versions of names, to avoid different practices in space stripping and space normalization in different parts of the code.
- Legacy-Id: 14128
2017-09-17 15:12:18 +00:00
Henrik Levkowetz 0fac54cb3e Merged in [14086] from rjsparks@nostrum.com:
Show whether a person is nomcom eligible on their edit_profile page. Related to #2257 and #2323.
 - Legacy-Id: 14090
Note: SVN reference [14086] has been migrated to Git commit dc20598999
2017-08-31 15:21:17 +00:00
Robert Sparks dc20598999 Show whether a person is nomcom eligible on their edit_profile page. Related to #2257 and #2323. Commit ready for merge.
- Legacy-Id: 14086
2017-08-28 19:28:44 +00:00
Henrik Levkowetz 4dad8cdf9a Added a test to check for missing admin models, and added the admin models that were currently missing.
- Legacy-Id: 14021
2017-08-01 14:00:29 +00:00
Robert Sparks 503199eaff Query unique person objects when searching for duplicates based on aliases. Fixes #2263. Commit ready for merge.
- Legacy-Id: 13978
2017-07-26 20:37:17 +00:00
Henrik Levkowetz f0317b3cff Added another surname particle.
- Legacy-Id: 13823
2017-07-10 20:08:33 +00:00
Henrik Levkowetz b8e11e7f71 Added a migration matching the new validator on Email.address.
- Legacy-Id: 13744
2017-07-01 10:26:49 +00:00
Henrik Levkowetz 70830e8705 Added email address validation for the Email.address field, now that we're not putting dummy addresses in it.
- Legacy-Id: 13739
2017-06-30 11:48:03 +00:00
Henrik Levkowetz 0d120fb46a Tweaked the EmailFactory to return an email address not longer than the Email.address field.
- Legacy-Id: 13738
2017-06-30 11:47:12 +00:00
Henrik Levkowetz ade8973015 Merged in [13726] from rcross@amsl.com:
Update secr.rolodex app to deactivate old Users when replaced on edit.  Fixes #2001.
 - Legacy-Id: 13735
Note: SVN reference [13726] has been migrated to Git commit a251076903
2017-06-29 20:39:52 +00:00
Henrik Levkowetz c4954e13b1 Added more test objects in test_merge_users(), and fixed an issue with merge_users() uncovered by the added testing.
- Legacy-Id: 13730
2017-06-29 14:53:00 +00:00
Ryan Cross a251076903 Update secr.rolodex app to deactivate old Users when replaced on edit. Fixes #2001. Commit ready for merge.
- Legacy-Id: 13726
2017-06-28 20:43:28 +00:00
Ryan Cross 7768e7c6b3 Improve User merge handling in merge_persons(). Fix minor bug in submit.checkers.py. Fixes #1900. Commit ready for merge.
- Legacy-Id: 13718
2017-06-26 23:13:58 +00:00
Henrik Levkowetz 7e68623c90 Additional tweaks to the surname particle handling.
- Legacy-Id: 13670
2017-06-18 03:22:52 +00:00
Henrik Levkowetz f1f32b41ae Additional tweaks to the surname particle handling.
- Legacy-Id: 13669
2017-06-18 03:20:49 +00:00
Henrik Levkowetz bb089c182d Improved the handling of surname particles, (prompted by failure to handle the name of Niels ten Oever correctly)
- Legacy-Id: 13663
2017-06-17 15:12:54 +00:00
Henrik Levkowetz 3d387e5ff6 Merged in [13567] from rcross@amsl.com:
Refactor ietf/bin/merge-person-records to facilitate testing.  Add tests.  Fixes #2162.
 - Legacy-Id: 13580
Note: SVN reference [13567] has been migrated to Git commit e9b7c5753e
2017-06-10 07:48:19 +00:00
Ryan Cross e9b7c5753e Refactor ietf/bin/merge-person-records to facilitate testing. Add tests. Fixes #2162. Commit ready for merge.
- Legacy-Id: 13567
2017-06-08 23:28:43 +00:00
Henrik Levkowetz 55d36e284d Changed the '%s <%s>'%(name,email) idiom in several places to use email.utils.formataddr() instead, to avoid future problems with names that would need quotes. Added comments in 2 places where this change would not be appropriate.
- Legacy-Id: 13536
2017-06-05 22:40:11 +00:00
Henrik Levkowetz 76628be3fd Merged in ^/branch/iola/author-stats-r13145 from olau@iola.dk, and fixed some tests in code which moved after the latest merge with trunk. The test suite passes, but the migrations are _not_ ready to run, because of numbering conflicts (again due to code changes on trunk since the latest sync).
- Legacy-Id: 13479
2017-05-31 20:59:26 +00:00
Henrik Levkowetz e6191432f3 Updated person migration 0017 to avoid database referential inconsistency due to missing person records.
- Legacy-Id: 13371
2017-05-16 15:17:07 +00:00
Henrik Levkowetz 48c275fa64 Merged in [13336] from rcross@amsl.com:
Fix issue where deferring a ballot causes 500 error in Telechat app.  Fixes #1342.
 - Legacy-Id: 13360
Note: SVN reference [13336] has been migrated to Git commit 7542e8daa3
2017-05-15 14:05:25 +00:00
Robert Sparks 8030fb3965 Created an EditedAuthorsDocEvent to capture the basis or reason for changing a documentauthor set. Mapped a large number of email addresses found from the RFC Editor database and from parsing RFCs to the right Person objects, and creating Person objects as necessary. Updated the documentauthors set for RFCs based on reconciling what the tracker already knew with the RFC Editor database and the parsing the RFCs. Manually inspected the Updates to ensure no author was inappropriately removed from the set. Commit ready for merge.
- Legacy-Id: 13289
2017-05-09 20:16:11 +00:00
Henrik Levkowetz 4803be0c03 Return a person search string instead of bombing out if there are multiple identical names where one don't have an active email address.
- Legacy-Id: 13245
2017-04-19 14:05:43 +00:00
Robert Sparks 0088273f5a Adds scripts to create/delete a dummy nomcom to use in development environments. Commit ready for merge. Fixes #2283.
- Legacy-Id: 13230
2017-04-18 16:13:14 +00:00
Ole Laursen 5b677dc6ba Merge author stats branch into new branch from trunk
- Legacy-Id: 13159
2017-03-28 14:36:40 +00:00
Lars Eggert 8a70e9773d Fix various "RemovedInDjango20Warning" warnings. Commit ready for merge.
- Legacy-Id: 13112
2017-03-25 19:16:45 +00:00
Henrik Levkowetz ce9f879500 Create all appropriate aliases when a Person record is created, both in ietfauth and in PersonFactory.
- Legacy-Id: 12958
2017-03-03 19:42:16 +00:00
Henrik Levkowetz ae8e07ef22 Merged in [12932] from rjsparks@nostrum.com:
Move primary but inactive email to the most recently touched active email for a Person. If a person has more than one primary email, make all but the first be not primary. Fixes #2214.
 - Legacy-Id: 12937
Note: SVN reference [12932] has been migrated to Git commit 572e373f95
2017-02-28 23:28:30 +00:00
Henrik Levkowetz 695bf37bc6 Merged in [12930] from rjsparks@nostrum.com:
Use the Role email associated with a reviewer as the From when emailing a review. Fixes #2201.
 - Legacy-Id: 12936
Note: SVN reference [12930] has been migrated to Git commit 8577a5e84a
2017-02-28 21:50:12 +00:00
Robert Sparks 572e373f95 Move primary but inactive email to the most recently touched active email for a Person. If a person has more than one primary email, make all but the first be not primary. Fixes #2214. Commit ready for merge.
- Legacy-Id: 12932
2017-02-28 19:39:01 +00:00
Robert Sparks 8577a5e84a Use the Role email associated with a reviewer as the From when emailing a review. Fixes #2201. Commit ready for merge.
- Legacy-Id: 12930
2017-02-28 18:40:09 +00:00
Henrik Levkowetz 5f053ad21a Cleaned up the remaining explicit url names, using dotted-paths to view
functions instead.  In all almost 700 changes.
 - Legacy-Id: 12923
2017-02-26 23:21:49 +00:00
Henrik Levkowetz 988a2b808f Added explicit charset setting and consistent encoding to several email paths through the datatracker. After introducing support for non-ascii names in email addresses, using role.formatted_email() to insert names into email content doesn't work any more, since uncode names will now be rfc2047-encoded in formatted_email(). Added another method role.name_and_email() for this case. Replaced another case of '%s <%s>' name-and-email formatting with formataddr(). Fixed some tests which did not send unicode email bodies to the email functions. Fixes issue #2017.
- Legacy-Id: 12896
2017-02-21 19:17:10 +00:00
Henrik Levkowetz 858d855eff Changed the new account creation to require a proper name at the same time as the account password is set, before actually creating the account. Also tweaked the password strength and confirmation code.
- Legacy-Id: 12892
2017-02-20 16:50:39 +00:00
Henrik Levkowetz 8930d29a8e Merged in Django-1.10 upgrade work from ^/personal/henrik/6.43.1-django-1.10
- Legacy-Id: 12881
2017-02-19 18:18:00 +00:00
Henrik Levkowetz cf4a4b02a7 Reworked the email address handling in order to be able to support non-ascii names as part of email address fields. Reworked the generation of user names in the test suite to generate names from multiple non-ascii locales. Fixes issue #2080.
- Legacy-Id: 12872
2017-02-18 21:50:18 +00:00
Ole Laursen c61babb418 Add citation and h-index statistics
- Legacy-Id: 12869
2017-02-17 17:43:14 +00:00
Ole Laursen b2ff10b0f2 Add support for extracting the country line from the author addresses
to the draft parser (incorporating patch from trunk), store the
extracted country instead of trying to turn it into an ISO country
code, add country and continent name models and add initial data for
those, add helper function for cleaning the countries, add author
country and continent charts, move the affiliation models to
stats/models.py, fix a bunch of bugs.
 - Legacy-Id: 12846
2017-02-15 18:43:57 +00:00
Henrik Levkowetz aa5e61d958 Updated all urlpatterns to use ietf.utils.urls.url() instead of django's,
in order to autogenerate dotted path url pattern names.  Updated a number
of url reverses to use dotted path, and removed explicit url pattern names
as needed.

Changed some imports to prevent import of ietf.urls before django
initialization was complete.


Changed 3 cases of form classes being curried to functions; django 1.10
didn't accept that.

Started converting old-style middleware classes to new-style middleware
functions (incomplete).

Tweaked a nomcom decorator to preserve function names and attributes, like
a good decorator should.

Replaced the removed django templatetag 'removetags' with our own version
which uses bleach, and does sanitizing in addition to removing explicitly
mentionied html tags.

Rewrote the filename argument handling in a management command which had
broken with the upgrade.
 - Legacy-Id: 12818
2017-02-11 14:43:01 +00:00
Henrik Levkowetz 761ed8a723 Extended name_parts to consider military rank prefixes.
- Legacy-Id: 12816
2017-02-10 21:58:39 +00:00
Henrik Levkowetz 19d585255a Django 1.10 does not permit the view argument to url() to be a string any more. Changed all cases view arguments to url() which were strings to instead reference the actual imported function.
- Legacy-Id: 12815
2017-02-10 14:49:16 +00:00
Ole Laursen 0d5bc33283 Fix some test errors
- Legacy-Id: 12787
2017-02-06 11:35:28 +00:00
Ole Laursen ea93001c63 Remove debug output
- Legacy-Id: 12786
2017-02-03 18:54:23 +00:00
Ole Laursen ef251c6bc7 Add author affiliation chart.
Also add a model for registering an alias for an affiliation so that
we can group affiliations that are considered the same for statistical
purposes, and a model for registering unimportant endings like Inc.
and GmbH.

Affiliation grouping is done through three means: stripping
uninteresting endings, merging entries that only differ in case and
aliases that map from case-insensitive alias to name.

Stripping endings and merging based on case seem to reduce the number
of needed manually maintained aliases greatly.
 - Legacy-Id: 12785
2017-02-03 18:49:43 +00:00