You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@allura.apache.org by ke...@apache.org on 2020/06/03 16:05:22 UTC

[allura] branch master updated (992db64 -> 9755507)

This is an automated email from the ASF dual-hosted git repository.

kentontaylor pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/allura.git.


    from 992db64  email_change_notif prevent new email notifications on site-admin
     new 03e2aac  [#8361] templates tests pass
     new 84a477c  [#8361] only strings for dispatch paths
     new 34e0aa0  [#8361] test_admin fixes; MaxBytesValidator should not convert
     new c0fa061  [#8361] use our own UnicodeString validator to work around problem, make sure everywhere uses it
     new c440188  [#8361] work around formencode problems with Files/bytes
     new 829ce8b  [#8361] e.message doesn't exist in py3
     new 6308ce0  [#8361] don't utf8 encode urls in tests
     new e659af3  [#8361] referer fixes (comes in as bytes, we want to use it as a string)
     new 8c45db8  [#8361] more header encoding fixes
     new 3bad27b  [#8361] work on test; will need Easywidgets version bump
     new 66462e7  [#8361] Bump Easywidgets version
     new 9755507  [#8361] updated requirements.txt

The 12 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 Allura/allura/app.py                               |   6 +-
 Allura/allura/controllers/attachments.py           |   4 +-
 Allura/allura/controllers/auth.py                  |  18 ++--
 Allura/allura/controllers/discuss.py               |  22 ++---
 Allura/allura/controllers/project.py               |   8 +-
 Allura/allura/controllers/repository.py            |   4 +-
 Allura/allura/controllers/rest.py                  |   6 +-
 Allura/allura/controllers/site_admin.py            |  20 ++--
 Allura/allura/ext/admin/admin_main.py              |   5 +-
 Allura/allura/ext/admin/widgets.py                 |  13 +--
 Allura/allura/ext/user_profile/user_main.py        |   9 +-
 Allura/allura/lib/repository.py                    |   8 +-
 Allura/allura/lib/utils.py                         |   2 +-
 Allura/allura/lib/validators.py                    |  32 +++++-
 Allura/allura/lib/widgets/auth_widgets.py          |   2 +-
 Allura/allura/lib/widgets/discuss.py               |   9 +-
 Allura/allura/lib/widgets/form_fields.py           |  15 +--
 Allura/allura/lib/widgets/forms.py                 |  68 +++++++------
 Allura/allura/lib/widgets/user_profile.py          |   7 +-
 Allura/allura/tests/decorators.py                  |  14 ++-
 Allura/allura/tests/functional/test_admin.py       |  31 +++---
 Allura/allura/tests/functional/test_auth.py        |   4 +-
 Allura/allura/tests/functional/test_discuss.py     |  20 ++--
 Allura/allura/tests/functional/test_root.py        |   7 +-
 .../tests/templates/jinja_master/test_lib.py       |   5 +-
 Allura/allura/tests/unit/patches.py                |   7 +-
 Allura/allura/webhooks.py                          |   7 +-
 AlluraTest/alluratest/controller.py                |  12 ---
 ForgeBlog/forgeblog/main.py                        |  15 +--
 ForgeBlog/forgeblog/tests/functional/test_rest.py  |   6 +-
 ForgeBlog/forgeblog/widgets.py                     |   3 +-
 ForgeChat/forgechat/main.py                        |   6 +-
 .../forgediscussion/controllers/root.py            |   9 +-
 ForgeDiscussion/forgediscussion/forum_main.py      |   2 +-
 ForgeDiscussion/forgediscussion/widgets/admin.py   |   6 +-
 ForgeImporters/forgeimporters/base.py              |   4 +-
 ForgeImporters/forgeimporters/github/code.py       |   4 +-
 ForgeImporters/forgeimporters/github/tracker.py    |   4 +-
 ForgeImporters/forgeimporters/github/wiki.py       |   7 +-
 ForgeSVN/forgesvn/model/svn.py                     |   2 +-
 ForgeSVN/forgesvn/svn_main.py                      |   4 +-
 ForgeShortUrl/forgeshorturl/main.py                |  13 ++-
 .../forgetracker/tests/functional/test_root.py     |  16 +--
 ForgeTracker/forgetracker/tracker_main.py          |  31 +++---
 ForgeTracker/forgetracker/widgets/admin.py         |   5 +-
 ForgeTracker/forgetracker/widgets/bin_form.py      |   9 +-
 ForgeTracker/forgetracker/widgets/ticket_form.py   |   2 +-
 ForgeWiki/forgewiki/wiki_main.py                   |  15 +--
 requirements.in                                    |   6 +-
 requirements.txt                                   | 108 ++++++++++-----------
 50 files changed, 356 insertions(+), 286 deletions(-)


[allura] 06/12: [#8361] e.message doesn't exist in py3

Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

kentontaylor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/allura.git

commit 829ce8bfa0ba5a14aeda36e49216d31973811ff9
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Tue May 12 11:56:15 2020 -0400

    [#8361] e.message doesn't exist in py3
---
 Allura/allura/lib/widgets/auth_widgets.py | 2 +-
 ForgeSVN/forgesvn/model/svn.py            | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Allura/allura/lib/widgets/auth_widgets.py b/Allura/allura/lib/widgets/auth_widgets.py
index a257599..3e6cbfb 100644
--- a/Allura/allura/lib/widgets/auth_widgets.py
+++ b/Allura/allura/lib/widgets/auth_widgets.py
@@ -90,7 +90,7 @@ class LoginForm(ForgeForm):
                 None)
         except exc.HTTPBadRequest as e:
             raise Invalid(
-                e.message,
+                e.args[0],
                 dict(username=value['username'], rememberme=value.get('rememberme'),
                      return_to=value.get('return_to')),
                 None)
diff --git a/ForgeSVN/forgesvn/model/svn.py b/ForgeSVN/forgesvn/model/svn.py
index 347e152..76ed47a 100644
--- a/ForgeSVN/forgesvn/model/svn.py
+++ b/ForgeSVN/forgesvn/model/svn.py
@@ -509,7 +509,7 @@ class SVNImplementation(M.RepositoryImplementation):
                     url, revision_start=rev, peg_revision=rev, limit=limit,
                     discover_changed_paths=True)
             except pysvn.ClientError as e:
-                if 'Unable to connect' in e.message:
+                if 'Unable to connect' in e.args[0]:
                     raise  # repo error
                 return  # no (more) history for this path
             for ci in logs:


[allura] 05/12: [#8361] work around formencode problems with Files/bytes

Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

kentontaylor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/allura.git

commit c44018882bd3edf210e74134722c18e0d0fdccd2
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Tue May 12 11:55:21 2020 -0400

    [#8361] work around formencode problems with Files/bytes
---
 Allura/allura/lib/validators.py    |  6 ++++++
 Allura/allura/lib/widgets/forms.py | 14 ++++++++++++++
 2 files changed, 20 insertions(+)

diff --git a/Allura/allura/lib/validators.py b/Allura/allura/lib/validators.py
index 2dee7fc..37e6cb2 100644
--- a/Allura/allura/lib/validators.py
+++ b/Allura/allura/lib/validators.py
@@ -458,6 +458,12 @@ def convertTime(timestring):
 
 class IconValidator(fev.FancyValidator):
     regex = '(jpg|jpeg|gif|png|bmp)$'
+
+    # https://github.com/formencode/formencode/issues/101 local fix
+    # formencode is_empty doesn't handle empty bytestring by default
+    def is_empty(self, val):
+        return val == b'' or super(IconValidator, self).is_empty(val)
+
     def _to_python(self, value, state):
         p = re.compile(self.regex, flags=re.I)
         result = p.search(value.filename)
diff --git a/Allura/allura/lib/widgets/forms.py b/Allura/allura/lib/widgets/forms.py
index 54082ce..d79b3ae 100644
--- a/Allura/allura/lib/widgets/forms.py
+++ b/Allura/allura/lib/widgets/forms.py
@@ -125,6 +125,20 @@ class ForgeForm(ew.SimpleForm):
                                                          ctx['errors'])
         return Markup(display)
 
+    # https://github.com/formencode/formencode/issues/101 local fix
+    def _make_schema(self):
+        schema = super(ForgeForm, self)._make_schema()
+
+        orig_val_is_iter = schema._value_is_iterator
+
+        def _value_is_iterator(val):
+            if isinstance(val, bytes):
+                return False
+            return orig_val_is_iter(val)
+        schema._value_is_iterator = _value_is_iterator
+
+        return schema
+
 
 class ForgeFormResponsive(ForgeForm):
     def __init__(self):


[allura] 07/12: [#8361] don't utf8 encode urls in tests

Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

kentontaylor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/allura.git

commit 6308ce07b38c993e85c367bbc3e08b6d85777437
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Mon May 18 11:19:50 2020 -0400

    [#8361] don't utf8 encode urls in tests
---
 Allura/allura/tests/functional/test_discuss.py          |  4 ++--
 ForgeBlog/forgeblog/main.py                             |  2 +-
 ForgeBlog/forgeblog/tests/functional/test_rest.py       |  6 +++---
 ForgeTracker/forgetracker/tests/functional/test_root.py | 16 ++++++++--------
 4 files changed, 14 insertions(+), 14 deletions(-)

diff --git a/Allura/allura/tests/functional/test_discuss.py b/Allura/allura/tests/functional/test_discuss.py
index 020ce0c..2849946 100644
--- a/Allura/allura/tests/functional/test_discuss.py
+++ b/Allura/allura/tests/functional/test_discuss.py
@@ -83,7 +83,7 @@ class TestDiscuss(TestDiscussBase):
             if field.has_attr('name'):
                 params[field['name']] = field.get('value') or ''
         params[f.find('textarea')['name']] = text
-        r = self.app.post(f['action'].encode('utf-8'), params=params,
+        r = self.app.post(f['action'], params=params,
                           headers={str('Referer'): str(thread_link.encode("utf-8"))},
                           extra_environ=dict(username=str('root')))
         r = r.follow()
@@ -416,7 +416,7 @@ class TestAttachment(TestDiscussBase):
             if field.has_attr('name'):
                 params[field['name']] = field.get('value') or ''
         params[f.find('textarea')['name']] = 'Test Post'
-        r = self.app.post(f['action'].encode('utf-8'), params=params,
+        r = self.app.post(f['action'], params=params,
                           headers={str('Referer'): str(self.thread_link.encode('utf-8'))})
         r = r.follow()
         self.post_link = str(
diff --git a/ForgeBlog/forgeblog/main.py b/ForgeBlog/forgeblog/main.py
index 344a4b1..52a01b7 100644
--- a/ForgeBlog/forgeblog/main.py
+++ b/ForgeBlog/forgeblog/main.py
@@ -608,7 +608,7 @@ class RootRestController(BaseController, AppRestControllerMixin):
     def _lookup(self, year=None, month=None, title=None, *rest):
         if not (year and month and title):
             raise exc.HTTPNotFound()
-        slug = '/'.join((year, month, six.moves.urllib.parse.unquote(title).decode('utf-8')))
+        slug = '/'.join((year, month, six.ensure_text(six.moves.urllib.parse.unquote(title))))
         post = BM.BlogPost.query.get(slug=slug, app_config_id=c.app.config._id)
         if not post:
             raise exc.HTTPNotFound()
diff --git a/ForgeBlog/forgeblog/tests/functional/test_rest.py b/ForgeBlog/forgeblog/tests/functional/test_rest.py
index 826a0ed..8b685ae 100644
--- a/ForgeBlog/forgeblog/tests/functional/test_rest.py
+++ b/ForgeBlog/forgeblog/tests/functional/test_rest.py
@@ -137,7 +137,7 @@ class TestBlogApi(TestRestApiBase):
         self.api_post('/rest/p/test/blog/', title='test',
                       text='test text', state='published')
         url = '/rest' + BM.BlogPost.query.find().first().url()
-        self.app.post(url.encode('utf-8'),
+        self.app.post(url,
                       params=dict(title='test2', text='test text2',
                                   state='published'),
                       extra_environ={'username': str('*anonymous')},
@@ -147,7 +147,7 @@ class TestBlogApi(TestRestApiBase):
         anon = M.ProjectRole.by_name('*anonymous')._id
         anon_write = M.ACE.allow(anon, 'write')
         acl.append(anon_write)
-        self.app.post(url.encode('utf-8'),
+        self.app.post(url,
                       params=dict(title='test2', text='test text2',
                                   state='published'),
                       extra_environ={'username': str('*anonymous')},
@@ -164,7 +164,7 @@ class TestBlogApi(TestRestApiBase):
                          extra_environ={'username': str('*anonymous')})
         assert_equal(r.json['posts'], [])
         url = '/rest' + BM.BlogPost.query.find().first().url()
-        self.app.post(url.encode('utf-8'),
+        self.app.post(url,
                       params=dict(title='test2', text='test text2',
                                   state='published'),
                       extra_environ={'username': str('*anonymous')},
diff --git a/ForgeTracker/forgetracker/tests/functional/test_root.py b/ForgeTracker/forgetracker/tests/functional/test_root.py
index bcb8069..eb341cc 100644
--- a/ForgeTracker/forgetracker/tests/functional/test_root.py
+++ b/ForgeTracker/forgetracker/tests/functional/test_root.py
@@ -926,7 +926,7 @@ class TestFunctionalController(TrackerTestController):
             if field.has_attr('name'):
                 params[field['name']] = field.get('value') or ''
         params[f.find('textarea')['name']] = 'test comment'
-        self.app.post(f['action'].encode('utf-8'), params=params,
+        self.app.post(f['action'], params=params,
                       headers={'Referer': '/bugs/1/'.encode("utf-8")})
         r = self.app.get('/bugs/1/', dict(page='1'))
         post_link = str(r.html.find('div', {'class': 'edit_post_form reply'}).find('form')['action'])
@@ -1522,7 +1522,7 @@ class TestFunctionalController(TrackerTestController):
             if field.has_attr('name'):
                 params[field['name']] = field.get('value') or ''
         params[f.find('textarea')['name']] = post_content
-        r = self.app.post(f['action'].encode('utf-8'), params=params,
+        r = self.app.post(f['action'], params=params,
                           headers={'Referer': '/bugs/1/'.encode("utf-8")})
         r = self.app.get('/bugs/1/', dict(page='1'))
         assert_true(post_content in r)
@@ -1538,7 +1538,7 @@ class TestFunctionalController(TrackerTestController):
             if field.has_attr('name'):
                 params[field['name']] = field.get('value') or ''
         params['ticket_form.summary'] = new_summary
-        r = self.app.post(f['action'].encode('utf-8'), params=params,
+        r = self.app.post(f['action'], params=params,
                           headers={'Referer': '/bugs/1/'.encode("utf-8")})
         r = self.app.get('/bugs/1/', dict(page='1'))
         assert_true(summary + ' --&gt; ' + new_summary in r)
@@ -1557,7 +1557,7 @@ class TestFunctionalController(TrackerTestController):
             if field.has_attr('name'):
                 params[field['name']] = field.get('value') or ''
         params[f.find('textarea')['name']] = post_content
-        r = self.app.post(f['action'].encode('utf-8'), params=params,
+        r = self.app.post(f['action'], params=params,
                           headers={'Referer': '/bugs/1/'.encode("utf-8")})
         r = self.app.get('/bugs/1/', dict(page='-1'))
         assert_true(summary in r)
@@ -1567,7 +1567,7 @@ class TestFunctionalController(TrackerTestController):
         assert_false('Page 1 of 1' in r)
         # add some more posts and check for pager
         for i in range(2):
-            r = self.app.post(f['action'].encode('utf-8'), params=params,
+            r = self.app.post(f['action'], params=params,
                               headers={'Referer': '/bugs/1/'.encode("utf-8")})
         r = self.app.get('/bugs/1/', dict(page='1', limit='2'))
         assert_true('Page 2 of 2' in r)
@@ -1585,7 +1585,7 @@ class TestFunctionalController(TrackerTestController):
             if field.has_attr('name'):
                 params[field['name']] = field.get('value') or ''
         params[f.find('textarea')['name']] = post_content
-        self.app.post(f['action'].encode('utf-8'), params=params,
+        self.app.post(f['action'], params=params,
                       headers={'Referer': '/bugs/1/'.encode("utf-8")})
         r = self.app.get('/bugs/feed.rss')
         post = M.Post.query.find().first()
@@ -2201,7 +2201,7 @@ class TestFunctionalController(TrackerTestController):
             if field.has_attr('name'):
                 params[field['name']] = field.get('value') or ''
         params[f.find('textarea')['name']] = post_content
-        r = self.app.post(f['action'].encode('utf-8'), params=params,
+        r = self.app.post(f['action'], params=params,
                           headers={'Referer': '/p/test2/bugs2/1/'.encode("utf-8")})
         r = self.app.get('/p/test2/bugs2/1/', dict(page='1'))
         assert_true(post_content in r)
@@ -2451,7 +2451,7 @@ class TestFunctionalController(TrackerTestController):
             if field.has_attr('name'):
                 params[field['name']] = field.get('value') or ''
         params[f.find('textarea')['name']] = 'test comment'
-        self.app.post(f['action'].encode('utf-8'), params=params,
+        self.app.post(f['action'], params=params,
                       headers={'Referer': '/bugs/1/'.encode("utf-8")})
         r = self.app.get('/bugs/1/', dict(page='1'))
         post_link = str(r.html.find('div', {'class': 'edit_post_form reply'}).find('form')['action'])


[allura] 02/12: [#8361] only strings for dispatch paths

Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

kentontaylor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/allura.git

commit 84a477c5fef2cd6617a4c671132479c9da4a1a8a
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Fri May 8 17:19:48 2020 -0400

    [#8361] only strings for dispatch paths
---
 Allura/allura/controllers/rest.py           | 6 +++++-
 Allura/allura/tests/functional/test_root.py | 7 +++++--
 2 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/Allura/allura/controllers/rest.py b/Allura/allura/controllers/rest.py
index a243f62..ee6548e 100644
--- a/Allura/allura/controllers/rest.py
+++ b/Allura/allura/controllers/rest.py
@@ -309,6 +309,10 @@ def nbhd_lookup_first_path(nbhd, name, current_user, remainder, api=False):
     prefix = nbhd.shortname_prefix
     pname = unquote(name)
     pname = six.ensure_text(pname)  # we don't support unicode names, but in case a url comes in with one
+    try:
+        pname.encode('ascii')
+    except UnicodeError:
+        raise exc.HTTPNotFound
     provider = plugin.ProjectRegistrationProvider.get()
     try:
         provider.shortname_validator.to_python(pname, check_allowed=False, neighborhood=nbhd)
@@ -324,7 +328,7 @@ def nbhd_lookup_first_path(nbhd, name, current_user, remainder, api=False):
     if project is None:
         # look for neighborhood tools matching the URL
         project = nbhd.neighborhood_project
-        return project, (pname.encode('utf-8'),) + remainder  # include pname in new remainder, it is actually the nbhd tool path
+        return project, (pname,) + remainder  # include pname in new remainder, it is actually the nbhd tool path
     if project and prefix == 'u/':
         # make sure user-projects are associated with an enabled user
         user = project.user_project_of
diff --git a/Allura/allura/tests/functional/test_root.py b/Allura/allura/tests/functional/test_root.py
index 014c093..a9cecb0 100644
--- a/Allura/allura/tests/functional/test_root.py
+++ b/Allura/allura/tests/functional/test_root.py
@@ -31,6 +31,8 @@ Please read http://pythonpaste.org/webtest/ for more information.
 from __future__ import unicode_literals
 from __future__ import absolute_import
 import os
+
+import six
 from six.moves.urllib.parse import quote
 
 from tg import tmpl_context as c
@@ -103,8 +105,9 @@ class TestRootController(TestController):
             self.app.get('/', headers=dict(Accept=str(hdr)), validate_skip=True)
 
     def test_encoded_urls(self):
-        # not valid unicode
-        self.app.get(b'/foo\xFF', status=400)
+        if six.PY2:
+            # not valid unicode
+            self.app.get(b'/foo\xFF', status=400)
         self.app.get('/foo%FF', status=400)
         # encoded
         self.app.get('/foo%C3%A9', status=404)


[allura] 09/12: [#8361] more header encoding fixes

Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

kentontaylor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/allura.git

commit 8c45db876bec1b3bdad3cfab1f43b2289c1b8c8f
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Mon May 18 17:23:06 2020 -0400

    [#8361] more header encoding fixes
---
 Allura/allura/tests/functional/test_discuss.py | 11 +++++------
 ForgeBlog/forgeblog/main.py                    |  2 +-
 2 files changed, 6 insertions(+), 7 deletions(-)

diff --git a/Allura/allura/tests/functional/test_discuss.py b/Allura/allura/tests/functional/test_discuss.py
index 80b8633..8518b00 100644
--- a/Allura/allura/tests/functional/test_discuss.py
+++ b/Allura/allura/tests/functional/test_discuss.py
@@ -31,7 +31,6 @@ from io import open
 from six.moves import range
 
 
-
 class TestDiscussBase(TestController):
 
     def _thread_link(self):
@@ -110,7 +109,7 @@ class TestDiscuss(TestDiscussBase):
         params[post_form.find('textarea')['name']] = 'This is a new post'
         r = self.app.post(post_link,
                           params=params,
-                          headers={str('Referer'): str(thread_link.encode("utf-8"))})
+                          headers={str('Referer'): str(thread_link)})
         r = r.follow()
         assert 'This is a new post' in r, r
         r = self.app.get(post_link)
@@ -124,10 +123,10 @@ class TestDiscuss(TestDiscussBase):
         params[post_form.find('textarea')['name']] = 'Tis a reply'
         r = self.app.post(post_link + 'reply',
                           params=params,
-                          headers={str('Referer'): str(post_link.encode("utf-8"))})
+                          headers={str('Referer'): str(post_link)})
         r = self.app.get(thread_link)
         assert 'Tis a reply' in r, r
-        permalinks = [post.find('form')['action'].encode('utf-8')
+        permalinks = [post.find('form')['action']
                       for post in r.html.findAll('div', {'class': 'edit_post_form reply'})]
         self.app.post(permalinks[1] + 'flag')
         self.app.post(permalinks[1] + 'moderate', params=dict(delete='delete'))
@@ -409,7 +408,7 @@ class TestAttachment(TestDiscussBase):
         for f in thread.html.findAll('form'):
             if f.get('action', '').endswith('/post'):
                 break
-        self.post_form_link = f['action'].encode('utf-8')
+        self.post_form_link = f['action']
         params = dict()
         inputs = f.findAll('input')
         for field in inputs:
@@ -417,7 +416,7 @@ class TestAttachment(TestDiscussBase):
                 params[field['name']] = field.get('value') or ''
         params[f.find('textarea')['name']] = 'Test Post'
         r = self.app.post(f['action'], params=params,
-                          headers={str('Referer'): str(self.thread_link.encode('utf-8'))})
+                          headers={str('Referer'): str(self.thread_link)})
         r = r.follow()
         self.post_link = str(
             r.html.find('div', {'class': 'edit_post_form reply'}).find('form')['action'])
diff --git a/ForgeBlog/forgeblog/main.py b/ForgeBlog/forgeblog/main.py
index c27ed0b..6e286c2 100644
--- a/ForgeBlog/forgeblog/main.py
+++ b/ForgeBlog/forgeblog/main.py
@@ -592,7 +592,7 @@ class RootRestController(BaseController, AppRestControllerMixin):
                 text=text,
                 labels=labels.split(','),
                 **kw)
-            return exc.HTTPCreated(headers=dict(Location=str(h.absurl('/rest' + post.url()).encode('utf-8'))))
+            return exc.HTTPCreated(headers=dict(Location=str(h.absurl('/rest' + post.url()))))
 
         else:
             result = RootController().index(limit=limit, page=page)


[allura] 08/12: [#8361] referer fixes (comes in as bytes, we want to use it as a string)

Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

kentontaylor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/allura.git

commit e659af303306f2dcc80491e5cd57ee75f0e13561
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Mon May 18 12:47:07 2020 -0400

    [#8361] referer fixes (comes in as bytes, we want to use it as a string)
---
 Allura/allura/app.py                               |  6 +++---
 Allura/allura/controllers/attachments.py           |  4 +++-
 Allura/allura/controllers/auth.py                  | 16 +++++++++-------
 Allura/allura/controllers/discuss.py               | 22 +++++++++++-----------
 Allura/allura/controllers/project.py               |  8 ++++----
 Allura/allura/controllers/repository.py            |  4 ++--
 Allura/allura/controllers/site_admin.py            | 12 ++++++------
 Allura/allura/ext/admin/admin_main.py              |  5 +++--
 Allura/allura/ext/user_profile/user_main.py        |  9 +++++----
 Allura/allura/lib/repository.py                    |  8 +++++---
 Allura/allura/lib/utils.py                         |  2 +-
 Allura/allura/tests/functional/test_discuss.py     |  2 +-
 Allura/allura/tests/unit/patches.py                |  7 +++++--
 ForgeBlog/forgeblog/main.py                        |  4 ++--
 ForgeChat/forgechat/main.py                        |  3 ++-
 .../forgediscussion/controllers/root.py            |  6 ++++--
 ForgeDiscussion/forgediscussion/forum_main.py      |  2 +-
 ForgeSVN/forgesvn/svn_main.py                      |  4 +++-
 ForgeShortUrl/forgeshorturl/main.py                | 10 ++++++----
 ForgeTracker/forgetracker/tracker_main.py          | 12 ++++++------
 ForgeWiki/forgewiki/wiki_main.py                   |  8 +++++---
 21 files changed, 87 insertions(+), 67 deletions(-)

diff --git a/Allura/allura/app.py b/Allura/allura/app.py
index a631fb7..ef751ef 100644
--- a/Allura/allura/app.py
+++ b/Allura/allura/app.py
@@ -918,7 +918,7 @@ class DefaultAdminController(BaseController, AdminControllerMixin):
         require_access(self.app, 'configure')
         self.app.config.options['mount_label'] = mount_label
         g.post_event('project_menu_updated')
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @expose('jinja:allura:templates/app_admin_options.html')
     def options(self):
@@ -973,7 +973,7 @@ class DefaultAdminController(BaseController, AdminControllerMixin):
                          + self.app.config.options.mount_point
                          + '/')
             else:
-                redirect(request.referer or '/')
+                redirect(six.ensure_text(request.referer or '/'))
 
     @without_trailing_slash
     @expose()
@@ -1026,7 +1026,7 @@ class DefaultAdminController(BaseController, AdminControllerMixin):
                 if (ace.permission == perm) and (ace.access == model.ACE.DENY):
                     self.app.config.acl.append(ace)
         g.post_event('project_menu_updated')  # since 'read' permission changes can affect what is visible in menu
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
 
 class WebhooksLookup(BaseController, AdminControllerMixin):
diff --git a/Allura/allura/controllers/attachments.py b/Allura/allura/controllers/attachments.py
index 427a329..f93c684 100644
--- a/Allura/allura/controllers/attachments.py
+++ b/Allura/allura/controllers/attachments.py
@@ -17,6 +17,8 @@
 
 from __future__ import unicode_literals
 from __future__ import absolute_import
+
+import six
 from six.moves.urllib.parse import unquote
 from webob import exc
 
@@ -117,7 +119,7 @@ class AttachmentController(BaseController):
             self.handle_post(delete, **kw)
             if is_ajax(request):
                 return
-            redirect(request.referer or '/')
+            redirect(six.ensure_text(request.referer or '/'))
         if self.artifact.deleted:
             raise exc.HTTPNotFound
         embed = False
diff --git a/Allura/allura/controllers/auth.py b/Allura/allura/controllers/auth.py
index 5c457b6..f70cfb7 100644
--- a/Allura/allura/controllers/auth.py
+++ b/Allura/allura/controllers/auth.py
@@ -137,10 +137,12 @@ class AuthController(BaseController):
         elif orig_request:
             return_to = orig_request.url
         else:
-            if request.referer is not None and request.referer.split('/')[-1] == 'neighborhood':
+            if request.referer is not None and six.ensure_text(request.referer).split('/')[-1] == 'neighborhood':
                 return_to = '/'
+            elif request.referer:
+                return_to = six.ensure_text(request.referer)
             else:
-                return_to = request.referer
+                return_to = None
         c.form = F.login_form
         return dict(return_to=return_to)
 
@@ -282,7 +284,7 @@ class AuthController(BaseController):
             flash('Verification link sent')
         else:
             flash('No such address', 'error')
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     def _verify_addr(self, addr, do_auth_check=True):
         confirmed_by_other = M.EmailAddress.find(dict(email=addr.email, confirmed=True)).all() if addr else []
@@ -386,7 +388,7 @@ class AuthController(BaseController):
 
         if 'multifactor-username' not in session:
             tg.flash('Your multifactor login was disrupted, please start over.', 'error')
-            redirect('/auth/', return_to=kwargs.get('return_to', ''))
+            redirect('/auth/', {'return_to': kwargs.get('return_to', '')})
 
         user = M.User.by_username(session['multifactor-username'])
         try:
@@ -729,7 +731,7 @@ class PreferencesController(BaseController):
     @require_post()
     def user_message(self, allow_user_messages=False):
         c.user.set_pref('disable_user_messages', not allow_user_messages)
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @expose('jinja:allura:templates/user_totp.html')
     @without_trailing_slash
@@ -1252,7 +1254,7 @@ class SubscriptionsController(BaseController):
         if email_format:
             c.user.set_pref('email_format', email_format)
 
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @expose()
     @require_post()
@@ -1263,7 +1265,7 @@ class SubscriptionsController(BaseController):
             h.auditlog_user('User mention notifications are enabled')
         else:
             h.auditlog_user('User mention notifications are disabled')
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
 
 class OAuthController(BaseController):
diff --git a/Allura/allura/controllers/discuss.py b/Allura/allura/controllers/discuss.py
index d6fa61e..c6f18e7 100644
--- a/Allura/allura/controllers/discuss.py
+++ b/Allura/allura/controllers/discuss.py
@@ -100,7 +100,7 @@ class DiscussionController(BaseController, FeedController):
             self.moderate = ModerationController(self)
 
     def error_handler(self, *args, **kwargs):
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @h.vardec
     @expose()
@@ -115,7 +115,7 @@ class DiscussionController(BaseController, FeedController):
                 thread.unsubscribe()
             session(self.M.Thread)._get().skip_mod_date = True
             session(self.M.Thread)._get().skip_last_updated = True
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     def get_feed(self, project, app, user):
         """Return a :class:`allura.controllers.feed.FeedArgs` object describing
@@ -215,7 +215,7 @@ class ThreadController(six.with_metaclass(h.ProxiedAttrMeta, BaseController, Fee
                     show_moderate=kw.get('show_moderate'))
 
     def error_handler(self, *args, **kwargs):
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @memorable_forget()
     @h.vardec
@@ -236,7 +236,7 @@ class ThreadController(six.with_metaclass(h.ProxiedAttrMeta, BaseController, Fee
         if self.thread.ref:
             require_access(self.thread.ref.artifact, 'post')
         self.thread.labels = labels.split(',')
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @expose()
     def flag_as_spam(self, **kw):
@@ -260,14 +260,14 @@ class ThreadController(six.with_metaclass(h.ProxiedAttrMeta, BaseController, Fee
 
 def handle_post_or_reply(thread, edit_widget, rate_limit, kw, parent_post_id=None):
     require_access(thread, 'post')
-    rate_limit(M.Post, "Comment", redir=request.referrer)
+    rate_limit(M.Post, "Comment", redir=six.ensure_text(request.referer or '/'))
     if thread.ref:
         require_access(thread.ref.artifact, 'post')
     kw = edit_widget.to_python(kw, None)  # could raise Invalid, but doesn't seem like it ever does
     if not kw['text']:
         flash('Your post was not saved. You must provide content.',
               'error')
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
     file_info = kw.get('file_info', None)
     p = thread.add_post(parent_id=parent_post_id, **kw)
     p.add_multiple_attachments(file_info)
@@ -275,7 +275,7 @@ def handle_post_or_reply(thread, edit_widget, rate_limit, kw, parent_post_id=Non
         thread.artifact.mod_date = datetime.utcnow()
     flash('Message posted')
     notification_tasks.send_usermentions_notification.post(p.index_id(), kw['text'])
-    redirect(request.referer or '/')
+    redirect(six.ensure_text(request.referer or '/'))
 
 
 class PostController(six.with_metaclass(h.ProxiedAttrMeta, BaseController)):
@@ -332,7 +332,7 @@ class PostController(six.with_metaclass(h.ProxiedAttrMeta, BaseController)):
                                        target=self.post.thread.artifact or self.post.thread,
                                        related_nodes=[self.post.app_config.project],
                                        tags=['comment'])
-            redirect(request.referer or '/')
+            redirect(six.ensure_text(request.referer or '/'))
         elif request.method == 'GET':
             if self.post.deleted:
                 raise exc.HTTPNotFound
@@ -406,7 +406,7 @@ class PostController(six.with_metaclass(h.ProxiedAttrMeta, BaseController)):
         return dict(status=status, counts=self.post.react_counts)
 
     def error_handler(self, *args, **kwargs):
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @memorable_forget()
     @h.vardec
@@ -450,7 +450,7 @@ class PostController(six.with_metaclass(h.ProxiedAttrMeta, BaseController)):
     def attach(self, file_info=None):
         require_access(self.post, 'moderate')
         self.post.add_multiple_attachments(file_info)
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @expose()
     def _lookup(self, id, *remainder):
@@ -564,7 +564,7 @@ class ModerationController(six.with_metaclass(h.ProxiedAttrMeta, BaseController)
                     posted.thread.post_to_feed(posted)
         flash('{} {}'.format(h.text.plural(count, 'post', 'posts'),
                               'deleted' if delete else 'marked as spam' if spam else 'approved'))
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @expose()
     @require_post()
diff --git a/Allura/allura/controllers/project.py b/Allura/allura/controllers/project.py
index 4240042..77fea7e 100644
--- a/Allura/allura/controllers/project.py
+++ b/Allura/allura/controllers/project.py
@@ -874,7 +874,7 @@ class NeighborhoodAwardsController(object):
                     icon.filename, icon.file, content_type=icon.type,
                     square=True, thumbnail_size=(48, 48),
                     thumbnail_meta=dict(award_id=award._id))
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @expose()
     @require_post()
@@ -895,7 +895,7 @@ class NeighborhoodAwardsController(object):
             award.comment = comment
             with h.push_context(recipient_q._id):
                 g.post_event('project_updated')
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
 
 class AwardController(object):
@@ -963,7 +963,7 @@ class AwardController(object):
                     g.post_event('project_updated')
             M.AwardFile.query.remove(dict(award_id=self.award._id))
             self.award.delete()
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
 
 class GrantController(object):
@@ -1005,7 +1005,7 @@ class GrantController(object):
         self.grant.delete()
         with h.push_context(self.project._id):
             g.post_event('project_updated')
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
 
 class ProjectImporterController(object):
diff --git a/Allura/allura/controllers/repository.py b/Allura/allura/controllers/repository.py
index 08fcde0..4959af7 100644
--- a/Allura/allura/controllers/repository.py
+++ b/Allura/allura/controllers/repository.py
@@ -116,7 +116,7 @@ class RepoRootController(BaseController, FeedController):
         allura.tasks.repo_tasks.refresh.post()
         if request.referer:
             flash('Repository is being refreshed')
-            redirect(request.referer or '/')
+            redirect(six.ensure_text(request.referer or '/'))
         else:
             return '%r refresh queued.\n' % c.app.repo
 
@@ -157,7 +157,7 @@ class RepoRootController(BaseController, FeedController):
                     raise
                 except Exception as ex:
                     flash(str(ex), 'error')
-                    redirect(request.referer or '/')
+                    redirect(six.ensure_text(request.referer or '/'))
 
     @property
     def mr_widget(self):
diff --git a/Allura/allura/controllers/site_admin.py b/Allura/allura/controllers/site_admin.py
index 4fa7ffc..7def94e 100644
--- a/Allura/allura/controllers/site_admin.py
+++ b/Allura/allura/controllers/site_admin.py
@@ -513,7 +513,7 @@ class SiteNotificationController(object):
     def delete(self):
         self.note.delete()
         ThreadLocalORMSession().flush_all()
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
 
 class TaskManagerController(object):
@@ -696,7 +696,7 @@ class AdminUserDetailsController(object):
             flash('Comment added', 'ok')
         else:
             flash('Can not add comment "%s" for user %s' % (comment, user))
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @expose()
     @require_post()
@@ -715,7 +715,7 @@ class AdminUserDetailsController(object):
             AuthenticationProvider.get(request).deactivate_user(user)
             AuthenticationProvider.get(request).enable_user(user, audit=False)
             flash('Set user status to pending')
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @expose()
     @require_post()
@@ -727,7 +727,7 @@ class AdminUserDetailsController(object):
         AuthenticationProvider.get(request).set_password(user, None, pwd)
         h.auditlog_user('Set random password', user=user)
         flash('Password is set', 'ok')
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @expose()
     @require_post()
@@ -740,7 +740,7 @@ class AdminUserDetailsController(object):
             allura.controllers.auth.AuthController().password_recovery_hash(email)
         except HTTPFound:
             pass  # catch redirect to '/'
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @expose()
     @require_post()
@@ -759,7 +759,7 @@ class AdminUserDetailsController(object):
         if not user or user.is_anonymous():
             raise HTTPNotFound()
         allura.controllers.auth.PreferencesController()._update_emails(user, admin=True, form_params=kw)
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
 
 class StatsSiteAdminExtension(SiteAdminExtension):
diff --git a/Allura/allura/ext/admin/admin_main.py b/Allura/allura/ext/admin/admin_main.py
index 3930a94..4fd8623 100644
--- a/Allura/allura/ext/admin/admin_main.py
+++ b/Allura/allura/ext/admin/admin_main.py
@@ -645,10 +645,11 @@ class ProjectAdminController(BaseController):
             flash('%s: %s' % (exc.__class__.__name__, exc.args[0]),
                   'error')
         if request.referer is not None and tool is not None and 'delete' in tool[0] and \
-            re.search(c.project.url() + r'(admin\/|)' + tool[0]['mount_point']+ r'\/*', request.referer):
+            re.search(c.project.url() + r'(admin\/|)' + tool[0]['mount_point']+ r'\/*',
+                      six.ensure_text(request.referer)):
             # Redirect to root when deleting currect module
             redirect('../')
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @expose('jinja:allura.ext.admin:templates/export.html')
     def export(self, tools=None, with_attachments=False):
diff --git a/Allura/allura/ext/user_profile/user_main.py b/Allura/allura/ext/user_profile/user_main.py
index bf96096..2125605 100644
--- a/Allura/allura/ext/user_profile/user_main.py
+++ b/Allura/allura/ext/user_profile/user_main.py
@@ -22,6 +22,7 @@ import logging
 import pkg_resources
 from datetime import datetime
 
+import six
 from formencode import validators
 from tg import request
 from tg import tmpl_context as c, app_globals as g
@@ -128,21 +129,21 @@ class UserProfileController(BaseController, FeedController):
     def _check_can_message(self, from_user, to_user):
         if from_user is User.anonymous():
             flash('You must be logged in to send user messages.', 'info')
-            redirect(request.referer or '/')
+            redirect(six.ensure_text(request.referer or '/'))
 
         if not (from_user and from_user.get_pref('email_address')):
             flash('In order to send messages, you must have an email address '
                   'associated with your account.', 'info')
-            redirect(request.referer or '/')
+            redirect(six.ensure_text(request.referer or '/'))
 
         if not (to_user and to_user.get_pref('email_address')):
             flash('This user can not receive messages because they do not have '
                   'an email address associated with their account.', 'info')
-            redirect(request.referer or '/')
+            redirect(six.ensure_text(request.referer or '/'))
 
         if to_user.get_pref('disable_user_messages'):
             flash('This user has disabled direct email messages', 'info')
-            redirect(request.referer or '/')
+            redirect(six.ensure_text(request.referer or '/'))
 
     @expose('jinja:allura.ext.user_profile:templates/user_index.html')
     def index(self, **kw):
diff --git a/Allura/allura/lib/repository.py b/Allura/allura/lib/repository.py
index af88d9b..be49df1 100644
--- a/Allura/allura/lib/repository.py
+++ b/Allura/allura/lib/repository.py
@@ -18,6 +18,8 @@
 from __future__ import unicode_literals
 from __future__ import absolute_import
 import logging
+
+import six
 from six.moves.urllib.parse import quote
 
 from tg import tmpl_context as c, app_globals as g
@@ -269,14 +271,14 @@ class RepoAdminController(DefaultAdminController):
     @require_post()
     def set_extensions(self, **post_data):
         self.repo.additional_viewable_extensions = post_data['additional_viewable_extensions']
-        redirect(request.referer)
+        redirect(six.ensure_text(request.referer or '/'))
 
     @without_trailing_slash
     @expose('jinja:allura:templates/repo/default_branch.html')
     def set_default_branch_name(self, branch_name=None, **kw):
         if (request.method == 'POST') and branch_name:
             self.repo.set_default_branch(branch_name)
-            redirect(request.referer or '/')
+            redirect(six.ensure_text(request.referer or '/'))
         else:
             return dict(app=self.app,
                         default_branch_name=self.app.default_branch_name)
@@ -312,7 +314,7 @@ class RepoAdminController(DefaultAdminController):
             flash(message,
                   'error' if 'Invalid' in message else 'ok')
 
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
 
 class RepoAdminRestController(BaseController):
diff --git a/Allura/allura/lib/utils.py b/Allura/allura/lib/utils.py
index a16fa91..a6f782e 100644
--- a/Allura/allura/lib/utils.py
+++ b/Allura/allura/lib/utils.py
@@ -383,7 +383,7 @@ class AntiSpam(object):
                 else:
                     # regular antispam failure handling
                     tg.flash(error_msg, 'error')
-                    redirect(error_url or tg.request.referer or '.')
+                    redirect(error_url or six.ensure_text(tg.request.referer or '.'))
         return before_validate(antispam_hook)
 
 
diff --git a/Allura/allura/tests/functional/test_discuss.py b/Allura/allura/tests/functional/test_discuss.py
index 2849946..80b8633 100644
--- a/Allura/allura/tests/functional/test_discuss.py
+++ b/Allura/allura/tests/functional/test_discuss.py
@@ -84,7 +84,7 @@ class TestDiscuss(TestDiscussBase):
                 params[field['name']] = field.get('value') or ''
         params[f.find('textarea')['name']] = text
         r = self.app.post(f['action'], params=params,
-                          headers={str('Referer'): str(thread_link.encode("utf-8"))},
+                          headers={str('Referer'): str(thread_link)},
                           extra_environ=dict(username=str('root')))
         r = r.follow()
         return r
diff --git a/Allura/allura/tests/unit/patches.py b/Allura/allura/tests/unit/patches.py
index f8818be..02897dc 100644
--- a/Allura/allura/tests/unit/patches.py
+++ b/Allura/allura/tests/unit/patches.py
@@ -17,7 +17,7 @@
 
 from __future__ import unicode_literals
 from __future__ import absolute_import
-from mock import Mock, patch
+from mock import Mock, patch, MagicMock
 from tg import tmpl_context as c
 
 from allura.tests.unit.factories import (
@@ -60,4 +60,7 @@ def fake_redirect_patch(test_case):
 
 
 def fake_request_patch(test_case):
-    return patch('allura.controllers.discuss.request')
+    return patch('allura.controllers.discuss.request',
+                 MagicMock(
+                     referer='.'
+                 ))
diff --git a/ForgeBlog/forgeblog/main.py b/ForgeBlog/forgeblog/main.py
index 52a01b7..c27ed0b 100644
--- a/ForgeBlog/forgeblog/main.py
+++ b/ForgeBlog/forgeblog/main.py
@@ -526,7 +526,7 @@ class BlogAdminController(DefaultAdminController):
         self.app.config.options[
             'AllowEmailPosting'] = allow_email_posting and True or False
         flash('Blog options updated')
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @without_trailing_slash
     @expose('jinja:forgeblog:templates/blog/admin_exfeed.html')
@@ -568,7 +568,7 @@ class BlogAdminController(DefaultAdminController):
             flash('Invalid link(s): %s' %
                   ','.join(link for link in invalid_list), 'error')
 
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
 
 class RootRestController(BaseController, AppRestControllerMixin):
diff --git a/ForgeChat/forgechat/main.py b/ForgeChat/forgechat/main.py
index e3b4765..92dc696 100644
--- a/ForgeChat/forgechat/main.py
+++ b/ForgeChat/forgechat/main.py
@@ -24,6 +24,7 @@ import logging
 from datetime import date, time, datetime, timedelta
 
 # Non-stdlib imports
+import six
 from tg import expose, validate, redirect, flash
 from tg.decorators import with_trailing_slash
 from tg import tmpl_context as c, request
@@ -127,7 +128,7 @@ class AdminController(DefaultAdminController):
 
     @with_trailing_slash
     def index(self, **kw):
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @expose()
     @require_post()
diff --git a/ForgeDiscussion/forgediscussion/controllers/root.py b/ForgeDiscussion/forgediscussion/controllers/root.py
index 0356f30..b636f7c 100644
--- a/ForgeDiscussion/forgediscussion/controllers/root.py
+++ b/ForgeDiscussion/forgediscussion/controllers/root.py
@@ -19,6 +19,8 @@ from __future__ import unicode_literals
 from __future__ import absolute_import
 import json
 import logging
+
+import six
 from six.moves.urllib.parse import unquote
 from datetime import date, datetime, timedelta, time
 import calendar
@@ -133,13 +135,13 @@ class RootController(BaseController, DispatchIndex, FeedController):
     @validate(W.new_topic, error_handler=create_topic)
     @AntiSpam.validate('Spambot protection engaged', error_url='create_topic')
     def save_new_topic(self, subject=None, text=None, forum=None, subscribe=False, **kw):
-        self.rate_limit(model.ForumPost, 'Topic creation', request.referer)
+        self.rate_limit(model.ForumPost, 'Topic creation', six.ensure_text(request.referer or '/'))
         discussion = model.Forum.query.get(
             app_config_id=c.app.config._id,
             shortname=forum)
         if discussion.deleted and not has_access(c.app, 'configure')():
             flash('This forum has been removed.')
-            redirect(request.referer or '/')
+            redirect(six.ensure_text(request.referer or '/'))
         require_access(discussion, 'post')
         thd = discussion.get_discussion_thread(dict(
             headers=dict(Subject=subject)))[0]
diff --git a/ForgeDiscussion/forgediscussion/forum_main.py b/ForgeDiscussion/forgediscussion/forum_main.py
index e481c23..5c6b19d 100644
--- a/ForgeDiscussion/forgediscussion/forum_main.py
+++ b/ForgeDiscussion/forgediscussion/forum_main.py
@@ -331,7 +331,7 @@ class ForumAdminController(DefaultAdminController):
                 else:
                     forum.acl = []
         flash('Forums updated')
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @h.vardec
     @expose()
diff --git a/ForgeSVN/forgesvn/svn_main.py b/ForgeSVN/forgesvn/svn_main.py
index 5fafbf7..315d910 100644
--- a/ForgeSVN/forgesvn/svn_main.py
+++ b/ForgeSVN/forgesvn/svn_main.py
@@ -19,6 +19,8 @@
 from __future__ import unicode_literals
 from __future__ import absolute_import
 import logging
+
+import six
 from tg import tmpl_context as c, request
 
 # Non-stdlib imports
@@ -184,7 +186,7 @@ class SVNImportController(BaseController, AdminControllerMixin):
             M.Notification.post_user(
                 c.user, self.app.repo, 'error',
                 text="Can't import into non empty repository.")
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
 
 class SVNCommitBrowserController(BaseController):
diff --git a/ForgeShortUrl/forgeshorturl/main.py b/ForgeShortUrl/forgeshorturl/main.py
index a471983..f03b9b6 100644
--- a/ForgeShortUrl/forgeshorturl/main.py
+++ b/ForgeShortUrl/forgeshorturl/main.py
@@ -17,6 +17,8 @@
 
 from __future__ import unicode_literals
 from __future__ import absolute_import
+
+import six
 from tg import expose, validate, redirect, flash, request
 from tg.decorators import without_trailing_slash
 
@@ -203,7 +205,7 @@ class ShortURLAdminController(DefaultAdminController):
 
     @expose()
     def index(self, **kw):
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @without_trailing_slash
     @expose('json:')
@@ -231,7 +233,7 @@ class ShortURLAdminController(DefaultAdminController):
                     names = {'short_url': 'Short url', 'full_url': 'Full URL'}
                     error_msg += '%s: %s ' % (names[msg], c.form_errors[msg])
                     flash(error_msg, 'error')
-                redirect(request.referer or '/')
+                redirect(six.ensure_text(request.referer or '/'))
 
             shorturl = ShortUrl.query.find({
                 'app_config_id': self.app.config._id,
@@ -240,7 +242,7 @@ class ShortURLAdminController(DefaultAdminController):
             if shorturl is not None:
                 if not update:
                     flash('Short url %s already exists' % short_url, 'error')
-                    redirect(request.referer or '/')
+                    redirect(six.ensure_text(request.referer or '/'))
                 else:
                     msg = ('update short url %s from %s to %s'
                            % (short_url, shorturl.full_url, full_url))
@@ -261,7 +263,7 @@ class ShortURLAdminController(DefaultAdminController):
             shorturl.last_updated = datetime.utcnow()
 
             M.AuditLog.log(msg)
-            redirect(request.referer or '/')
+            redirect(six.ensure_text(request.referer or '/'))
         return dict(
             app=self.app,
             url_len=len(ShortUrl.build_short_url(c.app, short_name='')))
diff --git a/ForgeTracker/forgetracker/tracker_main.py b/ForgeTracker/forgetracker/tracker_main.py
index 28c7727..2aa2b0e 100644
--- a/ForgeTracker/forgetracker/tracker_main.py
+++ b/ForgeTracker/forgetracker/tracker_main.py
@@ -1605,16 +1605,16 @@ class TicketController(BaseController, FeedController):
             tracker = M.AppConfig.query.get(_id=t_id)
             if tracker is None:
                 flash('Select valid tracker', 'error')
-                redirect(request.referer or '/')
+                redirect(six.ensure_text(request.referer or '/'))
 
             if tracker == self.ticket.app.config:
                 flash('Ticket already in a selected tracker', 'info')
-                redirect(request.referer or '/')
+                redirect(six.ensure_text(request.referer or '/'))
 
             if not has_access(tracker, 'admin')():
                 flash('You should have admin access to destination tracker',
                       'error')
-                redirect(request.referer or '/')
+                redirect(six.ensure_text(request.referer or '/'))
 
             new_ticket = self.ticket.move(tracker)
             c.app.globals.invalidate_bin_counts()
@@ -1702,7 +1702,7 @@ class TrackerAdminController(DefaultAdminController):
         for k, val in six.iteritems(kw):
             self.app.config.options[k] = val
         flash('Options updated')
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @expose()
     @require_post()
@@ -1712,7 +1712,7 @@ class TrackerAdminController(DefaultAdminController):
                 self.app.globals['show_in_search'][column] = True
             else:
                 self.app.globals['show_in_search'][column] = False
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @expose()
     def update_tickets(self, **post_data):
@@ -1805,7 +1805,7 @@ class TrackerAdminController(DefaultAdminController):
 
         self.app.globals.custom_fields = custom_fields
         flash('Fields updated')
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
 
 class RootRestController(BaseController, AppRestControllerMixin):
diff --git a/ForgeWiki/forgewiki/wiki_main.py b/ForgeWiki/forgewiki/wiki_main.py
index d635222..c30eb8e 100644
--- a/ForgeWiki/forgewiki/wiki_main.py
+++ b/ForgeWiki/forgewiki/wiki_main.py
@@ -22,6 +22,8 @@ import json
 import logging
 import os
 from pprint import pformat
+
+import six
 from six.moves.urllib.parse import unquote
 
 # Non-stdlib imports
@@ -521,7 +523,7 @@ class RootController(BaseController, DispatchIndex, FeedController):
             M.Mailbox.subscribe(type='direct')
         elif unsubscribe:
             M.Mailbox.unsubscribe()
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
 
 class PageController(BaseController, FeedController):
@@ -793,7 +795,7 @@ class PageController(BaseController, FeedController):
         self.page.add_multiple_attachments(file_info)
         if is_ajax(request):
             return
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))
 
     @expose('json:')
     @require_post()
@@ -940,4 +942,4 @@ class WikiAdminController(DefaultAdminController):
         self.app.show_right_bar = show_right_bar
         self.app.allow_email_posting = allow_email_posting
         flash('Wiki options updated')
-        redirect(request.referer or '/')
+        redirect(six.ensure_text(request.referer or '/'))


[allura] 03/12: [#8361] test_admin fixes; MaxBytesValidator should not convert

Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

kentontaylor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/allura.git

commit 34e0aa07200307806fa230668ba767411f2482e1
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Mon May 11 18:11:37 2020 -0400

    [#8361] test_admin fixes; MaxBytesValidator should not convert
---
 Allura/allura/lib/validators.py              |  4 ++--
 Allura/allura/tests/decorators.py            | 14 +++++++++----
 Allura/allura/tests/functional/test_admin.py | 31 ++++++++++++++--------------
 Allura/allura/tests/functional/test_auth.py  |  4 ++--
 4 files changed, 30 insertions(+), 23 deletions(-)

diff --git a/Allura/allura/lib/validators.py b/Allura/allura/lib/validators.py
index cd29024..f84ec5a 100644
--- a/Allura/allura/lib/validators.py
+++ b/Allura/allura/lib/validators.py
@@ -117,8 +117,8 @@ class MaxBytesValidator(fev.FancyValidator):
     max = 255
 
     def _to_python(self, value, state):
-        value = h.really_unicode(value or '').encode('utf-8')
-        if len(value) > self.max:
+        value_bytes = h.really_unicode(value or '').encode('utf-8')
+        if len(value_bytes) > self.max:
             raise fe.Invalid("Please enter a value less than %s bytes long." %
                              self.max, value, state)
         return value
diff --git a/Allura/allura/tests/decorators.py b/Allura/allura/tests/decorators.py
index 647674f..0a48ae7 100644
--- a/Allura/allura/tests/decorators.py
+++ b/Allura/allura/tests/decorators.py
@@ -173,11 +173,17 @@ def audits(*messages, **kwargs):
         preamble = '(Done by user: {}\n)?IP Address: {}\nUser-Agent: {}\n'.format(actor, ip_addr, user_agent)
     else:
         preamble = ''
+
     for message in messages:
-        assert M.AuditLog.query.find(dict(message=re.compile(preamble + message))).count(), \
-            'Could not find "%s"%s' % (message,
-                                       '\nYou may need to escape the regex chars in the text you are matching'
-                                       if message != re.escape(message) else '')
+        found = M.AuditLog.query.find(dict(message=re.compile(preamble + message))).count()
+        if not found:
+            hints = ''
+            all = M.AuditLog.query.find().all()
+            if len(all) < 10:
+                hints += '\nin these AuditLog messages:\n\t' + '\n\t'.join(a.message for a in all)
+            if message != re.escape(message):
+                hints += '\nYou may need to escape the regex chars in the text you are matching'
+            raise AssertionError('Could not find "%s"%s' % (message, hints))
 
 
 @contextlib.contextmanager
diff --git a/Allura/allura/tests/functional/test_admin.py b/Allura/allura/tests/functional/test_admin.py
index 15f862e..b02294b 100644
--- a/Allura/allura/tests/functional/test_admin.py
+++ b/Allura/allura/tests/functional/test_admin.py
@@ -20,8 +20,6 @@ from __future__ import unicode_literals
 from __future__ import absolute_import
 import os
 from datetime import datetime
-
-import allura
 import pkg_resources
 from io import BytesIO
 import logging
@@ -34,7 +32,9 @@ from ming.orm.ormsession import ThreadLocalORMSession
 from tg import expose
 from tg import tmpl_context as c, app_globals as g
 import mock
+import six
 
+import allura
 from allura.tests import TestController
 from allura.tests import decorators as td
 from allura.tests.decorators import audits, out_audits
@@ -59,12 +59,13 @@ class TestProjectAdmin(TestController):
                 'change summary to Milkshakes are for crazy monkeys',
                 'change project name to My Test Project',
                 'change short description to (\u00bf A Test Project \?){45}'):
-            self.app.post('/admin/update', status=302, params=dict(
+            r = self.app.post('/admin/update', params=dict(
                 name='My Test Project',
                 shortname='test',
                 summary='Milkshakes are for crazy monkeys',
                 short_description=('\u00bf A Test Project ?' * 45).encode('utf-8'),
                 labels='aaa,bbb'))
+            assert r.status_int == 302, (r.status, r.html.select('.error,.fielderror'))
         r = self.app.get('/admin/overview')
         assert b'A Test Project ?\xc2\xbf A' in r.body
         assert 'Test Subproject' not in r
@@ -168,7 +169,7 @@ class TestProjectAdmin(TestController):
     def test_features(self):
         proj = M.Project.query.get(shortname='test')
         assert_equals(proj.features, [])
-        with audits("change project features to \[u'One', u'Two'\]"):
+        with audits("change project features to \[{u}'One', {u}'Two'\]".format(u='u' if six.PY2 else '')):
             resp = self.app.post('/admin/update', params={
                 'features-0.feature': 'One',
                 'features-1.feature': '  ',
@@ -838,10 +839,10 @@ class TestProjectAdmin(TestController):
         dev_holder = r.html.find('table', {'id': 'usergroup_admin'}).findAll('tr')[2]
         mem_holder = r.html.find('table', {'id': 'usergroup_admin'}).findAll('tr')[3]
         mem_id = mem_holder['data-group']
-        # neither group has update permission
-        assert dev_holder.findAll('ul')[1].findAll('li')[2]['class'] == ["no"]
-        assert mem_holder.findAll('ul')[1].findAll('li')[2]['class'] == ["no"]
-        # add update permission to Member
+        # neither group has create permission
+        assert dev_holder.select_one('li[data-permission=create]')['class'] == ["no"]
+        assert mem_holder.select_one('li[data-permission=create]')['class'] == ["no"]
+        # add create permission to Member
         r = self.app.post('/admin/groups/change_perm', params={
             'role_id': mem_id,
             'permission': 'create',
@@ -849,10 +850,10 @@ class TestProjectAdmin(TestController):
         r = self.app.get('/admin/groups/')
         dev_holder = r.html.find('table', {'id': 'usergroup_admin'}).findAll('tr')[2]
         mem_holder = r.html.find('table', {'id': 'usergroup_admin'}).findAll('tr')[3]
-        # Member now has update permission
-        assert mem_holder.findAll('ul')[1].findAll('li')[2]['class'] == ["yes"]
-        # Developer has inherited update permission from Member
-        assert dev_holder.findAll('ul')[1].findAll('li')[2]['class'] == ["inherit"]
+        # Member now has create permission
+        assert mem_holder.select_one('li[data-permission=create]')['class'] == ["yes"]
+        # Developer has inherited create permission from Member
+        assert dev_holder.select_one('li[data-permission=create]')['class'] == ["inherit"]
         # remove update permission from Member
         r = self.app.post('/admin/groups/change_perm', params={
             'role_id': mem_id,
@@ -861,9 +862,9 @@ class TestProjectAdmin(TestController):
         r = self.app.get('/admin/groups/')
         dev_holder = r.html.find('table', {'id': 'usergroup_admin'}).findAll('tr')[2]
         mem_holder = r.html.find('table', {'id': 'usergroup_admin'}).findAll('tr')[3]
-        # neither group has update permission
-        assert dev_holder.findAll('ul')[1].findAll('li')[2]['class'] == ["no"]
-        assert mem_holder.findAll('ul')[1].findAll('li')[2]['class'] == ["no"]
+        # neither group has create permission
+        assert dev_holder.select_one('li[data-permission=create]')['class'] == ["no"]
+        assert mem_holder.select_one('li[data-permission=create]')['class'] == ["no"]
 
     def test_permission_inherit(self):
         r = self.app.get('/admin/groups/')
diff --git a/Allura/allura/tests/functional/test_auth.py b/Allura/allura/tests/functional/test_auth.py
index c85ca4a..c7dd677 100644
--- a/Allura/allura/tests/functional/test_auth.py
+++ b/Allura/allura/tests/functional/test_auth.py
@@ -882,9 +882,9 @@ class TestAuth(TestController):
         field_name = None
         for k, v in six.iteritems(form.fields):
             if subscribed:
-                check = c and v[0].value == 'on'
+                check = v and v[0].value == 'on'
             else:
-                check = c and v[0].value != 'on'
+                check = v and v[0].value != 'on'
             if k and k.endswith('.subscribed') and check:
                 field_name = k.replace('.subscribed', '')
         assert field_name, "Can't find unsubscribed tool for user"


[allura] 11/12: [#8361] Bump Easywidgets version

Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

kentontaylor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/allura.git

commit 66462e7f1fdab5101f8f1bc73080c4866aab26d5
Author: Kenton Taylor <kt...@slashdotmedia.com>
AuthorDate: Tue Jun 2 20:05:25 2020 +0000

    [#8361] Bump Easywidgets version
---
 requirements.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/requirements.in b/requirements.in
index b80e7b9..10f85c7 100644
--- a/requirements.in
+++ b/requirements.in
@@ -6,7 +6,7 @@ chardet
 colander==1.7.0
 cryptography==2.6.1
 decorator
-EasyWidgets>=0.3.3
+EasyWidgets>=0.3.6
 emoji
 faulthandler ; python_version < "3.3"
 feedgenerator


[allura] 01/12: [#8361] templates tests pass

Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

kentontaylor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/allura.git

commit 03e2aac98d93de4c0911f72a09544a60f16219e1
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Fri May 8 17:10:14 2020 -0400

    [#8361] templates tests pass
---
 Allura/allura/controllers/auth.py                      |  2 ++
 Allura/allura/tests/templates/jinja_master/test_lib.py |  5 +++--
 AlluraTest/alluratest/controller.py                    | 12 ------------
 3 files changed, 5 insertions(+), 14 deletions(-)

diff --git a/Allura/allura/controllers/auth.py b/Allura/allura/controllers/auth.py
index 9573993..5c457b6 100644
--- a/Allura/allura/controllers/auth.py
+++ b/Allura/allura/controllers/auth.py
@@ -113,6 +113,8 @@ class AuthController(BaseController):
             self.disable = DisableAccountController()
 
     def __getattr__(self, name):
+        if name == 'decoration':
+            raise AttributeError("TG decoratedcontroller calls this during import time, can't do anything complex")
         urls = plugin.UserPreferencesProvider.get().additional_urls()
         if name not in urls:
             raise AttributeError("'%s' object has no attribute '%s'" % (type(self).__name__, name))
diff --git a/Allura/allura/tests/templates/jinja_master/test_lib.py b/Allura/allura/tests/templates/jinja_master/test_lib.py
index 8b27438..67c0571 100644
--- a/Allura/allura/tests/templates/jinja_master/test_lib.py
+++ b/Allura/allura/tests/templates/jinja_master/test_lib.py
@@ -21,8 +21,9 @@ from tg import config, app_globals as g
 from mock import Mock
 from nose.tools import assert_equal
 
+import ming
 from allura.config.app_cfg import ForgeConfig, AlluraJinjaRenderer
-from alluratest.controller import setup_config_test
+from alluratest.controller import setup_basic_test
 
 
 def strip_space(s):
@@ -31,7 +32,7 @@ def strip_space(s):
 
 class TemplateTest(object):
     def setUp(self):
-        setup_config_test()
+        setup_basic_test()
         self.jinja2_env = AlluraJinjaRenderer.create(config, g)['jinja'].jinja2_env
 
 
diff --git a/AlluraTest/alluratest/controller.py b/AlluraTest/alluratest/controller.py
index b5e53de..2da93da 100644
--- a/AlluraTest/alluratest/controller.py
+++ b/AlluraTest/alluratest/controller.py
@@ -76,18 +76,6 @@ def get_config_file(config=None, current_pkg=None):
         return conf_file
 
 
-def setup_config_test(config_file=None, force=False):
-    '''
-    This may be necessary to use within test setup that needs the app config loaded
-    '''
-    if not config_file:
-        config_file = get_config_file()
-    already_loaded = tg.config.get('tg.app_globals')
-    if not already_loaded or force:
-        loadapp('config:' + config_file)
-setup_config_test.__test__ = False
-
-
 def setup_basic_test(config=None, app_name=DFL_APP_NAME):
     '''
     Create clean environment for running tests, includes mongodb connection with "mim" (mongo-in-memory) and sample


[allura] 10/12: [#8361] work on test; will need Easywidgets version bump

Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

kentontaylor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/allura.git

commit 3bad27b235c68f3268455d16d1ca664e7b3b9d91
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Mon May 18 17:27:29 2020 -0400

    [#8361] work on test; will need Easywidgets version bump
---
 Allura/allura/tests/functional/test_discuss.py | 3 ++-
 requirements.in                                | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/Allura/allura/tests/functional/test_discuss.py b/Allura/allura/tests/functional/test_discuss.py
index 8518b00..de143c4 100644
--- a/Allura/allura/tests/functional/test_discuss.py
+++ b/Allura/allura/tests/functional/test_discuss.py
@@ -292,7 +292,8 @@ class TestDiscuss(TestDiscussBase):
         r = self._make_post('Test post')
         post_link = str(
             r.html.find('div', {'class': 'edit_post_form reply'}).find('form')['action'])
-        self.app.post(post_link + 'moderate', params=dict(spam='spam'))
+        r = self.app.post(post_link + 'moderate', params=dict(spam='spam'))
+        assert_equal(r.json, {"result": "success"})
         post = M.Post.query.find().first()
         post_username = post.author().username
         moderate_link = '/p/test/wiki/_discuss/moderate'
diff --git a/requirements.in b/requirements.in
index cc30883..b80e7b9 100644
--- a/requirements.in
+++ b/requirements.in
@@ -42,7 +42,7 @@ requests
 requests-oauthlib
 # for taskd proc name switching
 setproctitle==1.1.9
-six==1.12.0
+six
 TimerMiddleware==0.5.1
 TurboGears2==2.3.12
 WebHelpers2


[allura] 04/12: [#8361] use our own UnicodeString validator to work around problem, make sure everywhere uses it

Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

kentontaylor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/allura.git

commit c0fa06111c27479566e6338bedae299f488ffa9b
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Mon May 11 18:11:59 2020 -0400

    [#8361] use our own UnicodeString validator to work around problem, make sure everywhere uses it
---
 Allura/allura/controllers/site_admin.py            |  8 ++--
 Allura/allura/ext/admin/widgets.py                 | 13 +++---
 Allura/allura/lib/validators.py                    | 22 +++++++--
 Allura/allura/lib/widgets/discuss.py               |  9 ++--
 Allura/allura/lib/widgets/form_fields.py           | 15 +++---
 Allura/allura/lib/widgets/forms.py                 | 54 +++++++++++-----------
 Allura/allura/lib/widgets/user_profile.py          |  7 +--
 Allura/allura/webhooks.py                          |  7 +--
 ForgeBlog/forgeblog/main.py                        |  7 +--
 ForgeBlog/forgeblog/widgets.py                     |  3 +-
 ForgeChat/forgechat/main.py                        |  3 +-
 .../forgediscussion/controllers/root.py            |  3 +-
 ForgeDiscussion/forgediscussion/widgets/admin.py   |  6 +--
 ForgeImporters/forgeimporters/base.py              |  4 +-
 ForgeImporters/forgeimporters/github/code.py       |  4 +-
 ForgeImporters/forgeimporters/github/tracker.py    |  4 +-
 ForgeImporters/forgeimporters/github/wiki.py       |  7 ++-
 ForgeShortUrl/forgeshorturl/main.py                |  3 +-
 ForgeTracker/forgetracker/tracker_main.py          | 19 ++++----
 ForgeTracker/forgetracker/widgets/admin.py         |  5 +-
 ForgeTracker/forgetracker/widgets/bin_form.py      |  9 ++--
 ForgeTracker/forgetracker/widgets/ticket_form.py   |  2 +-
 ForgeWiki/forgewiki/wiki_main.py                   |  7 +--
 requirements.in                                    |  2 +-
 24 files changed, 124 insertions(+), 99 deletions(-)

diff --git a/Allura/allura/controllers/site_admin.py b/Allura/allura/controllers/site_admin.py
index 644c9c9..4fa7ffc 100644
--- a/Allura/allura/controllers/site_admin.py
+++ b/Allura/allura/controllers/site_admin.py
@@ -312,7 +312,7 @@ class SiteAdminController(object):
 
     @without_trailing_slash
     @expose('jinja:allura:templates/site_admin_search.html')
-    @validate(validators=dict(q=validators.UnicodeString(if_empty=None),
+    @validate(validators=dict(q=v.UnicodeString(if_empty=None),
                               limit=validators.Int(if_invalid=None),
                               page=validators.Int(if_empty=0, if_invalid=0)))
     def search_projects(self, q=None, f=None, page=0, limit=None, **kw):
@@ -327,7 +327,7 @@ class SiteAdminController(object):
 
     @without_trailing_slash
     @expose('jinja:allura:templates/site_admin_search.html')
-    @validate(validators=dict(q=validators.UnicodeString(if_empty=None),
+    @validate(validators=dict(q=v.UnicodeString(if_empty=None),
                               limit=validators.Int(if_invalid=None),
                               page=validators.Int(if_empty=0, if_invalid=0)))
     def search_users(self, q=None, f=None, page=0, limit=None, **kw):
@@ -344,8 +344,8 @@ class SiteAdminController(object):
 
 class DeleteProjectsController(object):
     delete_form_validators = dict(
-        projects=validators.UnicodeString(if_empty=None),
-        reason=validators.UnicodeString(if_empty=None),
+        projects=v.UnicodeString(if_empty=None),
+        reason=v.UnicodeString(if_empty=None),
         disable_users=validators.StringBool(if_empty=False))
 
     def remove_comments(self, lines):
diff --git a/Allura/allura/ext/admin/widgets.py b/Allura/allura/ext/admin/widgets.py
index 82b3962..0eedae6 100644
--- a/Allura/allura/ext/admin/widgets.py
+++ b/Allura/allura/ext/admin/widgets.py
@@ -159,7 +159,6 @@ class FeaturesField(ew.CompoundField):
         yield ew.JSLink('allura/js/jquery-ui.min.js', location='body_top_js')
         yield ew.CSSLink('allura/css/smoothness/jquery-ui.min.css', compress=False)  # compress will also serve from a different location, breaking image refs
 
-
 class MetadataAdmin(ff.AdminForm):
     template = 'jinja:allura.ext.admin:templates/admin_widgets/metadata_admin.html'
     defaults = dict(
@@ -170,15 +169,16 @@ class MetadataAdmin(ff.AdminForm):
         name = ew.InputField(field_type='text',
                              label='Name',
                              validator=formencode.All(
-                                 fev.UnicodeString(not_empty=True, max=40),
-                                 V.MaxBytesValidator(max=40)),
+                                 V.UnicodeString(not_empty=True, max=40),
+                                 V.MaxBytesValidator(max=40),
+                             ),
                              attrs={'maxlength': 40,
                                     'title': "This is the publicly viewable name of the project, and will appear on project listings. It should be what you want to see as the project title in search listing.",
                                     'class': 'tooltip',
                                     })
         summary = ew.InputField(field_type="text", label='Short Summary',
                                 validator=formencode.All(
-                                    fev.UnicodeString(max=70),
+                                    V.UnicodeString(max=70),
                                     V.MaxBytesValidator(max=70)),
                                 attrs={'maxlength': 70,
                                        'title': 'Briefly state what your project is and what it does without repeating the project name. This summary appears in Google search results beneath the project name.',
@@ -186,8 +186,9 @@ class MetadataAdmin(ff.AdminForm):
                                        })
         short_description = ew.TextArea(label='Full Description',
                                         validator=formencode.All(
-                                            fev.UnicodeString(max=1000),
-                                            V.MaxBytesValidator(max=1000)),
+                                            V.UnicodeString(max=1000),
+                                            V.MaxBytesValidator(max=1000),
+                                            ),
                                         attrs={
                                             'title': 'Describe the full functionality of your project using related keywords. The first sentence has the most impact on search. Provide unique content that calls out keywords and describes the merits of your project.',
                                             'class': 'tooltip'
diff --git a/Allura/allura/lib/validators.py b/Allura/allura/lib/validators.py
index f84ec5a..2dee7fc 100644
--- a/Allura/allura/lib/validators.py
+++ b/Allura/allura/lib/validators.py
@@ -68,6 +68,22 @@ class NonHttpUrl(URL):
     ''', re.I | re.VERBOSE)
 
 
+class UnicodeString(fev.UnicodeString):
+    """
+    Override UnicodeString to fix bytes handling.
+    Specifically ran into problems with its 'tooLong' check is running on bytes incorrectly and getting wrong length
+
+    Fixed elsewhere like this too:
+        https://github.com/formencode/formencode/issues/2#issuecomment-378410047
+        https://github.com/sightmachine/formencode/commit/665c9d0141dacb2dc84fb7d20ad3f8c0a5fe5e2d
+    """
+    encoding = None
+
+
+# make UnicodeString fix above work through this String alias, just like formencode aliases String
+String = UnicodeString if str is str else fev.ByteString
+
+
 class Ming(fev.FancyValidator):
 
     def __init__(self, cls, **kw):
@@ -90,7 +106,7 @@ class Ming(fev.FancyValidator):
             return value._id
 
 
-class UniqueOAuthApplicationName(fev.UnicodeString):
+class UniqueOAuthApplicationName(UnicodeString):
 
     def _to_python(self, value, state):
         from allura import model as M
@@ -127,7 +143,7 @@ class MaxBytesValidator(fev.FancyValidator):
         return h.really_unicode(value or '')
 
 
-class MountPointValidator(fev.UnicodeString):
+class MountPointValidator(UnicodeString):
 
     def __init__(self, app_class,
                  reserved_mount_points=('feed', 'index', 'icon', '_nav.json'), **kw):
@@ -320,7 +336,7 @@ class CreateTaskSchema(fe.Schema):
 class CreateSiteNotificationSchema(fe.Schema):
     active = fev.StringBool(if_missing=False)
     impressions = fev.Int(not_empty=True)
-    content = fev.UnicodeString(not_empty=True)
+    content = UnicodeString(not_empty=True)
     user_role = fev.FancyValidator(not_empty=False, if_empty=None)
     page_regex = fev.FancyValidator(not_empty=False, if_empty=None)
     page_tool_type = fev.FancyValidator(not_empty=False, if_empty=None)
diff --git a/Allura/allura/lib/widgets/discuss.py b/Allura/allura/lib/widgets/discuss.py
index 25919cd..9338fbb 100644
--- a/Allura/allura/lib/widgets/discuss.py
+++ b/Allura/allura/lib/widgets/discuss.py
@@ -19,14 +19,11 @@ from __future__ import unicode_literals
 from __future__ import absolute_import
 from formencode import validators as fev
 
-import json
 import ew as ew_core
 import ew.jinja2_ew as ew
-from tg import app_globals as g
-from  tg import config
 
 from allura.lib import utils
-from allura.lib import validators as V
+from allura.lib import validators as v
 from allura.lib.widgets import form_fields as ffw
 from allura.lib.widgets import forms as ff
 from allura import model as M
@@ -169,7 +166,7 @@ class EditPost(ff.ForgeForm):
                     ew.TextField(name='subject', attrs=dict(style="width:97%")))
         else:
             # We are being validated
-            validator = fev.UnicodeString(not_empty=True, if_missing='')
+            validator = v.UnicodeString(not_empty=True, if_missing='')
             fields.append(ew.TextField(name='subject', validator=validator))
             fields.append(NullValidator(name=self.att_name))
         return fields
@@ -217,7 +214,7 @@ class _ThreadsTable(ew.TableField):
     template = 'jinja:allura:templates/widgets/threads_table.html'
 
     class fields(ew_core.NameList):
-        _id = ew.HiddenField(validator=V.Ming(M.Thread))
+        _id = ew.HiddenField(validator=v.Ming(M.Thread))
         subscription = ew.Checkbox(suppress_label=True)
         subject = ffw.DisplayOnlyField(label='Topic')
         url = ffw.DisplayOnlyField()
diff --git a/Allura/allura/lib/widgets/form_fields.py b/Allura/allura/lib/widgets/form_fields.py
index e02d6b8..5ef2ed0 100644
--- a/Allura/allura/lib/widgets/form_fields.py
+++ b/Allura/allura/lib/widgets/form_fields.py
@@ -30,6 +30,9 @@ import ew.jinja2_ew as ew
 import six
 from six.moves import range
 
+from allura.lib import validators as v
+
+
 log = logging.getLogger(__name__)
 
 
@@ -37,7 +40,7 @@ def onready(text):
     return ew.JSScript('$(function () {%s});' % text)
 
 
-class LabelList(fev.UnicodeString):
+class LabelList(v.UnicodeString):
 
     def __init__(self, *args, **kwargs):
         kwargs.setdefault('if_empty', [])
@@ -268,7 +271,7 @@ class AutoResizeTextarea(ew.TextArea):
 
 class MarkdownEdit(ew.TextArea):
     template = 'jinja:allura:templates/widgets/markdown_edit.html'
-    validator = fev.UnicodeString()
+    validator = v.UnicodeString()
     defaults = dict(
         ew.TextArea.defaults,
         name=None,
@@ -332,9 +335,9 @@ class PageList(ew_core.Widget):
     @property
     def url_params(self, **kw):
         url_params = dict()
-        for k, v in six.iteritems(request.params):
+        for k, val in six.iteritems(request.params):
             if k not in ['limit', 'count', 'page']:
-                url_params[k] = v
+                url_params[k] = val
         return url_params
 
 
@@ -350,9 +353,9 @@ class PageSize(ew_core.Widget):
     @property
     def url_params(self, **kw):
         url_params = dict()
-        for k, v in six.iteritems(request.params):
+        for k, val in six.iteritems(request.params):
             if k not in ['limit', 'count', 'page']:
-                url_params[k] = v
+                url_params[k] = val
         return url_params
 
     def resources(self):
diff --git a/Allura/allura/lib/widgets/forms.py b/Allura/allura/lib/widgets/forms.py
index ab0ef32..54082ce 100644
--- a/Allura/allura/lib/widgets/forms.py
+++ b/Allura/allura/lib/widgets/forms.py
@@ -151,14 +151,14 @@ class PasswordChangeBase(ForgeForm):
                     required=True,
                     autocomplete='new-password',
                 ),
-                validator=fev.UnicodeString(
+                validator=V.UnicodeString(
                     not_empty=True,
                     min=asint(tg.config.get('auth.min_password_len', 6)),
                     max=asint(tg.config.get('auth.max_password_len', 30)))),
             ew.PasswordField(
                 name='pw2',
                 label='New Password (again)',
-                validator=fev.UnicodeString(not_empty=True),
+                validator=V.UnicodeString(not_empty=True),
                 attrs=dict(
                     required=True,
                     autocomplete='new-password',
@@ -185,7 +185,7 @@ class PasswordChangeForm(PasswordChangeBase):
             ew.PasswordField(
                 name='oldpw',
                 label='Old Password',
-                validator=fev.UnicodeString(not_empty=True),
+                validator=V.UnicodeString(not_empty=True),
                 attrs=dict(
                     required=True,
                     autocomplete='current-password',
@@ -200,14 +200,14 @@ class PasswordChangeForm(PasswordChangeBase):
                     required=True,
                     autocomplete='new-password',
                 ),
-                validator=fev.UnicodeString(
+                validator=V.UnicodeString(
                     not_empty=True,
                     min=asint(tg.config.get('auth.min_password_len', 6)),
                     max=asint(tg.config.get('auth.max_password_len', 30)))),
             ew.PasswordField(
                 name='pw2',
                 label='New Password (again)',
-                validator=fev.UnicodeString(not_empty=True),
+                validator=V.UnicodeString(not_empty=True),
                 attrs=dict(
                     required=True,
                     autocomplete='new-password',
@@ -257,7 +257,7 @@ class PersonalDataForm(ForgeForm):
                          for v in ['Male', 'Female', 'Unknown', 'Other']],
                 validator=formencode.All(
                     V.OneOfValidator(['Male', 'Female', 'Unknown', 'Other']),
-                    fev.UnicodeString(not_empty=True))),
+                    V.UnicodeString(not_empty=True))),
             ew.SingleSelectField(
                 name='country',
                 label='Country of residence',
@@ -271,7 +271,7 @@ class PersonalDataForm(ForgeForm):
                 name='city',
                 label='City of residence',
                 attrs=dict(value=None),
-                validator=fev.UnicodeString(not_empty=False)),
+                validator=V.UnicodeString(not_empty=False)),
             ew.SingleSelectField(
                 name='timezone',
                 label='Timezone',
@@ -364,7 +364,7 @@ class AddTelNumberForm(ForgeForm):
         newnumber = ew.TextField(
             label='New telephone number',
             attrs={'value': ''},
-            validator=fev.UnicodeString(not_empty=True))
+            validator=V.UnicodeString(not_empty=True))
 
     def display(self, **kw):
         initial_value = kw.get('initial_value', '')
@@ -394,7 +394,7 @@ class SkypeAccountForm(ForgeForm):
         skypeaccount = ew.TextField(
             label='Skype account',
             attrs={'value': ''},
-            validator=fev.UnicodeString(not_empty=False))
+            validator=V.UnicodeString(not_empty=False))
 
     def display(self, **kw):
         initial_value = kw.get('initial_value', '')
@@ -450,13 +450,13 @@ class AddSocialNetworkForm(ForgeForm):
             ew.SingleSelectField(
                 name='socialnetwork',
                 label='Social network',
-                validator=fev.UnicodeString(not_empty=True),
+                validator=V.UnicodeString(not_empty=True),
                 options=[ew.Option(py_value=name, label=name)
                          for name in socialnetworks]),
             ew.TextField(
                 name='accounturl',
                 label='Account url',
-                validator=fev.UnicodeString(not_empty=True))
+                validator=V.UnicodeString(not_empty=True))
         ]
 
 
@@ -501,12 +501,12 @@ class AddInactivePeriodForm(ForgeForm):
             label='Start date',
             validator=formencode.All(
                 V.DateValidator(),
-                fev.UnicodeString(not_empty=True)))
+                V.UnicodeString(not_empty=True)))
         enddate = ew.TextField(
             label='End date',
             validator=formencode.All(
                 V.DateValidator(),
-                fev.UnicodeString(not_empty=True)))
+                V.UnicodeString(not_empty=True)))
 
     @ew_core.core.validator
     def to_python(self, kw, state):
@@ -560,17 +560,17 @@ class AddTimeSlotForm(ForgeForm):
                      for wd in weekdays],
             validator=formencode.All(
                 V.OneOfValidator(weekdays),
-                fev.UnicodeString(not_empty=True)))
+                V.UnicodeString(not_empty=True)))
         starttime = ew.TextField(
             label='Start time',
             validator=formencode.All(
                 V.TimeValidator(),
-                fev.UnicodeString(not_empty=True)))
+                V.UnicodeString(not_empty=True)))
         endtime = ew.TextField(
             label='End time',
             validator=formencode.All(
                 V.TimeValidator(),
-                fev.UnicodeString(not_empty=True)))
+                V.UnicodeString(not_empty=True)))
 
     @ew_core.core.validator
     def to_python(self, kw, state):
@@ -667,10 +667,10 @@ class AddTroveCategoryForm(ForgeForm):
         categoryname = ew.TextField(
             label="Category name",
             attrs={},
-            validator=fev.UnicodeString(not_empty=True))
+            validator=V.UnicodeString(not_empty=True))
         shortname = ew.TextField(
             label="Short name",
-            validator=fev.UnicodeString(),
+            validator=V.UnicodeString(),
             attrs={'placeholder': 'optional; unique identifier'})
 
     def display(self, **kw):
@@ -693,7 +693,7 @@ class AddUserSkillForm(ForgeForm):
         selected_skill = ew.HiddenField(
             attrs={'value': ''},
             show_errors=False,
-            validator=fev.UnicodeString(not_empty=True))
+            validator=V.UnicodeString(not_empty=True))
         level = ew.SingleSelectField(
             label="Level of knowledge",
             options=[
@@ -702,10 +702,10 @@ class AddUserSkillForm(ForgeForm):
                 ew.Option(py_value="high", label="Advanced level")],
             validator=formencode.All(
                 V.OneOfValidator(['low', 'medium', 'high']),
-                fev.UnicodeString(not_empty=True)))
+                V.UnicodeString(not_empty=True)))
         comment = ew.TextArea(
             label="Additional comments",
-            validator=fev.UnicodeString(not_empty=False),
+            validator=V.UnicodeString(not_empty=False),
             attrs={'rows': 5, 'cols': 30})
 
     def display(self, **kw):
@@ -732,7 +732,7 @@ class SelectSubCategoryForm(ForgeForm):
         ]
         self.fields['selected_category'].validator = formencode.All(
             V.OneOfValidator(categories),
-            fev.UnicodeString(not_empty=True))
+            V.UnicodeString(not_empty=True))
         return super(ForgeForm, self).display(**kw)
 
 
@@ -790,7 +790,7 @@ class RegistrationForm(ForgeForm):
             ew.TextField(
                 name='display_name',
                 label='Displayed Name',
-                validator=fev.UnicodeString(not_empty=True)),
+                validator=V.UnicodeString(not_empty=True)),
             username,
         ]
         if asbool(config.get('auth.require_email_addr', False)):
@@ -803,14 +803,14 @@ class RegistrationForm(ForgeForm):
                 name='pw',
                 label='New Password',
                 attrs=dict(minlength=asint(tg.config.get('auth.min_password_len', 6)), maxlength=asint(tg.config.get('auth.max_password_len', 30))),
-                validator=fev.UnicodeString(
+                validator=V.UnicodeString(
                     not_empty=True,
                     min=asint(tg.config.get('auth.min_password_len', 6)),
                     max=asint(tg.config.get('auth.max_password_len', 30)))),
             ew.PasswordField(
                 name='pw2',
                 label='New Password (again)',
-                validator=fev.UnicodeString(not_empty=True)),
+                validator=V.UnicodeString(not_empty=True)),
         ]
         return fields
 
@@ -995,7 +995,7 @@ class NeighborhoodAddProjectForm(ForgeForm):
             ew.InputField(name='project_name', label='Project Name',
                           field_type='text',
                           validator=formencode.All(
-                              fev.UnicodeString(not_empty=True, max=40),
+                              V.UnicodeString(not_empty=True, max=40),
                               V.MaxBytesValidator(max=40)),
                           ),
             ew.InputField(name='project_unixname',
@@ -1175,7 +1175,7 @@ class AwardGrantForm(ForgeForm):
             ew.TextArea(
                 name='comment',
                 label='Comment',
-                validator=fev.UnicodeString(not_empty=False),
+                validator=V.UnicodeString(not_empty=False),
                 attrs={'rows': 5, 'cols': 30}),
         ]
 
diff --git a/Allura/allura/lib/widgets/user_profile.py b/Allura/allura/lib/widgets/user_profile.py
index fd2e17e..587bf45 100644
--- a/Allura/allura/lib/widgets/user_profile.py
+++ b/Allura/allura/lib/widgets/user_profile.py
@@ -25,11 +25,12 @@ import ew.jinja2_ew as ew
 from jinja2 import Markup
 from paste.deploy.converters import asbool
 import tg
-from formencode import validators as fev
 from tg import app_globals as g
 from tg import request
 from tg import tmpl_context as c
+
 from allura.lib import helpers as h
+from allura.lib import validators as v
 from allura.lib.plugin import AuthenticationProvider
 from .forms import ForgeForm
 
@@ -42,7 +43,7 @@ class SendMessageForm(ForgeForm):
 
     class fields(ew_core.NameList):
         subject = ew.TextField(
-            validator=fev.UnicodeString(
+            validator=v.UnicodeString(
                 not_empty=True,
                 messages={'empty': "You must provide a Subject"}),
             attrs=dict(
@@ -52,7 +53,7 @@ class SendMessageForm(ForgeForm):
             label='Subject')
 
         message = ew.TextArea(
-            validator=fev.UnicodeString(
+            validator=v.UnicodeString(
                 not_empty=True,
                 messages={'empty': "You must provide a Message"}),
             attrs=dict(
diff --git a/Allura/allura/webhooks.py b/Allura/allura/webhooks.py
index a33a71b..c41d8c0 100644
--- a/Allura/allura/webhooks.py
+++ b/Allura/allura/webhooks.py
@@ -40,6 +40,7 @@ from paste.deploy.converters import asint, aslist
 from allura.app import AdminControllerMixin
 from allura.controllers import BaseController
 from allura.lib import helpers as h
+from allura.lib import validators as v
 from allura.lib.decorators import require_post, task
 from allura.lib.utils import DateJSONEncoder
 from allura import model as M
@@ -74,7 +75,7 @@ class WebhookValidator(fev.FancyValidator):
 
 class WebhookCreateForm(schema.Schema):
     url = fev.URL(not_empty=True)
-    secret = fev.UnicodeString()
+    secret = v.UnicodeString()
 
 
 class WebhookEditForm(WebhookCreateForm):
@@ -221,8 +222,8 @@ class WebhookRestController(BaseController):
         error = getattr(e, 'error_dict', None)
         if error:
             _error = {}
-            for k, v in six.iteritems(error):
-                _error[k] = six.text_type(v)
+            for k, val in six.iteritems(error):
+                _error[k] = six.text_type(val)
             return _error
         error = getattr(e, 'msg', None)
         if not error:
diff --git a/ForgeBlog/forgeblog/main.py b/ForgeBlog/forgeblog/main.py
index 04963f5..344a4b1 100644
--- a/ForgeBlog/forgeblog/main.py
+++ b/ForgeBlog/forgeblog/main.py
@@ -39,6 +39,7 @@ from ming.orm import session
 from allura.app import Application, SitemapEntry, ConfigOption
 from allura.app import DefaultAdminController
 from allura.lib import helpers as h
+from allura.lib import validators as v
 from allura.lib.utils import JSONForExport
 from allura.tasks import notification_tasks
 from allura.lib.search import search_app
@@ -271,7 +272,7 @@ class RootController(BaseController, FeedController):
 
     @with_trailing_slash
     @expose('jinja:forgeblog:templates/blog/search.html')
-    @validate(dict(q=validators.UnicodeString(if_empty=None),
+    @validate(dict(q=v.UnicodeString(if_empty=None),
                    history=validators.StringBool(if_empty=False),
                    search_comments=validators.StringBool(if_empty=False),
                    project=validators.StringBool(if_empty=False)))
@@ -434,8 +435,8 @@ class PostController(BaseController, FeedController):
         old_text = self.post.text
         if attachment is not None:
             self.post.add_multiple_attachments(attachment)
-        for k, v in six.iteritems(kw):
-            setattr(self.post, k, v)
+        for k, val in six.iteritems(kw):
+            setattr(self.post, k, val)
         self.post.commit()
         notification_tasks.send_usermentions_notification.post(self.post.index_id(), kw['text'], old_text)
         redirect('.')
diff --git a/ForgeBlog/forgeblog/widgets.py b/ForgeBlog/forgeblog/widgets.py
index 3d52bc5..b08df34 100644
--- a/ForgeBlog/forgeblog/widgets.py
+++ b/ForgeBlog/forgeblog/widgets.py
@@ -25,6 +25,7 @@ from formencode import validators as fev
 from allura.lib.widgets import form_fields as ffw
 from allura.lib.widgets import forms
 from allura import model as M
+from allura.lib import validators as v
 
 
 class BlogPager(ffw.PageList):
@@ -39,7 +40,7 @@ class BlogPostForm(forms.ForgeForm):
     def fields(self):
         return ew_core.NameList([
             ew.TextField(name='title',
-                         validator=fev.UnicodeString(not_empty=True,
+                         validator=v.UnicodeString(not_empty=True,
                                                      messages={'empty': "You must provide a Title"}),
                          attrs=dict(placeholder='Enter your title here',
                                     title='Enter your title here',
diff --git a/ForgeChat/forgechat/main.py b/ForgeChat/forgechat/main.py
index 8742489..e3b4765 100644
--- a/ForgeChat/forgechat/main.py
+++ b/ForgeChat/forgechat/main.py
@@ -33,6 +33,7 @@ from ming.utils import LazyProperty
 # Pyforge-specific imports
 from allura.app import Application, ConfigOption, SitemapEntry, DefaultAdminController
 from allura.lib import helpers as h
+from allura.lib import validators as v
 from allura.lib.search import search_app
 from allura.lib.decorators import require_post
 from allura.lib.security import require_access
@@ -150,7 +151,7 @@ class RootController(BaseController):
 
     @with_trailing_slash
     @expose('jinja:forgechat:templates/chat/search.html')
-    @validate(dict(q=validators.UnicodeString(if_empty=None),
+    @validate(dict(q=v.UnicodeString(if_empty=None),
                    project=validators.StringBool(if_empty=False)))
     def search(self, q=None, project=None, limit=None, page=0, **kw):
         c.search_results = SearchResults()
diff --git a/ForgeDiscussion/forgediscussion/controllers/root.py b/ForgeDiscussion/forgediscussion/controllers/root.py
index caa7ce7..0356f30 100644
--- a/ForgeDiscussion/forgediscussion/controllers/root.py
+++ b/ForgeDiscussion/forgediscussion/controllers/root.py
@@ -36,6 +36,7 @@ import pymongo
 from allura.lib.security import require_access, has_access, require_authenticated
 from allura.lib.search import search_app
 from allura.lib import helpers as h
+from allura.lib import validators as v
 from allura.lib.utils import AntiSpam
 from allura.lib.decorators import require_post, memorable_forget
 from allura.controllers import BaseController, DispatchIndex
@@ -151,7 +152,7 @@ class RootController(BaseController, DispatchIndex, FeedController):
 
     @with_trailing_slash
     @expose('jinja:forgediscussion:templates/discussionforums/search.html')
-    @validate(dict(q=validators.UnicodeString(if_empty=None),
+    @validate(dict(q=v.UnicodeString(if_empty=None),
                    history=validators.StringBool(if_empty=False),
                    project=validators.StringBool(if_empty=False),
                    limit=validators.Int(if_empty=None, if_invalid=None),
diff --git a/ForgeDiscussion/forgediscussion/widgets/admin.py b/ForgeDiscussion/forgediscussion/widgets/admin.py
index cec8eda..6c650b4 100644
--- a/ForgeDiscussion/forgediscussion/widgets/admin.py
+++ b/ForgeDiscussion/forgediscussion/widgets/admin.py
@@ -25,7 +25,7 @@ from bson import ObjectId
 import ew.jinja2_ew as ew
 
 from allura.lib.widgets import forms as ff
-from allura.lib.widgets import form_fields as ffw
+from allura.lib import validators as v
 from allura.lib import helpers as h
 from forgediscussion import model as DM
 
@@ -66,7 +66,7 @@ class AddForum(ff.AdminForm):
         fields = [
             ew.HiddenField(name='app_id', label='App'),
             ew.TextField(name='name', label='Name',
-                         validator=fev.UnicodeString()),
+                         validator=v.UnicodeString()),
             ew.TextField(name='shortname', label='Short Name',
                          validator=All(
                              fev.Regex(r"^[^\s\/\.]*$", not_empty=True, messages={
@@ -75,7 +75,7 @@ class AddForum(ff.AdminForm):
                              UniqueForumShortnameValidator())),
             ew.TextField(name='parent', label='Parent Forum'),
             ew.TextField(name='description', label='Description',
-                         validator=fev.UnicodeString()),
+                         validator=v.UnicodeString()),
             ew.TextField(name='monitoring_email',
                          label='Monitoring Email', validator=fev.Email()),
             ew.Checkbox(name="members_only", label="Developer Only"),
diff --git a/ForgeImporters/forgeimporters/base.py b/ForgeImporters/forgeimporters/base.py
index 13a7a7b..ea87738 100644
--- a/ForgeImporters/forgeimporters/base.py
+++ b/ForgeImporters/forgeimporters/base.py
@@ -71,7 +71,7 @@ class ProjectImportForm(schema.Schema):
         self.allow_extra_fields = True
 
     neighborhood = fev.NotEmpty()
-    project_name = fev.UnicodeString(not_empty=True, max=40)
+    project_name = v.UnicodeString(not_empty=True, max=40)
 
 
 class ToolImportForm(schema.Schema):
@@ -79,7 +79,7 @@ class ToolImportForm(schema.Schema):
     def __init__(self, tool_class):
         super(ToolImportForm, self).__init__()
         self.add_field('mount_point', v.MountPointValidator(tool_class))
-    mount_label = fev.UnicodeString()
+    mount_label = v.UnicodeString()
 
 
 class ImportErrorHandler(object):
diff --git a/ForgeImporters/forgeimporters/github/code.py b/ForgeImporters/forgeimporters/github/code.py
index 8a4449e..0fb06a4 100644
--- a/ForgeImporters/forgeimporters/github/code.py
+++ b/ForgeImporters/forgeimporters/github/code.py
@@ -19,7 +19,6 @@ from __future__ import unicode_literals
 from __future__ import absolute_import
 from tg import tmpl_context as c
 from tg import app_globals as g
-from formencode import validators as fev
 from tg import (
     expose,
     flash,
@@ -32,6 +31,7 @@ from tg.decorators import (
 
 from allura.lib.decorators import require_post
 from allura import model as M
+from allura.lib import validators as v
 
 from forgeimporters.base import (
     ToolImporter,
@@ -46,7 +46,7 @@ from forgeimporters.github import (
 
 
 class GitHubRepoImportForm(ToolImportForm):
-    gh_user_name = fev.UnicodeString(not_empty=True)
+    gh_user_name = v.UnicodeString(not_empty=True)
     gh_project_name = GitHubProjectNameValidator()
 
 
diff --git a/ForgeImporters/forgeimporters/github/tracker.py b/ForgeImporters/forgeimporters/github/tracker.py
index b796d3c..7e55a01 100644
--- a/ForgeImporters/forgeimporters/github/tracker.py
+++ b/ForgeImporters/forgeimporters/github/tracker.py
@@ -24,7 +24,6 @@ from six.moves.urllib.error import HTTPError
 import six
 from io import BytesIO
 
-from formencode import validators as fev
 from tg import (
     expose,
     flash,
@@ -37,6 +36,7 @@ from tg.decorators import (
 
 from allura import model as M
 from allura.lib import helpers as h
+from allura.lib import validators as v
 from allura.lib.plugin import ImportIdConverter
 from allura.lib.decorators import require_post
 from ming.orm import session, ThreadLocalORMSession
@@ -62,7 +62,7 @@ log = logging.getLogger(__name__)
 
 class GitHubTrackerImportForm(ToolImportForm):
     gh_project_name = GitHubProjectNameValidator()
-    gh_user_name = fev.UnicodeString(not_empty=True)
+    gh_user_name = v.UnicodeString(not_empty=True)
 
 
 class GitHubTrackerImportController(ToolImportController, GitHubOAuthMixin):
diff --git a/ForgeImporters/forgeimporters/github/wiki.py b/ForgeImporters/forgeimporters/github/wiki.py
index d96a3c1..3cc1ba4 100644
--- a/ForgeImporters/forgeimporters/github/wiki.py
+++ b/ForgeImporters/forgeimporters/github/wiki.py
@@ -24,14 +24,12 @@ from tempfile import mkdtemp
 from shutil import rmtree
 
 import six
-from paste.deploy.converters import aslist
 
 from bs4 import BeautifulSoup
 import git
 from tg import app_globals as g
 from tg import tmpl_context as c
 from ming.orm import ThreadLocalORMSession
-from formencode import validators as fev
 from tg import (
     expose,
     flash,
@@ -43,6 +41,7 @@ from tg.decorators import (
 )
 
 from allura.lib import helpers as h
+from allura.lib import validators as v
 from allura.lib import utils
 from allura.lib.plugin import ImportIdConverter
 from allura.lib.decorators import (
@@ -71,8 +70,8 @@ log = logging.getLogger(__name__)
 
 class GitHubWikiImportForm(ToolImportForm):
     gh_project_name = GitHubProjectNameValidator()
-    gh_user_name = fev.UnicodeString(not_empty=True)
-    tool_option = fev.UnicodeString(if_missing='')
+    gh_user_name = v.UnicodeString(not_empty=True)
+    tool_option = v.UnicodeString(if_missing='')
 
 
 class GitHubWikiImportController(ToolImportController, GitHubOAuthMixin):
diff --git a/ForgeShortUrl/forgeshorturl/main.py b/ForgeShortUrl/forgeshorturl/main.py
index 290b6f7..a471983 100644
--- a/ForgeShortUrl/forgeshorturl/main.py
+++ b/ForgeShortUrl/forgeshorturl/main.py
@@ -24,6 +24,7 @@ from allura.app import Application, SitemapEntry, DefaultAdminController
 from allura import model as M
 from allura.lib.security import require_access, has_access
 from allura.lib import helpers as h
+from allura.lib import validators as v
 from allura.lib.search import search_app
 from allura.controllers import BaseController
 from allura.lib.widgets import form_fields as ffw
@@ -156,7 +157,7 @@ class RootController(BaseController):
         }
 
     @expose('jinja:forgeshorturl:templates/search.html')
-    @validate(dict(q=validators.UnicodeString(if_empty=None),
+    @validate(dict(q=v.UnicodeString(if_empty=None),
                    project=validators.StringBool(if_empty=False)))
     def search(self, q=None, project=None, limit=None, page=0, **kw):
         c.search_results = W.search_results
diff --git a/ForgeTracker/forgetracker/tracker_main.py b/ForgeTracker/forgetracker/tracker_main.py
index 980ad5e..28c7727 100644
--- a/ForgeTracker/forgetracker/tracker_main.py
+++ b/ForgeTracker/forgetracker/tracker_main.py
@@ -47,6 +47,7 @@ from ming.utils import LazyProperty
 # Pyforge-specific imports
 from allura import model as M
 from allura.lib import helpers as h
+from allura.lib import validators as v
 from allura.lib import utils
 from allura.tasks import notification_tasks
 from allura.app import (
@@ -89,12 +90,12 @@ from six.moves import map
 log = logging.getLogger(__name__)
 
 search_validators = dict(
-    q=validators.UnicodeString(if_empty=None),
+    q=v.UnicodeString(if_empty=None),
     history=validators.StringBool(if_empty=False),
     project=validators.StringBool(if_empty=False),
     limit=validators.Int(if_invalid=None),
     page=validators.Int(if_empty=0, if_invalid=0),
-    sort=validators.UnicodeString(if_empty=None),
+    sort=v.UnicodeString(if_empty=None),
     filter=V.JsonConverter(if_empty={}, if_invalid={}),
     deleted=validators.StringBool(if_empty=False))
 
@@ -949,11 +950,11 @@ class RootController(BaseController, FeedController):
 
     @with_trailing_slash
     @expose('jinja:forgetracker:templates/tracker/mass_edit.html')
-    @validate(dict(q=validators.UnicodeString(if_empty=None),
+    @validate(dict(q=v.UnicodeString(if_empty=None),
                    filter=V.JsonConverter(if_empty={}),
                    limit=validators.Int(if_empty=10, if_invalid=10),
                    page=validators.Int(if_empty=0, if_invalid=0),
-                   sort=validators.UnicodeString(if_empty='ticket_num_i asc')))
+                   sort=v.UnicodeString(if_empty='ticket_num_i asc')))
     def edit(self, q=None, limit=None, page=None, sort=None, filter=None, **kw):
         require_access(c.app, 'update')
         result = TM.Ticket.paged_search(c.app.config, c.user, q, filter=filter,
@@ -977,11 +978,11 @@ class RootController(BaseController, FeedController):
 
     @with_trailing_slash
     @expose('jinja:forgetracker:templates/tracker/mass_move.html')
-    @validate(dict(q=validators.UnicodeString(if_empty=None),
+    @validate(dict(q=v.UnicodeString(if_empty=None),
                    filter=V.JsonConverter(if_empty={}),
                    limit=validators.Int(if_empty=10, if_invalid=10),
                    page=validators.Int(if_empty=0, if_invalid=0),
-                   sort=validators.UnicodeString(if_empty='ticket_num_i asc')))
+                   sort=v.UnicodeString(if_empty='ticket_num_i asc')))
     def move(self, q=None, limit=None, page=None, sort=None, filter=None, **kw):
         require_access(c.app, 'admin')
         result = TM.Ticket.paged_search(c.app.config, c.user, q, filter=filter,
@@ -1698,8 +1699,8 @@ class TrackerAdminController(DefaultAdminController):
     @validate(W.options_admin, error_handler=options)
     def set_options(self, **kw):
         require_access(self.app, 'configure')
-        for k, v in six.iteritems(kw):
-            self.app.config.options[k] = v
+        for k, val in six.iteritems(kw):
+            self.app.config.options[k] = val
         flash('Options updated')
         redirect(request.referer or '/')
 
@@ -1937,7 +1938,7 @@ class MilestoneController(BaseController):
     @validate(validators=dict(
         limit=validators.Int(if_invalid=None),
         page=validators.Int(if_empty=0, if_invalid=0),
-        sort=validators.UnicodeString(if_empty=''),
+        sort=v.UnicodeString(if_empty=''),
         filter=V.JsonConverter(if_empty={}),
         deleted=validators.StringBool(if_empty=False)))
     def index(self, q=None, columns=None, page=0, query=None, sort=None,
diff --git a/ForgeTracker/forgetracker/widgets/admin.py b/ForgeTracker/forgetracker/widgets/admin.py
index a36e11d..4e72838 100644
--- a/ForgeTracker/forgetracker/widgets/admin.py
+++ b/ForgeTracker/forgetracker/widgets/admin.py
@@ -20,6 +20,7 @@ from __future__ import absolute_import
 import ew.jinja2_ew as ew
 
 from formencode import validators as fev
+from allura.lib import validators as v
 from allura.lib.widgets import forms as ff
 from allura.lib.widgets import form_fields as ffw
 
@@ -61,13 +62,13 @@ class OptionsAdmin(ff.AdminForm):
             ffw.MarkdownEdit(
                 name='TicketHelpNew',
                 label='Help text to display on new ticket page',
-                validator=fev.String(),
+                validator=v.String(),
                 attrs={'style': 'width: 95%'}),
             ffw.MarkdownEdit(
                 name='TicketHelpSearch',
                 label='Help text to display on ticket list pages (index page,'
                       ' search results, milestone lists)',
-                validator=fev.String(),
+                validator=v.String(),
                 attrs={'style': 'width: 95%'}),
         ]
         return fields
diff --git a/ForgeTracker/forgetracker/widgets/bin_form.py b/ForgeTracker/forgetracker/widgets/bin_form.py
index a18cff2..31df17b 100644
--- a/ForgeTracker/forgetracker/widgets/bin_form.py
+++ b/ForgeTracker/forgetracker/widgets/bin_form.py
@@ -19,10 +19,9 @@ from __future__ import unicode_literals
 from __future__ import absolute_import
 import ew
 from ew import jinja2_ew
-from allura.lib import validators as V
+from allura.lib import validators as v
 
 from forgetracker import model
-from formencode import validators as fev
 
 
 class BinForm(ew.SimpleForm):
@@ -33,12 +32,12 @@ class BinForm(ew.SimpleForm):
 
     class hidden_fields(ew.NameList):
         _id = jinja2_ew.HiddenField(
-            validator=V.Ming(model.Bin), if_missing=None)
+            validator=v.Ming(model.Bin), if_missing=None)
 
     class fields(ew.NameList):
         summary = jinja2_ew.TextField(
             label='Bin Name',
-            validator=fev.UnicodeString(not_empty=True))
+            validator=v.UnicodeString(not_empty=True))
         terms = jinja2_ew.TextField(
             label='Search Terms',
-            validator=fev.UnicodeString(not_empty=True))
+            validator=v.UnicodeString(not_empty=True))
diff --git a/ForgeTracker/forgetracker/widgets/ticket_form.py b/ForgeTracker/forgetracker/widgets/ticket_form.py
index f1f04e1..fa1ddd4 100644
--- a/ForgeTracker/forgetracker/widgets/ticket_form.py
+++ b/ForgeTracker/forgetracker/widgets/ticket_form.py
@@ -106,7 +106,7 @@ class GenericTicketForm(ew.SimpleForm):
             ew.TextField(name='summary', label='Title',
                          attrs={'style': 'width: 425px', 'class':'memorable',
                                 'placeholder': 'Title'},
-                         validator=fev.UnicodeString(
+                         validator=v.UnicodeString(
                              not_empty=True, messages={
                                  'empty': "You must provide a Title"})),
             ffw.MarkdownEdit(label='Description', name='description',
diff --git a/ForgeWiki/forgewiki/wiki_main.py b/ForgeWiki/forgewiki/wiki_main.py
index 75f289a..d635222 100644
--- a/ForgeWiki/forgewiki/wiki_main.py
+++ b/ForgeWiki/forgewiki/wiki_main.py
@@ -36,6 +36,7 @@ from ming.orm import session
 # Pyforge-specific imports
 from allura import model as M
 from allura.lib import helpers as h
+from allura.lib import validators as v
 from allura.app import Application, SitemapEntry, DefaultAdminController, ConfigOption
 from allura.lib.search import search_app
 from allura.lib.decorators import require_post, memorable_forget
@@ -402,7 +403,7 @@ class RootController(BaseController, DispatchIndex, FeedController):
 
     @with_trailing_slash
     @expose('jinja:forgewiki:templates/wiki/search.html')
-    @validate(dict(q=validators.UnicodeString(if_empty=None),
+    @validate(dict(q=v.UnicodeString(if_empty=None),
                    history=validators.StringBool(if_empty=False),
                    search_comments=validators.StringBool(if_empty=False),
                    project=validators.StringBool(if_empty=False)))
@@ -424,7 +425,7 @@ class RootController(BaseController, DispatchIndex, FeedController):
 
     @with_trailing_slash
     @expose('jinja:forgewiki:templates/wiki/browse.html')
-    @validate(dict(sort=validators.UnicodeString(if_empty='alpha'),
+    @validate(dict(sort=v.UnicodeString(if_empty='alpha'),
                    show_deleted=validators.StringBool(if_empty=False),
                    page=validators.Int(if_empty=0, if_invalid=0),
                    limit=validators.Int(if_empty=None, if_invalid=None)))
@@ -468,7 +469,7 @@ class RootController(BaseController, DispatchIndex, FeedController):
 
     @with_trailing_slash
     @expose('jinja:forgewiki:templates/wiki/browse_tags.html')
-    @validate(dict(sort=validators.UnicodeString(if_empty='alpha'),
+    @validate(dict(sort=v.UnicodeString(if_empty='alpha'),
                    page=validators.Int(if_empty=0, if_invalid=0),
                    limit=validators.Int(if_empty=None, if_invalid=None)))
     def browse_tags(self, sort='alpha', page=0, limit=None, **kw):
diff --git a/requirements.in b/requirements.in
index 494ce9a..cc30883 100644
--- a/requirements.in
+++ b/requirements.in
@@ -11,7 +11,7 @@ emoji
 faulthandler ; python_version < "3.3"
 feedgenerator
 feedparser
-# FormEncode may need v2.0 to work past py3.3 or so?  https://github.com/formencode/formencode/issues/140
+# FormEncode is not currently maintained :(  Could go to 2.0.0a currently https://github.com/formencode/formencode/issues/140
 FormEncode
 GitPython==2.1.11
 html5lib==1.0.1


[allura] 12/12: [#8361] updated requirements.txt

Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

kentontaylor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/allura.git

commit 9755507198268804e4fd3839afd6574322d96fa3
Author: Kenton Taylor <kt...@slashdotmedia.com>
AuthorDate: Wed Jun 3 16:02:53 2020 +0000

    [#8361] updated requirements.txt
---
 requirements.txt | 108 +++++++++++++++++++++++++++----------------------------
 1 file changed, 54 insertions(+), 54 deletions(-)

diff --git a/requirements.txt b/requirements.txt
index 4c0135d..4030d32 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -4,99 +4,99 @@
 #
 #    pip-compile
 #
-activitystream==0.3.0
+activitystream==0.3.0     # via -r requirements.in
 asn1crypto==0.24.0        # via cryptography
-backlash==0.1.4
+backlash==0.1.4           # via -r requirements.in
 backports.functools-lru-cache==1.5  # via soupsieve
 backports.shutil-get-terminal-size==1.0.0  # via ipython
-beaker==1.10.1
-beautifulsoup4==4.8.0
+beaker==1.10.1            # via -r requirements.in
+beautifulsoup4==4.8.0     # via -r requirements.in, webtest
 bleach==3.1.4             # via pypeline
 certifi==2019.6.16        # via requests
 cffi==1.12.2              # via cryptography
-chardet==3.0.4
-colander==1.7.0
+chardet==3.0.4            # via -r requirements.in, requests
+colander==1.7.0           # via -r requirements.in
 crank==0.8.1              # via turbogears2
 creoleparser==0.7.5       # via pypeline
-cryptography==2.6.1
-datadiff==2.0.0
-decorator==4.4.0
+cryptography==2.6.1       # via -r requirements.in
+datadiff==2.0.0           # via -r requirements.in
+decorator==4.4.0          # via -r requirements.in, ipython, traitlets
 docutils==0.15.2          # via pypeline
-easywidgets==0.3.5
-emoji==0.5.3
+easywidgets==0.3.6        # via -r requirements.in
+emoji==0.5.3              # via -r requirements.in
 enum34==1.1.6             # via colander, cryptography, traitlets
-faulthandler==3.1 ; python_version < "3.3"
-feedgenerator==1.9.1
-feedparser==5.2.1
-formencode==1.3.1
+faulthandler==3.1 ; python_version < "3.3"  # via -r requirements.in
+feedgenerator==1.9.1      # via -r requirements.in
+feedparser==5.2.1         # via -r requirements.in
+formencode==1.3.1         # via -r requirements.in, easywidgets, ming
 funcsigs==1.0.2           # via beaker, mock
 future==0.17.1            # via timermiddleware
 genshi==0.7.3             # via creoleparser
 gitdb2==2.0.4             # via gitpython
-gitpython==2.1.11
-gunicorn==19.5.0
-html5lib==1.0.1
+gitpython==2.1.11         # via -r requirements.in
+gunicorn==19.5.0          # via -r requirements.in
+html5lib==1.0.1           # via -r requirements.in, pypeline, textile
 httplib2==0.18.1          # via oauth2
 idna==2.8                 # via requests
 inflection==0.3.1         # via profanityfilter
 ipaddress==1.0.22         # via cryptography
 ipython-genutils==0.2.0   # via traitlets
-ipython==5.8.0
+ipython==5.8.0            # via -r requirements.in
 iso8601==0.1.12           # via colander
-jinja2==2.11.2
-markdown-checklist==0.4.1
-markdown==2.6.11
-markupsafe==1.1.1
-ming==0.5.7
-mock==3.0.5
-nose==1.3.7
-oauth2==1.9.0.post1
+jinja2==2.11.2            # via -r requirements.in
+markdown-checklist==0.4.1  # via -r requirements.in
+markdown==2.6.11          # via -r requirements.in, markdown-checklist, pypeline
+markupsafe==1.1.1         # via -r requirements.in, easywidgets, jinja2, turbogears2, webhelpers2
+ming==0.5.7               # via -r requirements.in
+mock==3.0.5               # via -r requirements.in
+nose==1.3.7               # via -r requirements.in
+oauth2==1.9.0.post1       # via -r requirements.in
 oauthlib==3.0.2           # via requests-oauthlib
-paginate==0.5.6
-paste==3.1.0
-pastedeploy==2.0.1
-pastescript==3.1.0
+paginate==0.5.6           # via -r requirements.in
+paste==3.1.0              # via -r requirements.in, easywidgets, pastescript
+pastedeploy==2.0.1        # via -r requirements.in, pastescript
+pastescript==3.1.0        # via -r requirements.in
 pathlib2==2.3.4           # via ipython, pickleshare
 pexpect==4.7.0            # via ipython
 pickleshare==0.7.5        # via ipython
-pillow==6.2.2
-profanityfilter==2.0.6
+pillow==6.2.2             # via -r requirements.in
+profanityfilter==2.0.6    # via -r requirements.in
 prompt-toolkit==1.0.16    # via ipython
 ptyprocess==0.6.0         # via pexpect
 pycparser==2.19           # via cffi
-pyflakes==2.1.1
-pygments==2.4.2
-pymongo==2.8.1
-pypeline[creole,markdown,rst,textile]==0.5.2
-pysolr==3.6.0
-python-dateutil==2.8.0
-python-magic==0.4.15
-python-oembed==0.2.4
-pytz==2019.1
-qrcode==6.1
+pyflakes==2.1.1           # via -r requirements.in
+pygments==2.4.2           # via -r requirements.in, ipython
+pymongo==2.8.1            # via -r requirements.in, activitystream, ming
+pypeline[creole,markdown,rst,textile]==0.5.2  # via -r requirements.in
+pysolr==3.6.0             # via -r requirements.in
+python-dateutil==2.8.0    # via -r requirements.in, easywidgets
+python-magic==0.4.15      # via -r requirements.in
+python-oembed==0.2.4      # via -r requirements.in
+pytz==2019.1              # via -r requirements.in, feedgenerator, ming
+qrcode==6.1               # via -r requirements.in
 repoze.lru==0.7           # via turbogears2
-requests-oauthlib==1.2.0
-requests==2.22.0
+requests-oauthlib==1.2.0  # via -r requirements.in
+requests==2.22.0          # via -r requirements.in, pysolr, requests-oauthlib
 scandir==1.10.0           # via pathlib2
-setproctitle==1.1.9
+setproctitle==1.1.9       # via -r requirements.in
 simplegeneric==0.8.1      # via ipython
-six==1.12.0
+six==1.15.0               # via -r requirements.in, bleach, creoleparser, cryptography, easywidgets, feedgenerator, html5lib, ming, mock, paste, pastescript, pathlib2, prompt-toolkit, python-dateutil, qrcode, textile, traitlets, webhelpers2, webtest
 smmap2==2.0.4             # via gitdb2
 soupsieve==1.9.3          # via beautifulsoup4
-testfixtures==6.10.0
+testfixtures==6.10.0      # via -r requirements.in
 textile==3.0.4            # via pypeline
-timermiddleware==0.5.1
+timermiddleware==0.5.1    # via -r requirements.in
 traitlets==4.3.2          # via ipython
 translationstring==1.3    # via colander
-turbogears2==2.3.12
+turbogears2==2.3.12       # via -r requirements.in
 urllib3==1.25.8           # via requests
 waitress==1.4.3           # via webtest
 wcwidth==0.1.7            # via prompt-toolkit
 webencodings==0.5.1       # via bleach, html5lib
-webhelpers2==2.0
-webob==1.7.4
-webtest==2.0.33
-wrapt==1.11.2
+webhelpers2==2.0          # via -r requirements.in
+webob==1.7.4              # via -r requirements.in, backlash, easywidgets, timermiddleware, turbogears2, webtest
+webtest==2.0.33           # via -r requirements.in
+wrapt==1.11.2             # via -r requirements.in
 
 # The following packages are considered to be unsafe in a requirements file:
 # setuptools