You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@allura.apache.org by br...@apache.org on 2020/03/10 16:11:52 UTC
[allura] branch db/8354 updated (e02b2e4 -> 33d8446)
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a change to branch db/8354
in repository https://gitbox.apache.org/repos/asf/allura.git.
discard e02b2e4 [#8354] modernize run_tests script
discard 9aba823 [#8354] antispam fixes for py3
discard 75539c1 [#8354] make .ini settings compatible with py3 configparser
discard 2f3961b [#8354] avoid error: dictionary changed size during iteration
discard b5052ae [#8354] unicode/byte fixes encountered during nearly all tests setup
discard 670f81c [#8354] make sure jinja2 config values are of the right type
discard b7db90c [#8354] remove iteritems() usage in templates; assuming none of these are such huge lists that py2-non-iter will matter
discard a720ee8 [#8354] misc other import-time weird fixes
discard a9e5c2e [#8354] webhelpers -> webhelpers2
discard a5c0916 [#8354] webhelpers.feedgenerator -> standalone feedgenerator package
discard de2d4a5 [#8354] webhelpers.paginate -> standalone paginate package
discard e11520d [#8354] six-ify email mime imports
discard 7ed67d7 [#8354] change StringIO uses that really are BytesIO
discard 0c530a4 [#8354] fix six cookie import
new 5b371e0 [#8354] fix six cookie import
new dbebcd5 [#8354] change StringIO uses that really are BytesIO
new 6e0a2ce [#8354] six-ify email mime imports
new 61a80ca [#8354] webhelpers.paginate -> standalone paginate package
new 9c46fb0 [#8354] webhelpers.feedgenerator -> standalone feedgenerator package
new e270237 [#8354] webhelpers -> webhelpers2
new dcdfa27 [#8354] misc other import-time weird fixes
new fa63fe6 [#8354] remove iteritems() usage in templates; assuming none of these are such huge lists that py2-non-iter will matter
new 9ad5f3d [#8354] make sure jinja2 config values are of the right type
new 99a5ef8 [#8354] unicode/byte fixes encountered during nearly all tests setup
new 921c3b7 [#8354] avoid error: dictionary changed size during iteration
new e2aad50 [#8354] make .ini settings compatible with py3 configparser
new d5f5e01 [#8354] antispam fixes for py3
new 33d8446 [#8354] modernize run_tests script
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (e02b2e4)
\
N -- N -- N refs/heads/db/8354 (33d8446)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 14 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
[allura] 02/14: [#8354] change StringIO uses that really are BytesIO
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8354
in repository https://gitbox.apache.org/repos/asf/allura.git
commit dbebcd535eedca0f9279e6da694fac6eff319953
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Tue Mar 10 12:11:12 2020 -0400
[#8354] change StringIO uses that really are BytesIO
---
Allura/allura/app.py | 10 ++++----
Allura/allura/controllers/static.py | 5 ++--
Allura/allura/lib/helpers.py | 5 ++--
Allura/allura/lib/plugin.py | 4 +--
Allura/allura/model/filesystem.py | 4 +--
Allura/allura/tests/functional/test_admin.py | 10 ++++----
.../allura/tests/functional/test_neighborhood.py | 6 ++---
Allura/allura/tests/model/test_discussion.py | 26 +++++++++----------
Allura/allura/tests/model/test_filesystem.py | 21 ++++++++-------
ForgeBlog/forgeblog/tests/test_app.py | 4 +--
.../forgediscussion/tests/functional/test_forum.py | 6 ++---
.../tests/functional/test_forum_admin.py | 1 -
ForgeDiscussion/forgediscussion/tests/test_app.py | 4 +--
ForgeImporters/forgeimporters/base.py | 12 ++++-----
ForgeImporters/forgeimporters/github/tracker.py | 8 ++----
.../forgeimporters/tests/github/test_extractor.py | 30 ++++++++++------------
.../forgeimporters/tests/github/test_tracker.py | 2 +-
ForgeSVN/forgesvn/model/svn.py | 4 +--
ForgeSVN/forgesvn/tests/model/test_repository.py | 9 +++----
ForgeTracker/forgetracker/import_support.py | 4 +--
.../forgetracker/tests/functional/test_root.py | 6 ++---
ForgeTracker/forgetracker/tests/test_app.py | 4 +--
ForgeWiki/forgewiki/tests/functional/test_root.py | 6 ++---
ForgeWiki/forgewiki/tests/test_app.py | 4 +--
24 files changed, 93 insertions(+), 102 deletions(-)
diff --git a/Allura/allura/app.py b/Allura/allura/app.py
index 17e4943..056c776 100644
--- a/Allura/allura/app.py
+++ b/Allura/allura/app.py
@@ -20,7 +20,7 @@ from __future__ import absolute_import
import os
import logging
from urllib import basejoin
-from cStringIO import StringIO
+from io import BytesIO
from collections import defaultdict
from xml.etree import ElementTree as ET
from copy import copy
@@ -50,7 +50,7 @@ from allura.lib.utils import permanent_redirect, ConfigProxy
from allura import model as M
from allura.tasks import index_tasks
import six
-from io import open
+from io import open, BytesIO
from six.moves import map
log = logging.getLogger(__name__)
@@ -710,7 +710,7 @@ class Application(object):
if message.get('filename'):
# Special case - the actual post may not have been created yet
log.info('Saving attachment %s', message['filename'])
- fp = StringIO(message['payload'])
+ fp = BytesIO(six.ensure_binary(message['payload']))
self.AttachmentClass.save_attachment(
message['filename'], fp,
content_type=message.get(
@@ -728,9 +728,9 @@ class Application(object):
message_id)
try:
- fp = StringIO(message['payload'].encode('utf-8'))
+ fp = BytesIO(message['payload'].encode('utf-8'))
except UnicodeDecodeError:
- fp = StringIO(message['payload'])
+ fp = BytesIO(message['payload'])
post.attach(
'alternate', fp,
diff --git a/Allura/allura/controllers/static.py b/Allura/allura/controllers/static.py
index fd5404b..720ef77 100644
--- a/Allura/allura/controllers/static.py
+++ b/Allura/allura/controllers/static.py
@@ -17,8 +17,9 @@
from __future__ import unicode_literals
from __future__ import absolute_import
-from cStringIO import StringIO
+from io import BytesIO
+import six
from tg import expose
from tg.decorators import without_trailing_slash
from webob import exc
@@ -55,4 +56,4 @@ class NewForgeController(object):
"""
css, md5 = g.tool_icon_css
return utils.serve_file(
- StringIO(css), 'tool_icon_css', 'text/css', etag=md5)
+ BytesIO(six.ensure_binary(css)), 'tool_icon_css', 'text/css', etag=md5)
diff --git a/Allura/allura/lib/helpers.py b/Allura/allura/lib/helpers.py
index 13c14a8..d2506e7 100644
--- a/Allura/allura/lib/helpers.py
+++ b/Allura/allura/lib/helpers.py
@@ -23,6 +23,7 @@ import sys
import os
import os.path
import difflib
+
import six.moves.urllib.request, six.moves.urllib.parse, six.moves.urllib.error
import re
import unicodedata
@@ -37,7 +38,7 @@ from collections import defaultdict
import shlex
import socket
from functools import partial
-from cStringIO import StringIO
+from io import BytesIO
import cgi
import emoji
@@ -1211,7 +1212,7 @@ def rate_limit(cfg_opt, artifact_count, start_date, exception=None):
def base64uri(content_or_image, image_format='PNG', mimetype='image/png', windows_line_endings=False):
if hasattr(content_or_image, 'save'):
- output = StringIO()
+ output = BytesIO()
content_or_image.save(output, format=image_format)
content = output.getvalue()
else:
diff --git a/Allura/allura/lib/plugin.py b/Allura/allura/lib/plugin.py
index 32b6dae..a1b2629 100644
--- a/Allura/allura/lib/plugin.py
+++ b/Allura/allura/lib/plugin.py
@@ -29,7 +29,7 @@ import crypt
import random
from six.moves.urllib.request import urlopen
from six.moves.urllib.parse import urlparse
-from cStringIO import StringIO
+from io import BytesIO
from random import randint
from hashlib import sha256
from base64 import b64encode
@@ -1094,7 +1094,7 @@ class ProjectRegistrationProvider(object):
troves.append(
M.TroveCategory.query.get(trove_cat_id=trove_id)._id)
if 'icon' in project_template:
- icon_file = StringIO(urlopen(project_template['icon']['url']).read())
+ icon_file = BytesIO(urlopen(project_template['icon']['url']).read())
p.save_icon(project_template['icon']['filename'], icon_file)
if user_project:
diff --git a/Allura/allura/model/filesystem.py b/Allura/allura/model/filesystem.py
index 7bd726c..8392a92 100644
--- a/Allura/allura/model/filesystem.py
+++ b/Allura/allura/model/filesystem.py
@@ -19,7 +19,7 @@ from __future__ import unicode_literals
from __future__ import absolute_import
import os
import re
-from cStringIO import StringIO
+from io import BytesIO
import logging
import PIL
@@ -103,7 +103,7 @@ class File(MappedClass):
@classmethod
def from_data(cls, filename, data, **kw):
- return cls.from_stream(filename, StringIO(data), **kw)
+ return cls.from_stream(filename, BytesIO(data), **kw)
def delete(self):
self._fs().delete(self.file_id)
diff --git a/Allura/allura/tests/functional/test_admin.py b/Allura/allura/tests/functional/test_admin.py
index be3a353..cb3a182 100644
--- a/Allura/allura/tests/functional/test_admin.py
+++ b/Allura/allura/tests/functional/test_admin.py
@@ -21,7 +21,7 @@ from __future__ import absolute_import
import os
import allura
import pkg_resources
-import StringIO
+from io import BytesIO
import logging
from io import open
@@ -384,12 +384,12 @@ class TestProjectAdmin(TestController):
short_description='A Test Project'),
upload_files=[upload])
r = self.app.get('/p/test/icon')
- image = PIL.Image.open(StringIO.StringIO(r.body))
+ image = PIL.Image.open(BytesIO(r.body))
assert image.size == (48, 48)
r = self.app.get('/p/test/icon?foo=bar')
r = self.app.get('/p/test/icon?w=96')
- image = PIL.Image.open(StringIO.StringIO(r.body))
+ image = PIL.Image.open(BytesIO(r.body))
assert image.size == (96, 96)
r = self.app.get('/p/test/icon?w=12345', status=404)
@@ -411,10 +411,10 @@ class TestProjectAdmin(TestController):
filename = project.get_screenshots()[0].filename
r = self.app.get('/p/test/screenshot/' + filename)
uploaded = PIL.Image.open(file_path)
- screenshot = PIL.Image.open(StringIO.StringIO(r.body))
+ screenshot = PIL.Image.open(BytesIO(r.body))
assert uploaded.size == screenshot.size
r = self.app.get('/p/test/screenshot/' + filename + '/thumb')
- thumb = PIL.Image.open(StringIO.StringIO(r.body))
+ thumb = PIL.Image.open(BytesIO(r.body))
assert thumb.size == (150, 150)
# FIX: home pages don't currently support screenshots (now that they're a wiki);
# reinstate this code (or appropriate) when we have a macro for that
diff --git a/Allura/allura/tests/functional/test_neighborhood.py b/Allura/allura/tests/functional/test_neighborhood.py
index 7ebf528..1f7034c 100644
--- a/Allura/allura/tests/functional/test_neighborhood.py
+++ b/Allura/allura/tests/functional/test_neighborhood.py
@@ -20,7 +20,7 @@ from __future__ import unicode_literals
from __future__ import absolute_import
import json
import os
-from cStringIO import StringIO
+from io import BytesIO
import six.moves.urllib.request, six.moves.urllib.error, six.moves.urllib.parse
from io import open
@@ -276,7 +276,7 @@ class TestNeighborhood(TestController):
homepage='# MozQ1'),
extra_environ=dict(username=str('root')), upload_files=[upload])
r = self.app.get('/adobe/icon')
- image = PIL.Image.open(StringIO(r.body))
+ image = PIL.Image.open(BytesIO(r.body))
assert image.size == (48, 48)
r = self.app.get('/adobe/icon?foo=bar')
@@ -894,7 +894,7 @@ class TestNeighborhood(TestController):
foo_id, extra_environ=dict(username=str('root')))
r = self.app.get('/adobe/_admin/awards/%s/icon' %
foo_id, extra_environ=dict(username=str('root')))
- image = PIL.Image.open(StringIO(r.body))
+ image = PIL.Image.open(BytesIO(r.body))
assert image.size == (48, 48)
self.app.post('/adobe/_admin/awards/grant',
params=dict(grant='FOO', recipient='adobe-1',
diff --git a/Allura/allura/tests/model/test_discussion.py b/Allura/allura/tests/model/test_discussion.py
index b4e5f22..3f9906f 100644
--- a/Allura/allura/tests/model/test_discussion.py
+++ b/Allura/allura/tests/model/test_discussion.py
@@ -22,7 +22,7 @@ Model tests for artifact
"""
from __future__ import unicode_literals
from __future__ import absolute_import
-from cStringIO import StringIO
+from io import BytesIO
import time
from datetime import datetime, timedelta
from cgi import FieldStorage
@@ -178,14 +178,14 @@ def test_attachment_methods():
d = M.Discussion(shortname='test', name='test')
t = M.Thread.new(discussion_id=d._id, subject='Test Thread')
p = t.post('This is a post')
- p_att = p.attach('foo.text', StringIO('Hello, world!'),
+ p_att = p.attach('foo.text', BytesIO(b'Hello, world!'),
discussion_id=d._id,
thread_id=t._id,
post_id=p._id)
- t_att = p.attach('foo2.text', StringIO('Hello, thread!'),
+ t_att = p.attach('foo2.text', BytesIO(b'Hello, thread!'),
discussion_id=d._id,
thread_id=t._id)
- d_att = p.attach('foo3.text', StringIO('Hello, discussion!'),
+ d_att = p.attach('foo3.text', BytesIO(b'Hello, discussion!'),
discussion_id=d._id)
ThreadLocalORMSession.flush_all()
@@ -202,7 +202,7 @@ def test_attachment_methods():
fs.name = 'file_info'
fs.filename = 'fake.txt'
fs.type = 'text/plain'
- fs.file = StringIO('this is the content of the fake file\n')
+ fs.file = BytesIO(b'this is the content of the fake file\n')
p = t.post(text='test message', forum=None, subject='', file_info=fs)
ThreadLocalORMSession.flush_all()
n = M.Notification.query.get(
@@ -220,12 +220,12 @@ def test_multiple_attachments():
test_file1.name = 'file_info'
test_file1.filename = 'test1.txt'
test_file1.type = 'text/plain'
- test_file1.file = StringIO('test file1\n')
+ test_file1.file = BytesIO(b'test file1\n')
test_file2 = FieldStorage()
test_file2.name = 'file_info'
test_file2.filename = 'test2.txt'
test_file2.type = 'text/plain'
- test_file2.file = StringIO('test file2\n')
+ test_file2.file = BytesIO(b'test file2\n')
d = M.Discussion(shortname='test', name='test')
t = M.Thread.new(discussion_id=d._id, subject='Test Thread')
test_post = t.post('test post')
@@ -243,7 +243,7 @@ def test_add_attachment():
test_file.name = 'file_info'
test_file.filename = 'test.txt'
test_file.type = 'text/plain'
- test_file.file = StringIO('test file\n')
+ test_file.file = BytesIO(b'test file\n')
d = M.Discussion(shortname='test', name='test')
t = M.Thread.new(discussion_id=d._id, subject='Test Thread')
test_post = t.post('test post')
@@ -262,12 +262,12 @@ def test_notification_two_attaches():
fs1.name = 'file_info'
fs1.filename = 'fake.txt'
fs1.type = 'text/plain'
- fs1.file = StringIO('this is the content of the fake file\n')
+ fs1.file = BytesIO(b'this is the content of the fake file\n')
fs2 = FieldStorage()
fs2.name = 'file_info'
fs2.filename = 'fake2.txt'
fs2.type = 'text/plain'
- fs2.file = StringIO('this is the content of the fake file\n')
+ fs2.file = BytesIO(b'this is the content of the fake file\n')
p = t.post(text='test message', forum=None, subject='', file_info=[fs1, fs2])
ThreadLocalORMSession.flush_all()
n = M.Notification.query.get(
@@ -285,7 +285,7 @@ def test_discussion_delete():
d = M.Discussion(shortname='test', name='test')
t = M.Thread.new(discussion_id=d._id, subject='Test Thread')
p = t.post('This is a post')
- p.attach('foo.text', StringIO(''),
+ p.attach('foo.text', BytesIO(b''),
discussion_id=d._id,
thread_id=t._id,
post_id=p._id)
@@ -302,7 +302,7 @@ def test_thread_delete():
d = M.Discussion(shortname='test', name='test')
t = M.Thread.new(discussion_id=d._id, subject='Test Thread')
p = t.post('This is a post')
- p.attach('foo.text', StringIO(''),
+ p.attach('foo.text', BytesIO(b''),
discussion_id=d._id,
thread_id=t._id,
post_id=p._id)
@@ -315,7 +315,7 @@ def test_post_delete():
d = M.Discussion(shortname='test', name='test')
t = M.Thread.new(discussion_id=d._id, subject='Test Thread')
p = t.post('This is a post')
- p.attach('foo.text', StringIO(''),
+ p.attach('foo.text', BytesIO(b''),
discussion_id=d._id,
thread_id=t._id,
post_id=p._id)
diff --git a/Allura/allura/tests/model/test_filesystem.py b/Allura/allura/tests/model/test_filesystem.py
index d757a4f..3ff5727 100644
--- a/Allura/allura/tests/model/test_filesystem.py
+++ b/Allura/allura/tests/model/test_filesystem.py
@@ -21,7 +21,6 @@ from __future__ import unicode_literals
from __future__ import absolute_import
import os
from unittest import TestCase
-from cStringIO import StringIO
from io import BytesIO
from tg import tmpl_context as c
@@ -55,7 +54,7 @@ class TestFile(TestCase):
self.db.fs.chunks.remove()
def test_from_stream(self):
- f = File.from_stream('test1.txt', StringIO('test1'))
+ f = File.from_stream('test1.txt', BytesIO(b'test1'))
self.session.flush()
assert self.db.fs.count() == 1
assert self.db.fs.files.count() == 1
@@ -65,7 +64,7 @@ class TestFile(TestCase):
self._assert_content(f, 'test1')
def test_from_data(self):
- f = File.from_data('test2.txt', 'test2')
+ f = File.from_data('test2.txt', b'test2')
self.session.flush(f)
assert self.db.fs.count() == 1
assert self.db.fs.files.count() == 1
@@ -86,7 +85,7 @@ class TestFile(TestCase):
assert text.startswith('# -*-')
def test_delete(self):
- f = File.from_data('test1.txt', 'test1')
+ f = File.from_data('test1.txt', b'test1')
self.session.flush()
assert self.db.fs.count() == 1
assert self.db.fs.files.count() == 1
@@ -98,8 +97,8 @@ class TestFile(TestCase):
assert self.db.fs.chunks.count() == 0
def test_remove(self):
- File.from_data('test1.txt', 'test1')
- File.from_data('test2.txt', 'test2')
+ File.from_data('test1.txt', b'test1')
+ File.from_data('test2.txt', b'test2')
self.session.flush()
assert self.db.fs.count() == 2
assert self.db.fs.files.count() == 2
@@ -111,7 +110,7 @@ class TestFile(TestCase):
assert self.db.fs.chunks.count() == 1
def test_overwrite(self):
- f = File.from_data('test1.txt', 'test1')
+ f = File.from_data('test1.txt', b'test1')
self.session.flush()
assert self.db.fs.count() == 1
assert self.db.fs.files.count() == 1
@@ -126,7 +125,7 @@ class TestFile(TestCase):
self._assert_content(f, 'test2')
def test_serve_embed(self):
- f = File.from_data('te s\u0b6e1.txt', 'test1')
+ f = File.from_data('te s\u0b6e1.txt', b'test1')
self.session.flush()
with patch('allura.lib.utils.tg.request', Request.blank('/')), \
patch('allura.lib.utils.tg.response', Response()) as response, \
@@ -139,7 +138,7 @@ class TestFile(TestCase):
assert 'Content-Disposition' not in response.headers
def test_serve_embed_false(self):
- f = File.from_data('te s\u0b6e1.txt', 'test1')
+ f = File.from_data('te s\u0b6e1.txt', b'test1')
self.session.flush()
with patch('allura.lib.utils.tg.request', Request.blank('/')), \
patch('allura.lib.utils.tg.response', Response()) as response, \
@@ -175,7 +174,7 @@ class TestFile(TestCase):
def test_not_image(self):
f, t = File.save_image(
'file.txt',
- StringIO('blah'),
+ BytesIO(b'blah'),
thumbnail_size=(16, 16),
square=True,
save_original=True)
@@ -185,7 +184,7 @@ class TestFile(TestCase):
def test_invalid_image(self):
f, t = File.save_image(
'bogus.png',
- StringIO('bogus data here!'),
+ BytesIO(b'bogus data here!'),
thumbnail_size=(16, 16),
square=True,
save_original=True)
diff --git a/ForgeBlog/forgeblog/tests/test_app.py b/ForgeBlog/forgeblog/tests/test_app.py
index 9f32613..26089b9 100644
--- a/ForgeBlog/forgeblog/tests/test_app.py
+++ b/ForgeBlog/forgeblog/tests/test_app.py
@@ -23,10 +23,10 @@ import tempfile
import json
import os
from cgi import FieldStorage
+from io import BytesIO
from nose.tools import assert_equal
from tg import tmpl_context as c
-from cStringIO import StringIO
from ming.orm import ThreadLocalORMSession
from allura import model as M
@@ -112,7 +112,7 @@ class TestBulkExport(object):
test_file1 = FieldStorage()
test_file1.name = 'file_info'
test_file1.filename = 'test_file'
- test_file1.file = StringIO('test file1\n')
+ test_file1.file = BytesIO(b'test file1\n')
p = post.discussion_thread.add_post(text='test comment')
p.add_multiple_attachments(test_file1)
ThreadLocalORMSession.flush_all()
diff --git a/ForgeDiscussion/forgediscussion/tests/functional/test_forum.py b/ForgeDiscussion/forgediscussion/tests/functional/test_forum.py
index 8ad92f6..f4b59d6 100644
--- a/ForgeDiscussion/forgediscussion/tests/functional/test_forum.py
+++ b/ForgeDiscussion/forgediscussion/tests/functional/test_forum.py
@@ -212,13 +212,13 @@ class TestForumMessageHandling(TestController):
assert_equal(FM.ForumPost.query.find().count(), 3)
def test_attach(self):
- self._post('testforum', 'Attachment Thread', 'This is a text file',
+ self._post('testforum', 'Attachment Thread', 'This is text attachment',
message_id='test.attach.100@domain.net',
filename='test.txt',
content_type='text/plain')
- self._post('testforum', 'Test Thread', 'Nothing here',
+ self._post('testforum', 'Test Thread', b'Nothing here',
message_id='test.attach.100@domain.net')
- self._post('testforum', 'Attachment Thread', 'This is a text file',
+ self._post('testforum', 'Attachment Thread', 'This is binary ¶¬¡™£¢¢•º™™¶'.encode('utf-8'),
message_id='test.attach.100@domain.net',
content_type='text/plain')
diff --git a/ForgeDiscussion/forgediscussion/tests/functional/test_forum_admin.py b/ForgeDiscussion/forgediscussion/tests/functional/test_forum_admin.py
index 41682ec..127e2d2 100644
--- a/ForgeDiscussion/forgediscussion/tests/functional/test_forum_admin.py
+++ b/ForgeDiscussion/forgediscussion/tests/functional/test_forum_admin.py
@@ -19,7 +19,6 @@ from __future__ import unicode_literals
from __future__ import absolute_import
import os
import allura
-from StringIO import StringIO
import logging
import PIL
diff --git a/ForgeDiscussion/forgediscussion/tests/test_app.py b/ForgeDiscussion/forgediscussion/tests/test_app.py
index 4e842c3..30f9d90 100644
--- a/ForgeDiscussion/forgediscussion/tests/test_app.py
+++ b/ForgeDiscussion/forgediscussion/tests/test_app.py
@@ -26,10 +26,10 @@ import json
import os
from operator import attrgetter
from cgi import FieldStorage
+from io import BytesIO
from nose.tools import assert_equal
from tg import tmpl_context as c
-from cStringIO import StringIO
from forgediscussion.site_stats import posts_24hr
from ming.orm import ThreadLocalORMSession
@@ -96,7 +96,7 @@ class TestBulkExport(TestDiscussionApiBase):
test_file1 = FieldStorage()
test_file1.name = 'file_info'
test_file1.filename = 'test_file'
- test_file1.file = StringIO('test file1\n')
+ test_file1.file = BytesIO(b'test file1\n')
post.add_attachment(test_file1)
ThreadLocalORMSession.flush_all()
diff --git a/ForgeImporters/forgeimporters/base.py b/ForgeImporters/forgeimporters/base.py
index f4e78c9..9897910 100644
--- a/ForgeImporters/forgeimporters/base.py
+++ b/ForgeImporters/forgeimporters/base.py
@@ -20,6 +20,8 @@ from __future__ import absolute_import
import os
import errno
import logging
+from io import BytesIO
+
import six.moves.urllib.request, six.moves.urllib.parse, six.moves.urllib.error
import six.moves.urllib.request, six.moves.urllib.error, six.moves.urllib.parse
from collections import defaultdict
@@ -29,10 +31,6 @@ from datetime import datetime
import codecs
from six.moves import filter
import six
-try:
- from cStringIO import StringIO
-except ImportError:
- from StringIO import StringIO
from bs4 import BeautifulSoup
from tg import expose, validate, flash, redirect, config
@@ -616,17 +614,17 @@ class ImportAdminExtension(AdminExtension):
sidebar_links.append(link)
-def stringio_parser(page):
+def bytesio_parser(page):
return {
'content-type': page.info()['content-type'],
- 'data': StringIO(page.read()),
+ 'data': BytesIO(page.read()),
}
class File(object):
def __init__(self, url, filename=None):
- extractor = ProjectExtractor(None, url, parser=stringio_parser)
+ extractor = ProjectExtractor(None, url, parser=bytesio_parser)
self.url = url
self.filename = filename or os.path.basename(urlparse(url).path)
# try to get the mime-type from the filename first, because
diff --git a/ForgeImporters/forgeimporters/github/tracker.py b/ForgeImporters/forgeimporters/github/tracker.py
index 7db14e5..b796d3c 100644
--- a/ForgeImporters/forgeimporters/github/tracker.py
+++ b/ForgeImporters/forgeimporters/github/tracker.py
@@ -22,11 +22,7 @@ import logging
from datetime import datetime
from six.moves.urllib.error import HTTPError
import six
-
-try:
- from cStringIO import StringIO
-except ImportError:
- from StringIO import StringIO
+from io import BytesIO
from formencode import validators as fev
from tg import (
@@ -289,7 +285,7 @@ class Attachment(object):
def get_file(self, extractor):
try:
fp_ish = extractor.urlopen(self.url)
- fp = StringIO(fp_ish.read())
+ fp = BytesIO(fp_ish.read())
return fp
except HTTPError as e:
if e.code == 404:
diff --git a/ForgeImporters/forgeimporters/tests/github/test_extractor.py b/ForgeImporters/forgeimporters/tests/github/test_extractor.py
index 617e56b..2f31577 100644
--- a/ForgeImporters/forgeimporters/tests/github/test_extractor.py
+++ b/ForgeImporters/forgeimporters/tests/github/test_extractor.py
@@ -19,15 +19,13 @@ from __future__ import unicode_literals
from __future__ import absolute_import
import json
from unittest import TestCase
+from io import BytesIO
import six.moves.urllib.request, six.moves.urllib.error, six.moves.urllib.parse
from mock import patch, Mock
from ... import github
-# Can't use cStringIO here, because we cannot set attributes or subclass it,
-# and this is needed in mocked_urlopen below
-from StringIO import StringIO
from six.moves import zip
@@ -65,24 +63,24 @@ class TestGitHubProjectExtractor(TestCase):
def mocked_urlopen(self, url):
headers = {}
if url.endswith('/test_project'):
- response = StringIO(json.dumps(self.PROJECT_INFO))
+ response = BytesIO(json.dumps(self.PROJECT_INFO))
elif url.endswith('/issues?state=closed'):
- response = StringIO(json.dumps(self.CLOSED_ISSUES_LIST))
+ response = BytesIO(json.dumps(self.CLOSED_ISSUES_LIST))
elif url.endswith('/issues?state=open'):
- response = StringIO(json.dumps(self.OPENED_ISSUES_LIST))
+ response = BytesIO(json.dumps(self.OPENED_ISSUES_LIST))
headers = {'Link': '</issues?state=open&page=2>; rel="next"'}
elif url.endswith('/issues?state=open&page=2'):
- response = StringIO(json.dumps(self.OPENED_ISSUES_LIST_PAGE2))
+ response = BytesIO(json.dumps(self.OPENED_ISSUES_LIST_PAGE2))
elif url.endswith('/comments'):
- response = StringIO(json.dumps(self.ISSUE_COMMENTS))
+ response = BytesIO(json.dumps(self.ISSUE_COMMENTS))
headers = {'Link': '</comments?page=2>; rel="next"'}
elif url.endswith('/comments?page=2'):
- response = StringIO(json.dumps(self.ISSUE_COMMENTS_PAGE2))
+ response = BytesIO(json.dumps(self.ISSUE_COMMENTS_PAGE2))
elif url.endswith('/events'):
- response = StringIO(json.dumps(self.ISSUE_EVENTS))
+ response = BytesIO(json.dumps(self.ISSUE_EVENTS))
headers = {'Link': '</events?page=2>; rel="next"'}
elif url.endswith('/events?page=2'):
- response = StringIO(json.dumps(self.ISSUE_EVENTS_PAGE2))
+ response = BytesIO(json.dumps(self.ISSUE_EVENTS_PAGE2))
response.info = lambda: headers
return response
@@ -166,9 +164,9 @@ class TestGitHubProjectExtractor(TestCase):
'X-RateLimit-Remaining': '0',
'X-RateLimit-Reset': '1382693522',
}
- response_limit_exceeded = StringIO('{}')
+ response_limit_exceeded = BytesIO(b'{}')
response_limit_exceeded.info = lambda: limit_exceeded_headers
- response_ok = StringIO('{}')
+ response_ok = BytesIO(b'{}')
response_ok.info = lambda: {}
urlopen.side_effect = [response_limit_exceeded, response_ok]
e = github.GitHubProjectExtractor('test_project')
@@ -182,7 +180,7 @@ class TestGitHubProjectExtractor(TestCase):
sleep.reset_mock()
urlopen.reset_mock()
log.warn.reset_mock()
- response_ok = StringIO('{}')
+ response_ok = BytesIO(b'{}')
response_ok.info = lambda: {}
urlopen.side_effect = [response_ok]
e.get_page('fake 2')
@@ -202,11 +200,11 @@ class TestGitHubProjectExtractor(TestCase):
}
def urlopen_side_effect(*a, **kw):
- mock_resp = StringIO('{}')
+ mock_resp = BytesIO(b'{}')
mock_resp.info = lambda: {}
urlopen.side_effect = [mock_resp]
raise six.moves.urllib.error.HTTPError(
- 'url', 403, 'msg', limit_exceeded_headers, StringIO('{}'))
+ 'url', 403, 'msg', limit_exceeded_headers, BytesIO(b'{}'))
urlopen.side_effect = urlopen_side_effect
e = github.GitHubProjectExtractor('test_project')
e.get_page('fake')
diff --git a/ForgeImporters/forgeimporters/tests/github/test_tracker.py b/ForgeImporters/forgeimporters/tests/github/test_tracker.py
index 83da436..3c975a4 100644
--- a/ForgeImporters/forgeimporters/tests/github/test_tracker.py
+++ b/ForgeImporters/forgeimporters/tests/github/test_tracker.py
@@ -136,7 +136,7 @@ class TestTrackerImporter(TestCase):
def test_get_attachments(self):
importer = tracker.GitHubTrackerImporter()
extractor = mock.Mock()
- extractor.urlopen().read.return_value = 'data'
+ extractor.urlopen().read.return_value = b'data'
body = 'hello\n' \
'\r\n' \
''
diff --git a/ForgeSVN/forgesvn/model/svn.py b/ForgeSVN/forgesvn/model/svn.py
index 7370104..b66e1fc 100644
--- a/ForgeSVN/forgesvn/model/svn.py
+++ b/ForgeSVN/forgesvn/model/svn.py
@@ -27,7 +27,7 @@ import time
import operator as op
from subprocess import Popen, PIPE
from hashlib import sha1
-from cStringIO import StringIO
+from io import BytesIO
from datetime import datetime
import tempfile
from shutil import rmtree
@@ -578,7 +578,7 @@ class SVNImplementation(M.RepositoryImplementation):
data = self._svn.cat(
self._url + blob.path(),
revision=self._revision(blob.commit._id))
- return StringIO(data)
+ return BytesIO(data)
def blob_size(self, blob):
try:
diff --git a/ForgeSVN/forgesvn/tests/model/test_repository.py b/ForgeSVN/forgesvn/tests/model/test_repository.py
index 07b14a5..7ef9de7 100644
--- a/ForgeSVN/forgesvn/tests/model/test_repository.py
+++ b/ForgeSVN/forgesvn/tests/model/test_repository.py
@@ -26,8 +26,9 @@ import pkg_resources
from itertools import count, product
from datetime import datetime
from zipfile import ZipFile
-
+from io import BytesIO
from collections import defaultdict
+
from tg import tmpl_context as c, app_globals as g
import mock
from nose.tools import assert_equal, assert_in
@@ -921,8 +922,7 @@ class TestCommit(_TestWithRepo):
return counter.i
counter.i = 0
blobs = defaultdict(counter)
- from cStringIO import StringIO
- return lambda blob: StringIO(str(blobs[blob.path()]))
+ return lambda blob: BytesIO(str(blobs[blob.path()]))
def test_diffs_file_renames(self):
def open_blob(blob):
@@ -935,8 +935,7 @@ class TestCommit(_TestWithRepo):
# moved from /b/b and modified
'/b/a/z': 'Death Star will destroy you\nALL',
}
- from cStringIO import StringIO
- return StringIO(blobs.get(blob.path(), ''))
+ return BytesIO(blobs.get(blob.path(), ''))
self.repo._impl.open_blob = open_blob
self.repo._impl.commit = mock.Mock(return_value=self.ci)
diff --git a/ForgeTracker/forgetracker/import_support.py b/ForgeTracker/forgetracker/import_support.py
index 0fb4103..efe011f 100644
--- a/ForgeTracker/forgetracker/import_support.py
+++ b/ForgeTracker/forgetracker/import_support.py
@@ -21,7 +21,7 @@ from __future__ import absolute_import
import logging
import json
from datetime import datetime
-from cStringIO import StringIO
+from io import BytesIO
# Non-stdlib imports
from tg import tmpl_context as c
@@ -67,7 +67,7 @@ class ResettableStream(object):
def _read_header(self):
if self.buf is None:
data = self.fp.read(self.buf_size)
- self.buf = StringIO(data)
+ self.buf = BytesIO(data)
self.buf_len = len(data)
self.stream_pos = self.buf_len
diff --git a/ForgeTracker/forgetracker/tests/functional/test_root.py b/ForgeTracker/forgetracker/tests/functional/test_root.py
index 3ff8512..a5d9fe8 100644
--- a/ForgeTracker/forgetracker/tests/functional/test_root.py
+++ b/ForgeTracker/forgetracker/tests/functional/test_root.py
@@ -23,7 +23,7 @@ import six.moves.urllib.request, six.moves.urllib.parse, six.moves.urllib.error
import os
import time
import json
-import StringIO
+from io import BytesIO
import allura
import mock
from io import open
@@ -979,11 +979,11 @@ class TestFunctionalController(TrackerTestController):
uploaded = PIL.Image.open(file_path)
r = self.app.get('/bugs/1/attachment/' + filename)
- downloaded = PIL.Image.open(StringIO.StringIO(r.body))
+ downloaded = PIL.Image.open(BytesIO(r.body))
assert uploaded.size == downloaded.size
r = self.app.get('/bugs/1/attachment/' + filename + '/thumb')
- thumbnail = PIL.Image.open(StringIO.StringIO(r.body))
+ thumbnail = PIL.Image.open(BytesIO(r.body))
assert thumbnail.size == (100, 100)
def test_sidebar_static_page(self):
diff --git a/ForgeTracker/forgetracker/tests/test_app.py b/ForgeTracker/forgetracker/tests/test_app.py
index f1cfd2f..5053990 100644
--- a/ForgeTracker/forgetracker/tests/test_app.py
+++ b/ForgeTracker/forgetracker/tests/test_app.py
@@ -21,11 +21,11 @@ import tempfile
import json
import operator
import os
+from io import BytesIO
from nose.tools import assert_equal, assert_true
from tg import tmpl_context as c
from cgi import FieldStorage
-from cStringIO import StringIO
from alluratest.controller import setup_basic_test
from ming.orm import ThreadLocalORMSession
@@ -102,7 +102,7 @@ class TestBulkExport(TrackerTestController):
test_file1 = FieldStorage()
test_file1.name = 'file_info'
test_file1.filename = 'test_file'
- test_file1.file = StringIO('test file1\n')
+ test_file1.file = BytesIO(b'test file1\n')
self.post.add_attachment(test_file1)
ThreadLocalORMSession.flush_all()
diff --git a/ForgeWiki/forgewiki/tests/functional/test_root.py b/ForgeWiki/forgewiki/tests/functional/test_root.py
index b2a6022..77e87ef 100644
--- a/ForgeWiki/forgewiki/tests/functional/test_root.py
+++ b/ForgeWiki/forgewiki/tests/functional/test_root.py
@@ -21,7 +21,7 @@ from __future__ import unicode_literals
from __future__ import print_function
from __future__ import absolute_import
import os
-import StringIO
+from io import BytesIO
import allura
import json
from io import open
@@ -550,11 +550,11 @@ class TestRootController(TestController):
uploaded = PIL.Image.open(file_path)
r = self.app.get('/wiki/TEST/attachment/' + filename)
- downloaded = PIL.Image.open(StringIO.StringIO(r.body))
+ downloaded = PIL.Image.open(BytesIO(r.body))
assert uploaded.size == downloaded.size
r = self.app.get('/wiki/TEST/attachment/' + filename + '/thumb')
- thumbnail = PIL.Image.open(StringIO.StringIO(r.body))
+ thumbnail = PIL.Image.open(BytesIO(r.body))
assert thumbnail.size == (100, 100)
# Make sure thumbnail is absent
diff --git a/ForgeWiki/forgewiki/tests/test_app.py b/ForgeWiki/forgewiki/tests/test_app.py
index bd58982..d7f80e8 100644
--- a/ForgeWiki/forgewiki/tests/test_app.py
+++ b/ForgeWiki/forgewiki/tests/test_app.py
@@ -22,8 +22,8 @@ import tempfile
import json
import operator
import os
+from io import BytesIO
-from cStringIO import StringIO
from nose.tools import assert_equal
from tg import tmpl_context as c
from ming.orm import ThreadLocalORMSession
@@ -96,7 +96,7 @@ class TestBulkExport(object):
self.page.text = 'test_text'
self.page.mod_date = datetime.datetime(2013, 7, 5)
self.page.labels = ['test_label1', 'test_label2']
- self.page.attach('some/path/test_file', StringIO('test string'))
+ self.page.attach('some/path/test_file', BytesIO(b'test string'))
ThreadLocalORMSession.flush_all()
def test_bulk_export_with_attachmetns(self):
[allura] 10/14: [#8354] unicode/byte fixes encountered during
nearly all tests setup
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8354
in repository https://gitbox.apache.org/repos/asf/allura.git
commit 99a5ef8c2b74bd72f0c9d8dbbba14de8000c709b
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Fri Mar 6 17:51:53 2020 -0500
[#8354] unicode/byte fixes encountered during nearly all tests setup
---
Allura/allura/app.py | 2 --
Allura/allura/lib/custom_middleware.py | 2 +-
Allura/allura/lib/helpers.py | 7 ++++++-
Allura/allura/lib/plugin.py | 4 ++--
Allura/allura/lib/widgets/forms.py | 2 +-
Allura/allura/templates/jinja_master/lib.html | 4 ++--
Allura/allura/tests/test_plugin.py | 1 +
7 files changed, 13 insertions(+), 9 deletions(-)
diff --git a/Allura/allura/app.py b/Allura/allura/app.py
index 24ea6ff..effa59c 100644
--- a/Allura/allura/app.py
+++ b/Allura/allura/app.py
@@ -118,8 +118,6 @@ class SitemapEntry(object):
"""
self.label = label
self.className = className
- if url is not None:
- url = url.encode('utf-8')
self.url = url
self.small = small
self.ui_icon = ui_icon
diff --git a/Allura/allura/lib/custom_middleware.py b/Allura/allura/lib/custom_middleware.py
index 061d792..4ae9df1 100644
--- a/Allura/allura/lib/custom_middleware.py
+++ b/Allura/allura/lib/custom_middleware.py
@@ -241,7 +241,7 @@ class SSLMiddleware(object):
try:
request_uri = req.url
- request_uri.decode('ascii')
+ six.ensure_binary(request_uri).decode('ascii')
except UnicodeError:
resp = exc.HTTPBadRequest()
else:
diff --git a/Allura/allura/lib/helpers.py b/Allura/allura/lib/helpers.py
index 79e57b3..3f537ad 100644
--- a/Allura/allura/lib/helpers.py
+++ b/Allura/allura/lib/helpers.py
@@ -433,8 +433,13 @@ def nonce(length=4):
def cryptographic_nonce(length=40):
+ rand_bytes = os.urandom(length)
+ if six.PY2:
+ rand_ints = tuple(map(ord, rand_bytes))
+ else:
+ rand_ints = tuple(rand_bytes)
hex_format = '%.2x' * length
- return hex_format % tuple(map(ord, os.urandom(length)))
+ return hex_format % rand_ints
def random_password(length=20, chars=string.ascii_uppercase + string.digits):
diff --git a/Allura/allura/lib/plugin.py b/Allura/allura/lib/plugin.py
index a1b2629..a79750c 100644
--- a/Allura/allura/lib/plugin.py
+++ b/Allura/allura/lib/plugin.py
@@ -577,8 +577,8 @@ class LocalAuthenticationProvider(AuthenticationProvider):
if salt is None:
salt = ''.join(chr(randint(1, 0x7f))
for i in range(M.User.SALT_LEN))
- hashpass = sha256(salt + password.encode('utf-8')).digest()
- return 'sha256' + salt + b64encode(hashpass)
+ hashpass = sha256((salt + password).encode('utf-8')).digest()
+ return 'sha256' + salt + six.ensure_text(b64encode(hashpass))
def user_project_shortname(self, user):
# "_" isn't valid for subdomains (which project names are used with)
diff --git a/Allura/allura/lib/widgets/forms.py b/Allura/allura/lib/widgets/forms.py
index 888baa5..ab0ef32 100644
--- a/Allura/allura/lib/widgets/forms.py
+++ b/Allura/allura/lib/widgets/forms.py
@@ -82,7 +82,7 @@ class NeighborhoodProjectShortNameValidator(fev.FancyValidator):
"""
if neighborhood is None:
neighborhood = M.Neighborhood.query.get(name=state.full_dict['neighborhood'])
- value = h.really_unicode(value or '').encode('utf-8')
+ value = h.really_unicode(value or '')
self._validate_shortname(value, neighborhood, state)
if check_allowed:
self._validate_allowed(value, neighborhood, state)
diff --git a/Allura/allura/templates/jinja_master/lib.html b/Allura/allura/templates/jinja_master/lib.html
index 0a85183..9114986 100644
--- a/Allura/allura/templates/jinja_master/lib.html
+++ b/Allura/allura/templates/jinja_master/lib.html
@@ -18,13 +18,13 @@
-#}
{% macro csrf() -%}
- {% if request -%}
+ {% if request is defined -%}
{{ request.cookies['_session_id'] or request.environ['_session_id'] }}
{%- endif %}
{%- endmacro %}
{% macro csrf_token() -%}
- {% if request %}
+ {% if request is defined %}
<input name="_session_id" type="hidden" value="{{csrf()}}">
{% endif %}
{%- endmacro %}
diff --git a/Allura/allura/tests/test_plugin.py b/Allura/allura/tests/test_plugin.py
index 8e02010..ce23474 100644
--- a/Allura/allura/tests/test_plugin.py
+++ b/Allura/allura/tests/test_plugin.py
@@ -598,6 +598,7 @@ class TestLocalAuthenticationProvider(object):
ep = self.provider._encode_password
assert ep('test_pass') != ep('test_pass')
assert ep('test_pass', '0000') == ep('test_pass', '0000')
+ assert_equal(ep('test_pass', '0000'), 'sha2560000j7pRjKKZ5L8G0jScZKja9ECmYF2zBV82Mi+E3wkop30=')
def test_set_password_with_old_password(self):
user = Mock()
[allura] 13/14: [#8354] antispam fixes for py3
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8354
in repository https://gitbox.apache.org/repos/asf/allura.git
commit d5f5e01a5436ff084103732958eef81876a0e663
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Mon Mar 9 12:23:50 2020 -0400
[#8354] antispam fixes for py3
---
Allura/allura/lib/utils.py | 27 +++++++++++++++------------
1 file changed, 15 insertions(+), 12 deletions(-)
diff --git a/Allura/allura/lib/utils.py b/Allura/allura/lib/utils.py
index 2494c8b..81d3a26 100644
--- a/Allura/allura/lib/utils.py
+++ b/Allura/allura/lib/utils.py
@@ -213,14 +213,15 @@ class AntiSpam(object):
self.timestamp = timestamp if timestamp else int(time.time())
self.spinner = spinner if spinner else self.make_spinner()
self.timestamp_text = str(self.timestamp)
- self.spinner_text = self._wrap(self.spinner)
+ self.spinner_text = six.ensure_text(self._wrap(self.spinner))
else:
self.request = request
self.timestamp_text = request.params['timestamp']
self.spinner_text = request.params['spinner']
self.timestamp = int(self.timestamp_text)
self.spinner = self._unwrap(self.spinner_text)
- self.spinner_ord = list(map(ord, self.spinner))
+ trans_fn = ord if six.PY2 else int
+ self.spinner_ord = list(map(trans_fn, self.spinner))
self.random_padding = [random.randint(0, 255) for x in self.spinner]
self.honey_class = self.enc(self.spinner_text, css_safe=True)
@@ -241,20 +242,21 @@ class AntiSpam(object):
encoding doesn't use hyphens, underscores, colons, nor periods, so we'll
use these characters to replace its plus, slash, equals, and newline.
'''
- s = base64.b64encode(s)
- s = s.rstrip('=\n')
- s = s.replace('+', '-').replace('/', '_')
- s = 'X' + s
+ s = base64.b64encode(six.ensure_binary(s))
+ s = s.rstrip(b'=\n')
+ s = s.replace(b'+', b'-').replace(b'/', b'_')
+ s = b'X' + s
return s
@staticmethod
def _unwrap(s):
s = s[1:]
- s = s.replace('-', '+').replace('_', '/')
+ s = six.ensure_binary(s)
+ s = s.replace(b'-', b'+').replace(b'_', b'/')
i = len(s) % 4
if i > 0:
- s += '=' * (4 - i)
- s = base64.b64decode(s + '\n')
+ s += b'=' * (4 - i)
+ s = base64.b64decode(s + b'\n')
return s
def enc(self, plain, css_safe=False):
@@ -275,6 +277,7 @@ class AntiSpam(object):
enc = ''.join(six.unichr(p ^ s) for p, s in zip(plain, self.spinner_ord))
enc = six.ensure_binary(enc)
enc = self._wrap(enc)
+ enc = six.ensure_text(enc)
if css_safe:
enc = ''.join(ch for ch in enc if ch.isalpha())
return enc
@@ -315,7 +318,7 @@ class AntiSpam(object):
ip_chunk = '.'.join(octets[0:3])
plain = '%d:%s:%s' % (
timestamp, ip_chunk, tg.config.get('spinner_secret', 'abcdef'))
- return hashlib.sha1(plain).digest()
+ return hashlib.sha1(six.ensure_binary(plain)).digest()
@classmethod
def validate_request(cls, request=None, now=None, params=None):
@@ -339,11 +342,11 @@ class AntiSpam(object):
raise ValueError('Post from the distant past')
if obj.spinner != expected_spinner:
raise ValueError('Bad spinner value')
- for k in new_params.keys():
+ for k in list(new_params.keys()):
try:
new_params[obj.dec(k)] = new_params[k]
new_params.pop(k)
- except:
+ except Exception as ex:
pass
for fldno in range(obj.num_honey):
try:
[allura] 12/14: [#8354] make .ini settings compatible with py3
configparser
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8354
in repository https://gitbox.apache.org/repos/asf/allura.git
commit e2aad504f2f592deadc32b36cacc08e03a5089e8
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Fri Mar 6 17:59:14 2020 -0500
[#8354] make .ini settings compatible with py3 configparser
---
Allura/allura/model/project.py | 8 +++++++-
Allura/allura/tests/functional/test_admin.py | 13 +++++++++++--
Allura/allura/tests/test_tasks.py | 1 +
Allura/development.ini | 24 +++++++++++++++---------
Allura/docker-dev.ini | 3 ++-
Allura/production-docker-example.ini | 3 ++-
Allura/test.ini | 9 ++++-----
7 files changed, 42 insertions(+), 19 deletions(-)
diff --git a/Allura/allura/model/project.py b/Allura/allura/model/project.py
index 5c95d19..2a6c8ff 100644
--- a/Allura/allura/model/project.py
+++ b/Allura/allura/model/project.py
@@ -1103,7 +1103,13 @@ class Project(SearchIndexable, MappedClass, ActivityNode, ActivityObject):
elif not self.is_root:
shortname = self.shortname.split('/')[1]
- return config['bulk_export_filename'].format(project=shortname, date=datetime.utcnow())
+ filename_format = config['bulk_export_filename']
+ if six.PY2:
+ # in py3 ConfigParser requires %% to escape literal "%"
+ # since % has interpolation meaning within ConfigParser
+ # but in py2 the "%%" stays as "%%" so we have to switch it back to a single one
+ filename_format = filename_format.replace('%%', '%')
+ return filename_format.format(project=shortname, date=datetime.utcnow())
def bulk_export_status(self):
'''
diff --git a/Allura/allura/tests/functional/test_admin.py b/Allura/allura/tests/functional/test_admin.py
index cb3a182..2450152 100644
--- a/Allura/allura/tests/functional/test_admin.py
+++ b/Allura/allura/tests/functional/test_admin.py
@@ -19,6 +19,8 @@
from __future__ import unicode_literals
from __future__ import absolute_import
import os
+from datetime import datetime
+
import allura
import pkg_resources
from io import BytesIO
@@ -1030,6 +1032,7 @@ class TestExport(TestController):
assert_in('error', self.webflash(r))
@mock.patch('allura.ext.admin.admin_main.export_tasks')
+ @mock.patch.dict(tg.config, {'bulk_export_filename': '{project}.zip'})
def test_selected_one_tool(self, export_tasks):
r = self.app.post('/admin/export', {'tools': 'wiki'})
assert_in('ok', self.webflash(r))
@@ -1037,6 +1040,7 @@ class TestExport(TestController):
['wiki'], 'test.zip', send_email=True, with_attachments=False)
@mock.patch('allura.ext.admin.admin_main.export_tasks')
+ @mock.patch.dict(tg.config, {'bulk_export_filename': '{project}.zip'})
def test_selected_multiple_tools(self, export_tasks):
r = self.app.post('/admin/export', {'tools': ['wiki', 'wiki2']})
assert_in('ok', self.webflash(r))
@@ -1060,11 +1064,15 @@ class TestExport(TestController):
@td.with_user_project('test-user')
def test_bulk_export_filename_for_user_project(self):
project = M.Project.query.get(shortname='u/test-user')
- assert_equals(project.bulk_export_filename(), 'test-user.zip')
+ filename = project.bulk_export_filename()
+ assert filename.startswith('test-user-backup-{}-'.format(datetime.utcnow().year))
+ assert filename.endswith('.zip')
def test_bulk_export_filename_for_nbhd(self):
project = M.Project.query.get(name='Home Project for Projects')
- assert_equals(project.bulk_export_filename(), 'p.zip')
+ filename = project.bulk_export_filename()
+ assert filename.startswith('p-backup-{}-'.format(datetime.utcnow().year))
+ assert filename.endswith('.zip')
def test_bulk_export_path_for_nbhd(self):
project = M.Project.query.get(name='Home Project for Projects')
@@ -1129,6 +1137,7 @@ class TestRestExport(TestRestApiBase):
@mock.patch('allura.model.project.MonQTask')
@mock.patch('allura.ext.admin.admin_main.AdminApp.exportable_tools_for')
@mock.patch('allura.ext.admin.admin_main.export_tasks.bulk_export')
+ @mock.patch.dict(tg.config, {'bulk_export_filename': '{project}.zip'})
def test_export_ok(self, bulk_export, exportable_tools, MonQTask):
MonQTask.query.get.return_value = None
exportable_tools.return_value = [
diff --git a/Allura/allura/tests/test_tasks.py b/Allura/allura/tests/test_tasks.py
index 8f63478..c2d9117 100644
--- a/Allura/allura/tests/test_tasks.py
+++ b/Allura/allura/tests/test_tasks.py
@@ -641,6 +641,7 @@ class TestExportTasks(unittest.TestCase):
@mock.patch('allura.tasks.export_tasks.shutil')
@mock.patch('allura.tasks.export_tasks.zipdir')
+ @mock.patch.dict(tg.config, {'bulk_export_filename': '{project}.zip'})
@td.with_wiki
def test_bulk_export(self, zipdir, shutil):
M.MonQTask.query.remove()
diff --git a/Allura/development.ini b/Allura/development.ini
index ad56a7a..010b550 100644
--- a/Allura/development.ini
+++ b/Allura/development.ini
@@ -89,8 +89,8 @@ base_url = http://localhost:8080
; `Allura/allura/public/nf/images/` and specify file name below
; logo.link = /
; logo.path = sf10a.png
-; logo.width = 125 ; in px
-; logo.height = 18 ; in px
+; logo.width = 125 (in px)
+; logo.height = 18 (in px)
; Used to uniquify references to static resources, can be a timestamp or any unique value
; This should be updated each time you deploy (or make significant changes, like new tools, new css)
@@ -149,7 +149,8 @@ project_icon_sizes = 16 24 32 48 64 72 90 96 128 135 180 270
; For LDAP see https://forge-allura.apache.org/docs/getting_started/installation.html#using-ldap
;auth.method = ldap
auth.method = local
-auth.remember_for = 365 ; in days, for the "remember me" checkbox on login
+; in days, for the "remember me" checkbox on login
+auth.remember_for = 365
; Customize login/logout URLs only if you have some custom authentication set up.
auth.login_url = /auth/
@@ -163,7 +164,8 @@ auth.max_password_len = 30
; password expiration options (disabled if neither is set)
;auth.pwdexpire.days = 1
-;auth.pwdexpire.before = 1401949912 ; unix timestamp
+; unix timestamp:
+;auth.pwdexpire.before = 1401949912
; if using LDAP, also run `pip install python-ldap` in your Allura environment
@@ -261,7 +263,8 @@ site_admin_project_nbhd = Projects
; for stopforumspam, should be a listed_ip_*_all.txt file
;spam.stopforumspam.ip_addr_file =
;spam.stopforumspam.threshold = 20
-spam.form_post_expiration = 345600 ; 4 days
+; 4 days:
+spam.form_post_expiration = 345600
; Phone verification service: Nexmo Verify
; phone.method = nexmo
@@ -312,7 +315,8 @@ webhook.repo_push.max_hooks = {"git": 3, "hg": 3, "svn": 3}
; To use ssl if and only if a user is logged in:
;force_ssl.logged_in = true
; If you set force_ssl.logged_in, you probably want some URLs to be ssl when logged out:
-;force_ssl.pattern = ^/auth|^/[a-z0-9-]+/import_project/ ; import_project uses a login overlay
+; (import_project uses a login overlay)
+;force_ssl.pattern = ^/auth|^/[a-z0-9-]+/import_project/
; And to permit some URLs to be accessed over http anyway:
; /_test_vars is used when running `paster shell`
;no_redirect.pattern = ^/nf/\d+/_(ew|static)_/|^/rest/|^/nf/tool_icon_css|^/auth/refresh_repo|^/_test_vars
@@ -328,7 +332,8 @@ static.script_name = /nf/%(build_key)s/_static_/
static.url_base = /nf/%(build_key)s/_static_/
; Expires header for "static" resources served through allura (e.g. icons, attachments, /nf/tool_icon_css)
-files_expires_header_secs = 1209600 ; 2 weeks
+; 2 weeks:
+files_expires_header_secs = 1209600
; EasyWidgets settings
; This CORS header is necessary if serving webfonts via a different domain
@@ -432,7 +437,7 @@ scm.view.max_syntax_highlight_bytes = 500000
; If you keep bulk_export_enabled, you should set up your server to securely share bulk_export_path with users somehow
bulk_export_path = /tmp/bulk_export/{nbhd}/{project}
; bulk_export_tmpdir can be set to hold files before building the zip file. Defaults to use bulk_export_path
-bulk_export_filename = {project}-backup-{date:%Y-%m-%d-%H%M%S}.zip
+bulk_export_filename = {project}-backup-{date:%%Y-%%m-%%d-%%H%%M%%S}.zip
; You will need to specify site-specific instructions here for accessing the exported files.
bulk_export_download_instructions = Sample instructions for {project}
@@ -656,7 +661,8 @@ next=main
;
[app:task]
use = main
-override_root = task ; TurboGears will use controllers/task.py as root controller
+; TurboGears will use controllers/task.py as root controller
+override_root = task
diff --git a/Allura/docker-dev.ini b/Allura/docker-dev.ini
index 980d8b8..7a7faaf 100644
--- a/Allura/docker-dev.ini
+++ b/Allura/docker-dev.ini
@@ -66,7 +66,8 @@ forgemail.port = 8825
[app:task]
use = main
-override_root = task ; TurboGears will use controllers/task.py as root controller
+; TurboGears will use controllers/task.py as root controller
+override_root = task
[loggers]
keys = root, allura, sqlalchemy, paste, ew, taskdstatus, timermiddleware, tmw_details
diff --git a/Allura/production-docker-example.ini b/Allura/production-docker-example.ini
index b5ba53c..1824aa1 100644
--- a/Allura/production-docker-example.ini
+++ b/Allura/production-docker-example.ini
@@ -89,7 +89,8 @@ stats.sample_rate = .01
[app:task]
use = main
-override_root = task ; TurboGears will use controllers/task.py as root controller
+; TurboGears will use controllers/task.py as root controller
+override_root = task
diff --git a/Allura/test.ini b/Allura/test.ini
index 82132b5..efe4228 100644
--- a/Allura/test.ini
+++ b/Allura/test.ini
@@ -26,7 +26,8 @@
[app:main]
use = config:development.ini#main
-override_root=basetest_project_root ; TurboGears will use controllers/basetest_project_root.py as root controller
+; TurboGears will use controllers/basetest_project_root.py as root controller
+override_root=basetest_project_root
disable_template_overrides = True
; Use in-memory MongoDB
@@ -80,9 +81,6 @@ support_tool_choices = wiki tickets discussion
; tests expect max length of 40000
markdown_render_max_length = 40000
-; TODO: make this and tests match development.ini
-bulk_export_filename = {project}.zip
-
; TODO: update tests and let this be true
solr.use_new_types = false
@@ -91,7 +89,8 @@ auth.require_email_addr = false
[app:task]
use = main
-override_root = task ; TurboGears will use controllers/task.py as root controller
+; TurboGears will use controllers/task.py as root controller
+override_root = task
;
; Logging goes to a test.log file in current directory
[allura] 09/14: [#8354] make sure jinja2 config values are of the
right type
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8354
in repository https://gitbox.apache.org/repos/asf/allura.git
commit 9ad5f3d8ac13c359232ab533821a0f6ddcd4241d
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Fri Mar 6 17:33:10 2020 -0500
[#8354] make sure jinja2 config values are of the right type
---
Allura/allura/config/app_cfg.py | 3 ++-
Allura/allura/config/middleware.py | 4 ++--
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/Allura/allura/config/app_cfg.py b/Allura/allura/config/app_cfg.py
index fc9eb3e..91f89cf 100644
--- a/Allura/allura/config/app_cfg.py
+++ b/Allura/allura/config/app_cfg.py
@@ -42,6 +42,7 @@ import jinja2
from tg.configuration import AppConfig, config
from markupsafe import Markup
import ew
+from tg.support.converters import asint
import allura
# needed for tg.configuration to work
@@ -103,7 +104,7 @@ class AlluraJinjaRenderer(JinjaRenderer):
auto_reload=config['auto_reload_templates'],
autoescape=True,
bytecode_cache=bcc,
- cache_size=config.get('jinja_cache_size', -1),
+ cache_size=asint(config.get('jinja_cache_size', -1)),
extensions=['jinja2.ext.do', 'jinja2.ext.i18n'])
jinja2_env.install_gettext_translations(tg.i18n)
jinja2_env.filters['datetimeformat'] = helpers.datetimeformat
diff --git a/Allura/allura/config/middleware.py b/Allura/allura/config/middleware.py
index af52386..b46ee1b 100644
--- a/Allura/allura/config/middleware.py
+++ b/Allura/allura/config/middleware.py
@@ -175,9 +175,9 @@ def _make_core_app(root, global_conf, full_stack=True, **app_conf):
# (the Allura [easy_widgets.engines] entry point is named "jinja" (not jinja2) but it doesn't need
# any settings since it is a class that uses the same jinja env as the rest of allura)
**{
- 'jinja2.auto_reload': config['auto_reload_templates'],
+ 'jinja2.auto_reload': asbool(config['auto_reload_templates']),
'jinja2.bytecode_cache': AlluraJinjaRenderer._setup_bytecode_cache(),
- 'jinja2.cache_size': config.get('jinja_cache_size', -1),
+ 'jinja2.cache_size': asint(config.get('jinja_cache_size', -1)),
}
)
# Handle static files (by tool)
[allura] 01/14: [#8354] fix six cookie import
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8354
in repository https://gitbox.apache.org/repos/asf/allura.git
commit 5b371e0933214b5c35abc24d16102d0b68be0d43
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Fri Mar 6 12:29:23 2020 -0500
[#8354] fix six cookie import
---
Allura/allura/lib/decorators.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/Allura/allura/lib/decorators.py b/Allura/allura/lib/decorators.py
index 6a33304..a4b9bd2 100644
--- a/Allura/allura/lib/decorators.py
+++ b/Allura/allura/lib/decorators.py
@@ -21,7 +21,11 @@ import inspect
import sys
import json
import logging
-from six.moves.http_cookies import Cookie
+import six
+if six.PY3:
+ from http.cookies import SimpleCookie as Cookie
+else:
+ from Cookie import Cookie
from collections import defaultdict
from six.moves.urllib.parse import unquote
from datetime import datetime
[allura] 06/14: [#8354] webhelpers -> webhelpers2
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8354
in repository https://gitbox.apache.org/repos/asf/allura.git
commit e270237f614a2152c0dea94700ef200428fc9434
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Fri Mar 6 16:23:16 2020 -0500
[#8354] webhelpers -> webhelpers2
---
Allura/allura/lib/helpers.py | 2 +-
requirements.in | 2 +-
requirements.txt | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/Allura/allura/lib/helpers.py b/Allura/allura/lib/helpers.py
index 6bfe048..79e57b3 100644
--- a/Allura/allura/lib/helpers.py
+++ b/Allura/allura/lib/helpers.py
@@ -65,7 +65,7 @@ import formencode
from jinja2 import Markup
from jinja2.filters import contextfilter, escape, do_filesizeformat
from paste.deploy.converters import asbool, aslist, asint
-from webhelpers import date, html, number, misc, text
+from webhelpers2 import date, text
from webob.exc import HTTPUnauthorized
from allura.lib import exceptions as exc
diff --git a/requirements.in b/requirements.in
index 83f8ccf..d25b8d1 100644
--- a/requirements.in
+++ b/requirements.in
@@ -45,7 +45,7 @@ setproctitle==1.1.9
six==1.12.0
TimerMiddleware==0.5.1
TurboGears2==2.3.12
-WebHelpers==1.3
+WebHelpers2
WebOb==1.7.4
wrapt==1.11.2
diff --git a/requirements.txt b/requirements.txt
index 97d5881..3ad0464 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -93,7 +93,7 @@ urllib3==1.25.3 # via requests
waitress==1.3.0 # via webtest
wcwidth==0.1.7 # via prompt-toolkit
webencodings==0.5.1 # via bleach, html5lib
-webhelpers==1.3
+webhelpers2==2.0
webob==1.7.4
webtest==2.0.33
wrapt==1.11.2
[allura] 11/14: [#8354] avoid error: dictionary changed size during
iteration
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8354
in repository https://gitbox.apache.org/repos/asf/allura.git
commit 921c3b7dd616e1aaced245de7464a98218d0aa15
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Fri Mar 6 17:52:12 2020 -0500
[#8354] avoid error: dictionary changed size during iteration
---
Allura/allura/websetup/bootstrap.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/Allura/allura/websetup/bootstrap.py b/Allura/allura/websetup/bootstrap.py
index 46c8832..9280175 100644
--- a/Allura/allura/websetup/bootstrap.py
+++ b/Allura/allura/websetup/bootstrap.py
@@ -290,7 +290,7 @@ def wipe_database():
continue
log.info('Wiping database %s', database)
db = conn[database]
- for coll in db.collection_names():
+ for coll in list(db.collection_names()):
if coll.startswith('system.'):
continue
log.info('Dropping collection %s:%s', database, coll)
@@ -304,7 +304,7 @@ def clear_all_database_tables():
conn = M.main_doc_session.bind.conn
for db in conn.database_names():
db = conn[db]
- for coll in db.collection_names():
+ for coll in list(db.collection_names()):
if coll == 'system.indexes':
continue
db.drop_collection(coll)
[allura] 14/14: [#8354] modernize run_tests script
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8354
in repository https://gitbox.apache.org/repos/asf/allura.git
commit 33d84464113d1aff2d263b469de92c6a5456bcbd
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Tue Mar 10 11:56:05 2020 -0400
[#8354] modernize run_tests script
---
run_tests | 25 ++++++++++++++++---------
1 file changed, 16 insertions(+), 9 deletions(-)
diff --git a/run_tests b/run_tests
index d3b30b7..c4ff7b4 100755
--- a/run_tests
+++ b/run_tests
@@ -17,17 +17,20 @@
# specific language governing permissions and limitations
# under the License.
+from __future__ import unicode_literals
+from __future__ import absolute_import
+from __future__ import print_function
import argparse
-from copy import copy
from glob import glob
import multiprocessing
from multiprocessing.pool import ThreadPool
import subprocess
import sys
import threading
-import textwrap
import os
+import six
+
CPUS = multiprocessing.cpu_count()
CONCURRENT_SUITES = (CPUS // 4) or CPUS
CONCURRENT_TESTS = CPUS // CONCURRENT_SUITES
@@ -42,10 +45,14 @@ NOT_MULTIPROC_SAFE = [
'ForgeSVN',
]
+# unless you want to mess with changing stdout's own encoding, this works well:
+# py2 gets utf-8 encoded (binary) and py3 gets unicode text
+print_ensured = six.ensure_binary if six.PY2 else six.ensure_text
+
def run_one(cmd, **popen_kwargs):
- cmd_to_show = u'`{}` in {}'.format(cmd, popen_kwargs.get('cwd', '.'))
- print u'{} running {}\n'.format(threading.current_thread(), cmd_to_show)
+ cmd_to_show = '`{}` in {}'.format(cmd, popen_kwargs.get('cwd', '.'))
+ print('{} running {}\n'.format(threading.current_thread(), cmd_to_show))
sys.stdout.flush()
all_popen_kwargs = dict(shell=True, stderr=subprocess.STDOUT,
@@ -56,15 +63,15 @@ def run_one(cmd, **popen_kwargs):
proc = subprocess.Popen(cmd, **all_popen_kwargs)
while proc.poll() is None:
line = proc.stdout.readline()
- sys.stdout.write(line)
- if 'No data to combine' in line:
+ sys.stdout.write(print_ensured(line))
+ if b'No data to combine' in line:
sys.stdout.write('^^ error from "coverage combine" command. Make sure your package has a setup.cfg with coverage settings like other packages\n')
sys.stdout.flush()
# wait for completion and get remainder of output
out_remainder, _ = proc.communicate()
- sys.stdout.write(out_remainder)
+ sys.stdout.write(print_ensured(out_remainder))
sys.stdout.flush()
- print u'finished {}'.format(cmd_to_show)
+ print('finished {}'.format(cmd_to_show))
sys.stdout.flush()
return proc
@@ -108,7 +115,7 @@ def check_packages(packages):
try:
__import__(pkg.lower())
except ImportError:
- print "Not running tests for {}, since it isn't set up".format(pkg)
+ print("Not running tests for {}, since it isn't set up".format(pkg))
else:
yield pkg
[allura] 08/14: [#8354] remove iteritems() usage in templates;
assuming none of these are such huge lists that py2-non-iter will
matter
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8354
in repository https://gitbox.apache.org/repos/asf/allura.git
commit fa63fe6057c222028705dcbdb23b30ff2b69e335
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Fri Mar 6 17:15:08 2020 -0500
[#8354] remove iteritems() usage in templates; assuming none of these are such huge lists that py2-non-iter will matter
---
Allura/allura/ext/admin/templates/project_trove.html | 2 +-
Allura/allura/templates/browse_trove_categories.html | 4 ++--
Allura/allura/templates/reconfirm_auth.html | 2 +-
Allura/allura/templates/repo/merge_request_changed.html | 2 +-
Allura/allura/templates/widgets/forge_form.html | 2 +-
Allura/allura/templates/widgets/page_size.html | 2 +-
Allura/allura/templates/widgets/search_help.html | 2 +-
Allura/allura/templates/widgets/state_field.html | 2 +-
Allura/allura/templates_responsive/widgets/forge_form.html | 2 +-
ForgeImporters/forgeimporters/templates/project_base.html | 4 ++--
ForgeTracker/forgetracker/templates/tracker/admin_fields.html | 2 +-
.../templates/tracker_widgets/custom_field_admin_detail.html | 2 +-
12 files changed, 14 insertions(+), 14 deletions(-)
diff --git a/Allura/allura/ext/admin/templates/project_trove.html b/Allura/allura/ext/admin/templates/project_trove.html
index 23af654..e93aac4 100644
--- a/Allura/allura/ext/admin/templates/project_trove.html
+++ b/Allura/allura/ext/admin/templates/project_trove.html
@@ -50,7 +50,7 @@
<div class="grid-19 trove_add_container">
{% if trove_recommendations[base.shortname] %}
Recommended to choose from:
- {% for trove_id, label in trove_recommendations[base.shortname].iteritems() %}
+ {% for trove_id, label in trove_recommendations[base.shortname].items() %}
<a href="#" data-id="{{ trove_id }}" data-trove="{{ base.shortname }}" class="recommended_trove"><i class="fa fa-plus-circle"></i> {{ label }}</a>
{% endfor %}
<br>
diff --git a/Allura/allura/templates/browse_trove_categories.html b/Allura/allura/templates/browse_trove_categories.html
index 9e2ab07..cbf8c82 100644
--- a/Allura/allura/templates/browse_trove_categories.html
+++ b/Allura/allura/templates/browse_trove_categories.html
@@ -25,10 +25,10 @@
<h3><a href="/categories">(back)</a></h3>
<h3>Trove Categories</h3>
<ul>
- {% for key, value in tree.iteritems() recursive %}
+ {% for key, value in tree.items() recursive %}
<li>{{ key }}</li>
{% if value %}
- <ul>{{ loop(value.iteritems()) }}</ul>
+ <ul>{{ loop(value.items()) }}</ul>
{% endif %}
{% endfor %}
</ul>
diff --git a/Allura/allura/templates/reconfirm_auth.html b/Allura/allura/templates/reconfirm_auth.html
index 26d6fd2..a3ea322 100644
--- a/Allura/allura/templates/reconfirm_auth.html
+++ b/Allura/allura/templates/reconfirm_auth.html
@@ -35,7 +35,7 @@
<input type="submit" value="Submit">
{# include any post params again, so that their original form submit can go through successfully #}
- {% for key, val in request.POST.iteritems() %}
+ {% for key, val in request.POST.items() %}
{% if key != 'password' %}
<input type="hidden" name="{{ key }}" value="{{ val }}">
{% endif %}
diff --git a/Allura/allura/templates/repo/merge_request_changed.html b/Allura/allura/templates/repo/merge_request_changed.html
index dd4059c..479bab3 100644
--- a/Allura/allura/templates/repo/merge_request_changed.html
+++ b/Allura/allura/templates/repo/merge_request_changed.html
@@ -16,7 +16,7 @@
specific language governing permissions and limitations
under the License.
-#}
-{% for field, values in changes.iteritems() %}
+{% for field, values in changes.items() %}
{% if field == 'Description': %}
- **{{ field }}**:
diff --git a/Allura/allura/templates/widgets/forge_form.html b/Allura/allura/templates/widgets/forge_form.html
index 1ff6702..4f182d1 100644
--- a/Allura/allura/templates/widgets/forge_form.html
+++ b/Allura/allura/templates/widgets/forge_form.html
@@ -25,7 +25,7 @@
{% if style == 'wide' %}
{% set extra_width = 4 %}
{% endif %}
- {% if errors and not errors.iteritems and show_errors %}
+ {% if errors and not errors.items and show_errors %}
<div class="grid-{{19 + extra_width}}"><span {{widget.j2_attrs({'class':error_class})}}>{{errors|nl2br}}</span></div>
{% endif %}
{% for field in widget.fields %}
diff --git a/Allura/allura/templates/widgets/page_size.html b/Allura/allura/templates/widgets/page_size.html
index baed90a..623a087 100644
--- a/Allura/allura/templates/widgets/page_size.html
+++ b/Allura/allura/templates/widgets/page_size.html
@@ -17,7 +17,7 @@
under the License.
-#}
<form method="get">
- {% for k,v in widget.url_params.iteritems() %}
+ {% for k,v in widget.url_params.items() %}
<input type="hidden" name="{{k}}" value="{{v}}"/>
{% endfor %}
{% if limit %}
diff --git a/Allura/allura/templates/widgets/search_help.html b/Allura/allura/templates/widgets/search_help.html
index a936e4c..23a85c1 100644
--- a/Allura/allura/templates/widgets/search_help.html
+++ b/Allura/allura/templates/widgets/search_help.html
@@ -24,7 +24,7 @@
{% if fields %}
<p>To search on specific fields, use these field names instead of a general text search. You can group with <code>AND</code> or <code>OR</code>.</p>
<ul>
- {% for fld, descr in fields.iteritems() %}
+ {% for fld, descr in fields.items() %}
<li><b>{{ fld }}:</b>{{ descr }}</li>
{% endfor %}
</ul>
diff --git a/Allura/allura/templates/widgets/state_field.html b/Allura/allura/templates/widgets/state_field.html
index 59f6843..3dca679 100644
--- a/Allura/allura/templates/widgets/state_field.html
+++ b/Allura/allura/templates/widgets/state_field.html
@@ -25,7 +25,7 @@
<span class="{{ error_class }}">{{ ctx.errors }}</span><br/>
{% endif %}
{{ selector.display(css_class=selector_cls, **ctx) }}
- {% for name, field in states.iteritems() %}
+ {% for name, field in states.items() %}
{% set ctx = widget.context_for(field) %}
<div class="{{ field_cls }}" data-name="{{ name }}">
{% if field.show_label %}<label for="{{ ctx.name }}">{{ field.label }}</label><br/>{% endif %}
diff --git a/Allura/allura/templates_responsive/widgets/forge_form.html b/Allura/allura/templates_responsive/widgets/forge_form.html
index 43b72e7..1678a12 100644
--- a/Allura/allura/templates_responsive/widgets/forge_form.html
+++ b/Allura/allura/templates_responsive/widgets/forge_form.html
@@ -29,7 +29,7 @@
{% if enctype %}enctype="{{enctype}}"{% endif %}
{% if target %}target="{{target}}"{% endif %}
action="{{action}}">
- {% if errors and not errors.iteritems and show_errors %}
+ {% if errors and not errors.items and show_errors %}
<div class=""><span {{widget.j2_attrs({'class':error_class})}}>{{errors|nl2br}}</span></div>
{% endif %}
{% for field in widget.fields %}
diff --git a/ForgeImporters/forgeimporters/templates/project_base.html b/ForgeImporters/forgeimporters/templates/project_base.html
index 09b3d80..65b33b8 100644
--- a/ForgeImporters/forgeimporters/templates/project_base.html
+++ b/ForgeImporters/forgeimporters/templates/project_base.html
@@ -113,14 +113,14 @@
{% if c.form_errors['tools'] %}
<div class="error">{{c.form_errors['tools']}}</div>
{% endif %}
- {% for name, tool_importer in importer.tool_importers.iteritems() %}
+ {% for name, tool_importer in importer.tool_importers.items() %}
<div class="tool">
<img src="{{ tool_importer.tool_icon(g.theme, 48) }}" alt="{{ tool_importer.tool_label }} icon">
<label>
<input name="tools" value="{{name}}" type="checkbox"{% if not c.form_errors or name in c.form_values['tools'] %} checked="checked"{% endif %}/>
{{tool_importer.tool_label}}
</label>
- {% for option_name, option_label in tool_importer.tool_option.iteritems() %}
+ {% for option_name, option_label in tool_importer.tool_option.items() %}
<label>
<input name="tool_option" value="{{option_name}}" type="checkbox"{% if not c.form_errors or name in c.form_values['tool_option'] %} checked="checked"{% endif %}/>
{{option_label}}
diff --git a/ForgeTracker/forgetracker/templates/tracker/admin_fields.html b/ForgeTracker/forgetracker/templates/tracker/admin_fields.html
index 33f05a7..751bb85 100644
--- a/ForgeTracker/forgetracker/templates/tracker/admin_fields.html
+++ b/ForgeTracker/forgetracker/templates/tracker/admin_fields.html
@@ -37,7 +37,7 @@
<th>Show in list views (e.g. search results, milestone views)</th>
</tr>
</thead>
- {%for column, full_name in columns.iteritems() %}
+ {%for column, full_name in columns.items() %}
<tr>
<td>{{full_name}}</td> <td><input type="checkbox" name="{{column}}" {%if globals.show_in_search[column]%}checked {%endif%}></td>
</tr>
diff --git a/ForgeTracker/forgetracker/templates/tracker_widgets/custom_field_admin_detail.html b/ForgeTracker/forgetracker/templates/tracker_widgets/custom_field_admin_detail.html
index 4dab13d..ef6e5d0 100644
--- a/ForgeTracker/forgetracker/templates/tracker_widgets/custom_field_admin_detail.html
+++ b/ForgeTracker/forgetracker/templates/tracker_widgets/custom_field_admin_detail.html
@@ -19,7 +19,7 @@
<div class="{{container_cls}}">
{% set ctx=widget.context_for(selector) %}
{{selector.display(css_class=selector_cls, **ctx)}}
- {% for name, field in states.iteritems() %}
+ {% for name, field in states.items() %}
{% set ctx=widget.context_for(field) %}
<div class="{{field_cls}}" data-name="{{name}}">
{{field.display(**ctx)}}
[allura] 03/14: [#8354] six-ify email mime imports
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8354
in repository https://gitbox.apache.org/repos/asf/allura.git
commit 6e0a2ceba266134135cdb572bc5a9fecc8892661
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Fri Mar 6 14:55:16 2020 -0500
[#8354] six-ify email mime imports
---
Allura/allura/lib/mail_util.py | 4 ++--
Allura/allura/tests/test_mail_util.py | 4 ++--
ForgeDiscussion/forgediscussion/tests/functional/test_forum.py | 6 +++---
3 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/Allura/allura/lib/mail_util.py b/Allura/allura/lib/mail_util.py
index cce61a9..bdde46c 100644
--- a/Allura/allura/lib/mail_util.py
+++ b/Allura/allura/lib/mail_util.py
@@ -21,8 +21,8 @@ import re
import logging
import smtplib
import email.feedparser
-from email.MIMEMultipart import MIMEMultipart
-from email.MIMEText import MIMEText
+from six.moves.email_mime_multipart import MIMEMultipart
+from six.moves.email_mime_text import MIMEText
from email import header
import six
diff --git a/Allura/allura/tests/test_mail_util.py b/Allura/allura/tests/test_mail_util.py
index 4b3d21e..80ab989 100644
--- a/Allura/allura/tests/test_mail_util.py
+++ b/Allura/allura/tests/test_mail_util.py
@@ -20,8 +20,8 @@
from __future__ import unicode_literals
from __future__ import absolute_import
import unittest
-from email.MIMEMultipart import MIMEMultipart
-from email.MIMEText import MIMEText
+from six.moves.email_mime_multipart import MIMEMultipart
+from six.moves.email_mime_text import MIMEText
import mock
from nose.tools import raises, assert_equal, assert_false, assert_true, assert_in
diff --git a/ForgeDiscussion/forgediscussion/tests/functional/test_forum.py b/ForgeDiscussion/forgediscussion/tests/functional/test_forum.py
index f4b59d6..900f190 100644
--- a/ForgeDiscussion/forgediscussion/tests/functional/test_forum.py
+++ b/ForgeDiscussion/forgediscussion/tests/functional/test_forum.py
@@ -22,9 +22,9 @@ from __future__ import absolute_import
import mock
import random
import logging
-from email.mime.text import MIMEText
-from email.mime.image import MIMEImage
-from email.mime.multipart import MIMEMultipart
+from six.moves.email_mime_text import MIMEText
+from six.moves.email_mime_image import MIMEImage
+from six.moves.email_mime_multipart import MIMEMultipart
import pkg_resources
import pymongo
[allura] 07/14: [#8354] misc other import-time weird fixes
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8354
in repository https://gitbox.apache.org/repos/asf/allura.git
commit dcdfa27db50d064261aad3717eae971b857eb026
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Fri Mar 6 17:11:43 2020 -0500
[#8354] misc other import-time weird fixes
---
Allura/allura/app.py | 4 ++--
Allura/allura/model/multifactor.py | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/Allura/allura/app.py b/Allura/allura/app.py
index 056c776..24ea6ff 100644
--- a/Allura/allura/app.py
+++ b/Allura/allura/app.py
@@ -19,7 +19,7 @@ from __future__ import unicode_literals
from __future__ import absolute_import
import os
import logging
-from urllib import basejoin
+from six.moves.urllib_parse import urljoin
from io import BytesIO
from collections import defaultdict
from xml.etree import ElementTree as ET
@@ -169,7 +169,7 @@ class SitemapEntry(object):
if callable(lbl):
lbl = lbl(app)
if url is not None:
- url = basejoin(app.url, url)
+ url = urljoin(app.url, url)
return SitemapEntry(lbl, url,
[ch.bind_app(app) for ch in self.children],
className=self.className,
diff --git a/Allura/allura/model/multifactor.py b/Allura/allura/model/multifactor.py
index 433c246..1486a8e 100644
--- a/Allura/allura/model/multifactor.py
+++ b/Allura/allura/model/multifactor.py
@@ -40,7 +40,7 @@ class TotpKey(MappedClass):
_id = FieldProperty(S.ObjectId)
user_id = FieldProperty(S.ObjectId, required=True)
- key = FieldProperty(bytes, required=True)
+ key = FieldProperty(str, required=True)
class RecoveryCode(MappedClass):
[allura] 05/14: [#8354] webhelpers.feedgenerator -> standalone
feedgenerator package
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8354
in repository https://gitbox.apache.org/repos/asf/allura.git
commit 9c46fb0fba9b9d7b9f6dba9c72e9838a2041f206
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Fri Mar 6 16:22:51 2020 -0500
[#8354] webhelpers.feedgenerator -> standalone feedgenerator package
---
Allura/allura/lib/helpers.py | 3 +--
Allura/allura/model/artifact.py | 4 ++--
ForgeActivity/forgeactivity/main.py | 2 +-
ForgeBlog/forgeblog/tests/functional/test_feeds.py | 4 ++--
ForgeTracker/forgetracker/tracker_main.py | 2 +-
requirements.in | 1 +
requirements.txt | 1 +
7 files changed, 9 insertions(+), 8 deletions(-)
diff --git a/Allura/allura/lib/helpers.py b/Allura/allura/lib/helpers.py
index d2506e7..6bfe048 100644
--- a/Allura/allura/lib/helpers.py
+++ b/Allura/allura/lib/helpers.py
@@ -65,8 +65,7 @@ import formencode
from jinja2 import Markup
from jinja2.filters import contextfilter, escape, do_filesizeformat
from paste.deploy.converters import asbool, aslist, asint
-
-from webhelpers import date, feedgenerator, html, number, misc, text
+from webhelpers import date, html, number, misc, text
from webob.exc import HTTPUnauthorized
from allura.lib import exceptions as exc
diff --git a/Allura/allura/model/artifact.py b/Allura/allura/model/artifact.py
index d3a6f1c..cc98b45 100644
--- a/Allura/allura/model/artifact.py
+++ b/Allura/allura/model/artifact.py
@@ -29,7 +29,7 @@ from ming.orm import state, session
from ming.orm import FieldProperty, ForeignIdProperty, RelationProperty
from ming.orm.declarative import MappedClass
from ming.utils import LazyProperty
-from webhelpers import feedgenerator as FG
+import feedgenerator as FG
from allura.lib import helpers as h
from allura.lib import security
@@ -961,7 +961,7 @@ class Feed(MappedClass):
@classmethod
def feed(cls, q, feed_type, title, link, description,
since=None, until=None, page=None, limit=None):
- "Produces webhelper.feedgenerator Feed"
+ "Produces feedgenerator Feed"
d = dict(title=title, link=h.absurl(h.urlquote(link)),
description=description, language='en',
feed_url=request.url)
diff --git a/ForgeActivity/forgeactivity/main.py b/ForgeActivity/forgeactivity/main.py
index 5f1c78b..0ee91d5 100644
--- a/ForgeActivity/forgeactivity/main.py
+++ b/ForgeActivity/forgeactivity/main.py
@@ -30,7 +30,7 @@ from tg import expose, validate, config
from tg.decorators import with_trailing_slash, without_trailing_slash
from paste.deploy.converters import asbool, asint
from webob import exc
-from webhelpers import feedgenerator as FG
+import feedgenerator as FG
from activitystream.storage.mingstorage import Activity
from allura.app import Application
diff --git a/ForgeBlog/forgeblog/tests/functional/test_feeds.py b/ForgeBlog/forgeblog/tests/functional/test_feeds.py
index 2608ea8..f75ea4e 100644
--- a/ForgeBlog/forgeblog/tests/functional/test_feeds.py
+++ b/ForgeBlog/forgeblog/tests/functional/test_feeds.py
@@ -71,10 +71,10 @@ class TestFeeds(TestController):
def test_rss_feed_contains_self_link(self):
r = self.app.get('/blog/feed.rss')
# atom namespace included
- assert_in('<rss xmlns:atom="http://www.w3.org/2005/Atom" version="2.0">', r)
+ assert_in('<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">', r)
# ...and atom:link points to feed url
assert_in('<atom:link href="http://localhost/blog/feed.rss" '
- 'type="application/rss+xml" rel="self"></atom:link>', r)
+ 'rel="self" type="application/rss+xml"></atom:link>', r)
def test_post_feeds(self):
self._post()
diff --git a/ForgeTracker/forgetracker/tracker_main.py b/ForgeTracker/forgetracker/tracker_main.py
index 06cf18e..980ad5e 100644
--- a/ForgeTracker/forgetracker/tracker_main.py
+++ b/ForgeTracker/forgetracker/tracker_main.py
@@ -37,7 +37,7 @@ from formencode import validators
from bson import ObjectId
from bson.son import SON
from bson.errors import InvalidId
-from webhelpers import feedgenerator as FG
+import feedgenerator as FG
from ming import schema
from ming.odm import session
diff --git a/requirements.in b/requirements.in
index 98e4dc7..83f8ccf 100644
--- a/requirements.in
+++ b/requirements.in
@@ -9,6 +9,7 @@ decorator
EasyWidgets>=0.3.3
emoji
faulthandler ; python_version < "3.3"
+feedgenerator
feedparser
# FormEncode may need v2.0 to work past py3.3 or so? https://github.com/formencode/formencode/issues/140
FormEncode
diff --git a/requirements.txt b/requirements.txt
index 04f722d..97d5881 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -26,6 +26,7 @@ easywidgets==0.3.5
emoji==0.5.3
enum34==1.1.6 # via colander, cryptography, traitlets
faulthandler==3.1 ; python_version < "3.3"
+feedgenerator==1.9.1
feedparser==5.2.1
formencode==1.3.1
funcsigs==1.0.2 # via beaker, mock
[allura] 04/14: [#8354] webhelpers.paginate -> standalone paginate
package
Posted by br...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
brondsem pushed a commit to branch db/8354
in repository https://gitbox.apache.org/repos/asf/allura.git
commit 61a80cafa4b9b3339cbf1b3a014d2a429db1be2e
Author: Dave Brondsema <da...@brondsema.net>
AuthorDate: Fri Mar 6 15:31:43 2020 -0500
[#8354] webhelpers.paginate -> standalone paginate package
---
Allura/allura/controllers/site_admin.py | 2 +-
Allura/allura/lib/widgets/form_fields.py | 6 ++++--
Allura/allura/templates/widgets/page_list.html | 2 +-
ForgeBlog/forgeblog/templates/blog_widgets/page_list.html | 4 ++--
requirements.in | 1 +
requirements.txt | 1 +
6 files changed, 10 insertions(+), 6 deletions(-)
diff --git a/Allura/allura/controllers/site_admin.py b/Allura/allura/controllers/site_admin.py
index 14b5ffd..644c9c9 100644
--- a/Allura/allura/controllers/site_admin.py
+++ b/Allura/allura/controllers/site_admin.py
@@ -33,6 +33,7 @@ from tg import request
from formencode import validators, Invalid
from webob.exc import HTTPNotFound, HTTPFound
from ming.odm import ThreadLocalORMSession
+import paginate
from allura.app import SitemapEntry
from allura.lib import helpers as h
@@ -50,7 +51,6 @@ from allura.scripts.delete_projects import DeleteProjects
import allura
from six.moves.urllib.parse import urlparse
-from webhelpers import paginate
import six
from six.moves import range
from six.moves import map
diff --git a/Allura/allura/lib/widgets/form_fields.py b/Allura/allura/lib/widgets/form_fields.py
index 7610325..b60cee1 100644
--- a/Allura/allura/lib/widgets/form_fields.py
+++ b/Allura/allura/lib/widgets/form_fields.py
@@ -23,7 +23,7 @@ import json
import logging
from formencode import validators as fev
-from webhelpers import paginate
+import paginate
import ew as ew_core
import ew.jinja2_ew as ew
@@ -312,7 +312,9 @@ class PageList(ew_core.Widget):
params['page'] = page - page_offset
return url(request.path, params)
return paginate.Page(list(range(count)), page + page_offset, int(limit),
- url=page_url)
+ url=page_url,
+ url_maker=lambda pagenum: '?page={}&limit={}'.format(pagenum-1, limit)
+ )
def prepare_context(self, context):
context = super(PageList, self).prepare_context(context)
diff --git a/Allura/allura/templates/widgets/page_list.html b/Allura/allura/templates/widgets/page_list.html
index 2621601..fbe25a2 100644
--- a/Allura/allura/templates/widgets/page_list.html
+++ b/Allura/allura/templates/widgets/page_list.html
@@ -23,7 +23,7 @@
{% if pager_output.strip() %}
<div>
<div class="page_list">
- {{ pager_output }}
+ {{ pager_output|safe }}
</div>
<div class="clear"></div>
</div>
diff --git a/ForgeBlog/forgeblog/templates/blog_widgets/page_list.html b/ForgeBlog/forgeblog/templates/blog_widgets/page_list.html
index bbc9725..0681277 100644
--- a/ForgeBlog/forgeblog/templates/blog_widgets/page_list.html
+++ b/ForgeBlog/forgeblog/templates/blog_widgets/page_list.html
@@ -19,10 +19,10 @@
<div style="margin: 10px">
{% set paginator = widget.paginator(count, page, limit) %}
<div style="float: right">
- {{paginator.pager('$link_previous', symbol_previous='Newer Entries >>')}}
+ {{paginator.pager('$link_previous', symbol_previous='Newer Entries >>')|safe}}
</div>
<div style="float: left">
- {{paginator.pager('$link_next', symbol_next='<< Older Entries')}}
+ {{paginator.pager('$link_next', symbol_next='<< Older Entries')|safe}}
</div>
<div style="clear: both"></div>
</div>
diff --git a/requirements.in b/requirements.in
index fdb3fd4..98e4dc7 100644
--- a/requirements.in
+++ b/requirements.in
@@ -21,6 +21,7 @@ MarkupSafe
Ming==0.5.6
# oauth2 doesn't have py3.6 support. There's a fork with fixes but no pypi releases I can find. https://github.com/joestump/python-oauth2/issues/223
oauth2
+paginate
Paste
PasteDeploy
PasteScript
diff --git a/requirements.txt b/requirements.txt
index 04c5a8e..04f722d 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -51,6 +51,7 @@ mock==3.0.5
nose==1.3.7
oauth2==1.9.0.post1
oauthlib==3.0.2 # via requests-oauthlib
+paginate==0.5.6
paste==3.1.0
pastedeploy==2.0.1
pastescript==3.1.0