You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Julian Foad <ju...@apache.org> on 2019/10/07 09:13:19 UTC
Re: PMCs: any Hackathon requests? (deadline 11 October)
Moving this thread to dev@ as it should be public [1].
See my responses inline.
> Sally Khudairi <sk...@apache.org> wrote:
>> We received an email from hackCBS 2.0, a hackathon that will be taking place 19-20 October at the University of Delhi with 700+ students.
>>
>> They are interested in our participation by providing a list of tasks from various Apache projects for them to work on.
>>
>> If you would like to submit a list of work areas, please let me know your interest and forward your list(s) no later than 11 October.
>
> Johan Corveleyn wrote:
>> Nathan Hartman wrote:
>>> We should totally take advantage of this hackathon request, if at all feasible.
>>
>> I'm not sure if we will be able to get enough attention, but there's
>> no harm in trying. Some ideas are listed on
>> http://subversion.apache.org/ideas.html.
>> But of course there are many other possible suggestions (perhaps some
>> more fresh ideas, appealing to University students, ...).
>
> I propose to reply to Sally's message with something along the
> following lines, pending thoughts / suggestions / criticisms...
> Opportunities for new devs must not slip through our fingers!
>
> [[[
> [...]
> The Apache Subversion PMC is always interested in attracting new
> developers to the project.
>
> Between our 'ideas' page and issue tracker, we have some opportunities
> [...]
> ]]]
Johan Corveleyn wrote:
> Nathan Hartman wrote:
>> Daniel Shahaf wrote:
>>> I read Sally's request as asking for a (reviewed/edited) list of tasks,
>>> not just asking each project to tell her where their issue tracker is. Seeing
>>> as the duration of the hackathon is two days, perhaps linking to the
>>> "bite-sized" label could work, though?
>>
>> I agree.
>>
>> Can we come up with 2 or 3 suggested tasks? I'll go first: I nominate
>> https://issues.apache.org/jira/browse/SVN-2858.
>
> Ah, that one :-). Note that Neels Hofmeyr did work on a design for
> that back in 2011 (see
> https://svn.apache.org/repos/asf/subversion/trunk/notes/hold). You can
> also find several discussion threads about it in 2011 if you search
> for "svn:hold".
That's a bad choice because it's a new feature that isn't clearly agreed
and defined, so it requires proposing and debating a design and all its
interactions with existing behaviour.
We're in a period where we should be working instead toward stability
and availability.
> Another potential task might be: overhaul of our website, for which I
> think you had some ideas, right, Nathan? Maybe someone with web design
> skills might be able to do part of the work (under your guidance)?
A subtask of that might be suitable.
> Another suggestion:
> https://issues.apache.org/jira/browse/SVN-4753 (SVN viewspec feature)
> We now have "svn info --x-viewspec classic|svn11". We need a
> corresponding way to "consume" viewspecs (e.g. "svn update
> --apply-viewspec some.spec" [...]
I would consider that more of a missing feature, and so potentially
something I'd like us to do, but unfortunately it requires deep
understanding and careful changing of the core APIs.
- Julian
[1] This discussion should have been public from the start. Generally
one would need to ask all participants before making it public, but in
this case I'm confident we'll agree and it's just that unfortunately
Sally sent it to private@ by default, as an easy way of contacting all
the PMCs. See a long thread about the problem, with nobody agreeing to
do anything about it, on dev@community.apache.org 2018-08-23 "Keeping
PMC communications public when possible":
https://lists.apache.org/thread.html/9faff8ffc1e5017b241c47a4e62bf55a5b8b2e9365bb3f48f9700ec4@%3Cdev.community.apache.org%3E
Re: SWIG Python bindings build with SWIG 4.0
Posted by Branko Čibej <br...@apache.org>.
On 21.10.2019 04:11, Branko Čibej wrote:
> You're right. Fixed in r1868674, this makes the Python bindings tests
> work on my laptop, with both the system Python 2 and the Homebrewed
> Python 3. I've forced the tests to run on the buildslave.
I merged trunk to swig-py3 to get the fixes from trunk for the regular
test suite.
-- Brane
Re: SWIG Python bindings build with SWIG 4.0
Posted by Branko Čibej <br...@apache.org>.
On 21.10.2019 01:15, Yasuhito FUTATSUKI wrote:
> On 2019/10/20 21:58, Branko Čibej wrote:
>> On 20.10.2019 06:35, Yasuhito FUTATSUKI wrote:
>>> On 2019/10/20 8:37, Branko Čibej wrote:
>>>> On 20.10.2019 01:10, Yasuhito FUTATSUKI wrote:
>>>>> On 2019/10/20 6:59, Branko Čibej wrote:
>>>>>> On 19.10.2019 23:06, Branko Čibej wrote:
>>>>>>> On 19.10.2019 19:55, Branko Čibej wrote:
>>>>>>>> On 19.10.2019 11:45, Yasuhito FUTATSUKI wrote:
>>>>>>>>> On 2019/10/18 8:39, Branko Čibej wrote:
>>>>>>>>>> On 17.10.2019 23:46, Branko Čibej wrote:
>>>>>>>>>>> On 17.10.2019 23:01, Daniel Shahaf wrote:
>>>>>>>>>>>> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>>>>>>>>>>>>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>>>>>>>>>>>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>>>>>>>>>>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>>>>>>>>>>>>> Which reminds me: it'd be nice to set up a buildbot to
>>>>>>>>>>>>>>>> run the
>>>>>>>>>>>>>>>> build and
>>>>>>>>>>>>>>>> test process under Python 3. Any committer can edit the
>>>>>>>>>>>>>>>> buildbot
>>>>>>>>>>>>>>>> scripts[1], but the question is which of the buildbot
>>>>>>>>>>>>>>>> slaves
>>>>>>>>>>>>>>>> has Python 3
>>>>>>>>>>>>>>>> installed?
>>>>>>>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Been there, done that, bought the DVD.
>>>>>>>>>>>>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing
>>>>>>>>>>>>>> on that
>>>>>>>>>>>>>> bot? They (currently) fail under python3.
>>>>>>>>>>>>> I don't know. It's possible, I suppose, that the
>>>>>>>>>>>>> activation of
>>>>>>>>>>>>> the
>>>>>>>>>>>>> python3 virtual environment has no effect on the test
>>>>>>>>>>>>> driver. It
>>>>>>>>>>>>> might
>>>>>>>>>>>>> not be a bad idea to have the tests print the Python version
>>>>>>>>>>>>> in the
>>>>>>>>>>>>> summary and possibly in the log of every test case.
>>>>>>>>>>>> What's the value of $(PYTHON) in Makefile? That's what the
>>>>>>>>>>>> «check»
>>>>>>>>>>>> target uses.
>>>>>>>>>>> Yep, apparently that's the bug ... I'm testing script changes
>>>>>>>>>>> now,
>>>>>>>>>>> along
>>>>>>>>>>> with r1868566 for good measure.
>>>>>>>>>> I've set up this:
>>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
>>>>>>>>>>
>>>>>>>>>> It will build and test the core libraries and swig-py bindings
>>>>>>>>>> with
>>>>>>>>>> Python 3 on the swig-py3 branch.
>>>>>>>>>>
>>>>>>>>>> I hope ... build #0 running as we speak.
>>>>>>>>> Unfortunately build #2, which ran after upgrading SWIG and
>>>>>>>>> Python,
>>>>>>>>> failed
>>>>>>>>> to build SWIG Python bindings because of SWIG 4.0, as reported on
>>>>>>>>> SVN-4818.
>>>>>>>>>
>>>>>>>>> This also affects on trunk with SWIG 4.0.
>>>>>>>>> (e.g.
>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-full/builds/2418)
>>>>>>>>>
>>>>>>>>> With attached patch on trunk
>>>>>>>>> (trunk_build_with_swig4_patch.txt) and
>>>>>>>>> on swig-py3 (swig_py3_build_with_swig4_patch.txt), *.so can be
>>>>>>>>> produced,
>>>>>>>>> but the modules don't work correctly.
>>>>>>>>>
>>>>>>>>> It seems they were caused by incompatibility of Python code for
>>>>>>>>> proxy
>>>>>>>>> object generated by SWIG, and it can not be resolved so
>>>>>>>>> simple....
>>>>>>>>> (importlib module vs simply use 'import', absense of
>>>>>>>>> _swig_setattr()
>>>>>>>>> and _swig_getattr(), etc.)
>>>>>>>> My bad. I downgraded to Swig 3.0.12 and restarted the failed
>>>>>>>> builds.
>>>>>>>
>>>>>>> Hm, that did not really help. On the swig-py3 branch:
>>>>>>>
>>>>>>> /bin/sh: line 1: 62481 Segmentation fault: 11
>>>>>>> /srv/virtualenv-3/bin/python
>>>>>>> /srv/buildbot/slave/svn-x64-macosx-local-python3-swig/build/subversion/bindings/swig/python/tests/run_all.py
>>>>>>>
>>>>>>>
>>>>>>> make: *** [check-swig-py] Error 139
>>>>>>>
>>>>>>>
>>>>>>> So, the Python bindings seem to have crashed the Python 3
>>>>>>> interpreter on
>>>>>>> macOS?
>>>>>>
>>>>>> Yes, they did:
>>>>>>
>>>>>> Process 28278 stopped
>>>>>> * thread #2, queue = 'com.apple.main-thread', stop reason =
>>>>>> EXC_BAD_ACCESS (code=1, address=0x48)
>>>>>> frame #0: 0x00007fff373ab746 Python`PyErr_Restore + 61
>>>>>> Python`PyErr_Restore:
>>>>>> -> 0x7fff373ab746 <+61>: movq 0x48(%rbx), %rdi
>>>>>> 0x7fff373ab74a <+65>: movq 0x50(%rbx), %r13
>>>>>> 0x7fff373ab74e <+69>: movq 0x58(%rbx), %r12
>>>>>> 0x7fff373ab752 <+73>: movq %r15, 0x48(%rbx)
>>>>>> Target 0: (Python) stopped.
>>>>>>
>>>>>> frame #3: 0x0000000103800fea _core.so`PyInit__core at
>>>>>> core.c:40158:12
>>>>>> 40155 m = Py_InitModule((char *) SWIG_name, SwigMethods);
>>>>>> 40156 #endif
>>>>>> 40157
>>>>>> -> 40158 md = d = PyModule_GetDict(m);
>>>>>> 40159 (void)md;
>>>>>> 40160
>>>>>> 40161 SWIG_InitializeModule(0);
>>>>>> (lldb) print m
>>>>>> (PyObject *) $0 = 0x000000010279ad70
>>>>>> (lldb) print *m
>>>>>> (PyObject) $1 = {
>>>>>> ob_refcnt = 785
>>>>>> ob_type = 0x000000010026ea30
>>>>>> }
>>>>>
>>>>> I guess the _core module linked incompatible Python library to
>>>>> execute.
>>>>>
>>>>> on
>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig/builds/6/steps/Build/logs/stdio
>>>>>
>>>>>
>>>>> :
>>>>> [[[
>>>>> checking for compiling Python extensions... clang
>>>>> checking for linking Python extensions... clang -bundle -undefined
>>>>> dynamic_lookup -isysroot
>>>>> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
>>>>>
>>>>>
>>>>> -F/usr/local/opt/python/Frameworks -framework Python
>>>>> checking for linking Python libraries... -bundle -undefined
>>>>> dynamic_lookup -F/usr/local/opt/python/Frameworks -framework Python
>>>>> ]]]
>>>>
>>>>
>>>> Good catch. This certainly looks correct, it's where Homebrew installs
>>>> python. However ...
>>>>
>>>> $ otool -L /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so
>>>> /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so:
>>>> /System/Library/Frameworks/Python.framework/Versions/2.7/Python
>>>> (compatibility version 2.7.0, current version 2.7.10)
>>>> ...
>>>>
>>>>
>>>> That is wrong. Looks like we'll have to set DYLD_LIBRARY_PATH so
>>>> that it
>>>> finds the Python 3 installation.
>>
>> (Actually that would be DYLD_FRAMEWORK_PATH.)
>>
>>> ... or have to use Run-Path Dependent Libraries feature on link time,
>>> if DYLD_LIBRARY_PATH cannot work well due to SIP.
>>>
>>> https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/RunpathDependentLibraries.html
>>>
>>>
>>>
>>> The link option in log above comes from result of
>>> 'build/get-py-info.py --link', so should we tweak build/get-py-info.py,
>>> or add option to configure to pass Python dynamic library path?
>>
>>
>> Using RPATH for the Python framework on macOS would be the right
>> solution, but it doesn't solve the problem. It looks like we need some
>> magic incantation on macOS to explicitly pick the Python 3 framework;
>> using just the -F option doesn't work as expected, the path to the
>> system default Python 2.7 framework is hard-coded in to the extension
>> libraries.
>
> Umm.... it seems extension libraries actually don't need to link
> "Python" dynamic library.
>
> e.g.
> $ otool -L `python -c 'import _socket;print(_socket.__file__)'`
> /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_socket.so:
>
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0,
> current version 1252.250.1)
You're right. Fixed in r1868674, this makes the Python bindings tests
work on my laptop, with both the system Python 2 and the Homebrewed
Python 3. I've forced the tests to run on the buildslave.
-- Brane
Re: SWIG Python bindings build with SWIG 4.0
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/10/20 21:58, Branko Čibej wrote:
> On 20.10.2019 06:35, Yasuhito FUTATSUKI wrote:
>> On 2019/10/20 8:37, Branko Čibej wrote:
>>> On 20.10.2019 01:10, Yasuhito FUTATSUKI wrote:
>>>> On 2019/10/20 6:59, Branko Čibej wrote:
>>>>> On 19.10.2019 23:06, Branko Čibej wrote:
>>>>>> On 19.10.2019 19:55, Branko Čibej wrote:
>>>>>>> On 19.10.2019 11:45, Yasuhito FUTATSUKI wrote:
>>>>>>>> On 2019/10/18 8:39, Branko Čibej wrote:
>>>>>>>>> On 17.10.2019 23:46, Branko Čibej wrote:
>>>>>>>>>> On 17.10.2019 23:01, Daniel Shahaf wrote:
>>>>>>>>>>> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>>>>>>>>>>>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>>>>>>>>>>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>>>>>>>>>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>>>>>>>>>>>> Which reminds me: it'd be nice to set up a buildbot to
>>>>>>>>>>>>>>> run the
>>>>>>>>>>>>>>> build and
>>>>>>>>>>>>>>> test process under Python 3. Any committer can edit the
>>>>>>>>>>>>>>> buildbot
>>>>>>>>>>>>>>> scripts[1], but the question is which of the buildbot slaves
>>>>>>>>>>>>>>> has Python 3
>>>>>>>>>>>>>>> installed?
>>>>>>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Been there, done that, bought the DVD.
>>>>>>>>>>>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing
>>>>>>>>>>>>> on that
>>>>>>>>>>>>> bot? They (currently) fail under python3.
>>>>>>>>>>>> I don't know. It's possible, I suppose, that the activation of
>>>>>>>>>>>> the
>>>>>>>>>>>> python3 virtual environment has no effect on the test
>>>>>>>>>>>> driver. It
>>>>>>>>>>>> might
>>>>>>>>>>>> not be a bad idea to have the tests print the Python version
>>>>>>>>>>>> in the
>>>>>>>>>>>> summary and possibly in the log of every test case.
>>>>>>>>>>> What's the value of $(PYTHON) in Makefile? That's what the
>>>>>>>>>>> «check»
>>>>>>>>>>> target uses.
>>>>>>>>>> Yep, apparently that's the bug ... I'm testing script changes
>>>>>>>>>> now,
>>>>>>>>>> along
>>>>>>>>>> with r1868566 for good measure.
>>>>>>>>> I've set up this:
>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
>>>>>>>>>
>>>>>>>>> It will build and test the core libraries and swig-py bindings
>>>>>>>>> with
>>>>>>>>> Python 3 on the swig-py3 branch.
>>>>>>>>>
>>>>>>>>> I hope ... build #0 running as we speak.
>>>>>>>> Unfortunately build #2, which ran after upgrading SWIG and Python,
>>>>>>>> failed
>>>>>>>> to build SWIG Python bindings because of SWIG 4.0, as reported on
>>>>>>>> SVN-4818.
>>>>>>>>
>>>>>>>> This also affects on trunk with SWIG 4.0.
>>>>>>>> (e.g.
>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-full/builds/2418)
>>>>>>>>
>>>>>>>> With attached patch on trunk (trunk_build_with_swig4_patch.txt) and
>>>>>>>> on swig-py3 (swig_py3_build_with_swig4_patch.txt), *.so can be
>>>>>>>> produced,
>>>>>>>> but the modules don't work correctly.
>>>>>>>>
>>>>>>>> It seems they were caused by incompatibility of Python code for
>>>>>>>> proxy
>>>>>>>> object generated by SWIG, and it can not be resolved so simple....
>>>>>>>> (importlib module vs simply use 'import', absense of
>>>>>>>> _swig_setattr()
>>>>>>>> and _swig_getattr(), etc.)
>>>>>>> My bad. I downgraded to Swig 3.0.12 and restarted the failed builds.
>>>>>>
>>>>>> Hm, that did not really help. On the swig-py3 branch:
>>>>>>
>>>>>> /bin/sh: line 1: 62481 Segmentation fault: 11
>>>>>> /srv/virtualenv-3/bin/python
>>>>>> /srv/buildbot/slave/svn-x64-macosx-local-python3-swig/build/subversion/bindings/swig/python/tests/run_all.py
>>>>>>
>>>>>> make: *** [check-swig-py] Error 139
>>>>>>
>>>>>>
>>>>>> So, the Python bindings seem to have crashed the Python 3
>>>>>> interpreter on
>>>>>> macOS?
>>>>>
>>>>> Yes, they did:
>>>>>
>>>>> Process 28278 stopped
>>>>> * thread #2, queue = 'com.apple.main-thread', stop reason =
>>>>> EXC_BAD_ACCESS (code=1, address=0x48)
>>>>> frame #0: 0x00007fff373ab746 Python`PyErr_Restore + 61
>>>>> Python`PyErr_Restore:
>>>>> -> 0x7fff373ab746 <+61>: movq 0x48(%rbx), %rdi
>>>>> 0x7fff373ab74a <+65>: movq 0x50(%rbx), %r13
>>>>> 0x7fff373ab74e <+69>: movq 0x58(%rbx), %r12
>>>>> 0x7fff373ab752 <+73>: movq %r15, 0x48(%rbx)
>>>>> Target 0: (Python) stopped.
>>>>>
>>>>> frame #3: 0x0000000103800fea _core.so`PyInit__core at core.c:40158:12
>>>>> 40155 m = Py_InitModule((char *) SWIG_name, SwigMethods);
>>>>> 40156 #endif
>>>>> 40157
>>>>> -> 40158 md = d = PyModule_GetDict(m);
>>>>> 40159 (void)md;
>>>>> 40160
>>>>> 40161 SWIG_InitializeModule(0);
>>>>> (lldb) print m
>>>>> (PyObject *) $0 = 0x000000010279ad70
>>>>> (lldb) print *m
>>>>> (PyObject) $1 = {
>>>>> ob_refcnt = 785
>>>>> ob_type = 0x000000010026ea30
>>>>> }
>>>>
>>>> I guess the _core module linked incompatible Python library to execute.
>>>>
>>>> on
>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig/builds/6/steps/Build/logs/stdio
>>>>
>>>> :
>>>> [[[
>>>> checking for compiling Python extensions... clang
>>>> checking for linking Python extensions... clang -bundle -undefined
>>>> dynamic_lookup -isysroot
>>>> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
>>>>
>>>> -F/usr/local/opt/python/Frameworks -framework Python
>>>> checking for linking Python libraries... -bundle -undefined
>>>> dynamic_lookup -F/usr/local/opt/python/Frameworks -framework Python
>>>> ]]]
>>>
>>>
>>> Good catch. This certainly looks correct, it's where Homebrew installs
>>> python. However ...
>>>
>>> $ otool -L /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so
>>> /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so:
>>> /System/Library/Frameworks/Python.framework/Versions/2.7/Python
>>> (compatibility version 2.7.0, current version 2.7.10)
>>> ...
>>>
>>>
>>> That is wrong. Looks like we'll have to set DYLD_LIBRARY_PATH so that it
>>> finds the Python 3 installation.
>
> (Actually that would be DYLD_FRAMEWORK_PATH.)
>
>> ... or have to use Run-Path Dependent Libraries feature on link time,
>> if DYLD_LIBRARY_PATH cannot work well due to SIP.
>>
>> https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/RunpathDependentLibraries.html
>>
>>
>> The link option in log above comes from result of
>> 'build/get-py-info.py --link', so should we tweak build/get-py-info.py,
>> or add option to configure to pass Python dynamic library path?
>
>
> Using RPATH for the Python framework on macOS would be the right
> solution, but it doesn't solve the problem. It looks like we need some
> magic incantation on macOS to explicitly pick the Python 3 framework;
> using just the -F option doesn't work as expected, the path to the
> system default Python 2.7 framework is hard-coded in to the extension
> libraries.
Umm.... it seems extension libraries actually don't need to link
"Python" dynamic library.
e.g.
$ otool -L `python -c 'import _socket;print(_socket.__file__)'`
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_socket.so:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
_core.so built on FreeBSD:
$ ldd subversion/bindings/swig/python/.libs/_core.so
subversion/bindings/swig/python/.libs/_core.so:
libsvn_swig_py-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py/.libs/libsvn_swig_py-1.so.0 (0x801298000)
libsvn_client-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_client/.libs/libsvn_client-1.so.0 (0x8014ac000)
libsvn_wc-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_wc/.libs/libsvn_wc-1.so.0 (0x801730000)
libsvn_diff-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_diff/.libs/libsvn_diff-1.so.0 (0x8019d5000)
libsvn_ra-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_ra/.libs/libsvn_ra-1.so.0 (0x801bec000)
libsvn_ra_local-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_ra_local/.libs/libsvn_ra_local-1.so.0 (0x801df8000)
libsvn_repos-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_repos/.libs/libsvn_repos-1.so.0 (0x802001000)
libsvn_fs-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_fs/.libs/libsvn_fs-1.so.0 (0x80223c000)
libsvn_fs_fs-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_fs_fs/.libs/libsvn_fs_fs-1.so.0 (0x802449000)
libsvn_fs_x-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_fs_x/.libs/libsvn_fs_x-1.so.0 (0x802698000)
libsvn_fs_util-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_fs_util/.libs/libsvn_fs_util-1.so.0 (0x8028e7000)
libsvn_ra_svn-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_ra_svn/.libs/libsvn_ra_svn-1.so.0 (0x802ae9000)
libsvn_ra_serf-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_ra_serf/.libs/libsvn_ra_serf-1.so.0 (0x802d0d000)
libserf-1.so.1 => /usr/local/lib/libserf-1.so.1 (0x802f3e000)
libsvn_delta-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_delta/.libs/libsvn_delta-1.so.0 (0x803159000)
libsvn_subr-1.so.0 => /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/libsvn_subr/.libs/libsvn_subr-1.so.0 (0x803377000)
libaprutil-1.so.0 => /usr/local/lib/libaprutil-1.so.0 (0x803608000)
libexpat.so.1 => /usr/local/lib/libexpat.so.1 (0x803835000)
libz.so.6 => /lib/libz.so.6 (0x803a5f000)
libsqlite3.so.0 => /usr/local/lib/libsqlite3.so.0 (0x803c77000)
libmagic.so.4 => /usr/lib/libmagic.so.4 (0x803fd9000)
liblz4.so.1 => /usr/local/lib/liblz4.so.1 (0x8041fc000)
libutf8proc.so.2 => /usr/local/lib/libutf8proc.so.2 (0x80442b000)
libapr-1.so.0 => /usr/local/lib/libapr-1.so.0 (0x804675000)
libintl.so.8 => /usr/local/lib/libintl.so.8 (0x8048af000)
libthr.so.3 => /lib/libthr.so.3 (0x804aba000)
libc.so.7 => /lib/libc.so.7 (0x800823000)
libssl.so.8 => /usr/lib/libssl.so.8 (0x804ce2000)
libcrypto.so.8 => /lib/libcrypto.so.8 (0x805000000)
libcrypt.so.5 => /lib/libcrypt.so.5 (0x80546f000)
libdb-5.3.so.0 => /usr/local/lib/libdb-5.3.so.0 (0x80568e000)
libgdbm.so.6 => /usr/local/lib/libgdbm.so.6 (0x805a33000)
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: SWIG Python bindings build with SWIG 4.0
Posted by Branko Čibej <br...@apache.org>.
On 20.10.2019 06:35, Yasuhito FUTATSUKI wrote:
> On 2019/10/20 8:37, Branko Čibej wrote:
>> On 20.10.2019 01:10, Yasuhito FUTATSUKI wrote:
>>> On 2019/10/20 6:59, Branko Čibej wrote:
>>>> On 19.10.2019 23:06, Branko Čibej wrote:
>>>>> On 19.10.2019 19:55, Branko Čibej wrote:
>>>>>> On 19.10.2019 11:45, Yasuhito FUTATSUKI wrote:
>>>>>>> On 2019/10/18 8:39, Branko Čibej wrote:
>>>>>>>> On 17.10.2019 23:46, Branko Čibej wrote:
>>>>>>>>> On 17.10.2019 23:01, Daniel Shahaf wrote:
>>>>>>>>>> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>>>>>>>>>>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>>>>>>>>>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>>>>>>>>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>>>>>>>>>>> Which reminds me: it'd be nice to set up a buildbot to
>>>>>>>>>>>>>> run the
>>>>>>>>>>>>>> build and
>>>>>>>>>>>>>> test process under Python 3. Any committer can edit the
>>>>>>>>>>>>>> buildbot
>>>>>>>>>>>>>> scripts[1], but the question is which of the buildbot slaves
>>>>>>>>>>>>>> has Python 3
>>>>>>>>>>>>>> installed?
>>>>>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>>>>>>>>>>
>>>>>>>>>>>>> Been there, done that, bought the DVD.
>>>>>>>>>>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing
>>>>>>>>>>>> on that
>>>>>>>>>>>> bot? They (currently) fail under python3.
>>>>>>>>>>> I don't know. It's possible, I suppose, that the activation of
>>>>>>>>>>> the
>>>>>>>>>>> python3 virtual environment has no effect on the test
>>>>>>>>>>> driver. It
>>>>>>>>>>> might
>>>>>>>>>>> not be a bad idea to have the tests print the Python version
>>>>>>>>>>> in the
>>>>>>>>>>> summary and possibly in the log of every test case.
>>>>>>>>>> What's the value of $(PYTHON) in Makefile? That's what the
>>>>>>>>>> «check»
>>>>>>>>>> target uses.
>>>>>>>>> Yep, apparently that's the bug ... I'm testing script changes
>>>>>>>>> now,
>>>>>>>>> along
>>>>>>>>> with r1868566 for good measure.
>>>>>>>> I've set up this:
>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
>>>>>>>>
>>>>>>>> It will build and test the core libraries and swig-py bindings
>>>>>>>> with
>>>>>>>> Python 3 on the swig-py3 branch.
>>>>>>>>
>>>>>>>> I hope ... build #0 running as we speak.
>>>>>>> Unfortunately build #2, which ran after upgrading SWIG and Python,
>>>>>>> failed
>>>>>>> to build SWIG Python bindings because of SWIG 4.0, as reported on
>>>>>>> SVN-4818.
>>>>>>>
>>>>>>> This also affects on trunk with SWIG 4.0.
>>>>>>> (e.g.
>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-full/builds/2418)
>>>>>>>
>>>>>>> With attached patch on trunk (trunk_build_with_swig4_patch.txt) and
>>>>>>> on swig-py3 (swig_py3_build_with_swig4_patch.txt), *.so can be
>>>>>>> produced,
>>>>>>> but the modules don't work correctly.
>>>>>>>
>>>>>>> It seems they were caused by incompatibility of Python code for
>>>>>>> proxy
>>>>>>> object generated by SWIG, and it can not be resolved so simple....
>>>>>>> (importlib module vs simply use 'import', absense of
>>>>>>> _swig_setattr()
>>>>>>> and _swig_getattr(), etc.)
>>>>>> My bad. I downgraded to Swig 3.0.12 and restarted the failed builds.
>>>>>
>>>>> Hm, that did not really help. On the swig-py3 branch:
>>>>>
>>>>> /bin/sh: line 1: 62481 Segmentation fault: 11
>>>>> /srv/virtualenv-3/bin/python
>>>>> /srv/buildbot/slave/svn-x64-macosx-local-python3-swig/build/subversion/bindings/swig/python/tests/run_all.py
>>>>>
>>>>> make: *** [check-swig-py] Error 139
>>>>>
>>>>>
>>>>> So, the Python bindings seem to have crashed the Python 3
>>>>> interpreter on
>>>>> macOS?
>>>>
>>>> Yes, they did:
>>>>
>>>> Process 28278 stopped
>>>> * thread #2, queue = 'com.apple.main-thread', stop reason =
>>>> EXC_BAD_ACCESS (code=1, address=0x48)
>>>> frame #0: 0x00007fff373ab746 Python`PyErr_Restore + 61
>>>> Python`PyErr_Restore:
>>>> -> 0x7fff373ab746 <+61>: movq 0x48(%rbx), %rdi
>>>> 0x7fff373ab74a <+65>: movq 0x50(%rbx), %r13
>>>> 0x7fff373ab74e <+69>: movq 0x58(%rbx), %r12
>>>> 0x7fff373ab752 <+73>: movq %r15, 0x48(%rbx)
>>>> Target 0: (Python) stopped.
>>>>
>>>> frame #3: 0x0000000103800fea _core.so`PyInit__core at core.c:40158:12
>>>> 40155 m = Py_InitModule((char *) SWIG_name, SwigMethods);
>>>> 40156 #endif
>>>> 40157
>>>> -> 40158 md = d = PyModule_GetDict(m);
>>>> 40159 (void)md;
>>>> 40160
>>>> 40161 SWIG_InitializeModule(0);
>>>> (lldb) print m
>>>> (PyObject *) $0 = 0x000000010279ad70
>>>> (lldb) print *m
>>>> (PyObject) $1 = {
>>>> ob_refcnt = 785
>>>> ob_type = 0x000000010026ea30
>>>> }
>>>
>>> I guess the _core module linked incompatible Python library to execute.
>>>
>>> on
>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig/builds/6/steps/Build/logs/stdio
>>>
>>> :
>>> [[[
>>> checking for compiling Python extensions... clang
>>> checking for linking Python extensions... clang -bundle -undefined
>>> dynamic_lookup -isysroot
>>> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
>>>
>>> -F/usr/local/opt/python/Frameworks -framework Python
>>> checking for linking Python libraries... -bundle -undefined
>>> dynamic_lookup -F/usr/local/opt/python/Frameworks -framework Python
>>> ]]]
>>
>>
>> Good catch. This certainly looks correct, it's where Homebrew installs
>> python. However ...
>>
>> $ otool -L /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so
>> /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so:
>> /System/Library/Frameworks/Python.framework/Versions/2.7/Python
>> (compatibility version 2.7.0, current version 2.7.10)
>> ...
>>
>>
>> That is wrong. Looks like we'll have to set DYLD_LIBRARY_PATH so that it
>> finds the Python 3 installation.
(Actually that would be DYLD_FRAMEWORK_PATH.)
> ... or have to use Run-Path Dependent Libraries feature on link time,
> if DYLD_LIBRARY_PATH cannot work well due to SIP.
>
> https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/RunpathDependentLibraries.html
>
>
> The link option in log above comes from result of
> 'build/get-py-info.py --link', so should we tweak build/get-py-info.py,
> or add option to configure to pass Python dynamic library path?
Using RPATH for the Python framework on macOS would be the right
solution, but it doesn't solve the problem. It looks like we need some
magic incantation on macOS to explicitly pick the Python 3 framework;
using just the -F option doesn't work as expected, the path to the
system default Python 2.7 framework is hard-coded in to the extension
libraries.
-- Brane
Re: SWIG Python bindings build with SWIG 4.0
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/10/20 8:37, Branko Čibej wrote:
> On 20.10.2019 01:10, Yasuhito FUTATSUKI wrote:
>> On 2019/10/20 6:59, Branko Čibej wrote:
>>> On 19.10.2019 23:06, Branko Čibej wrote:
>>>> On 19.10.2019 19:55, Branko Čibej wrote:
>>>>> On 19.10.2019 11:45, Yasuhito FUTATSUKI wrote:
>>>>>> On 2019/10/18 8:39, Branko Čibej wrote:
>>>>>>> On 17.10.2019 23:46, Branko Čibej wrote:
>>>>>>>> On 17.10.2019 23:01, Daniel Shahaf wrote:
>>>>>>>>> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>>>>>>>>>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>>>>>>>>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>>>>>>>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>>>>>>>>>> Which reminds me: it'd be nice to set up a buildbot to run the
>>>>>>>>>>>>> build and
>>>>>>>>>>>>> test process under Python 3. Any committer can edit the
>>>>>>>>>>>>> buildbot
>>>>>>>>>>>>> scripts[1], but the question is which of the buildbot slaves
>>>>>>>>>>>>> has Python 3
>>>>>>>>>>>>> installed?
>>>>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>>>>>>>>>
>>>>>>>>>>>> Been there, done that, bought the DVD.
>>>>>>>>>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing
>>>>>>>>>>> on that
>>>>>>>>>>> bot? They (currently) fail under python3.
>>>>>>>>>> I don't know. It's possible, I suppose, that the activation of
>>>>>>>>>> the
>>>>>>>>>> python3 virtual environment has no effect on the test driver. It
>>>>>>>>>> might
>>>>>>>>>> not be a bad idea to have the tests print the Python version
>>>>>>>>>> in the
>>>>>>>>>> summary and possibly in the log of every test case.
>>>>>>>>> What's the value of $(PYTHON) in Makefile? That's what the
>>>>>>>>> «check»
>>>>>>>>> target uses.
>>>>>>>> Yep, apparently that's the bug ... I'm testing script changes now,
>>>>>>>> along
>>>>>>>> with r1868566 for good measure.
>>>>>>> I've set up this:
>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
>>>>>>>
>>>>>>> It will build and test the core libraries and swig-py bindings with
>>>>>>> Python 3 on the swig-py3 branch.
>>>>>>>
>>>>>>> I hope ... build #0 running as we speak.
>>>>>> Unfortunately build #2, which ran after upgrading SWIG and Python,
>>>>>> failed
>>>>>> to build SWIG Python bindings because of SWIG 4.0, as reported on
>>>>>> SVN-4818.
>>>>>>
>>>>>> This also affects on trunk with SWIG 4.0.
>>>>>> (e.g. https://ci.apache.org/builders/svn-x64-macosx-full/builds/2418)
>>>>>>
>>>>>> With attached patch on trunk (trunk_build_with_swig4_patch.txt) and
>>>>>> on swig-py3 (swig_py3_build_with_swig4_patch.txt), *.so can be
>>>>>> produced,
>>>>>> but the modules don't work correctly.
>>>>>>
>>>>>> It seems they were caused by incompatibility of Python code for proxy
>>>>>> object generated by SWIG, and it can not be resolved so simple....
>>>>>> (importlib module vs simply use 'import', absense of _swig_setattr()
>>>>>> and _swig_getattr(), etc.)
>>>>> My bad. I downgraded to Swig 3.0.12 and restarted the failed builds.
>>>>
>>>> Hm, that did not really help. On the swig-py3 branch:
>>>>
>>>> /bin/sh: line 1: 62481 Segmentation fault: 11
>>>> /srv/virtualenv-3/bin/python
>>>> /srv/buildbot/slave/svn-x64-macosx-local-python3-swig/build/subversion/bindings/swig/python/tests/run_all.py
>>>> make: *** [check-swig-py] Error 139
>>>>
>>>>
>>>> So, the Python bindings seem to have crashed the Python 3
>>>> interpreter on
>>>> macOS?
>>>
>>> Yes, they did:
>>>
>>> Process 28278 stopped
>>> * thread #2, queue = 'com.apple.main-thread', stop reason =
>>> EXC_BAD_ACCESS (code=1, address=0x48)
>>> frame #0: 0x00007fff373ab746 Python`PyErr_Restore + 61
>>> Python`PyErr_Restore:
>>> -> 0x7fff373ab746 <+61>: movq 0x48(%rbx), %rdi
>>> 0x7fff373ab74a <+65>: movq 0x50(%rbx), %r13
>>> 0x7fff373ab74e <+69>: movq 0x58(%rbx), %r12
>>> 0x7fff373ab752 <+73>: movq %r15, 0x48(%rbx)
>>> Target 0: (Python) stopped.
>>>
>>> frame #3: 0x0000000103800fea _core.so`PyInit__core at core.c:40158:12
>>> 40155 m = Py_InitModule((char *) SWIG_name, SwigMethods);
>>> 40156 #endif
>>> 40157
>>> -> 40158 md = d = PyModule_GetDict(m);
>>> 40159 (void)md;
>>> 40160
>>> 40161 SWIG_InitializeModule(0);
>>> (lldb) print m
>>> (PyObject *) $0 = 0x000000010279ad70
>>> (lldb) print *m
>>> (PyObject) $1 = {
>>> ob_refcnt = 785
>>> ob_type = 0x000000010026ea30
>>> }
>>
>> I guess the _core module linked incompatible Python library to execute.
>>
>> on
>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig/builds/6/steps/Build/logs/stdio
>> :
>> [[[
>> checking for compiling Python extensions... clang
>> checking for linking Python extensions... clang -bundle -undefined
>> dynamic_lookup -isysroot
>> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
>> -F/usr/local/opt/python/Frameworks -framework Python
>> checking for linking Python libraries... -bundle -undefined
>> dynamic_lookup -F/usr/local/opt/python/Frameworks -framework Python
>> ]]]
>
>
> Good catch. This certainly looks correct, it's where Homebrew installs
> python. However ...
>
> $ otool -L /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so
> /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so:
> /System/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.10)
> ...
>
>
> That is wrong. Looks like we'll have to set DYLD_LIBRARY_PATH so that it
> finds the Python 3 installation.
... or have to use Run-Path Dependent Libraries feature on link time,
if DYLD_LIBRARY_PATH cannot work well due to SIP.
https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/RunpathDependentLibraries.html
The link option in log above comes from result of
'build/get-py-info.py --link', so should we tweak build/get-py-info.py,
or add option to configure to pass Python dynamic library path?
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: SWIG Python bindings build with SWIG 4.0
Posted by Branko Čibej <br...@apache.org>.
On 20.10.2019 01:10, Yasuhito FUTATSUKI wrote:
> On 2019/10/20 6:59, Branko Čibej wrote:
>> On 19.10.2019 23:06, Branko Čibej wrote:
>>> On 19.10.2019 19:55, Branko Čibej wrote:
>>>> On 19.10.2019 11:45, Yasuhito FUTATSUKI wrote:
>>>>> On 2019/10/18 8:39, Branko Čibej wrote:
>>>>>> On 17.10.2019 23:46, Branko Čibej wrote:
>>>>>>> On 17.10.2019 23:01, Daniel Shahaf wrote:
>>>>>>>> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>>>>>>>>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>>>>>>>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>>>>>>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>>>>>>>>> Which reminds me: it'd be nice to set up a buildbot to run the
>>>>>>>>>>>> build and
>>>>>>>>>>>> test process under Python 3. Any committer can edit the
>>>>>>>>>>>> buildbot
>>>>>>>>>>>> scripts[1], but the question is which of the buildbot slaves
>>>>>>>>>>>> has Python 3
>>>>>>>>>>>> installed?
>>>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>>>>>>>>
>>>>>>>>>>> Been there, done that, bought the DVD.
>>>>>>>>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing
>>>>>>>>>> on that
>>>>>>>>>> bot? They (currently) fail under python3.
>>>>>>>>> I don't know. It's possible, I suppose, that the activation of
>>>>>>>>> the
>>>>>>>>> python3 virtual environment has no effect on the test driver. It
>>>>>>>>> might
>>>>>>>>> not be a bad idea to have the tests print the Python version
>>>>>>>>> in the
>>>>>>>>> summary and possibly in the log of every test case.
>>>>>>>> What's the value of $(PYTHON) in Makefile? That's what the
>>>>>>>> «check»
>>>>>>>> target uses.
>>>>>>> Yep, apparently that's the bug ... I'm testing script changes now,
>>>>>>> along
>>>>>>> with r1868566 for good measure.
>>>>>> I've set up this:
>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
>>>>>>
>>>>>> It will build and test the core libraries and swig-py bindings with
>>>>>> Python 3 on the swig-py3 branch.
>>>>>>
>>>>>> I hope ... build #0 running as we speak.
>>>>> Unfortunately build #2, which ran after upgrading SWIG and Python,
>>>>> failed
>>>>> to build SWIG Python bindings because of SWIG 4.0, as reported on
>>>>> SVN-4818.
>>>>>
>>>>> This also affects on trunk with SWIG 4.0.
>>>>> (e.g. https://ci.apache.org/builders/svn-x64-macosx-full/builds/2418)
>>>>>
>>>>> With attached patch on trunk (trunk_build_with_swig4_patch.txt) and
>>>>> on swig-py3 (swig_py3_build_with_swig4_patch.txt), *.so can be
>>>>> produced,
>>>>> but the modules don't work correctly.
>>>>>
>>>>> It seems they were caused by incompatibility of Python code for proxy
>>>>> object generated by SWIG, and it can not be resolved so simple....
>>>>> (importlib module vs simply use 'import', absense of _swig_setattr()
>>>>> and _swig_getattr(), etc.)
>>>> My bad. I downgraded to Swig 3.0.12 and restarted the failed builds.
>>>
>>> Hm, that did not really help. On the swig-py3 branch:
>>>
>>> /bin/sh: line 1: 62481 Segmentation fault: 11
>>> /srv/virtualenv-3/bin/python
>>> /srv/buildbot/slave/svn-x64-macosx-local-python3-swig/build/subversion/bindings/swig/python/tests/run_all.py
>>> make: *** [check-swig-py] Error 139
>>>
>>>
>>> So, the Python bindings seem to have crashed the Python 3
>>> interpreter on
>>> macOS?
>>
>> Yes, they did:
>>
>> Process 28278 stopped
>> * thread #2, queue = 'com.apple.main-thread', stop reason =
>> EXC_BAD_ACCESS (code=1, address=0x48)
>> frame #0: 0x00007fff373ab746 Python`PyErr_Restore + 61
>> Python`PyErr_Restore:
>> -> 0x7fff373ab746 <+61>: movq 0x48(%rbx), %rdi
>> 0x7fff373ab74a <+65>: movq 0x50(%rbx), %r13
>> 0x7fff373ab74e <+69>: movq 0x58(%rbx), %r12
>> 0x7fff373ab752 <+73>: movq %r15, 0x48(%rbx)
>> Target 0: (Python) stopped.
>>
>> frame #3: 0x0000000103800fea _core.so`PyInit__core at core.c:40158:12
>> 40155 m = Py_InitModule((char *) SWIG_name, SwigMethods);
>> 40156 #endif
>> 40157
>> -> 40158 md = d = PyModule_GetDict(m);
>> 40159 (void)md;
>> 40160
>> 40161 SWIG_InitializeModule(0);
>> (lldb) print m
>> (PyObject *) $0 = 0x000000010279ad70
>> (lldb) print *m
>> (PyObject) $1 = {
>> ob_refcnt = 785
>> ob_type = 0x000000010026ea30
>> }
>
> I guess the _core module linked incompatible Python library to execute.
>
> on
> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig/builds/6/steps/Build/logs/stdio
> :
> [[[
> checking for compiling Python extensions... clang
> checking for linking Python extensions... clang -bundle -undefined
> dynamic_lookup -isysroot
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
> -F/usr/local/opt/python/Frameworks -framework Python
> checking for linking Python libraries... -bundle -undefined
> dynamic_lookup -F/usr/local/opt/python/Frameworks -framework Python
> ]]]
Good catch. This certainly looks correct, it's where Homebrew installs
python. However ...
$ otool -L /opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so
/opt/mine/svn-swig-py3/lib/svn-python/libsvn/_core.so:
/System/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.10)
...
That is wrong. Looks like we'll have to set DYLD_LIBRARY_PATH so that it
finds the Python 3 installation.
-- Brane
Re: SWIG Python bindings build with SWIG 4.0
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/10/20 6:59, Branko Čibej wrote:
> On 19.10.2019 23:06, Branko Čibej wrote:
>> On 19.10.2019 19:55, Branko Čibej wrote:
>>> On 19.10.2019 11:45, Yasuhito FUTATSUKI wrote:
>>>> On 2019/10/18 8:39, Branko Čibej wrote:
>>>>> On 17.10.2019 23:46, Branko Čibej wrote:
>>>>>> On 17.10.2019 23:01, Daniel Shahaf wrote:
>>>>>>> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>>>>>>>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>>>>>>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>>>>>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>>>>>>>> Which reminds me: it'd be nice to set up a buildbot to run the
>>>>>>>>>>> build and
>>>>>>>>>>> test process under Python 3. Any committer can edit the buildbot
>>>>>>>>>>> scripts[1], but the question is which of the buildbot slaves
>>>>>>>>>>> has Python 3
>>>>>>>>>>> installed?
>>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>>>>>>>
>>>>>>>>>> Been there, done that, bought the DVD.
>>>>>>>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing on that
>>>>>>>>> bot? They (currently) fail under python3.
>>>>>>>> I don't know. It's possible, I suppose, that the activation of the
>>>>>>>> python3 virtual environment has no effect on the test driver. It
>>>>>>>> might
>>>>>>>> not be a bad idea to have the tests print the Python version in the
>>>>>>>> summary and possibly in the log of every test case.
>>>>>>> What's the value of $(PYTHON) in Makefile? That's what the «check»
>>>>>>> target uses.
>>>>>> Yep, apparently that's the bug ... I'm testing script changes now,
>>>>>> along
>>>>>> with r1868566 for good measure.
>>>>> I've set up this:
>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
>>>>>
>>>>> It will build and test the core libraries and swig-py bindings with
>>>>> Python 3 on the swig-py3 branch.
>>>>>
>>>>> I hope ... build #0 running as we speak.
>>>> Unfortunately build #2, which ran after upgrading SWIG and Python, failed
>>>> to build SWIG Python bindings because of SWIG 4.0, as reported on
>>>> SVN-4818.
>>>>
>>>> This also affects on trunk with SWIG 4.0.
>>>> (e.g. https://ci.apache.org/builders/svn-x64-macosx-full/builds/2418)
>>>>
>>>> With attached patch on trunk (trunk_build_with_swig4_patch.txt) and
>>>> on swig-py3 (swig_py3_build_with_swig4_patch.txt), *.so can be produced,
>>>> but the modules don't work correctly.
>>>>
>>>> It seems they were caused by incompatibility of Python code for proxy
>>>> object generated by SWIG, and it can not be resolved so simple....
>>>> (importlib module vs simply use 'import', absense of _swig_setattr()
>>>> and _swig_getattr(), etc.)
>>> My bad. I downgraded to Swig 3.0.12 and restarted the failed builds.
>>
>> Hm, that did not really help. On the swig-py3 branch:
>>
>> /bin/sh: line 1: 62481 Segmentation fault: 11 /srv/virtualenv-3/bin/python /srv/buildbot/slave/svn-x64-macosx-local-python3-swig/build/subversion/bindings/swig/python/tests/run_all.py
>> make: *** [check-swig-py] Error 139
>>
>>
>> So, the Python bindings seem to have crashed the Python 3 interpreter on
>> macOS?
>
> Yes, they did:
>
> Process 28278 stopped
> * thread #2, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x48)
> frame #0: 0x00007fff373ab746 Python`PyErr_Restore + 61
> Python`PyErr_Restore:
> -> 0x7fff373ab746 <+61>: movq 0x48(%rbx), %rdi
> 0x7fff373ab74a <+65>: movq 0x50(%rbx), %r13
> 0x7fff373ab74e <+69>: movq 0x58(%rbx), %r12
> 0x7fff373ab752 <+73>: movq %r15, 0x48(%rbx)
> Target 0: (Python) stopped.
>
> frame #3: 0x0000000103800fea _core.so`PyInit__core at core.c:40158:12
> 40155 m = Py_InitModule((char *) SWIG_name, SwigMethods);
> 40156 #endif
> 40157
> -> 40158 md = d = PyModule_GetDict(m);
> 40159 (void)md;
> 40160
> 40161 SWIG_InitializeModule(0);
> (lldb) print m
> (PyObject *) $0 = 0x000000010279ad70
> (lldb) print *m
> (PyObject) $1 = {
> ob_refcnt = 785
> ob_type = 0x000000010026ea30
> }
I guess the _core module linked incompatible Python library to execute.
on https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig/builds/6/steps/Build/logs/stdio :
[[[
checking for compiling Python extensions... clang
checking for linking Python extensions... clang -bundle -undefined dynamic_lookup -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk -F/usr/local/opt/python/Frameworks -framework Python
checking for linking Python libraries... -bundle -undefined dynamic_lookup -F/usr/local/opt/python/Frameworks -framework Python
]]]
Is this correct?
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: SWIG Python bindings build with SWIG 4.0 (was: Re: Issue tracker
housecleaning: SVN-1722)
Posted by Branko Čibej <br...@apache.org>.
On 19.10.2019 23:06, Branko Čibej wrote:
> On 19.10.2019 19:55, Branko Čibej wrote:
>> On 19.10.2019 11:45, Yasuhito FUTATSUKI wrote:
>>> On 2019/10/18 8:39, Branko Čibej wrote:
>>>> On 17.10.2019 23:46, Branko Čibej wrote:
>>>>> On 17.10.2019 23:01, Daniel Shahaf wrote:
>>>>>> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>>>>>>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>>>>>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>>>>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>>>>>>> Which reminds me: it'd be nice to set up a buildbot to run the
>>>>>>>>>> build and
>>>>>>>>>> test process under Python 3. Any committer can edit the buildbot
>>>>>>>>>> scripts[1], but the question is which of the buildbot slaves
>>>>>>>>>> has Python 3
>>>>>>>>>> installed?
>>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>>>>>>
>>>>>>>>> Been there, done that, bought the DVD.
>>>>>>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing on that
>>>>>>>> bot? They (currently) fail under python3.
>>>>>>> I don't know. It's possible, I suppose, that the activation of the
>>>>>>> python3 virtual environment has no effect on the test driver. It
>>>>>>> might
>>>>>>> not be a bad idea to have the tests print the Python version in the
>>>>>>> summary and possibly in the log of every test case.
>>>>>> What's the value of $(PYTHON) in Makefile? That's what the «check»
>>>>>> target uses.
>>>>> Yep, apparently that's the bug ... I'm testing script changes now,
>>>>> along
>>>>> with r1868566 for good measure.
>>>> I've set up this:
>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
>>>>
>>>> It will build and test the core libraries and swig-py bindings with
>>>> Python 3 on the swig-py3 branch.
>>>>
>>>> I hope ... build #0 running as we speak.
>>> Unfortunately build #2, which ran after upgrading SWIG and Python, failed
>>> to build SWIG Python bindings because of SWIG 4.0, as reported on
>>> SVN-4818.
>>>
>>> This also affects on trunk with SWIG 4.0.
>>> (e.g. https://ci.apache.org/builders/svn-x64-macosx-full/builds/2418)
>>>
>>> With attached patch on trunk (trunk_build_with_swig4_patch.txt) and
>>> on swig-py3 (swig_py3_build_with_swig4_patch.txt), *.so can be produced,
>>> but the modules don't work correctly.
>>>
>>> It seems they were caused by incompatibility of Python code for proxy
>>> object generated by SWIG, and it can not be resolved so simple....
>>> (importlib module vs simply use 'import', absense of _swig_setattr()
>>> and _swig_getattr(), etc.)
>> My bad. I downgraded to Swig 3.0.12 and restarted the failed builds.
>
> Hm, that did not really help. On the swig-py3 branch:
>
> /bin/sh: line 1: 62481 Segmentation fault: 11 /srv/virtualenv-3/bin/python /srv/buildbot/slave/svn-x64-macosx-local-python3-swig/build/subversion/bindings/swig/python/tests/run_all.py
> make: *** [check-swig-py] Error 139
>
>
> So, the Python bindings seem to have crashed the Python 3 interpreter on
> macOS?
Yes, they did:
Process 28278 stopped
* thread #2, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x48)
frame #0: 0x00007fff373ab746 Python`PyErr_Restore + 61
Python`PyErr_Restore:
-> 0x7fff373ab746 <+61>: movq 0x48(%rbx), %rdi
0x7fff373ab74a <+65>: movq 0x50(%rbx), %r13
0x7fff373ab74e <+69>: movq 0x58(%rbx), %r12
0x7fff373ab752 <+73>: movq %r15, 0x48(%rbx)
Target 0: (Python) stopped.
frame #3: 0x0000000103800fea _core.so`PyInit__core at core.c:40158:12
40155 m = Py_InitModule((char *) SWIG_name, SwigMethods);
40156 #endif
40157
-> 40158 md = d = PyModule_GetDict(m);
40159 (void)md;
40160
40161 SWIG_InitializeModule(0);
(lldb) print m
(PyObject *) $0 = 0x000000010279ad70
(lldb) print *m
(PyObject) $1 = {
ob_refcnt = 785
ob_type = 0x000000010026ea30
}
Full traceback attached, if that helps.
-- Brane
Re: SWIG Python bindings build with SWIG 4.0 (was: Re: Issue tracker
housecleaning: SVN-1722)
Posted by Branko Čibej <br...@apache.org>.
On 19.10.2019 19:55, Branko Čibej wrote:
> On 19.10.2019 11:45, Yasuhito FUTATSUKI wrote:
>> On 2019/10/18 8:39, Branko Čibej wrote:
>>> On 17.10.2019 23:46, Branko Čibej wrote:
>>>> On 17.10.2019 23:01, Daniel Shahaf wrote:
>>>>> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>>>>>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>>>>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>>>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>>>>>> Which reminds me: it'd be nice to set up a buildbot to run the
>>>>>>>>> build and
>>>>>>>>> test process under Python 3. Any committer can edit the buildbot
>>>>>>>>> scripts[1], but the question is which of the buildbot slaves
>>>>>>>>> has Python 3
>>>>>>>>> installed?
>>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>>>>>
>>>>>>>> Been there, done that, bought the DVD.
>>>>>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing on that
>>>>>>> bot? They (currently) fail under python3.
>>>>>> I don't know. It's possible, I suppose, that the activation of the
>>>>>> python3 virtual environment has no effect on the test driver. It
>>>>>> might
>>>>>> not be a bad idea to have the tests print the Python version in the
>>>>>> summary and possibly in the log of every test case.
>>>>> What's the value of $(PYTHON) in Makefile? That's what the «check»
>>>>> target uses.
>>>> Yep, apparently that's the bug ... I'm testing script changes now,
>>>> along
>>>> with r1868566 for good measure.
>>> I've set up this:
>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
>>>
>>> It will build and test the core libraries and swig-py bindings with
>>> Python 3 on the swig-py3 branch.
>>>
>>> I hope ... build #0 running as we speak.
>> Unfortunately build #2, which ran after upgrading SWIG and Python, failed
>> to build SWIG Python bindings because of SWIG 4.0, as reported on
>> SVN-4818.
>>
>> This also affects on trunk with SWIG 4.0.
>> (e.g. https://ci.apache.org/builders/svn-x64-macosx-full/builds/2418)
>>
>> With attached patch on trunk (trunk_build_with_swig4_patch.txt) and
>> on swig-py3 (swig_py3_build_with_swig4_patch.txt), *.so can be produced,
>> but the modules don't work correctly.
>>
>> It seems they were caused by incompatibility of Python code for proxy
>> object generated by SWIG, and it can not be resolved so simple....
>> (importlib module vs simply use 'import', absense of _swig_setattr()
>> and _swig_getattr(), etc.)
> My bad. I downgraded to Swig 3.0.12 and restarted the failed builds.
Hm, that did not really help. On the swig-py3 branch:
/bin/sh: line 1: 62481 Segmentation fault: 11 /srv/virtualenv-3/bin/python /srv/buildbot/slave/svn-x64-macosx-local-python3-swig/build/subversion/bindings/swig/python/tests/run_all.py
make: *** [check-swig-py] Error 139
So, the Python bindings seem to have crashed the Python 3 interpreter on
macOS?
(I installed the python3 package from Homebrew; it works perfectly on my
laptop.)
-- Brane
Re: SWIG Python bindings build with SWIG 4.0 (was: Re: Issue tracker
housecleaning: SVN-1722)
Posted by Branko Čibej <br...@apache.org>.
On 19.10.2019 11:45, Yasuhito FUTATSUKI wrote:
> On 2019/10/18 8:39, Branko Čibej wrote:
>> On 17.10.2019 23:46, Branko Čibej wrote:
>>> On 17.10.2019 23:01, Daniel Shahaf wrote:
>>>> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>>>>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>>>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>>>>> Which reminds me: it'd be nice to set up a buildbot to run the
>>>>>>>> build and
>>>>>>>> test process under Python 3. Any committer can edit the buildbot
>>>>>>>> scripts[1], but the question is which of the buildbot slaves
>>>>>>>> has Python 3
>>>>>>>> installed?
>>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>>>>
>>>>>>> Been there, done that, bought the DVD.
>>>>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing on that
>>>>>> bot? They (currently) fail under python3.
>>>>> I don't know. It's possible, I suppose, that the activation of the
>>>>> python3 virtual environment has no effect on the test driver. It
>>>>> might
>>>>> not be a bad idea to have the tests print the Python version in the
>>>>> summary and possibly in the log of every test case.
>>>> What's the value of $(PYTHON) in Makefile? That's what the «check»
>>>> target uses.
>>>
>>> Yep, apparently that's the bug ... I'm testing script changes now,
>>> along
>>> with r1868566 for good measure.
>>
>> I've set up this:
>> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
>>
>> It will build and test the core libraries and swig-py bindings with
>> Python 3 on the swig-py3 branch.
>>
>> I hope ... build #0 running as we speak.
>
> Unfortunately build #2, which ran after upgrading SWIG and Python, failed
> to build SWIG Python bindings because of SWIG 4.0, as reported on
> SVN-4818.
>
> This also affects on trunk with SWIG 4.0.
> (e.g. https://ci.apache.org/builders/svn-x64-macosx-full/builds/2418)
>
> With attached patch on trunk (trunk_build_with_swig4_patch.txt) and
> on swig-py3 (swig_py3_build_with_swig4_patch.txt), *.so can be produced,
> but the modules don't work correctly.
>
> It seems they were caused by incompatibility of Python code for proxy
> object generated by SWIG, and it can not be resolved so simple....
> (importlib module vs simply use 'import', absense of _swig_setattr()
> and _swig_getattr(), etc.)
My bad. I downgraded to Swig 3.0.12 and restarted the failed builds.
-- Brane
SWIG Python bindings build with SWIG 4.0 (was: Re: Issue tracker
housecleaning: SVN-1722)
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/10/18 8:39, Branko Čibej wrote:
> On 17.10.2019 23:46, Branko Čibej wrote:
>> On 17.10.2019 23:01, Daniel Shahaf wrote:
>>> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>>>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>>>> Which reminds me: it'd be nice to set up a buildbot to run the build and
>>>>>>> test process under Python 3. Any committer can edit the buildbot
>>>>>>> scripts[1], but the question is which of the buildbot slaves has Python 3
>>>>>>> installed?
>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>>>
>>>>>> Been there, done that, bought the DVD.
>>>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing on that
>>>>> bot? They (currently) fail under python3.
>>>> I don't know. It's possible, I suppose, that the activation of the
>>>> python3 virtual environment has no effect on the test driver. It might
>>>> not be a bad idea to have the tests print the Python version in the
>>>> summary and possibly in the log of every test case.
>>> What's the value of $(PYTHON) in Makefile? That's what the «check» target uses.
>>
>> Yep, apparently that's the bug ... I'm testing script changes now, along
>> with r1868566 for good measure.
>
> I've set up this:
> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
>
> It will build and test the core libraries and swig-py bindings with
> Python 3 on the swig-py3 branch.
>
> I hope ... build #0 running as we speak.
Unfortunately build #2, which ran after upgrading SWIG and Python, failed
to build SWIG Python bindings because of SWIG 4.0, as reported on SVN-4818.
This also affects on trunk with SWIG 4.0.
(e.g. https://ci.apache.org/builders/svn-x64-macosx-full/builds/2418)
With attached patch on trunk (trunk_build_with_swig4_patch.txt) and
on swig-py3 (swig_py3_build_with_swig4_patch.txt), *.so can be produced,
but the modules don't work correctly.
It seems they were caused by incompatibility of Python code for proxy
object generated by SWIG, and it can not be resolved so simple....
(importlib module vs simply use 'import', absense of _swig_setattr()
and _swig_getattr(), etc.)
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Issue tracker housecleaning: SVN-1722
Posted by Julian Foad <ju...@apache.org>.
Branko Čibej wrote:
> Yasuhito FUTATSUKI wrote:
>> However, it seems there is more general question, "What versions
>> do we support on Python 3?" [...]
>
> To be honest, I wouldn't care about any Python 3 older than 3.5 [...]
Let's move this to a new email thread. I'll open one. Subject: "What
versions of Python 3 do we support?"
- Julian
Re: Issue tracker housecleaning: SVN-1722
Posted by Branko Čibej <br...@apache.org>.
On 01.11.2019 09:58, Yasuhito FUTATSUKI wrote:
> On 2019/11/01 14:23, Nathan Hartman wrote:
>> On Fri, Oct 18, 2019 at 9:22 AM Branko Čibej <br...@apache.org> wrote:
>>
>>> Running the build scripts and tests with Python3 works now on trunk,
>>> with the latest fixes. Except for this warning:
>>>
>>> .../run_tests.py:53: DeprecationWarning: the imp module is
>>> deprecated in
>>> favour of importlib; see the module's documentation for alternative
>>> uses
>>> import optparse, subprocess, imp, threading, traceback
>>>
>>>
>>> I know we make 'imp' vs. 'importlib' choices elsewhere in the code, we
>>> probably just missed a case here.
>>>
>>
>> Where?
>>
>> I searched but did not find any other 'imp' vs 'importlib' choices in
>> any of the *.py files, neither on trunk nor on the branch swig-py3,
>> except for the instance you note above in run_tests.py.
>
> I saw them in build tree, however they were not our code but the code
> generated by SWIG (< 4.0) for swig Python bindings.
>
>> 'imp' is only used in TestHarness._run_py_test to call
>> imp.load_module. In that function, there is one version of the call
>> for Py < 3.0, another for Py >= 3.0.
>>
>> But imp was deprecated in Py 3.4, not 3.0, and imp.PY_SOURCE was
>> deprecated in 3.3, so there are too many different versions at play
>> here.
>>
>> Suggestions?
>
> In this case, if you can rewrite it with importlib.import_module()
> for Python 2.7 and Python 3.1, perhaps it will also work with
> Python 3.2 and later.
>
>
> However, it seems there is more general question, "What versions
> do we support on Python 3?"
>
> It seems we don't promise to support any version of Python 3 yet.
> So I think we can restrict version to support for Python 3,
> comparatively safely.
>
> Python 3.4 had reached end of life[1]. And developers might not
> have test environment with older Python 3.
To be honest, I wouldn't care about any Python 3 older than 3.5. IMO it
took the 3.x series quite a while to mature from "wow, a new major
version!" to "a better scripting language". 3.5 or thereabouts was the
turning point.
-- Brane
Re: Issue tracker housecleaning: SVN-1722
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/11/01 14:23, Nathan Hartman wrote:
> On Fri, Oct 18, 2019 at 9:22 AM Branko Čibej <br...@apache.org> wrote:
>
>> Running the build scripts and tests with Python3 works now on trunk,
>> with the latest fixes. Except for this warning:
>>
>> .../run_tests.py:53: DeprecationWarning: the imp module is deprecated in
>> favour of importlib; see the module's documentation for alternative uses
>> import optparse, subprocess, imp, threading, traceback
>>
>>
>> I know we make 'imp' vs. 'importlib' choices elsewhere in the code, we
>> probably just missed a case here.
>>
>
> Where?
>
> I searched but did not find any other 'imp' vs 'importlib' choices in
> any of the *.py files, neither on trunk nor on the branch swig-py3,
> except for the instance you note above in run_tests.py.
I saw them in build tree, however they were not our code but the code
generated by SWIG (< 4.0) for swig Python bindings.
> 'imp' is only used in TestHarness._run_py_test to call
> imp.load_module. In that function, there is one version of the call
> for Py < 3.0, another for Py >= 3.0.
>
> But imp was deprecated in Py 3.4, not 3.0, and imp.PY_SOURCE was
> deprecated in 3.3, so there are too many different versions at play
> here.
>
> Suggestions?
In this case, if you can rewrite it with importlib.import_module()
for Python 2.7 and Python 3.1, perhaps it will also work with
Python 3.2 and later.
However, it seems there is more general question, "What versions
do we support on Python 3?"
It seems we don't promise to support any version of Python 3 yet.
So I think we can restrict version to support for Python 3,
comparatively safely.
Python 3.4 had reached end of life[1]. And developers might not
have test environment with older Python 3.
[1] Python 3.4.10
https://www.python.org/downloads/release/python-3410/
Actually, the code I wrote for Python 3 may not work with Python 3.5
and prior, because I tested with Python 3.6 and Python 3.7 only,
although I intend to care backward comatibility (and perhaps
sometimes I ignore compatibility with Python 3.3 and prior,
but I don't remember, sorry).
Anyway, I think if we can't test with older version of Python 3,
we can't support it (as just same I said for SWIG).
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Issue tracker housecleaning: SVN-1722
Posted by Nathan Hartman <ha...@gmail.com>.
On Fri, Oct 18, 2019 at 9:22 AM Branko Čibej <br...@apache.org> wrote:
> Running the build scripts and tests with Python3 works now on trunk,
> with the latest fixes. Except for this warning:
>
> .../run_tests.py:53: DeprecationWarning: the imp module is deprecated in
> favour of importlib; see the module's documentation for alternative uses
> import optparse, subprocess, imp, threading, traceback
>
>
> I know we make 'imp' vs. 'importlib' choices elsewhere in the code, we
> probably just missed a case here.
>
Where?
I searched but did not find any other 'imp' vs 'importlib' choices in
any of the *.py files, neither on trunk nor on the branch swig-py3,
except for the instance you note above in run_tests.py.
'imp' is only used in TestHarness._run_py_test to call
imp.load_module. In that function, there is one version of the call
for Py < 3.0, another for Py >= 3.0.
But imp was deprecated in Py 3.4, not 3.0, and imp.PY_SOURCE was
deprecated in 3.3, so there are too many different versions at play
here.
Suggestions?
Re: Issue tracker housecleaning: SVN-1722
Posted by Branko Čibej <br...@apache.org>.
On 18.10.2019 01:39, Branko Čibej wrote:
> On 17.10.2019 23:46, Branko Čibej wrote:
>> On 17.10.2019 23:01, Daniel Shahaf wrote:
>>> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>>>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>>>> Which reminds me: it'd be nice to set up a buildbot to run the build and
>>>>>>> test process under Python 3. Any committer can edit the buildbot
>>>>>>> scripts[1], but the question is which of the buildbot slaves has Python 3
>>>>>>> installed?
>>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>>>
>>>>>> Been there, done that, bought the DVD.
>>>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing on that
>>>>> bot? They (currently) fail under python3.
>>>> I don't know. It's possible, I suppose, that the activation of the
>>>> python3 virtual environment has no effect on the test driver. It might
>>>> not be a bad idea to have the tests print the Python version in the
>>>> summary and possibly in the log of every test case.
>>> What's the value of $(PYTHON) in Makefile? That's what the «check» target uses.
>> Yep, apparently that's the bug ... I'm testing script changes now, along
>> with r1868566 for good measure.
> I've set up this:
> https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
>
> It will build and test the core libraries and swig-py bindings with
> Python 3 on the swig-py3 branch.
>
> I hope ... build #0 running as we speak.
Running the build scripts and tests with Python3 works now on trunk,
with the latest fixes. Except for this warning:
.../run_tests.py:53: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import optparse, subprocess, imp, threading, traceback
I know we make 'imp' vs. 'importlib' choices elsewhere in the code, we
probably just missed a case here.
-- Brane
Re: Issue tracker housecleaning: SVN-1722
Posted by Branko Čibej <br...@apache.org>.
On 17.10.2019 23:46, Branko Čibej wrote:
> On 17.10.2019 23:01, Daniel Shahaf wrote:
>> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>>> Which reminds me: it'd be nice to set up a buildbot to run the build and
>>>>>> test process under Python 3. Any committer can edit the buildbot
>>>>>> scripts[1], but the question is which of the buildbot slaves has Python 3
>>>>>> installed?
>>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>>
>>>>> Been there, done that, bought the DVD.
>>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing on that
>>>> bot? They (currently) fail under python3.
>>> I don't know. It's possible, I suppose, that the activation of the
>>> python3 virtual environment has no effect on the test driver. It might
>>> not be a bad idea to have the tests print the Python version in the
>>> summary and possibly in the log of every test case.
>> What's the value of $(PYTHON) in Makefile? That's what the «check» target uses.
>
> Yep, apparently that's the bug ... I'm testing script changes now, along
> with r1868566 for good measure.
I've set up this:
https://ci.apache.org/builders/svn-x64-macosx-local-python3-swig
It will build and test the core libraries and swig-py bindings with
Python 3 on the swig-py3 branch.
I hope ... build #0 running as we speak.
-- Brane
Re: Issue tracker housecleaning: SVN-1722
Posted by Branko Čibej <br...@apache.org>.
On 17.10.2019 23:01, Daniel Shahaf wrote:
> Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
>> On 17.10.2019 22:37, Daniel Shahaf wrote:
>>> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>>>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>>>> Which reminds me: it'd be nice to set up a buildbot to run the build and
>>>>> test process under Python 3. Any committer can edit the buildbot
>>>>> scripts[1], but the question is which of the buildbot slaves has Python 3
>>>>> installed?
>>>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>>>
>>>> Been there, done that, bought the DVD.
>>> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing on that
>>> bot? They (currently) fail under python3.
>> I don't know. It's possible, I suppose, that the activation of the
>> python3 virtual environment has no effect on the test driver. It might
>> not be a bad idea to have the tests print the Python version in the
>> summary and possibly in the log of every test case.
> What's the value of $(PYTHON) in Makefile? That's what the «check» target uses.
Yep, apparently that's the bug ... I'm testing script changes now, along
with r1868566 for good measure.
-- Brane
Re: Issue tracker housecleaning: SVN-1722
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Branko Čibej wrote on Thu, Oct 17, 2019 at 22:56:29 +0200:
> On 17.10.2019 22:37, Daniel Shahaf wrote:
> > Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
> >> On 16.10.2019 10:01, Daniel Shahaf wrote:
> >>> Which reminds me: it'd be nice to set up a buildbot to run the build and
> >>> test process under Python 3. Any committer can edit the buildbot
> >>> scripts[1], but the question is which of the buildbot slaves has Python 3
> >>> installed?
> >> https://ci.apache.org/builders/svn-x64-macosx-local-python3
> >>
> >> Been there, done that, bought the DVD.
> > Why are svnadmin_tests 69 and tree_conflict_tests 26 passing on that
> > bot? They (currently) fail under python3.
>
> I don't know. It's possible, I suppose, that the activation of the
> python3 virtual environment has no effect on the test driver. It might
> not be a bad idea to have the tests print the Python version in the
> summary and possibly in the log of every test case.
What's the value of $(PYTHON) in Makefile? That's what the «check» target uses.
Re: Issue tracker housecleaning: SVN-1722
Posted by Nathan Hartman <ha...@gmail.com>.
On Thu, Oct 17, 2019 at 3:03 PM Nathan Hartman <ha...@gmail.com> wrote:
> On Thu, Oct 17, 2019 at 2:30 PM Julian Foad <ju...@apache.org> wrote:
> > Close buttons are at top of page when logged in.
>
> Nope. I am logged in. I have a feeling my account isn't setup
> correctly with permissions. I might have to bother infra... :-(
Infra fixed it. SVN-1722 is FINALLY closed! On to the next one......
Thanks to everyone for your help.
Re: Issue tracker housecleaning: SVN-1722
Posted by Julian Foad <ju...@apache.org>.
Close buttons are at top of page when logged in.
Re: Issue tracker housecleaning: SVN-1722
Posted by Nathan Hartman <ha...@gmail.com>.
On Thu, Oct 17, 2019 at 11:44 AM Julian Foad <ju...@apache.org> wrote:
> Looks good to me.
Thanks! Regression test added in r1868561.
Now, here's a dumb question...
How do I close the issue?! There doesn't seem to be a button for it.
Re: Issue tracker housecleaning: SVN-1722
Posted by Julian Foad <ju...@apache.org>.
Nathan Hartman wrote:
> Test below. Please review...
Looks good to me.
- Julian
Run tests against an arbitrary installed svn
Posted by Julian Foad <ju...@apache.org>.
It would be good if we could run the test suite against an arbitrary
installed set of Subversion binaries and libraries.
We have this for a starting point:
> make check SVN_BIN_DIR=/opt/foo/some/path
For the Python tests, it mostly uses the given path for loading 'svn*'
executables. In a few cases it doesn't, probably where they are loaded
indirectly, e.g. in "ra-test 3: test ra_svn tunnel creation callbacks".
I found this by changing
-check: bin @TRANSFORM_LIBTOOL_SCRIPTS@ $(TEST_DEPS) @BDB_TEST_DEPS@
+check: @TRANSFORM_LIBTOOL_SCRIPTS@ $(TEST_DEPS) @BDB_TEST_DEPS@
so the main "svn*" are not built in the source tree, and running "make
clean; make check SVN_BIN_DIR=...".
For the C tests, "make check" first builds the svn libraries in the
source tree and links the tests against them. However, at run time, if
we set
LD_LIBRARY_PATH=/path/to/installed/svn/lib
we can delete the libs from the source tree, and it will load dynamic
libs from the target installation. I am not sure if that covers
everything or whether some static linking was used as well.
It would be good if we could make a clearer separation and be able to do
a test run that doesn't need to build libs in the source tree, so we can
see it is only using the desired target installation.
- Julian
Re: Issue tracker housecleaning: SVN-1722
Posted by Nathan Hartman <ha...@gmail.com>.
On Thu, Oct 17, 2019 at 8:18 AM Branko Čibej <br...@apache.org> wrote:
> On 17.10.2019 13:55, Julian Foad wrote:
> >> Is it possible, through some incantation that I've overlooked, to run
> >> the cmdline tests against an arbitrary svn binary?
> >
> > Unfortunately that's something we never got around to. I would love
> > to make that possible.
> ???
>
> make check SVN_BIN_DIR=/opt/foo/some/path
>
> See Makefile.in, line 617.
Thanks!
>
Re: Issue tracker housecleaning: SVN-1722
Posted by Branko Čibej <br...@apache.org>.
On 17.10.2019 13:55, Julian Foad wrote:
>> Is it possible, through some incantation that I've overlooked, to run
>> the cmdline tests against an arbitrary svn binary?
>
> Unfortunately that's something we never got around to. I would love
> to make that possible.
???
make check SVN_BIN_DIR=/opt/foo/some/path
See Makefile.in, line 617.
-- Brane
Re: Issue tracker housecleaning: SVN-1722
Posted by Julian Foad <ju...@apache.org>.
Nathan Hartman wrote:
> I have the enthusiasm:-)
Great!
> More importantly this is an opportunity to learn more about the test
> program.
>
> Is it possible, through some incantation that I've overlooked, to run
> the cmdline tests against an arbitrary svn binary?
Unfortunately that's something we never got around to. I would love to
make that possible.
> Or, perhaps a better question is: how do you verify that a test fails on
> an old version and passes on a new version? (I happen to have 1.8
> installed so I'd like to verify that my new test fails on the 1.8 client
> and passes on 1.13.)
I do it by copying the new test to the old branch by diff-and-patch.
- Julian
Re: Issue tracker housecleaning: SVN-1722
Posted by Nathan Hartman <ha...@gmail.com>.
Test below. Please review...
On Thu, Oct 17, 2019 at 8:18 AM Branko Čibej <br...@apache.org> wrote:
> >> Is it possible, through some incantation that I've overlooked, to run
> >> the cmdline tests against an arbitrary svn binary?
> make check SVN_BIN_DIR=/opt/foo/some/path
>
> See Makefile.in, line 617.
Btw, apparently, 1.8.19 is far too old to run with the current test suite.
When I attempted:
make check TESTS=subversion/tests/cmdline/diff_tests.py \
SVN_BIN_DIR=(path to 1.8.19 bin dir here)
I got this:
[1/1] diff_tests.py......................................................FAILURE
Summary of test results:
SUMMARY: Some tests failed.
(It says 1/1 because I commented out everything except my test in test_list.)
It doesn't print details of what failed and tests.log contains only this:
START: diff_tests.py
END: diff_tests.py
ELAPSED: diff_tests.py 0:00:00.015756
There is no fails.log, no traceback, nothing.
So I tried copying my test over to a working copy of 1.8.19. That doesn't
work because too much of the test harness has changed since then.
Oh well. Not worth the effort. All I wanted was to prove that the test
will fail on a version known to have the bug, but given how pedantic
run_and_verify_svn was about my expected output EXACTLY matching
(it took some head scratching to figure out that I was missing a newline),
I'm 100% sure this would definitely fail on 1.8.19.
Anyway this passes on svn trunk as of r1868559.
This being my first test, if someone could just glance and see if I've
made some obvious faux pas, I'd appreciate that...
[[[
#----------------------------------------------------------------------
# Regression test for issue #1722: 'svn diff' produced a wrong header,
# indicating one revision as being in the working copy when it should
# be 'nonexistent'
@Issue(1722)
def diff_nonexistent_in_wc(sbox):
"nonexistent in working copy"
sbox.build(empty=True)
wc_dir = sbox.wc_dir
# We mirror the actions of the reproduction script (with one exception:
# we 'svn up -r 0' instead of checking out a second working copy)
sbox.simple_add_text('test\n', 'file')
sbox.simple_commit()
sbox.simple_update(revision=0)
# Expected output is empty for these cases:
# svn diff -r BASE
# svn diff -r 0:BASE
# svn diff -r 0
# Expected output for:
# svn diff -r BASE:HEAD
# svn diff -r 0:HEAD
# svn diff -r 0:1
expected_output_base_head = make_diff_header("file", "nonexistent",
"revision 1") + [
"@@ -0,0 +1 @@\n",
"+test\n",
]
# Expected output for:
# svn diff -r HEAD:BASE
# svn diff -r HEAD
# svn diff -r 1:0
# svn diff -r 1
expected_output_head_base = make_diff_header("file", "revision 1",
"nonexistent") + [
"@@ -1 +0,0 @@\n",
"-test\n"
]
os.chdir(wc_dir)
svntest.actions.run_and_verify_svn(expected_output_base_head, [],
'diff', '-r', 'BASE:HEAD')
svntest.actions.run_and_verify_svn(expected_output_head_base, [],
'diff', '-r', 'HEAD:BASE')
svntest.actions.run_and_verify_svn([], [],
'diff', '-r', 'BASE')
svntest.actions.run_and_verify_svn(expected_output_head_base, [],
'diff', '-r', 'HEAD')
svntest.actions.run_and_verify_svn([], [],
'diff', '-r', '0:BASE')
svntest.actions.run_and_verify_svn(expected_output_base_head, [],
'diff', '-r', '0:HEAD')
svntest.actions.run_and_verify_svn(expected_output_base_head, [],
'diff', '-r', '0:1')
svntest.actions.run_and_verify_svn(expected_output_head_base, [],
'diff', '-r', '1:0')
svntest.actions.run_and_verify_svn([], [],
'diff', '-r', '0')
svntest.actions.run_and_verify_svn(expected_output_head_base, [],
'diff', '-r', '1')
]]]
Thanks,
Nathan
Re: Issue tracker housecleaning: SVN-1722
Posted by Nathan Hartman <ha...@gmail.com>.
On Wed, Oct 16, 2019 at 4:46 AM Julian Foad <ju...@apache.org> wrote:
> Of course, ideally, it is lovely to improve the test coverage, if you
> have spare time and spare enthusiasm, or can do so really quickly so it
> doesn't impact other activities. However, it's not required and not
> necessarily worth it. A perfectly valid choice would be to prioritize
> moving on instead to reviewing another old open issue.
I have the enthusiasm:-)
More importantly this is an opportunity to learn more about the test
program.
Is it possible, through some incantation that I've overlooked, to run the
cmdline tests against an arbitrary svn binary?
Or, perhaps a better question is: how do you verify that a test fails on an
old version and passes on a new version? (I happen to have 1.8 installed so
I'd like to verify that my new test fails on the 1.8 client and passes on
1.13.)
Thanks,
Nathan
Re: Issue tracker housecleaning: SVN-1722
Posted by Julian Foad <ju...@apache.org>.
Daniel Shahaf wrote:
> Nathan Hartman wrote on Wed, 16 Oct 2019 04:47 +00:00:
>> Is it sensible to add a regression test?
>
> Yes, unless other tests already cover this.
Of course, ideally, it is lovely to improve the test coverage, if you
have spare time and spare enthusiasm, or can do so really quickly so it
doesn't impact other activities. However, it's not required and not
necessarily worth it. A perfectly valid choice would be to prioritize
moving on instead to reviewing another old open issue.
- Julian
Re: Issue tracker housecleaning: SVN-1722
Posted by Branko Čibej <br...@apache.org>.
On 17.10.2019 22:37, Daniel Shahaf wrote:
> Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
>> On 16.10.2019 10:01, Daniel Shahaf wrote:
>>> Which reminds me: it'd be nice to set up a buildbot to run the build and
>>> test process under Python 3. Any committer can edit the buildbot
>>> scripts[1], but the question is which of the buildbot slaves has Python 3
>>> installed?
>> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>>
>> Been there, done that, bought the DVD.
> Why are svnadmin_tests 69 and tree_conflict_tests 26 passing on that
> bot? They (currently) fail under python3.
I don't know. It's possible, I suppose, that the activation of the
python3 virtual environment has no effect on the test driver. It might
not be a bad idea to have the tests print the Python version in the
summary and possibly in the log of every test case.
-- Brane
Re: Issue tracker housecleaning: SVN-1722
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Branko Čibej wrote on Thu, 17 Oct 2019 06:49 +00:00:
> On 16.10.2019 10:01, Daniel Shahaf wrote:
> > Which reminds me: it'd be nice to set up a buildbot to run the build and
> > test process under Python 3. Any committer can edit the buildbot
> > scripts[1], but the question is which of the buildbot slaves has Python 3
> > installed?
>
> https://ci.apache.org/builders/svn-x64-macosx-local-python3
>
> Been there, done that, bought the DVD.
Why are svnadmin_tests 69 and tree_conflict_tests 26 passing on that
bot? They (currently) fail under python3.
Re: Issue tracker housecleaning: SVN-1722
Posted by Branko Čibej <br...@apache.org>.
On 16.10.2019 10:01, Daniel Shahaf wrote:
> Which reminds me: it'd be nice to set up a buildbot to run the build and
> test process under Python 3. Any committer can edit the buildbot
> scripts[1], but the question is which of the buildbot slaves has Python 3
> installed?
https://ci.apache.org/builders/svn-x64-macosx-local-python3
Been there, done that, bought the DVD.
-- Brane
Re: Issue tracker housecleaning: SVN-1722
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Nathan Hartman wrote on Wed, 16 Oct 2019 04:47 +00:00:
> Is it sensible to add a regression test?
Yes, unless other tests already cover this.
Such tests would probably be either in diff_tests.py or in subversion/tests/libsvn_diff/.
In general, Python tests may be tagged with «@Issue(1722)» decorators,
but lack of such a decorator doesn't mean there are no tests for this
issue. Other tests may exist, as well as tests for this specific issue
that predate the @Issue decorator.
> If so, should I wait until after the Py3 work in progress on the
> test suite?
No need to wait. Any new tests should pass under Python 3, but I doubt
you'll even have to test that explicitly — SVN-1722 doesn't look like an
issue whose test would involve bytes/str differences, or any other
py2/py3 differences.
Which reminds me: it'd be nice to set up a buildbot to run the build and
test process under Python 3. Any committer can edit the buildbot
scripts[1], but the question is which of the buildbot slaves has Python 3
installed?
Cheers,
Daniel
[1] https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/subversion.conf
Re: Issue tracker housecleaning: SVN-1722
Posted by Nathan Hartman <ha...@gmail.com>.
On Tue, Oct 15, 2019 at 2:40 AM Julian Foad <ju...@foad.me.uk> wrote:
> Your analysis is good enough for me. I don't need to double check the details.
>
> I'd say, add a short summary and a link to that email, and close it.
Will do...
FWIW I tested the SVN-1722 reproduction script on 1.13.0-rc1 earlier
today and all is well.
Is it sensible to add a regression test? If so, should I wait until
after the Py3 work in progress on the test suite?
Thanks,
Nathan
Re: Issue tracker housecleaning: SVN-1722
Posted by Julian Foad <ju...@foad.me.uk>.
Nathan,
Your analysis is good enough for me. I don't need to double check the details.
I'd say, add a short summary and a link to that email, and close it.
- Julian
Issue tracker housecleaning: SVN-1722
Posted by Nathan Hartman <ha...@gmail.com>.
A short time ago, in a galaxy nearby... or, in the thread
"Fwd: PMCs: any Hackathon requests? (deadline 11 October)"...
Julian Foad wrote:
>> Nathan Hartman wrote:
>> https://issues.apache.org/jira/browse/SVN-1722 "svn
>> diff may missreport a revision as the working copy."
>
> Errm... last reviewed 9 years ago. Is it really what it says it is?
>
> Sadly, we have hundreds of unreviewed (or not recently reviewed) open
> issues. Some are misleadingly labeled "bite-sized" but are really
> symptoms of deeper problems. Can we take this opportunity to do
> something about that? Like, maybe start bringing one old issue to the
> dev list for review each week?
I later said I would bring one issue to the list for review each week.
So here's the first one... SVN-1722 is as good a candidate as any.
Summary: I think this issue should be closed.
It looks as though this issue was fixed somewhere between 1.8.19 and
1.12.2.
I think the issue tracker was not updated because the fix might have
occurred as part of other refactorings or new features in the code.
In 2004, when the issue was opened, it was reported that the problem
manifest when svn diff was run with the following arguments in the
reproduction script:
+ svn diff -r BASE:HEAD
+ svn diff -r HEAD:BASE
+ svn diff -r HEAD
+ svn diff -r 0:HEAD
+ svn diff -r 1
Later, in 2010, C. Michael Pilato commented in the issue tracker and
showed one form that was fixed and another that that was still broken.
The correct form was output by:
+ svn diff -r BASE:HEAD
The broken form was still being output by:
+ svn diff -r HEAD:BASE
I ran the reproduction script against 1.8.19 and against 1.12.2
(respectively the oldest and newest installed versions I happen to
have at the moment):
When run against 1.8.19, the following forms produce the broken output:
+ svn diff -r HEAD:BASE
+ svn diff -r HEAD
+ svn diff -r 1
When run against 1.12.2, all of the forms appear correct.
I did not setup and run a bisect to find the exact commit where the
issue gets fixed, but some manual investigation turned up this very
promising-looking candidate:
r1570053
and these slightly less-promising-looking ones:
r1619452
r1570266
The reproduction script (copied from issue tracker):
$ cat runtest
rm -rf repo test test2
svnadmin create repo
svn checkout file:///`pwd`/repo test
svn checkout file:///`pwd`/repo test2
cd test
echo "test" >file
svn add file
svn -m test commit
cd ../test2
svn diff -r BASE:HEAD
svn diff -r HEAD:BASE
svn diff -r BASE
svn diff -r HEAD
svn diff -r 0:BASE
svn diff -r 0:HEAD
svn diff -r 0:1
svn diff -r 1:0
svn diff -r 0
svn diff -r 1
Following is the output when run against 1.8.19; note the incorrect
"working copy" labels when -r is HEAD:BASE, HEAD, and 1:
$ sh -x runtest
+ rm -rf repo test test2
+ svnadmin create repo
++ pwd
+ svn checkout file:////Users/nate/mount/ramdrive/test/repo test
Checked out revision 0.
++ pwd
+ svn checkout file:////Users/nate/mount/ramdrive/test/repo test2
Checked out revision 0.
+ cd test
+ echo test
+ svn add file
A file
+ svn -m test commit
Adding file
Transmitting file data .
Committed revision 1.
+ cd ../test2
+ svn diff -r BASE:HEAD
Index: file
===================================================================
--- file (revision 0)
+++ file (revision 1)
@@ -0,0 +1 @@
+test
+ svn diff -r HEAD:BASE
Index: file
===================================================================
--- file (revision 1)
+++ file (working copy)
@@ -1 +0,0 @@
-test
+ svn diff -r BASE
+ svn diff -r HEAD
Index: file
===================================================================
--- file (revision 1)
+++ file (working copy)
@@ -1 +0,0 @@
-test
+ svn diff -r 0:BASE
+ svn diff -r 0:HEAD
Index: file
===================================================================
--- file (revision 0)
+++ file (revision 1)
@@ -0,0 +1 @@
+test
+ svn diff -r 0:1
Index: file
===================================================================
--- file (revision 0)
+++ file (revision 1)
@@ -0,0 +1 @@
+test
+ svn diff -r 1:0
Index: file
===================================================================
--- file (revision 1)
+++ file (revision 0)
@@ -1 +0,0 @@
-test
+ svn diff -r 0
+ svn diff -r 1
Index: file
===================================================================
--- file (revision 1)
+++ file (working copy)
@@ -1 +0,0 @@
-test
$
Following is the output when run against 1.12.2:
$ sh -x runtest
+ rm -rf repo test test2
+ svnadmin create repo
+ pwd
+ svn checkout file:////home/nate/ramdrive/test/repo test
Checked out revision 0.
+ pwd
+ svn checkout file:////home/nate/ramdrive/test/repo test2
Checked out revision 0.
+ cd test
+ echo test
+ svn add file
A file
+ svn -m test commit
Adding file
Transmitting file data .done
Committing transaction...
Committed revision 1.
+ cd ../test2
+ svn diff -r BASE:HEAD
Index: file
===================================================================
--- file (nonexistent)
+++ file (revision 1)
@@ -0,0 +1 @@
+test
+ svn diff -r HEAD:BASE
Index: file
===================================================================
--- file (revision 1)
+++ file (nonexistent)
@@ -1 +0,0 @@
-test
+ svn diff -r BASE
+ svn diff -r HEAD
Index: file
===================================================================
--- file (revision 1)
+++ file (nonexistent)
@@ -1 +0,0 @@
-test
+ svn diff -r 0:BASE
+ svn diff -r 0:HEAD
Index: file
===================================================================
--- file (nonexistent)
+++ file (revision 1)
@@ -0,0 +1 @@
+test
+ svn diff -r 0:1
Index: file
===================================================================
--- file (nonexistent)
+++ file (revision 1)
@@ -0,0 +1 @@
+test
+ svn diff -r 1:0
Index: file
===================================================================
--- file (revision 1)
+++ file (nonexistent)
@@ -1 +0,0 @@
-test
+ svn diff -r 0
+ svn diff -r 1
Index: file
===================================================================
--- file (revision 1)
+++ file (nonexistent)
@@ -1 +0,0 @@
-test
$
Thoughts?
Nathan
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Julian Foad <ju...@apache.org>.
Nathan Hartman wrote:
> Perhaps:
> * 1 "byte size" code quality issue from the issue tracker
> * 1 website task
> * 1 Py3 support task
Generally, sure, sounds good. Go for it.
> https://issues.apache.org/jira/browse/SVN-1722 "svn
> diff may missreport a revision as the working copy."
Errm... last reviewed 9 years ago. Is it really what it says it is?
Sadly, we have hundreds of unreviewed (or not recently reviewed) open
issues. Some are misleadingly labeled "bite-sized" but are really
symptoms of deeper problems. Can we take this opportunity to do
something about that? Like, maybe start bringing one old issue to the
dev list for review each week?
- Julian
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Daniel Shahaf wrote on Thu, Oct 10, 2019 at 20:47:05 +0000:
> Nathan Hartman wrote on Thu, Oct 10, 2019 at 16:12:01 -0400:
> > On Thu, Oct 10, 2019 at 1:48 PM Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> > > > Should these be re-merged into one item, as you had it before?
> > >
> > > That depends. _Do_ the Windows build scripts support libmagic?
> >
> > I think they don't. Unless I missed something.
> >
> > It looks as though the Windows build uses the makefile at
> > tools/dev/windows-build/Makefile. I am guessing that packagers for the
> > Windows platform are using this as their starting point? Anyway, this
> > makefile makes absolutely no mention of libmagic.
>
> That makefile is not the primary way to build on windows. The primary way is
> what INSTALL documents, i.e., gen-make.py and build/**/*.py.
>
> Given that the makefile specifies serf 1.1.0 and configure.ac enforces ≥1.3.4,
> I dare say that no one uses that makefile any more.
There's also tools/dev/build-svn-deps-win.pl, but that, too, might have bitrotten.
Cheers,
Daniel
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Nathan Hartman wrote on Thu, Oct 10, 2019 at 16:12:01 -0400:
> On Thu, Oct 10, 2019 at 1:48 PM Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> > > Should these be re-merged into one item, as you had it before?
> >
> > That depends. _Do_ the Windows build scripts support libmagic?
>
> I think they don't. Unless I missed something.
>
> It looks as though the Windows build uses the makefile at
> tools/dev/windows-build/Makefile. I am guessing that packagers for the
> Windows platform are using this as their starting point? Anyway, this
> makefile makes absolutely no mention of libmagic.
That makefile is not the primary way to build on windows. The primary way is
what INSTALL documents, i.e., gen-make.py and build/**/*.py.
Given that the makefile specifies serf 1.1.0 and configure.ac enforces ≥1.3.4,
I dare say that no one uses that makefile any more.
Cheers,
Daniel
> I'll merge the two items into one:
>
> * Python programming:
>
> - SVN-3914 - Implement building with libmagic on Windows in the build
> scripts and document it in INSTALL.
> https://issues.apache.org/jira/browse/SVN-3914?issueNumber=3914
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Nathan Hartman <ha...@gmail.com>.
On Fri, Oct 11, 2019 at 6:52 PM Daniel Shahaf <d....@daniel.shahaf.name>
wrote:
> Nathan, I appreciate the intent, but Yasuhito and I are both familiar with
> the
> semantics of str and bytes objects in Python
I misread the earlier message. Apologies for the noise.
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Nathan Hartman wrote on Fri, Oct 11, 2019 at 16:50:39 -0400:
> On Fri, Oct 11, 2019 at 4:32 PM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
> > This is caused by mixing bytes object drived from file contents and str
> > object to construct log message.
>
> Does something like this answer help:
>
> https://stackoverflow.com/questions/31058055/how-do-i-convert-a-python-3-byte-string-variable-into-a-regular-string/31060836
>
> Something like:
> str(bytes_string, 'utf-8')
Nathan, I appreciate the intent, but Yasuhito and I are both familiar with the
semantics of str and bytes objects in Python 3. We're not asking what the
difference between bytes and str is, or how to work with them; we are simply
trying to resolve two particular test failures, in svnadmin_ and
tree_conflict_tests.py. Specifically, we're trying to determine whether file
contents should be handled as str or as bytes, both in the test function and in
the test framework.
Cheers,
Daniel
Re: 1.13.x and swig-py3
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
Though it is somewhat off topic....
On 2019-10-24 18:38, Stefan Sperling wrote:
> FreeBSD ships 1.10 as part of their base system (it would make sense for
> them to stick to LTS).
Subversion in FreeBSD base system are only command line programs, which
link libsvn_* statically, and not depend on Python to build. FreeBSD also
provides Subversion as ports/pkg both of newest release and LTS, however
langage bindings are provided only for newest release.
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: 1.13.x and swig-py3
Posted by Stefan Sperling <st...@elego.de>.
On Thu, Oct 24, 2019 at 02:46:39AM +0000, Daniel Shahaf wrote:
> Nathan Hartman wrote on Wed, Oct 23, 2019 at 01:11:19 -0400:
> > James McCoy wrote:
> > > FWIW, since we switched to the new release model, I (in my capacity as
> > > packager for Debian) basically ignore non-LTS releases. It's easier to
> > > ignore them, since I know I only want an LTS release in Debian's
> > > release, and would rather not risk getting "stuck" with a non-LTS
> > > release when Debian freezes.
> >
> > And this is completely okay!
> >
> > I use Debian myself. Most Debian packages "lag" behind the latest and
> > greatest. The same is true of other stability-oriented OSes. When you
> > have 50,000 packages in your repositories and a whole dependency hell
> > between them (circular / diamond dependencies, anyone?), you have to
> > move forward cautiously. This is where users must decide if they favor
> > stability or new features and choose their distro accordingly. It's a
> > trade-off.
> >
> > Thank you for maintaining the Debian package. It works great!
>
> Nathan, if I understood James correctly, he's saying that he doesn't package
> non-LTS releases even for Debian unstable and Debian testing, so users of those
> flavours of Debian will only be served LTS versions of Subversion by Debian.
> That _is_ noteworthy: those two flavours of Debian normally have little lag behind
> upstream. It's only the "stable" flavour that typically (by design) has lag.
>
> The upshot of this is that users of Debian testing/unstable, which are by and
> large "low lag" distros, don't get STS releases of Subversion.
>
> James, please correct me if I'm wrong.
I do not think that this is a problem. LTS releases are the equivalent
of releases we used to publish before the release schedule was changed.
Non-LTS releases are just additional releases which give us a chance to
get more feedback and give users regular access to new features and fixes.
What matters is that some of the systems out there run our regular releases.
And indeed, it is not as if everyone was using our LTS releases only.
OpenBSD releases usually ship a very recent release of SVN, which works
well because their release cycle is also 6 months long.
FreeBSD ships 1.10 as part of their base system (it would make sense for
them to stick to LTS).
NetBSD pkgsrc ships 1.12.
Fedora seems to ship 1.10 and 1.12.
ArchLinux ships 1.12.
OpenSUSE has 1.12, 1.9, and 1.8 available via their build service (I am not
sure what shipped in their latest release).
Slackware ships 1.9 in their release and carries 1.12 in -current.
I am not going to go on with this list. I think it already shows that there
is sufficient interest in our non-LTS releases, and that downstream consumers
are making choices for their users about LTS/non-LTS SVN releases, such that
the version of SVN shipped aligns with their respective project release cycles
and stability goals. This is good.
Re: 1.13.x and swig-py3
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Nathan Hartman wrote on Wed, Oct 23, 2019 at 01:11:19 -0400:
> James McCoy wrote:
> > FWIW, since we switched to the new release model, I (in my capacity as
> > packager for Debian) basically ignore non-LTS releases. It's easier to
> > ignore them, since I know I only want an LTS release in Debian's
> > release, and would rather not risk getting "stuck" with a non-LTS
> > release when Debian freezes.
>
> And this is completely okay!
>
> I use Debian myself. Most Debian packages "lag" behind the latest and
> greatest. The same is true of other stability-oriented OSes. When you
> have 50,000 packages in your repositories and a whole dependency hell
> between them (circular / diamond dependencies, anyone?), you have to
> move forward cautiously. This is where users must decide if they favor
> stability or new features and choose their distro accordingly. It's a
> trade-off.
>
> Thank you for maintaining the Debian package. It works great!
Nathan, if I understood James correctly, he's saying that he doesn't package
non-LTS releases even for Debian unstable and Debian testing, so users of those
flavours of Debian will only be served LTS versions of Subversion by Debian.
That _is_ noteworthy: those two flavours of Debian normally have little lag behind
upstream. It's only the "stable" flavour that typically (by design) has lag.
The upshot of this is that users of Debian testing/unstable, which are by and
large "low lag" distros, don't get STS releases of Subversion.
James, please correct me if I'm wrong.
Cheers,
Daniel
Re: 1.13.x and swig-py3
Posted by Nathan Hartman <ha...@gmail.com>.
I'll respond to Mark, Julian, Stefan, and James below...
Mark Phippard wrote:
> I thought the frequent release/LTS plan was a worth a try, I just do
> not see where it is working out and yielding benefits.
I'd like to point out that I have made several proposals but was
(politely) reminded that we are in a period of stabilization. So those
items and others are on my TODO list for later.
I think we need to clarify what being in a period of stabilization
means. It better not mean forever! I assume it means that we don't want
to rock the boat (too much) until after 1.14-LTS is released. After
that, I assume we have 2 STS releases for new developments followed by
1 "stabilization" STS to iron out and polish everything for the next
LTS. But this is all a subject for another thread.
Also, very much to the point:
Julian Foad wrote:
> I would like to point out we have gained two new enthusiastic
> contributors (Nathan, Yasuhito) this year.
Yes!
We wouldn't even be having this conversation, nor would we have swig-
py3, if it were not for the beautiful work Yasuhito has been doing. We
owe Yasuhito a very big Thank You!
I'm the other new developer and I have quite a lot in the pipeline,
much of it aimed at attracting new users and developers. I'm working on
a redesign and reorganization of the website, with more big items
planned afterwards. Regarding actual Subversion code, I'm focusing on
code quality at this time. I've uncovered an edge case in the tree
conflict resolver that I'm working to pinpoint. You'll see either a fix
or at least a reproduction script soon. Julian suggested and I have
promised to bring an old issue to the list every week. Last week we
closed our first weekly old issue. This week I posted the second (SVN-
1804) and I'm working on a fix that I'll propose soon. Like timed
releases, this process of "timed issue resolution" will continue for as
long as it takes to get our issue tracker cleaned up. I believe that
having 15 year old issues in there is a psychological drag on everyone.
It will be cleaned up!!
Also, we've recently reached out to a hackathon; not sure what (if any)
will materialize from that, but it's a good first step and much more
will follow. Stay tuned!
Stefan Sperling wrote:
> I also had concerns in the beginning when Julian brought up the
> proposal of time-based releases. I didn't expect it to work.
>
> But I think what Julian has shown us is that the calendar keeps us honest
> in our commitment to actually do releases.
<snip>
> It would be interesting to learn what our new developers think about this.
> I am sure that getting their changes released is important to them.
> Time-based releases provide a certain amount of anticipation and clarity
when
> a release is supposed to happen. In the past, we used to discuss what to
> release and when, every single time, and often delayed releases for
months.
> That could put new contributors off because they might not feel like they
> have a huge standing when arguing for their changes to be released soon.
> While the results of their work are sitting on trunk which nobody is
actually
> running in production. That certainly would not motivate me, if I try to
put
> myself in their shoes.
Stefan, that is exactly what I think. I couldn't have stated it better
myself.
We *should* do time-based releases. It's really important that we do.
IIRC one of the motivations for time-based releases was the new tree
conflict resolver. It has made Subversion so much better and so much
easier to use, yet it waited, largely untested, for two years. That
should never be allowed to happen!
There are myriad reasons why the managers of any organization must keep
things on schedule.
For us, the developers: As you and others have said, it "keeps us
honest." It imposes deadlines to get things done. It prevents
procrastinating on making a release. It makes expectations clear. It
avoids endless discussions on whether to make a release or not.
For our customers, the end users, server operators, IT departments,
decision makers, and packagers: It gives a tangible schedule to aid in
planning.
It also shows current and potential customers that Subversion is alive,
being maintained, and that future releases are scheduled. I can't
stress the importance of this point enough, especially given that we're
in the version control business, where the key is long-term safekeeping
of data and its history, and longevity of the version control software
itself.
I'd like to thank Julian for being diligent about driving the time-
based releases and for standing firm that they should happen on
schedule!
James McCoy wrote:
> FWIW, since we switched to the new release model, I (in my capacity as
> packager for Debian) basically ignore non-LTS releases. It's easier to
> ignore them, since I know I only want an LTS release in Debian's
> release, and would rather not risk getting "stuck" with a non-LTS
> release when Debian freezes.
And this is completely okay!
I use Debian myself. Most Debian packages "lag" behind the latest and
greatest. The same is true of other stability-oriented OSes. When you
have 50,000 packages in your repositories and a whole dependency hell
between them (circular / diamond dependencies, anyone?), you have to
move forward cautiously. This is where users must decide if they favor
stability or new features and choose their distro accordingly. It's a
trade-off.
Thank you for maintaining the Debian package. It works great!
And thanks to everyone here, for all that you do.
Nathan
Re: 1.13.x and swig-py3
Posted by James McCoy <ja...@jamessan.com>.
On Mon, Oct 21, 2019 at 01:35:45PM +0200, Branko Čibej wrote:
> On 21.10.2019 13:16, Julian Foad wrote:
> > Johan Corveleyn wrote:
> >> Nathan Hartman wrote:
> >> Branko Čibej wrote:
> >> > By the principle of least surprise, I think it
> >> > would be better to merge to trunk, create a
> >> > new 1.13.0 release candidate
> >>
> >> +1
> >>
> >> > and (maybe?) restart the soak.
> >>
> >> I support this idea even if the soak must restart or be extended.
> >>
> >> +1. Since 1.13 contains so very little, I think it's good to be a bit
> >> flexible with our planned timing here, to get this on board. I.e.
> >> let's merge it in, cut a new rc, and restart the soak.
> >
> > Dear all, with respect,
> >
> > I would love to see the Py3 support released ASAP.
> >
> > But, have we not learned from our past mistakes? We have prepared a
> > regular release that is right now looking to be ready to deploy next
> > week, on time. If we postpone and destabilize it now [1], this would
> > make mockery of "regular" releases. I would love to trust that
> > merging the branch will go smoothly with no follow-up required and no
> > extra time taken, but history has taught us that it is foolish to
> > assume so.
> >
> > Surely the right approach is to release what we have got (the
> > currently soaking 1.13), then release the new one as soon as we can
> > get it ready. It sounds like it's not suitable for a patch release, so
> > we'll make it a new minor release, calling it 1.14.
>
>
> If you (or some other RM volunteer) is prepared to roll 1.14 with Python
> 3 support in, say, a month after 1.13 -- with all that implies for
> downstream packagers -- then sure.
FWIW, since we switched to the new release model, I (in my capacity as
packager for Debian) basically ignore non-LTS releases. It's easier to
ignore them, since I know I only want an LTS release in Debian's
release, and would rather not risk getting "stuck" with a non-LTS
release when Debian freezes.
The downside to this is that the non-LTS releases don't get any of the
feedback of going through the various builds/tests on Debian's
infrastructure.
I'll likely make an exception for the Python 3 support since there's a
significant push within Debian to switch over to Python 3 ASAP, and
definitely in time for the next release.
Cheers,
--
James
GPG Key: 4096R/91BF BF4D 6956 BD5D F7B7 2D23 DFE6 91AE 331B A3DB
Re: 1.13.x and swig-py3
Posted by Stefan Sperling <st...@elego.de>.
On Tue, Oct 22, 2019 at 10:13:04AM -0400, Mark Phippard wrote:
> I do not think the current 1.13 release is worth releasing. I think we
> should merge the Py 3 branch and release that as 1.13 once we think it is
> ready. That should essentially become the release we maintain until there
> is something new worth generating a new minor release, whenever that may be.
Since the process for 1.13 has already started, why don't we finish the
release as planned? And discuss release policy changes for 1.14, which
is happening in only 6 months from now?
That would give us plenty of time to conclude this discussion without
rushing through it. And 1.14 is supposed to be the next LTS anyway.
I could be wrong but it seems like we're having this discussion only because
new work on python 3 and the new release cycle happened to overlap, and
otherwise we would not be having it?
If getting the python 3 changes out quickly is important, could we merge
some or all python 3 fixes into 1.13.1, or are they all breaking API?
I haven't had time to actually look at the patches and this entire thread,
so please excuse me if this has already been discussed and ruled out.
I also agree with Julian that it will take time to see results from our
release policy changes. We are still at the stage where we are learning
a new process and can fine-tune it based on new experience and insights.
As long as Julian is willing to keep up his role as a time-based release
manager, I believe we should keep trying. If we reach a point where there
is nothing at all to release after 6 months, we can still reconsider.
It would be interesting to learn what our new developers think about this.
I am sure that getting their changes released is important to them.
Time-based releases provide a certain amount of anticipation and clarity when
a release is supposed to happen. In the past, we used to discuss what to
release and when, every single time, and often delayed releases for months.
That could put new contributors off because they might not feel like they
have a huge standing when arguing for their changes to be released soon.
While the results of their work are sitting on trunk which nobody is actually
running in production. That certainly would not motivate me, if I try to put
myself in their shoes.
Re: 1.13.x and swig-py3
Posted by Mark Phippard <ma...@gmail.com>.
Julian's email reminded me I planned to reply to this one.
On Mon, Oct 21, 2019 at 10:59 AM Stefan Sperling <st...@elego.de> wrote:
> On Mon, Oct 21, 2019 at 10:25:01AM -0400, Mark Phippard wrote:
> > Yes. Go back to having new releases when there is enough interesting
> > content to justify the release. We can lower the bar from the old days.
> > There does not need to always be a big ticket feature or two driving the
> > release, but if there is nothing worthy of a release we should not do one
> > just because the calendar says so.
>
> I also had concerns in the beginning when Julian brought up the
> proposal of time-based releases. I didn't expect it to work.
>
> But I think what Julian has shown us is that the calendar keeps us honest
> in our commitment to actually do releases. If Julian hadn't been driving
> this, I believe we would be in a situation where fixes would accumulate
> on trunk and not be released at all, because a general lack of activity
> makes doing releases look like a lot of work unless we're used to doing
> them regularly and the process is streamlined accordingly.
>
> I have experience with time-based releases in another community (OpenBSD).
> Releases happen every 6 months, like clock work. There's not much worry
> about fixes and features missing the boat, because the next boat will be
> prepared and ready soon enough. Avoiding the introduction of regressions
> in new releases is a much bigger concern there, everything else can wait
> when in doubt.
>
> I think this would also work well for Subversion, but we have to adopt a
> mind set that makes this work.
>
I do not agree with this last statement (for the most part). Time-based
releases are great. I wish we had pushed for this back when 1.4 landed.
The difference is that OpenBSD and all of the other projects that follow
this are active. We are not. I do not think this is about mindset, it is
just facing the reality that there are not many changes happening. If
OpenBSD only landed 2 bugfixes in a 6-month release cycle and the previous
two releases were also pretty sparse, they might be having the same
conversations.
You raise a fair point that the clock does force some stuff. I am not sure
the Python 3 work would have seen the same uptick if a release was not
happening to stir things up again. I do not have any real objections if we
want to continue to use time based releases and enough people are willing
to put in the work to make that happen. I just do not think it is working
and we ought to consider a new approach.
I do not think the current 1.13 release is worth releasing. I think we
should merge the Py 3 branch and release that as 1.13 once we think it is
ready. That should essentially become the release we maintain until there
is something new worth generating a new minor release, whenever that may be.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: 1.13.x and swig-py3
Posted by Julian Foad <ju...@apache.org>.
Stefan Sperling wrote:
> [...] it sounds like we
> are very close to having full support for Python 3 on trunk already or
> will have it soon. Which means it will be ready in time for 1.14 LTS as
> originally scheduled. Is that really not enough?
Stefan, thank you for articulating so clearly, especially about Python2
EOL not being the end of the world and about the value of adhering to
regular releases.
Further thoughts...
Mark wrote,
> I thought the frequent release/LTS plan was a worth a try, I
> just do not see where it is working out and yielding benefits.
> Our activity and contributions continues to go down, not up.
> The releases are underwhelming.
It's true that 1.13 is "essentially empty" in terms of new features.
However, after making changes aimed at improving participation, it takes
time to see results. I would like to point out we have gained two new
enthusiastic contributors (Nathan, Yasuhito) this year.
Prompted by Nathan's "wacky idea": When the time comes for a regular
release, if there are no changes that require a new minor version, then
it makes sense we should just issue a patch release and not bump the
minor version number.
I compared {1.12.x,1.13.x}/subversion/include/ and saw one public API
change: the new svn_fs_ioctl() and its related declarations. That means
we do need a new minor release. On another occasion, if we were to
review sooner and find a situation like that, we might decide to remove
that public change (if that's an option), on the principle that we
should prefer a patch release. It is too late to consider changing it
for this release, but we should start looking out for that situation
arising in future, if we decide that's a good variant of the release
process.
While I object to postponing the current release, I am very open to
agreeing a wide range of changes to our release strategy for the next
releases. I agree with Mark's point that the current release strategy
is not right for the project and I think we should try to find a better
approach. That needs discussion in a separate thread.
- Julian
Re: 1.13.x and swig-py3
Posted by Stefan Sperling <st...@elego.de>.
On Mon, Oct 21, 2019 at 10:25:01AM -0400, Mark Phippard wrote:
> Yes. Go back to having new releases when there is enough interesting
> content to justify the release. We can lower the bar from the old days.
> There does not need to always be a big ticket feature or two driving the
> release, but if there is nothing worthy of a release we should not do one
> just because the calendar says so.
I also had concerns in the beginning when Julian brought up the
proposal of time-based releases. I didn't expect it to work.
But I think what Julian has shown us is that the calendar keeps us honest
in our commitment to actually do releases. If Julian hadn't been driving
this, I believe we would be in a situation where fixes would accumulate
on trunk and not be released at all, because a general lack of activity
makes doing releases look like a lot of work unless we're used to doing
them regularly and the process is streamlined accordingly.
I have experience with time-based releases in another community (OpenBSD).
Releases happen every 6 months, like clock work. There's not much worry
about fixes and features missing the boat, because the next boat will be
prepared and ready soon enough. Avoiding the introduction of regressions
in new releases is a much bigger concern there, everything else can wait
when in doubt.
I think this would also work well for Subversion, but we have to adopt a
mind set that makes this work.
In my opinion the world will not end for Subversion when Python 2 falls
out of support, even if no current release of SVN fully supports Python 3.
This is because our userbase has been slowing down with us. For the most
part, Subversion is not run on the latest version of Linux of the day.
It is run on server operating systems like Red Hat and Windows 20xx Server
which will still have Python 2 around. Our new LTS release scheme is a
good fit for this.
Most SVN clients are TortoiseSVN on corporate desktops. The Linux client
install base has moved on to Git/Hg. People being hired as software
developers on Linux today have usually never worked with SVN and won't
be expected to.
I have not followed these Python 3 patches closely but it sounds like we
are very close to having full support for Python 3 on trunk already or
will have it soon. Which means it will be ready in time for 1.14 LTS as
originally scheduled. Is that really not enough?
Re: 1.13.x and swig-py3
Posted by Mark Phippard <ma...@gmail.com>.
On Mon, Oct 21, 2019 at 8:29 AM Julian Foad <ju...@apache.org> wrote:
> Mark Phippard wrote:
> > I think it is just another example that "regular" releases are not
> appropriate for this project at this stage of its life with a lack of
> regular activity.
> [...]
> > Also, as brane already pointed out, this would also be making a mockery
> of regular releases.
>
> Mark, I do understand that for your case, each minor release is a
> problem. So far, we haven't been able to understand and resolve how we
> could alleviate that, other than by making fewer or no minor releases.
> In the mean time, making an extra one would be sore.
>
FWIW, I am trying to remove that hat for this discussion and am just
speaking with my PMC hat on. Obviously my opinions may be colored by the
release pain I feel, but I am trying not to factor that in.
I thought the frequent release/LTS plan was a worth a try, I just do not
see where it is working out and yielding benefits. Our activity and
contributions continues to go down, not up. The releases are underwhelming.
The release notes for 1.13 are essentially empty. You note there are 2
important fixes.
We actually have something highly relevant now with the Python 3 support
and due to unfortunate timing our release policy is getting in the way.
> Given that you're about the only downstream maintainer who discusses the
> issues here, I take that seriously.
>
> So, that takes us back to
>
> 1.) Proposing we stop or change the way we do minor releases, and do
> something different?
>
Yes. Go back to having new releases when there is enough interesting
content to justify the release. We can lower the bar from the old days.
There does not need to always be a big ticket feature or two driving the
release, but if there is nothing worthy of a release we should not do one
just because the calendar says so. Important fixes should be backported to
supported releases and we can go back to issuing bug fix releases when
needed. I would backport these important fixes to 1.10 and 1.12 and do new
fix releases and then release 1.13 with the Python 3 changes when we think
it is ready.
Moving forward we could continue to backport fixes and do new 1.13 releases
and do a 1.14 only when there was enough interest in doing so.
I do not recall what these two fixes are, but putting them in a 1.13
release is kind of hiding them from our maintainers. I will not pretend to
be highly fluent in Red Hat policies, as an example, but I know they are
good at backporting security fixes. They seem to be including our current
LTS release in 1.10. If we backported these fixes to our 1.10 then it
seems a lot more likely someone at Red Hat might see and consider including
these fixes in their version. By just having them in our 1.13 release it
seems a lot less likely they would see them. I would assume that is true
for other maintainers.
Anyway, rolling releases make more sense when there is more activity. Even
if we did not have a big ticket feature you could justify a release just
based on the cumulative amount of activity. I do not think this is the
case for our project where things have slowed to a crawl for the most part.
> Julian Foad wrote:
> >> Surely the right approach is to release what we have got [...], then
> >> [...] a new minor release, calling it 1.14.
> >
> > Would this be the LTS release it is supposed to be?
>
> Do you mean, "Would this new '1.14' be an LTS release?" The answer to
> that is no. It would be neither LTS nor regular. The next LTS will be
> released next April; we previously assumed it would be numbered 1.14-LTS
> but if we do things this way then it would probably be numbered 1.15-LTS
> instead.
>
Yeah, I was basically just referencing our past "declarations" that 1.14
would be an LTS release.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: 1.13.x and swig-py3
Posted by Nathan Hartman <ha...@gmail.com>.
On Mon, Oct 21, 2019 at 8:29 AM Julian Foad <ju...@apache.org> wrote:
> Mark Phippard wrote:
> > Also, as brane already pointed out, this would also be making a mockery
> of regular releases.
>
I'll throw this wacky idea out there...
(With endless respect for everyone here, always.)
Suppose we backport the contents of 1.13-rc1 to 1.12.3 such that 1.12.3
is identical to 1.13-rc1 except for version numbers. Afterwards, merge
swig-py3 to trunk, backport it to 1.13, issue 1.13-rc2, start a 4-week
soak.
This (wacky idea) is meant to accomplish several things:
1. We can get the important changes Julian mentioned released, as
1.12.3.
2. If I understand HACKING correctly, we can release 1.12.3 ON TIME --
meaning around the same time that we would have released 1.13, since it
would be a patch release that doesn't require a 4-week soak. Please
correct me if I'm mistaken on this point. And, for all practical
purposes, we did a soak.
3. Packagers who have worked on 1.13-rc1 won't lose the work they've
done -- PROVIDED that we communicate well that we decided to renumber
it as a patch release and that they'll find all contents identical
except for version numbers. I hope Mark will tell us if this would be
an acceptable compromise?
4. We wouldn't "make a mockery" of regular releases, since 1.12.3 would
occur at the regular time, only with a different version number than we
previously thought. Some might balk at that different version number
but I think that's a much better scenario than 1.14 suddenly not being
the next LTS release, since we've been communicating that it would be
LTS for quite some time and it suddenly not being LTS would confuse a
lot of people. No one will be confused by a 1.12.3 that fixes bugs.
5. We can test the heck out of 1.13 with swig-py3 before rolling 1.13-
rc2. Even if we find blocking breakage, we have time to restart the
soak and still get it out there before Python 2 goes EOL.
Justification for all this:
While we have a release schedule (and YES, I do think we should stick
to it!), Python 2 going EOL is an exceptional circumstance; and so, if
this event causes our release schedule to be a bit different, I don't
see that as a negative; rather, I see it as being responsive to a
pretty significant change that is coming and to users who have written
to us with questions and concerns about this change.
Now, like I said:
* I say all of this with endless respect for everyone.
* I'm throwing it out there as a wacky idea.
Nathan
Re: 1.13.x and swig-py3
Posted by Julian Foad <ju...@apache.org>.
Mark Phippard wrote:
> I think it is just another example that "regular" releases are not appropriate for this project at this stage of its life with a lack of regular activity.
[...]
> Also, as brane already pointed out, this would also be making a mockery of regular releases.
Mark, I do understand that for your case, each minor release is a
problem. So far, we haven't been able to understand and resolve how we
could alleviate that, other than by making fewer or no minor releases.
In the mean time, making an extra one would be sore.
Given that you're about the only downstream maintainer who discusses the
issues here, I take that seriously.
So, that takes us back to
1.) Proposing we stop or change the way we do minor releases, and do
something different?
2.) Maybe not insert an extra release. Instead,
2a.) postpone and change the release we've already prepared?; or
2b.) get the work ready for the next LTS instead?
What do you think we should do?
> Julian Foad wrote:
>> Surely the right approach is to release what we have got [...], then
>> [...] a new minor release, calling it 1.14.
>
> Would this be the LTS release it is supposed to be?
Do you mean, "Would this new '1.14' be an LTS release?" The answer to
that is no. It would be neither LTS nor regular. The next LTS will be
released next April; we previously assumed it would be numbered 1.14-LTS
but if we do things this way then it would probably be numbered 1.15-LTS
instead.
- Julian
Re: 1.13.x and swig-py3
Posted by Mark Phippard <ma...@gmail.com>.
> On Oct 21, 2019, at 7:16 AM, Julian Foad <ju...@apache.org> wrote:
>
> Johan Corveleyn wrote:
>> Nathan Hartman wrote:
>> Branko Čibej wrote:
>> > By the principle of least surprise, I think it
>> > would be better to merge to trunk, create a
>> > new 1.13.0 release candidate
>> +1
>> > and (maybe?) restart the soak.
>> I support this idea even if the soak must restart or be extended.
>> +1. Since 1.13 contains so very little, I think it's good to be a bit flexible with our planned timing here, to get this on board. I.e. let's merge it in, cut a new rc, and restart the soak.
>
> Dear all, with respect,
>
> I would love to see the Py3 support released ASAP.
>
> But, have we not learned from our past mistakes? We have prepared a regular release that is right now looking to be ready to deploy next week, on time. If we postpone and destabilize it now [1], this would make mockery of "regular" releases. I would love to trust that merging the branch will go smoothly with no follow-up required and no extra time taken, but history has taught us that it is foolish to assume so.
I think it is just another example that "regular" releases are not appropriate for this project at this stage of its life with a lack of regular activity.
> Surely the right approach is to release what we have got (the currently soaking 1.13), then release the new one as soon as we can get it ready. It sounds like it's not suitable for a patch release, so we'll make it a new minor release, calling it 1.14.
Would this be the LTS release it is supposed to be? Also, as brane already pointed out, this would also be making a mockery of regular releases.
Mark
Re: 1.13.x and swig-py3
Posted by Johan Corveleyn <jc...@gmail.com>.
On Mon, Oct 21, 2019 at 1:35 PM Branko Čibej <br...@apache.org> wrote:
> On 21.10.2019 13:16, Julian Foad wrote:
...
> > Surely the right approach is to release what we have got (the
> > currently soaking 1.13), then release the new one as soon as we can
> > get it ready. It sounds like it's not suitable for a patch release, so
> > we'll make it a new minor release, calling it 1.14.
>
>
> If you (or some other RM volunteer) is prepared to roll 1.14 with Python
> 3 support in, say, a month after 1.13 -- with all that implies for
> downstream packagers -- then sure.
I had the same thought. Creating more releases implies a lot of work
for downstream.
I'd say: if we don't want to postpone 1.13 for py3 (and we can't
backport it to a patch release), then we should indeed fix it for
1.14, but not make an extra release out of it. It will simply be part
of 1.14 LTS, to be released somewhere around April 2020.
Now, can someone please find a release blocking bug in 1.13, so we
have a valid reason for restarting the soak? ;-)
Seriously: we always have the option to retract / re-cut / ... 1.13 as
long as it has not been released. The projected release date is just
that, a _projected_ release date, which we aim for. If there is a
serious reason we can postpone the release ... (I understand that this
may not be a "serious reason" ... but downstreamers anyway have to
account for the possibility that an RC is thrown away even at the last
minute).
--
Johan
Re: 1.13.x and swig-py3
Posted by Branko Čibej <br...@apache.org>.
On 21.10.2019 13:16, Julian Foad wrote:
> Johan Corveleyn wrote:
>> Nathan Hartman wrote:
>> Branko Čibej wrote:
>> > By the principle of least surprise, I think it
>> > would be better to merge to trunk, create a
>> > new 1.13.0 release candidate
>>
>> +1
>>
>> > and (maybe?) restart the soak.
>>
>> I support this idea even if the soak must restart or be extended.
>>
>> +1. Since 1.13 contains so very little, I think it's good to be a bit
>> flexible with our planned timing here, to get this on board. I.e.
>> let's merge it in, cut a new rc, and restart the soak.
>
> Dear all, with respect,
>
> I would love to see the Py3 support released ASAP.
>
> But, have we not learned from our past mistakes? We have prepared a
> regular release that is right now looking to be ready to deploy next
> week, on time. If we postpone and destabilize it now [1], this would
> make mockery of "regular" releases. I would love to trust that
> merging the branch will go smoothly with no follow-up required and no
> extra time taken, but history has taught us that it is foolish to
> assume so.
>
> Surely the right approach is to release what we have got (the
> currently soaking 1.13), then release the new one as soon as we can
> get it ready. It sounds like it's not suitable for a patch release, so
> we'll make it a new minor release, calling it 1.14.
If you (or some other RM volunteer) is prepared to roll 1.14 with Python
3 support in, say, a month after 1.13 -- with all that implies for
downstream packagers -- then sure.
> Nothing says we shouldn't release an extra minor release, or that we
> shouldn't make two minor releases close together.
Well, other than that it is also, strictly speaking, a "mockery of
regular releases." :)
-- Brane
Re: 1.13.x and swig-py3
Posted by Julian Foad <ju...@apache.org>.
Johan Corveleyn wrote:
> Nathan Hartman wrote:
> Branko Čibej wrote:
> > By the principle of least surprise, I think it
> > would be better to merge to trunk, create a
> > new 1.13.0 release candidate
>
> +1
>
> > and (maybe?) restart the soak.
>
> I support this idea even if the soak must restart or be extended.
>
> +1. Since 1.13 contains so very little, I think it's good to be a bit
> flexible with our planned timing here, to get this on board. I.e. let's
> merge it in, cut a new rc, and restart the soak.
Dear all, with respect,
I would love to see the Py3 support released ASAP.
But, have we not learned from our past mistakes? We have prepared a
regular release that is right now looking to be ready to deploy next
week, on time. If we postpone and destabilize it now [1], this would
make mockery of "regular" releases. I would love to trust that merging
the branch will go smoothly with no follow-up required and no extra time
taken, but history has taught us that it is foolish to assume so.
Surely the right approach is to release what we have got (the currently
soaking 1.13), then release the new one as soon as we can get it ready.
It sounds like it's not suitable for a patch release, so we'll make it a
new minor release, calling it 1.14.
Nothing says we shouldn't release an extra minor release, or that we
shouldn't make two minor releases close together.
Sure, there are no major developments in 1.13, but there are important
fixes [2]. And sure, 1.14 was previously listed as expected to be the
next LTS, and now it won't be. And people will need to choose whether
to upgrade to both or just one of them. And we will have to adjust our
terminology and docs a little as we will be calling it neither "regular"
nor "LTS". All small things to sort out. No big deal.
Then we will achieve everything we wanted. The regular releases remain
regular. The new stuff gets released ASAP, and before the next LTS release.
The existing 1.13 release has value in itself. In the role of release
manager, I plan to go ahead with the 1.13 release, and I would urge you
to support this by helping provide the final tests and signatures. Then
I will be willing to roll another release, including all the web site
updates etc, as soon as we can get it ready.
Can we do it that way, please?
- Julian
[1] Yes, from a release manager's POV, this is destabilizing, no matter
how much it has been tested already.
[2] There are important fixes in 1.13 that, if 1.13 were not to be
available, should otherwise be backported and released in 1.12. We are
not releasing them in a 1.12 patch because 1.13 is "known" to be coming
soon.
Re: 1.13.x and swig-py3 (was: Re: Test failures with Python 3 (Re:
PMCs: any Hackathon requests? (deadline 11 October)))
Posted by Johan Corveleyn <jc...@gmail.com>.
Op zo 20 okt. 2019 03:51 schreef Nathan Hartman <ha...@gmail.com>:
> On Sat, Oct 19, 2019 at 2:11 PM Branko Čibej <br...@apache.org> wrote:
> > By the principle of least surprise, I think it
> > would be better to merge to trunk, create a
> > new 1.13.0 release candidate
>
> +1
>
> > and (maybe?) restart the soak.
>
> I support this idea even if the soak must restart or be extended.
>
+1. Since 1.13 contains so very little, I think it's good to be a bit
flexible with our planned timing here, to get this on board. I.e. let's
merge it in, cut a new rc, and restart the soak.
--
Johan
Re: 1.13.x and swig-py3 (was: Re: Test failures with Python 3 (Re:
PMCs: any Hackathon requests? (deadline 11 October)))
Posted by Branko Čibej <br...@apache.org>.
On 20.10.2019 04:05, Daniel Shahaf wrote:
> Nathan Hartman wrote on Sun, 20 Oct 2019 01:49 +00:00:
>> I support this idea even if the soak must restart or be extended.
> Brainstorming: How about letting the soak continue but documenting in
> the release notes, release announcements, etc., that we'll be adding swig-
> py3 support in a patch release? This way, swig-py2 users will know not
> to upgrade from 1.12.x until after the destabilizing changes are made.
The thing is that the swig-py3 branch introduces a new dependency (py3c)
for Py2 bindings, not just for Py3. That's a big deal even for
packagers, IMO.
(Which reminds me: get-deps.sh still needs to be updated to download py3c.)
> I assume the parts of the swig-py3 branch outside subversion/bindings/
> aren't destabilizing and would not be a concern to backport in a patch
> release.
Those are mainly test fixes, so sure.
-- Brane
Re: 1.13.x and swig-py3 (was: Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests? (deadline 11 October)))
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Nathan Hartman wrote on Sun, 20 Oct 2019 01:49 +00:00:
> I support this idea even if the soak must restart or be extended.
Brainstorming: How about letting the soak continue but documenting in
the release notes, release announcements, etc., that we'll be adding swig-
py3 support in a patch release? This way, swig-py2 users will know not
to upgrade from 1.12.x until after the destabilizing changes are made.
I assume the parts of the swig-py3 branch outside subversion/bindings/
aren't destabilizing and would not be a concern to backport in a patch
release.
Cheers,
Daniel
Re: 1.13.x and swig-py3 (was: Re: Test failures with Python 3 (Re:
PMCs: any Hackathon requests? (deadline 11 October)))
Posted by Nathan Hartman <ha...@gmail.com>.
On Sat, Oct 19, 2019 at 2:11 PM Branko Čibej <br...@apache.org> wrote:
> By the principle of least surprise, I think it
> would be better to merge to trunk, create a
> new 1.13.0 release candidate
+1
> and (maybe?) restart the soak.
I support this idea even if the soak must restart or be extended.
Rationale:
* we still get 1.13 out in time for the holidays
* we get one STS release with Py3 before the
next LTS release
* we support Py3 before Py2 goes "out of
support" -- there's no "lapse in coverage" for
customers who have IT policies about that
(One corporate or government customer wrote
to users@ some months ago and specifically
mentioned concern because of such a policy;
I'd love to reply and say that Py3 is supported
on time.)
Bottom line: we'll be a little late in the release but in exchange we'll
have one really whiz-bang compelling new feature -- that people have been
asking for -- to write about in the
release notes.
Nathan
Re: 1.13.x and swig-py3 (was: Re: Test failures with Python 3 (Re:
PMCs: any Hackathon requests? (deadline 11 October)))
Posted by Branko Čibej <br...@apache.org>.
On 19.10.2019 12:06, Johan Corveleyn wrote:
> On Sat, Oct 19, 2019 at 1:23 AM Daniel Shahaf <d....@daniel.shahaf.name> wrote:
>> Yasuhito FUTATSUKI wrote on Fri, Oct 18, 2019 at 23:09:19 +0900:
>>> On 2019/10/16 21:12, Johan Corveleyn wrote:
>>>> This makes me wonder: should that be fixed specifically on trunk, and
>>>> nominated for backport to 1.13, so we can possibly claim basic support
>>>> for Python 3 in our build and test processes (in at least one released
>>>> version) before the end of this year?
>>>>
>>>> Or should we reintegrate the swig-py3 branch ASAP, and nominate *that*
>>>> for backport to 1.13, so we can have Python 3 support, including swig
>>>> bindings?
>>> I prefer the latter, as one of users :) I want to use
>>> tools/hook-scripts/mailer/mailer.py with Python 3.
>> If we want this to happen, we should first of all complete the swig-py3 branch
>> and merge it to trunk.
> Yes, and I think the branch is now ready for merge to trunk.
>
>> What's not clear to me is what would happen afterwards. Is anyone proposing to
>> delay 1.13.0 until swig-py3 is merged (remember that we are already more than
>> halfway through the soak)? If not, how would merging swig-py3 to 1.13.x
>> coexist with the premise of "no destabilizing changes in patch releases"?
>> Would we have to delay merging swig-py3 to 1.13.x until January, when py2 has
>> finally gone out of support?
> I wouldn't postpone 1.13.0 for it. I suppose the best way is that we
> propose it for backport for 1.13.1, shortly after 1.13.0 has been
> released.
> Also: the swig-py3 branch does not break our support for py2. With
> that branch, both py2 and py3 swig-bindings can be built and run fine.
But it does introduce major changes in the bindings, for Py2 as well.
I'd really, really rather not drop them on unsuspecting users in a patch
release.
By the principle of least surprise, I think it would be better to merge
to trunk, create a new 1.13.0 release candidate and (maybe?) restart the
soak.
>> What should we do about swig-py3 support in 1.12.x and 1.10.x, which are both LTS?
> Only 1.10.x is LTS (and 1.9.x perhaps? But ISTR we eol'ed that). I
> suppose a backport to 1.10.x would be a good idea, so we have at least
> one LTS release this year that supports py3.
-0, for the same reason as above. We'll have the LTS that supports
Python 3 in a few months. Anyone who cares so much about Python 3
support will be able to use 1.13.0 (or .x). And of course ... EOL or
not, Python 2 will be around for a while yet.
>> Should we say anything about swig-py2 / swig-py3 in the release notes _today_,
>> before 1.13.0 has been released, about our plans for 1.13.x patch releases?
Given that I prefer releasing this with 1.13.0, the answer is, of
course, "yes."
-- Brane
Re: 1.13.x and swig-py3 (was: Re: Test failures with Python 3 (Re:
PMCs: any Hackathon requests? (deadline 11 October)))
Posted by Johan Corveleyn <jc...@gmail.com>.
On Sat, Oct 19, 2019 at 1:23 AM Daniel Shahaf <d....@daniel.shahaf.name> wrote:
>
> Yasuhito FUTATSUKI wrote on Fri, Oct 18, 2019 at 23:09:19 +0900:
> > On 2019/10/16 21:12, Johan Corveleyn wrote:
> > > This makes me wonder: should that be fixed specifically on trunk, and
> > > nominated for backport to 1.13, so we can possibly claim basic support
> > > for Python 3 in our build and test processes (in at least one released
> > > version) before the end of this year?
> > >
> > > Or should we reintegrate the swig-py3 branch ASAP, and nominate *that*
> > > for backport to 1.13, so we can have Python 3 support, including swig
> > > bindings?
> >
> > I prefer the latter, as one of users :) I want to use
> > tools/hook-scripts/mailer/mailer.py with Python 3.
>
> If we want this to happen, we should first of all complete the swig-py3 branch
> and merge it to trunk.
Yes, and I think the branch is now ready for merge to trunk.
> What's not clear to me is what would happen afterwards. Is anyone proposing to
> delay 1.13.0 until swig-py3 is merged (remember that we are already more than
> halfway through the soak)? If not, how would merging swig-py3 to 1.13.x
> coexist with the premise of "no destabilizing changes in patch releases"?
> Would we have to delay merging swig-py3 to 1.13.x until January, when py2 has
> finally gone out of support?
I wouldn't postpone 1.13.0 for it. I suppose the best way is that we
propose it for backport for 1.13.1, shortly after 1.13.0 has been
released.
Also: the swig-py3 branch does not break our support for py2. With
that branch, both py2 and py3 swig-bindings can be built and run fine.
> What should we do about swig-py3 support in 1.12.x and 1.10.x, which are both LTS?
Only 1.10.x is LTS (and 1.9.x perhaps? But ISTR we eol'ed that). I
suppose a backport to 1.10.x would be a good idea, so we have at least
one LTS release this year that supports py3.
> Should we say anything about swig-py2 / swig-py3 in the release notes _today_,
> before 1.13.0 has been released, about our plans for 1.13.x patch releases?
I think we should mention it at least in the 1.13.0 release notes, in
the known issues section, and announce our plan to address it in
1.13.1 or something like that?
--
Johan
1.13.x and swig-py3 (was: Re: Test failures with Python 3 (Re: PMCs:
any Hackathon requests? (deadline 11 October)))
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Yasuhito FUTATSUKI wrote on Fri, Oct 18, 2019 at 23:09:19 +0900:
> On 2019/10/16 21:12, Johan Corveleyn wrote:
> > This makes me wonder: should that be fixed specifically on trunk, and
> > nominated for backport to 1.13, so we can possibly claim basic support
> > for Python 3 in our build and test processes (in at least one released
> > version) before the end of this year?
> >
> > Or should we reintegrate the swig-py3 branch ASAP, and nominate *that*
> > for backport to 1.13, so we can have Python 3 support, including swig
> > bindings?
>
> I prefer the latter, as one of users :) I want to use
> tools/hook-scripts/mailer/mailer.py with Python 3.
If we want this to happen, we should first of all complete the swig-py3 branch
and merge it to trunk.
What's not clear to me is what would happen afterwards. Is anyone proposing to
delay 1.13.0 until swig-py3 is merged (remember that we are already more than
halfway through the soak)? If not, how would merging swig-py3 to 1.13.x
coexist with the premise of "no destabilizing changes in patch releases"?
Would we have to delay merging swig-py3 to 1.13.x until January, when py2 has
finally gone out of support?
What should we do about swig-py3 support in 1.12.x and 1.10.x, which are both LTS?
Should we say anything about swig-py2 / swig-py3 in the release notes _today_,
before 1.13.0 has been released, about our plans for 1.13.x patch releases?
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests? (deadline 11 October))
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Yasuhito FUTATSUKI wrote on Thu, 17 Oct 2019 22:57 +00:00:
> I'm sorry, I missed to answer to these questions.
>
> On 2019/10/16 16:55, Daniel Shahaf wrote:
> > Good catch. Yes, we should update INSTALL to reflect that Python 3 is
> > supported for the build and test process, even if it's not yet supported
> > by the swig bindings in trunk and 1.13.x. Would you be able to update
> > that? You're welcome to just commit changes to trunk/INSTALL directly,
> > if that's easier for you.
>
> Yes, I'll do it, after resolved issues on test with Python3, the false
> positive issue and the test on Windows issue.
Thank you.
> > As to the test suite patches, I think they're ready to be committed,
> > aren't they?
>
> Yes, at least I also think so, putting aside commit log message.
In this case, please do go ahead and commit them :-)
Cheers,
Daniel
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
I'm sorry, I missed to answer to these questions.
On 2019/10/16 16:55, Daniel Shahaf wrote:
> Good catch. Yes, we should update INSTALL to reflect that Python 3 is
> supported for the build and test process, even if it's not yet supported
> by the swig bindings in trunk and 1.13.x. Would you be able to update
> that? You're welcome to just commit changes to trunk/INSTALL directly,
> if that's easier for you.
Yes, I'll do it, after resolved issues on test with Python3, the false
positive issue and the test on Windows issue.
> As to the test suite patches, I think they're ready to be committed,
> aren't they?
Yes, at least I also think so, putting aside commit log message.
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests? (deadline 11 October))
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Johan Corveleyn wrote on Wed, 16 Oct 2019 12:12 +00:00:
> This makes me wonder: should that be fixed specifically on trunk, and
> nominated for backport to 1.13, so we can possibly claim basic support
> for Python 3 in our build and test processes (in at least one released
> version) before the end of this year?
+1
> Or should we reintegrate the swig-py3 branch ASAP, and nominate *that*
> for backport to 1.13, so we can have Python 3 support, including swig
> bindings?
Good question.
Cheers,
Daniel
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/10/16 21:12, Johan Corveleyn wrote:
> On Wed, Oct 16, 2019 at 1:37 PM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
>>
>> On 2019/10/16 18:10, Johan Corveleyn wrote:
>>>> Python 3 for the build and test process is only supported on *nix, not
>>> on Windows.
>>>
>>> [[[
>>> Traceback (most recent call last):
>>> File "win-tests.py", line 134, in <module>
>>> cp.items('options'))
>>> File "build\generator\gen_win_dependencies.py", line 306, in __init__
>>> self.find_libraries(False)
>>> File "build\generator\gen_win_dependencies.py", line 327, in find_libraries
>>> self._find_jdk(show_warnings)
>>> File "build\generator\gen_win_dependencies.py", line 1085, in _find_jdk
>>> vermatch = re.search(r'(([0-9]+(\.[0-9]+)+)(_[._0-9]+)?)', line, re.M)
>>> File "C:\Python37\lib\re.py", line 183, in search
>>> return _compile(pattern, flags).search(string)
>>> TypeError: cannot use a string pattern on a bytes-like object
>>> ]]]
>>>
>>> The fix is probably easy, but I'm just noting it here so we don't get
>>> ahead of ourselves.
>>>
>>
>> It seems the change addressing for it on swig-py3 branch is a part of
>> r1822485, the hunks attached.
>
> Ack.
I've looked over the log on swig-py3 branch and picked up more hunks,
and I've made a patch against trunk@1868582.
Could you please test on trunk on Windows with this patch?
> This makes me wonder: should that be fixed specifically on trunk, and
> nominated for backport to 1.13, so we can possibly claim basic support
> for Python 3 in our build and test processes (in at least one released
> version) before the end of this year?
>
> Or should we reintegrate the swig-py3 branch ASAP, and nominate *that*
> for backport to 1.13, so we can have Python 3 support, including swig
> bindings?
I prefer the latter, as one of users :) I want to use
tools/hook-scripts/mailer/mailer.py with Python 3.
Thanks,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Johan Corveleyn <jc...@gmail.com>.
On Wed, Oct 16, 2019 at 1:37 PM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
>
> On 2019/10/16 18:10, Johan Corveleyn wrote:
> > > Python 3 for the build and test process is only supported on *nix, not
> > on Windows.
> >
> > [[[
> > Traceback (most recent call last):
> > File "win-tests.py", line 134, in <module>
> > cp.items('options'))
> > File "build\generator\gen_win_dependencies.py", line 306, in __init__
> > self.find_libraries(False)
> > File "build\generator\gen_win_dependencies.py", line 327, in find_libraries
> > self._find_jdk(show_warnings)
> > File "build\generator\gen_win_dependencies.py", line 1085, in _find_jdk
> > vermatch = re.search(r'(([0-9]+(\.[0-9]+)+)(_[._0-9]+)?)', line, re.M)
> > File "C:\Python37\lib\re.py", line 183, in search
> > return _compile(pattern, flags).search(string)
> > TypeError: cannot use a string pattern on a bytes-like object
> > ]]]
> >
> > The fix is probably easy, but I'm just noting it here so we don't get
> > ahead of ourselves.
> >
>
> It seems the change addressing for it on swig-py3 branch is a part of
> r1822485, the hunks attached.
Ack.
This makes me wonder: should that be fixed specifically on trunk, and
nominated for backport to 1.13, so we can possibly claim basic support
for Python 3 in our build and test processes (in at least one released
version) before the end of this year?
Or should we reintegrate the swig-py3 branch ASAP, and nominate *that*
for backport to 1.13, so we can have Python 3 support, including swig
bindings?
--
Johan
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/10/16 18:10, Johan Corveleyn wrote:
> > Python 3 for the build and test process is only supported on *nix, not
> on Windows.
>
> [[[
> Traceback (most recent call last):
> File "win-tests.py", line 134, in <module>
> cp.items('options'))
> File "build\generator\gen_win_dependencies.py", line 306, in __init__
> self.find_libraries(False)
> File "build\generator\gen_win_dependencies.py", line 327, in find_libraries
> self._find_jdk(show_warnings)
> File "build\generator\gen_win_dependencies.py", line 1085, in _find_jdk
> vermatch = re.search(r'(([0-9]+(\.[0-9]+)+)(_[._0-9]+)?)', line, re.M)
> File "C:\Python37\lib\re.py", line 183, in search
> return _compile(pattern, flags).search(string)
> TypeError: cannot use a string pattern on a bytes-like object
> ]]]
>
> The fix is probably easy, but I'm just noting it here so we don't get
> ahead of ourselves.
>
It seems the change addressing for it on swig-py3 branch is a part of
r1822485, the hunks attached.
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Johan Corveleyn <jc...@gmail.com>.
On Wed, Oct 16, 2019 at 9:55 AM Daniel Shahaf <d....@daniel.shahaf.name> wrote:
>
> Yasuhito FUTATSUKI wrote on Wed, 16 Oct 2019 04:40 +00:00:
> > On 2019/10/16 10:36, Daniel Shahaf wrote:
> > > Yasuhito FUTATSUKI wrote on Tue, Oct 15, 2019 at 16:19:46 +0900:
> > >> On 2019-10-15 07:04, Daniel Shahaf wrote:
> > >>> Yasuhito FUTATSUKI wrote on Sun, 13 Oct 2019 04:01 +00:00:
> > >>>> On 2019/10/13 7:24, Daniel Shahaf wrote:
> >
> > >>> Also, what about the svnadmin_tests.py patch you posted upthread? Is
> > >>> there a reason not to go ahead and commit it to trunk? (and even nominate
> > >>> it for backport in 1.13.x/STATUS)
> > >>
> > >> I think that is no probrem.
> > >
> > > Let me elaborate. By my reckoning, «make check» on trunk has a false positive
> > > when ${PYTHON} is python3; and furthermore, python2 will reach EOL during the
> > > lifetime of the 1.13 branch; therefore, we should fix the false positive and
> > > backport the fix to supported branches. What's your view?
> >
> > I agree with you, because I think we should publish at least one relase
> > support Python 3, so that we can remove the sentence "Note that Python 3.x
> > is not supported and most likely won't work." in "Dependencies in Detail"
> > section of INSTALL file, before EOL of Python 2.
>
> Good catch. Yes, we should update INSTALL to reflect that Python 3 is
> supported for the build and test process, even if it's not yet supported
> by the swig bindings in trunk and 1.13.x. Would you be able to update
> that? You're welcome to just commit changes to trunk/INSTALL directly,
> if that's easier for you.
Python 3 for the build and test process is only supported on *nix, not
on Windows.
[[[
Traceback (most recent call last):
File "win-tests.py", line 134, in <module>
cp.items('options'))
File "build\generator\gen_win_dependencies.py", line 306, in __init__
self.find_libraries(False)
File "build\generator\gen_win_dependencies.py", line 327, in find_libraries
self._find_jdk(show_warnings)
File "build\generator\gen_win_dependencies.py", line 1085, in _find_jdk
vermatch = re.search(r'(([0-9]+(\.[0-9]+)+)(_[._0-9]+)?)', line, re.M)
File "C:\Python37\lib\re.py", line 183, in search
return _compile(pattern, flags).search(string)
TypeError: cannot use a string pattern on a bytes-like object
]]]
The fix is probably easy, but I'm just noting it here so we don't get
ahead of ourselves.
--
Johan
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests? (deadline 11 October))
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Yasuhito FUTATSUKI wrote on Wed, 16 Oct 2019 04:40 +00:00:
> On 2019/10/16 10:36, Daniel Shahaf wrote:
> > Yasuhito FUTATSUKI wrote on Tue, Oct 15, 2019 at 16:19:46 +0900:
> >> On 2019-10-15 07:04, Daniel Shahaf wrote:
> >>> Yasuhito FUTATSUKI wrote on Sun, 13 Oct 2019 04:01 +00:00:
> >>>> On 2019/10/13 7:24, Daniel Shahaf wrote:
>
> >>> Also, what about the svnadmin_tests.py patch you posted upthread? Is
> >>> there a reason not to go ahead and commit it to trunk? (and even nominate
> >>> it for backport in 1.13.x/STATUS)
> >>
> >> I think that is no probrem.
> >
> > Let me elaborate. By my reckoning, «make check» on trunk has a false positive
> > when ${PYTHON} is python3; and furthermore, python2 will reach EOL during the
> > lifetime of the 1.13 branch; therefore, we should fix the false positive and
> > backport the fix to supported branches. What's your view?
>
> I agree with you, because I think we should publish at least one relase
> support Python 3, so that we can remove the sentence "Note that Python 3.x
> is not supported and most likely won't work." in "Dependencies in Detail"
> section of INSTALL file, before EOL of Python 2.
Good catch. Yes, we should update INSTALL to reflect that Python 3 is
supported for the build and test process, even if it's not yet supported
by the swig bindings in trunk and 1.13.x. Would you be able to update
that? You're welcome to just commit changes to trunk/INSTALL directly,
if that's easier for you.
As to the test suite patches, I think they're ready to be committed,
aren't they?
Thanks,
Daniel
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/10/16 10:36, Daniel Shahaf wrote:
> Yasuhito FUTATSUKI wrote on Tue, Oct 15, 2019 at 16:19:46 +0900:
>> On 2019-10-15 07:04, Daniel Shahaf wrote:
>>> Yasuhito FUTATSUKI wrote on Sun, 13 Oct 2019 04:01 +00:00:
>>>> On 2019/10/13 7:24, Daniel Shahaf wrote:
>>> Also, what about the svnadmin_tests.py patch you posted upthread? Is
>>> there a reason not to go ahead and commit it to trunk? (and even nominate
>>> it for backport in 1.13.x/STATUS)
>>
>> I think that is no probrem.
>
> Let me elaborate. By my reckoning, «make check» on trunk has a false positive
> when ${PYTHON} is python3; and furthermore, python2 will reach EOL during the
> lifetime of the 1.13 branch; therefore, we should fix the false positive and
> backport the fix to supported branches. What's your view?
I agree with you, because I think we should publish at least one relase
support Python 3, so that we can remove the sentence "Note that Python 3.x
is not supported and most likely won't work." in "Dependencies in Detail"
section of INSTALL file, before EOL of Python 2.
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Yasuhito FUTATSUKI wrote on Tue, Oct 15, 2019 at 16:19:46 +0900:
> On 2019-10-15 07:04, Daniel Shahaf wrote:
> > Yasuhito FUTATSUKI wrote on Sun, 13 Oct 2019 04:01 +00:00:
> > > On 2019/10/13 7:24, Daniel Shahaf wrote:
>
> > > I see. Now I agree it would suffice here.
> > > > So, how about:
> > > >
> > > > 1. Make the test use non-binary mode for changing and reading the
> > > > file 'lambda'.
> > > >
> > > > 2. Locally revert the C part of r1841731 and make sure the modified test
> > > > still (correctly) fails. (That revision both added the test and
> > > > fixed the bug the test checks for.)
> > >
> > > So it looks sufficient to me.
> >
> > Cool. Will you perchance have time to do this? No worries if not.
>
> Yes, I'll do it on FreeBSD on tonight or tomorrow night (in JST :)).
Thanks.
> However I think it is also need to test for each 1 and 2 on Windows,
> because r1841736 and r1841743 also were attempt to fix this test
> on Windows, with Python 2.
I trust your judgement on this.
> > Also, what about the svnadmin_tests.py patch you posted upthread? Is
> > there a reason not to go ahead and commit it to trunk? (and even nominate
> > it for backport in 1.13.x/STATUS)
>
> I think that is no probrem.
Let me elaborate. By my reckoning, «make check» on trunk has a false positive
when ${PYTHON} is python3; and furthermore, python2 will reach EOL during the
lifetime of the 1.13 branch; therefore, we should fix the false positive and
backport the fix to supported branches. What's your view?
Cheers,
Daniel
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Johan Corveleyn wrote on Wed, Oct 16, 2019 at 01:07:32 +0200:
> I did have some trouble testing it with Python 3.7 though:
>
> - First, I had to try it on the swig-py3 branch, because on trunk I
> get this when trying to run any test with py 3.7:
⋮
> I guess that's one of the issues fixed by the swig-py3 branch.
>
> - Then, on the swig-py3 branch, with py 3.7.4 I ran into this issue:
> https://bugs.python.org/issue37549 (os.dup() fails for standard
> streams on Windows 7)
> This fails for any *.py test, because of line 836 in build/run_tests.py:
⋮
>
> - Upgraded to py 3.7.5, in which the above issue seems to be fixed.
> Now, *.py tests still don't work. I get no output at all:
⋮
> However, if I run it with --log-to-stdout, the tests do work (with a
> lot of output on stdout). […]
Drive-by comment here, but: since Python 2 reaches EOL during the lifetime of
the 1.13.x branch, I think it would be nice for us to make sure «make check»
passes on Windows under Python 3…
> Conclusion: I can confirm your patch works on Windows, for both Pyton
> 2.7.16 and 3.7.5 on the swig-py3 branch. As for the stdout
> redirection, I guess there might still be a problem ... perhaps the
> fix for https://bugs.python.org/issue37549 is not sufficient for
> Windows 7 ... dunno. Maybe someone can try this on Windows 10 and see
> if it makes a difference.
… although then again, Windows 7 reaches EOL rather soon too, so if the
problem manifests only under Windows 7, it might well not be worth fixing.
Cheers,
Daniel
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Johan Corveleyn <jc...@gmail.com>.
On Tue, Oct 15, 2019 at 6:20 PM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
>
> On 2019-10-15 17:17, Johan Corveleyn wrote:
> > On Tue, Oct 15, 2019 at 9:26 AM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
> >>
> >> On 2019-10-15 07:04, Daniel Shahaf wrote:
> >>> Yasuhito FUTATSUKI wrote on Sun, 13 Oct 2019 04:01 +00:00:
> >>>> On 2019/10/13 7:24, Daniel Shahaf wrote:
> >>
> >>>> I see. Now I agree it would suffice here.
> >>>>
> >>>>> So, how about:
> >>>>>
> >>>>> 1. Make the test use non-binary mode for changing and reading the
> >>>>> file 'lambda'.
>
> >>>>> 2. Locally revert the C part of r1841731 and make sure the modified test
> >>>>> still (correctly) fails. (That revision both added the test and
> >>>>> fixed the bug the test checks for.)
>
> I overlooked comments in this test. On step 2 the test will continue to loop
> as far as resource is available, or until signaled.
>
> And yes, after 'svn merge -r1841731:1841730 subversion/libsvn_client/conflicts.c',
> the test can't reach patched line. So no more test is needed on step 2, both
> on Unix/Linux and on Windows.
>
> >>>>
> >>>> So it looks sufficient to me.
> >>>
> >>> Cool. Will you perchance have time to do this? No worries if not.
> >>
> >> Yes, I'll do it on FreeBSD on tonight or tomorrow night (in JST :)).
>
> with the attached patch, both with Python 2.7.15 and Python 3.7.0 on FreeBSD,
> tree_conflict_tests passed.
>
> >> However I think it is also need to test for each 1 and 2 on Windows,
> >> because r1841736 and r1841743 also were attempt to fix this test
> >> on Windows, with Python 2.
> >
> > Feel free to let me know if I need to test something on Windows.
>
> Thank you. Could you please test the tree_conflict_tests with this patch,
> both with Python 2 and Python 3 on Windows?
Okay, I can confirm that tree_conflict_tests works with Python 2.7.16
(both with and without the patch) and with 3.7.5 (with the patch) on
Windows 7.
I did have some trouble testing it with Python 3.7 though:
- First, I had to try it on the swig-py3 branch, because on trunk I
get this when trying to run any test with py 3.7:
[[[
Traceback (most recent call last):
File "win-tests.py", line 134, in <module>
cp.items('options'))
File "build\generator\gen_win_dependencies.py", line 306, in __init__
self.find_libraries(False)
File "build\generator\gen_win_dependencies.py", line 327, in find_libraries
self._find_jdk(show_warnings)
File "build\generator\gen_win_dependencies.py", line 1085, in _find_jdk
vermatch = re.search(r'(([0-9]+(\.[0-9]+)+)(_[._0-9]+)?)', line, re.M)
File "C:\Python37\lib\re.py", line 183, in search
return _compile(pattern, flags).search(string)
TypeError: cannot use a string pattern on a bytes-like object
]]]
I guess that's one of the issues fixed by the swig-py3 branch.
- Then, on the swig-py3 branch, with py 3.7.4 I ran into this issue:
https://bugs.python.org/issue37549 (os.dup() fails for standard
streams on Windows 7)
This fails for any *.py test, because of line 836 in build/run_tests.py:
old_stdout = os.dup(sys.stdout.fileno())
It errors out with:
OSError: [WinError 87] The parameter is incorrect
- Upgraded to py 3.7.5, in which the above issue seems to be fixed.
Now, *.py tests still don't work. I get no output at all:
[[[
C:\research\svn\dev\swig-py3>python win-tests.py --release -t tree_conflict .
'ruby' is not recognized as an internal or external command,
operable program or batch file.
Testing Release configuration on local repository.
[1/1] tree_conflict_tests.py
C:\research\svn\dev\swig-py3>
]]]
tests.log only contains one line:
START: tree_conflict_tests.py
However, if I run it with --log-to-stdout, the tests do work (with a
lot of output on stdout). I.e. I get some fails without your
fix_tree_conflict_tests_patch.txt, and all tests successful if I apply
the patch.
Conclusion: I can confirm your patch works on Windows, for both Pyton
2.7.16 and 3.7.5 on the swig-py3 branch. As for the stdout
redirection, I guess there might still be a problem ... perhaps the
fix for https://bugs.python.org/issue37549 is not sufficient for
Windows 7 ... dunno. Maybe someone can try this on Windows 10 and see
if it makes a difference.
--
Johan
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019-10-15 17:17, Johan Corveleyn wrote:
> On Tue, Oct 15, 2019 at 9:26 AM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
>>
>> On 2019-10-15 07:04, Daniel Shahaf wrote:
>>> Yasuhito FUTATSUKI wrote on Sun, 13 Oct 2019 04:01 +00:00:
>>>> On 2019/10/13 7:24, Daniel Shahaf wrote:
>>
>>>> I see. Now I agree it would suffice here.
>>>>
>>>>> So, how about:
>>>>>
>>>>> 1. Make the test use non-binary mode for changing and reading the
>>>>> file 'lambda'.
>>>>> 2. Locally revert the C part of r1841731 and make sure the modified test
>>>>> still (correctly) fails. (That revision both added the test and
>>>>> fixed the bug the test checks for.)
I overlooked comments in this test. On step 2 the test will continue to loop
as far as resource is available, or until signaled.
And yes, after 'svn merge -r1841731:1841730 subversion/libsvn_client/conflicts.c',
the test can't reach patched line. So no more test is needed on step 2, both
on Unix/Linux and on Windows.
>>>>
>>>> So it looks sufficient to me.
>>>
>>> Cool. Will you perchance have time to do this? No worries if not.
>>
>> Yes, I'll do it on FreeBSD on tonight or tomorrow night (in JST :)).
with the attached patch, both with Python 2.7.15 and Python 3.7.0 on FreeBSD,
tree_conflict_tests passed.
>> However I think it is also need to test for each 1 and 2 on Windows,
>> because r1841736 and r1841743 also were attempt to fix this test
>> on Windows, with Python 2.
>
> Feel free to let me know if I need to test something on Windows.
Thank you. Could you please test the tree_conflict_tests with this patch,
both with Python 2 and Python 3 on Windows?
Thanks,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Johan Corveleyn <jc...@gmail.com>.
On Tue, Oct 15, 2019 at 9:26 AM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
>
> On 2019-10-15 07:04, Daniel Shahaf wrote:
> > Yasuhito FUTATSUKI wrote on Sun, 13 Oct 2019 04:01 +00:00:
> >> On 2019/10/13 7:24, Daniel Shahaf wrote:
>
> >> I see. Now I agree it would suffice here.
> >>
> >>> So, how about:
> >>>
> >>> 1. Make the test use non-binary mode for changing and reading the
> >>> file 'lambda'.
> >>>
> >>> 2. Locally revert the C part of r1841731 and make sure the modified test
> >>> still (correctly) fails. (That revision both added the test and
> >>> fixed the bug the test checks for.)
> >>
> >> So it looks sufficient to me.
> >
> > Cool. Will you perchance have time to do this? No worries if not.
>
> Yes, I'll do it on FreeBSD on tonight or tomorrow night (in JST :)).
> However I think it is also need to test for each 1 and 2 on Windows,
> because r1841736 and r1841743 also were attempt to fix this test
> on Windows, with Python 2.
Feel free to let me know if I need to test something on Windows.
--
Johan
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019-10-15 07:04, Daniel Shahaf wrote:
> Yasuhito FUTATSUKI wrote on Sun, 13 Oct 2019 04:01 +00:00:
>> On 2019/10/13 7:24, Daniel Shahaf wrote:
>> I see. Now I agree it would suffice here.
>>
>>> So, how about:
>>>
>>> 1. Make the test use non-binary mode for changing and reading the
>>> file 'lambda'.
>>>
>>> 2. Locally revert the C part of r1841731 and make sure the modified test
>>> still (correctly) fails. (That revision both added the test and
>>> fixed the bug the test checks for.)
>>
>> So it looks sufficient to me.
>
> Cool. Will you perchance have time to do this? No worries if not.
Yes, I'll do it on FreeBSD on tonight or tomorrow night (in JST :)).
However I think it is also need to test for each 1 and 2 on Windows,
because r1841736 and r1841743 also were attempt to fix this test
on Windows, with Python 2.
> Also, what about the svnadmin_tests.py patch you posted upthread? Is
> there a reason not to go ahead and commit it to trunk? (and even nominate
> it for backport in 1.13.x/STATUS)
I think that is no probrem.
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests? (deadline 11 October))
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Yasuhito FUTATSUKI wrote on Sun, 13 Oct 2019 04:01 +00:00:
> On 2019/10/13 7:24, Daniel Shahaf wrote:
> > Yasuhito FUTATSUKI wrote on Sat, 12 Oct 2019 03:01 +00:00:
> >> If textual comparison is sufficient here, it is right to open file
> >> text mode (with suitable, unified set of `encoding', `errors', and `newline'
> >> parameter). Otherwise, if strict comparison is needed, we must avoid unwanted,
> >> not one-on-one corresponding conversion from bytes to str applied by Python.
> >> In the latter case, it may be rather incorrect to use
> >> compare_and_display_lines().
> >
> > Good question. I suspect textual comparison would suffice here, because
> > this is a tree conflicts test, not a keywords semantics test, and the
> > test case seems to revolve around the tree changes, not around the
> > newline characters.
>
> I see. Now I agree it would suffice here.
>
> > So, how about:
> >
> > 1. Make the test use non-binary mode for changing and reading the
> > file 'lambda'.
> >
> > 2. Locally revert the C part of r1841731 and make sure the modified test
> > still (correctly) fails. (That revision both added the test and
> > fixed the bug the test checks for.)
>
> So it looks sufficient to me.
Cool. Will you perchance have time to do this? No worries if not.
Also, what about the svnadmin_tests.py patch you posted upthread? Is
there a reason not to go ahead and commit it to trunk? (and even nominate
it for backport in 1.13.x/STATUS)
Cheers,
Daniel
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/10/13 7:24, Daniel Shahaf wrote:
> Yasuhito FUTATSUKI wrote on Sat, 12 Oct 2019 03:01 +00:00:
>> If textual comparison is sufficient here, it is right to open file
>> text mode (with suitable, unified set of `encoding', `errors', and `newline'
>> parameter). Otherwise, if strict comparison is needed, we must avoid unwanted,
>> not one-on-one corresponding conversion from bytes to str applied by Python.
>> In the latter case, it may be rather incorrect to use
>> compare_and_display_lines().
>
> Good question. I suspect textual comparison would suffice here, because
> this is a tree conflicts test, not a keywords semantics test, and the
> test case seems to revolve around the tree changes, not around the
> newline characters.
I see. Now I agree it would suffice here.
> So, how about:
>
> 1. Make the test use non-binary mode for changing and reading the
> file 'lambda'.
>
> 2. Locally revert the C part of r1841731 and make sure the modified test
> still (correctly) fails. (That revision both added the test and
> fixed the bug the test checks for.)
So it looks sufficient to me.
Thanks,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests? (deadline 11 October))
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Yasuhito FUTATSUKI wrote on Sat, 12 Oct 2019 03:01 +00:00:
> If textual comparison is sufficient here, it is right to open file
> text mode (with suitable, unified set of `encoding', `errors', and `newline'
> parameter). Otherwise, if strict comparison is needed, we must avoid unwanted,
> not one-on-one corresponding conversion from bytes to str applied by Python.
> In the latter case, it may be rather incorrect to use
> compare_and_display_lines().
Good question. I suspect textual comparison would suffice here, because
this is a tree conflicts test, not a keywords semantics test, and the
test case seems to revolve around the tree changes, not around the
newline characters.
So, how about:
1. Make the test use non-binary mode for changing and reading the
file 'lambda'.
2. Locally revert the C part of r1841731 and make sure the modified test
still (correctly) fails. (That revision both added the test and
fixed the bug the test checks for.)
Alternatively, we could make the test verify the contents of 'lambda' in binary
mode without using compare_and_display_lines.
Makes sense?
Cheers,
Daniel
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019-10-12 07:47, Daniel Shahaf wrote:
> Yasuhito FUTATSUKI wrote on Sat, Oct 12, 2019 at 05:31:53 +0900:
>> Yes, it will fix local_missing_dir_endless_loop() itself correctly.
>> But the stack trace before fix indicate there is at least one problem
>> in svntest.verify.compare_and_display_lines().
>>
>> Assume the file contents is broken here. This situation can be simulate
>> by patch like:
>>
>> Index: subversion/tests/cmdline/tree_conflict_tests.py
>> ===================================================================
>> --- subversion/tests/cmdline/tree_conflict_tests.py (revision 1868264)
>> +++ subversion/tests/cmdline/tree_conflict_tests.py (working copy)
>> @@ -1544,7 +1544,7 @@
>> contents = open(sbox.ospath('A1/B/lambda'), 'rb').readlines()
>> svntest.verify.compare_and_display_lines(
>> "A1/B/lambda has unexpectected contents", sbox.ospath("A1/B/lambda"),
>> - [ "This is the file 'lambda'.\n", "This is more content.\n"], contents)
>> + [ b"This is the file 'lambda'.\n", b"This is not more content.\n"], contents)
>> #######################################################################
>>
>>
>> then we will got fails.log, contains stack trace for unexpected exception
>> within the code to construct log message.
>>
>> [[[
>> W: A1/B/lambda has unexpectected contents
>> W: EXPECTED svn-test-work/working_copies/tree_conflict_tests-26/A1/B/lambda (match_all=True):
>> W: CWD: /home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline
>> Traceback (most recent call last):
>> File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/main.py", line 1931, in run
>> rc = self.pred.run(sandbox)
>> File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/testcase.py", line 178, in run
>> result = self.func(sandbox)
>> File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/tree_conflict_tests.py", line 1547, in local_missing_dir_endless_loop
>> [ b"This is the file 'lambda'.\n", b"This is not more content.\n"], contents)
>> File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/verify.py", line 503, in compare_and_display_lines
>> expected.display_differences(message, label, actual)
>> File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/verify.py", line 154, in display_differences
>> display_lines(message, self.expected, actual, e_label, label)
>> File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/verify.py", line 474, in display_lines
>> logger.warn('| ' + x.rstrip())
>> TypeError: can only concatenate str (not "bytes") to str
>> FAIL: tree_conflict_tests.py 26: endless loop when resolving local-missing dir
>> ]]]
>>
>> This is caused by mixing bytes object drived from file contents and str
>> object to construct log message.
>
> I agree: this FAIL indicates str and bytes are mixed. My question is:
> Why do you think svntest.verify.compare_and_display_lines() needs to be
> changed? That function's names implies it deals with text files, so,
> why should compare_and_display_lines() support the case that its third
> and fourth parameters (EXPECTED and ACTUAL) are both lists of bytes objects?
>
> In other words, why would changing «'rb'» to «'r'» on line 1544 —
> without changing the str literals to bytes literals on line 1547 — not
> be a correct solution?
Ah, I thought strict comparison is needed here because of raw mode file I/O.
Index: subversion/tests/cmdline/tree_conflict_tests.py
===================================================================
--- subversion/tests/cmdline/tree_conflict_tests.py (revision 1868264)
+++ subversion/tests/cmdline/tree_conflict_tests.py (working copy)
@@ -1518,7 +1518,7 @@
sbox.simple_move('A/B', 'A/B2')
sbox.simple_commit()
sbox.simple_update()
- main.file_append_binary(sbox.ospath("A/B2/lambda"), "This is more content.\n")
+ main.file_append_binary(sbox.ospath("A/B2/lambda"), "This is more content.\r\n")
sbox.simple_commit()
sbox.simple_update()
@@ -1544,7 +1544,7 @@
contents = open(sbox.ospath('A1/B/lambda'), 'rb').readlines()
svntest.verify.compare_and_display_lines(
"A1/B/lambda has unexpectected contents", sbox.ospath("A1/B/lambda"),
- [ "This is the file 'lambda'.\n", "This is more content.\n"], contents)
+ [ b"This is the file 'lambda'.\n", b"This is more content.\n"], contents)
#######################################################################
Above patch will make local_missing_dir_endless_loop test fail because of
strictness of comparison. However,
Index: subversion/tests/cmdline/tree_conflict_tests.py
===================================================================
--- subversion/tests/cmdline/tree_conflict_tests.py (revision 1868264)
+++ subversion/tests/cmdline/tree_conflict_tests.py (working copy)
@@ -1518,7 +1518,7 @@
sbox.simple_move('A/B', 'A/B2')
sbox.simple_commit()
sbox.simple_update()
- main.file_append_binary(sbox.ospath("A/B2/lambda"), "This is more content.\n")
+ main.file_append_binary(sbox.ospath("A/B2/lambda"), "This is more content.\r\n")
sbox.simple_commit()
sbox.simple_update()
@@ -1541,7 +1541,7 @@
# If everything works as expected the resolver will recommended a
# resolution option and 'svn' will resolve the conflict automatically.
# Verify that 'A1/B/lambda' contains the merged content:
- contents = open(sbox.ospath('A1/B/lambda'), 'rb').readlines()
+ contents = open(sbox.ospath('A1/B/lambda'), 'r').readlines()
svntest.verify.compare_and_display_lines(
"A1/B/lambda has unexpectected contents", sbox.ospath("A1/B/lambda"),
[ "This is the file 'lambda'.\n", "This is more content.\n"], contents)
this will make it succcess because of "universal newline" feature on Python.
If textual comparison is sufficient here, it is right to open file
text mode (with suitable, unified set of `encoding', `errors', and `newline'
parameter). Otherwise, if strict comparison is needed, we must avoid unwanted,
not one-on-one corresponding conversion from bytes to str applied by Python.
In the latter case, it may be rather incorrect to use
compare_and_display_lines().
> Hope I'm clearer this time. If not, I'd be happy to clarify.
>
> Cheers,
>
> Daniel
>
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Yasuhito FUTATSUKI wrote on Sat, Oct 12, 2019 at 05:31:53 +0900:
> Yes, it will fix local_missing_dir_endless_loop() itself correctly.
> But the stack trace before fix indicate there is at least one problem
> in svntest.verify.compare_and_display_lines().
>
> Assume the file contents is broken here. This situation can be simulate
> by patch like:
>
> Index: subversion/tests/cmdline/tree_conflict_tests.py
> ===================================================================
> --- subversion/tests/cmdline/tree_conflict_tests.py (revision 1868264)
> +++ subversion/tests/cmdline/tree_conflict_tests.py (working copy)
> @@ -1544,7 +1544,7 @@
> contents = open(sbox.ospath('A1/B/lambda'), 'rb').readlines()
> svntest.verify.compare_and_display_lines(
> "A1/B/lambda has unexpectected contents", sbox.ospath("A1/B/lambda"),
> - [ "This is the file 'lambda'.\n", "This is more content.\n"], contents)
> + [ b"This is the file 'lambda'.\n", b"This is not more content.\n"], contents)
> #######################################################################
>
>
> then we will got fails.log, contains stack trace for unexpected exception
> within the code to construct log message.
>
> [[[
> W: A1/B/lambda has unexpectected contents
> W: EXPECTED svn-test-work/working_copies/tree_conflict_tests-26/A1/B/lambda (match_all=True):
> W: CWD: /home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline
> Traceback (most recent call last):
> File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/main.py", line 1931, in run
> rc = self.pred.run(sandbox)
> File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/testcase.py", line 178, in run
> result = self.func(sandbox)
> File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/tree_conflict_tests.py", line 1547, in local_missing_dir_endless_loop
> [ b"This is the file 'lambda'.\n", b"This is not more content.\n"], contents)
> File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/verify.py", line 503, in compare_and_display_lines
> expected.display_differences(message, label, actual)
> File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/verify.py", line 154, in display_differences
> display_lines(message, self.expected, actual, e_label, label)
> File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/verify.py", line 474, in display_lines
> logger.warn('| ' + x.rstrip())
> TypeError: can only concatenate str (not "bytes") to str
> FAIL: tree_conflict_tests.py 26: endless loop when resolving local-missing dir
> ]]]
>
> This is caused by mixing bytes object drived from file contents and str
> object to construct log message.
I agree: this FAIL indicates str and bytes are mixed. My question is:
Why do you think svntest.verify.compare_and_display_lines() needs to be
changed? That function's names implies it deals with text files, so,
why should compare_and_display_lines() support the case that its third
and fourth parameters (EXPECTED and ACTUAL) are both lists of bytes objects?
In other words, why would changing «'rb'» to «'r'» on line 1544 —
without changing the str literals to bytes literals on line 1547 — not
be a correct solution?
Hope I'm clearer this time. If not, I'd be happy to clarify.
Cheers,
Daniel
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Nathan Hartman <ha...@gmail.com>.
On Fri, Oct 11, 2019 at 4:32 PM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
> This is caused by mixing bytes object drived from file contents and str
> object to construct log message.
Does something like this answer help:
https://stackoverflow.com/questions/31058055/how-do-i-convert-a-python-3-byte-string-variable-into-a-regular-string/31060836
Something like:
str(bytes_string, 'utf-8')
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019-10-12 02:56, Daniel Shahaf wrote:
> Yasuhito FUTATSUKI wrote on Fri, Oct 11, 2019 at 16:35:19 +0900:
>> The latter also can be fixed by fix_tree_conflict_tests_patch.txt
>> at least so that the test can be passed on Python 3. However, from above
>> stack trace, I think it is incomplete because there still exists some sort of
>> confusion between bytes and str in subversion/tests/cmdline/svntest/*.py
>>
>> Index: subversion/tests/cmdline/tree_conflict_tests.py
>> ===================================================================
>> --- subversion/tests/cmdline/tree_conflict_tests.py (revision 1868264)
>> +++ subversion/tests/cmdline/tree_conflict_tests.py (working copy)
>> @@ -1544,7 +1544,7 @@
>> contents = open(sbox.ospath('A1/B/lambda'), 'rb').readlines()
>> svntest.verify.compare_and_display_lines(
>> "A1/B/lambda has unexpectected contents", sbox.ospath("A1/B/lambda"),
>> - [ "This is the file 'lambda'.\n", "This is more content.\n"], contents)
>> + [ b"This is the file 'lambda'.\n", b"This is more content.\n"], contents)
>
> Why do you think this is incomplete? The open() call uses mode='rb', so
> «contents» will be set to an array of bytes objects, so it'll need to be
> compared to an array of bytes objects. Which is to say, this patch, too, looks
> correct to me.
Yes, it will fix local_missing_dir_endless_loop() itself correctly.
But the stack trace before fix indicate there is at least one problem
in svntest.verify.compare_and_display_lines().
Assume the file contents is broken here. This situation can be simulate
by patch like:
Index: subversion/tests/cmdline/tree_conflict_tests.py
===================================================================
--- subversion/tests/cmdline/tree_conflict_tests.py (revision 1868264)
+++ subversion/tests/cmdline/tree_conflict_tests.py (working copy)
@@ -1544,7 +1544,7 @@
contents = open(sbox.ospath('A1/B/lambda'), 'rb').readlines()
svntest.verify.compare_and_display_lines(
"A1/B/lambda has unexpectected contents", sbox.ospath("A1/B/lambda"),
- [ "This is the file 'lambda'.\n", "This is more content.\n"], contents)
+ [ b"This is the file 'lambda'.\n", b"This is not more content.\n"], contents)
#######################################################################
then we will got fails.log, contains stack trace for unexpected exception
within the code to construct log message.
[[[
W: A1/B/lambda has unexpectected contents
W: EXPECTED svn-test-work/working_copies/tree_conflict_tests-26/A1/B/lambda (match_all=True):
W: CWD: /home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline
Traceback (most recent call last):
File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/main.py", line 1931, in run
rc = self.pred.run(sandbox)
File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/testcase.py", line 178, in run
result = self.func(sandbox)
File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/tree_conflict_tests.py", line 1547, in local_missing_dir_endless_loop
[ b"This is the file 'lambda'.\n", b"This is not more content.\n"], contents)
File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/verify.py", line 503, in compare_and_display_lines
expected.display_differences(message, label, actual)
File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/verify.py", line 154, in display_differences
display_lines(message, self.expected, actual, e_label, label)
File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/verify.py", line 474, in display_lines
logger.warn('| ' + x.rstrip())
TypeError: can only concatenate str (not "bytes") to str
FAIL: tree_conflict_tests.py 26: endless loop when resolving local-missing dir
]]]
This is caused by mixing bytes object drived from file contents and str
object to construct log message.
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Yasuhito FUTATSUKI wrote on Fri, Oct 11, 2019 at 16:35:19 +0900:
> On 2019-10-11 06:45, Daniel Shahaf wrote:
> ...
> > From autogen.sh to 'make check', all steps pass with PYTHON=/usr/bin/python3.
>
> There were two failures in 1.11 on tests.
>
> https://lists.apache.org/thread.html/366e77c7443a0c575b47de150a2c394e1e685fde8887c805a656a33d@%3Cusers.subversion.apache.org%3E
>
> and it seems they still exist in trunk.
Sorry, my bad. I did see them in my test run too, but incorrectly determined
they were an artifact of my build environment.
> The former failure can be fixed by attached patch, fix_svnadmin_tests_patch.txt
> Index: subversion/tests/cmdline/svnadmin_tests.py
> ===================================================================
> --- subversion/tests/cmdline/svnadmin_tests.py (revision 1868264)
> +++ subversion/tests/cmdline/svnadmin_tests.py (working copy)
> @@ -3859,7 +3859,7 @@
> sbox.repo_url)
>
> dump_lines = svntest.actions.run_and_verify_dump(sbox.repo_dir)
> - assert propval + '\n' in dump_lines
> + assert propval.encode() + b'\n' in dump_lines
+1 to commit.
> The latter also can be fixed by fix_tree_conflict_tests_patch.txt
> at least so that the test can be passed on Python 3. However, from above
> stack trace, I think it is incomplete because there still exists some sort of
> confusion between bytes and str in subversion/tests/cmdline/svntest/*.py
>
> Index: subversion/tests/cmdline/tree_conflict_tests.py
> ===================================================================
> --- subversion/tests/cmdline/tree_conflict_tests.py (revision 1868264)
> +++ subversion/tests/cmdline/tree_conflict_tests.py (working copy)
> @@ -1544,7 +1544,7 @@
> contents = open(sbox.ospath('A1/B/lambda'), 'rb').readlines()
> svntest.verify.compare_and_display_lines(
> "A1/B/lambda has unexpectected contents", sbox.ospath("A1/B/lambda"),
> - [ "This is the file 'lambda'.\n", "This is more content.\n"], contents)
> + [ b"This is the file 'lambda'.\n", b"This is more content.\n"], contents)
Why do you think this is incomplete? The open() call uses mode='rb', so
«contents» will be set to an array of bytes objects, so it'll need to be
compared to an array of bytes objects. Which is to say, this patch, too, looks
correct to me.
Cheers,
Daniel
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Nathan Hartman wrote on Fri, Oct 11, 2019 at 12:53:23 -0400:
> (2) Once this is more-or-less under control, I'd like to start actively
> looking for volunteers, hackathon events, and any other opportunities
> to get more involvement.
This might be easier to do on an ASF-wide scale. comdev¹ would be a good place
to start.
Cheers,
Daniel
¹ dev@community.apache.org / https://community.apache.org/
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Nathan Hartman <ha...@gmail.com>.
On Fri, Oct 11, 2019 at 11:45 AM Julian Foad <ju...@apache.org>
wrote:
> Daniel Shahaf wrote:
> > Julian, do you have any release.py tasks to suggest?
>
> There are certainly some things need doing in release.py.
Unfortunately I didn't hear back in time to put that on the list for
this hackathon. If the event organizer communicates with us, we
might be able to add that if you can describe exactly what you'd
like worked on.
> One thing I've taken from this discussion is there are sometimes
> potential contributors coming along who might take a well defined and
> contained task, so it's useful if we can have tasks well defined and
> discoverable, something we need to improve.
Agreed.
(1) Housecleaning on the issues database and properly marking items as
"bite size," "hackathon ready," or some other appropriate label, will
make it much easier to respond when we get such a request.
(2) Once this is more-or-less under control, I'd like to start actively
looking for volunteers, hackathon events, and any other opportunities
to get more involvement.
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Julian Foad <ju...@apache.org>.
Daniel Shahaf wrote:
> Julian, do you have any release.py tasks to suggest?
There are certainly some things need doing in release.py.
One thing I've taken from this discussion is there are sometimes
potential contributors coming along who might take a well defined and
contained task, so it's useful if we can have tasks well defined and
discoverable, something we need to improve.
- Julian
Re: Supported SWIG version on swig-py3
Posted by James McCoy <ja...@jamessan.com>.
On Tue, Oct 29, 2019 at 11:40:35AM +0100, Stefan Sperling wrote:
> On Tue, Oct 29, 2019 at 11:37:17AM +0100, Branko Čibej wrote:
> > Our .tar.bz2 and .tar.gz packages contain source files generated by
> > Swig. A packager /can/ decide to regenerate them; I don't know why they
> > would want to, but they can.
>
> I had to do just that for quite some time in the OpenBSD package because
> swig versions were out of sync with those used by Subversion's release
> manager at the time, which led to problems during the build.
> I don't remember the details but regenerating those files with the
> installed version of swig fixed things.
We also do that in Debian, but it's more to ensure that we _can_
regenerate them if needed. If we have to patch something in the swig
files, we want to know that we'll be able to successfully rebuild the
resulting output.
Cheers,
--
James
GPG Key: 4096R/91BF BF4D 6956 BD5D F7B7 2D23 DFE6 91AE 331B A3DB
Re: Supported SWIG version on swig-py3
Posted by Stefan Sperling <st...@elego.de>.
On Tue, Oct 29, 2019 at 11:37:17AM +0100, Branko Čibej wrote:
> Our .tar.bz2 and .tar.gz packages contain source files generated by
> Swig. A packager /can/ decide to regenerate them; I don't know why they
> would want to, but they can.
I had to do just that for quite some time in the OpenBSD package because
swig versions were out of sync with those used by Subversion's release
manager at the time, which led to problems during the build.
I don't remember the details but regenerating those files with the
installed version of swig fixed things.
Re: [Patch] Supported SWIG version(Re: Supported SWIG version on
swig-py3)
Posted by Branko Čibej <br...@apache.org>.
On 12.11.2019 16:19, Nathan Hartman wrote:
> On Tue, Nov 12, 2019 at 7:26 AM Branko Čibej <brane@apache.org
> <ma...@apache.org>> wrote:
>
> On 12.11.2019 13:08, Julian Foad wrote:
> > Daniel Shahaf wrote:
> >> There should be a comma after "e.g.".
> >
> > In my experience (in England)[,] a comma can be used[,] but is not
> > often used. I appreciate [that] we like to get grammar
> > {right|"Right"}, but ... :-)
>
>
> In MY experience, my English teacher (i.e., someone from England, not
> just someone who taught me English, in a British school) hammered the
> comma after 'e.g.' and 'i.e.' into my brainstem in no uncertain
> terms. :)
>
>
> Thank you for that. :-)
>
> What about my "SWIG version 2.0.0 and later" singular / plural question
> and mix of treating it as singular in some places and plural in others?
In the specific example you quote, I believe the singular is correct
since you only use one version at a time, but have a choice of which
version that may be.
> may I suggest:
>
> * SWIG installation is optional if you are using a Subversion
> distribution tarball, because it contains the source files generated
> by SWIG.
Rule number one: active voice, not passive voice.
You do not have to install SWIG if you are using a Subversion
distribution tarball, etc.
> may I suggest:
>
> * Currently, SWIG versions 2.0.0 and later are supported, with the
> following notes:
See rule number one.
We currently support SWIG version 2.0.0 and later, etc.
Or:
We currently support SWIG versions from 2.0.0 onwards, etc.
> I suggest adding "one of the":
>
> * To verify you have SWIG installed correctly, run "swig -version"
> from the command line.
English relies on prepositions, not on verb conjugations (since has
none). Also see rule number one.
To verify that you installed SWIG correctly, etc.
-- Brane
Re: [Patch] Supported SWIG version(Re: Supported SWIG version on swig-py3)
Posted by Nathan Hartman <ha...@gmail.com>.
On Wed, Nov 13, 2019 at 1:20 AM Yasuhito FUTATSUKI <fu...@poem.co.jp>
wrote:
> On 2019/11/13 14:16, Nathan Hartman wrote:
> > Yasuhito, if possible could you please commit your patch?
> > Then, I will work on the description in the INSTALL file and I'll
> > incorporate the good input from Daniel, Julian, and Brane... :-)
>
> Thanks a lot, for your reviews, helps, and advices.
>
> I've committed my patch, with update for messages in build/ac-macros
> by Nathan's suggestion and URLs update for ViewVC and Trac in INSTALL
> file only, in r1869722.
Thank you!
I have followed up with r1869745, to improve the text in
subversion/bindings/swig/INSTALL.
If any mistakes are found, let me know.
Note: If the patch contributed by Jun Omae a little earlier is
committed, the text regarding SWIG 4.0.0+ in INSTALL will need to be
adjusted accordingly. I have not changed it because the patch has not
been committed yet. See the thread "[Patch] Support building with
SWIG 4 on Python 3.x" --
https://lists.apache.org/thread.html/49766e2f6b78b6a89cbb398526b202d248ea90aa2b7c7e73cb8b22c0@%3Cdev.subversion.apache.org%3E
Cheers,
Nathan
Re: [Patch] Supported SWIG version(Re: Supported SWIG version on
swig-py3)
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/11/13 14:16, Nathan Hartman wrote:
> On Tue, Nov 12, 2019 at 10:39 AM Julian Foad <ju...@apache.org> wrote:
>
>> Nathan Hartman wrote:
>>> What about my "SWIG version 2.0.0 and later" singular / plural question
>>> and mix of treating it as singular in some places and plural in others?
>>
>> "SWIG versions 2 and later are ..."
>> "SWIG version 2 or later is ..."
>>
>> If there are no concerns about the substance of the patch, I suggest a
>> good course of action is to commit it ASAP, then just make your edits in
>> follow-up commits, and optionally mention in email that you did so.
>> Even when some of the language is a bit unclear, most of this sort of
>> thing doesn't need pre-commit review.
>
>
> I agree.
>
> Yasuhito, if possible could you please commit your patch?
> Then, I will work on the description in the INSTALL file and I'll
> incorporate the good input from Daniel, Julian, and Brane... :-)
Thanks a lot, for your reviews, helps, and advices.
I've committed my patch, with update for messages in build/ac-macros
by Nathan's suggestion and URLs update for ViewVC and Trac in INSTALL
file only, in r1869722.
Thanks,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: [Patch] Supported SWIG version(Re: Supported SWIG version on swig-py3)
Posted by Nathan Hartman <ha...@gmail.com>.
On Tue, Nov 12, 2019 at 10:39 AM Julian Foad <ju...@apache.org> wrote:
> Nathan Hartman wrote:
> > What about my "SWIG version 2.0.0 and later" singular / plural question
> > and mix of treating it as singular in some places and plural in others?
>
> "SWIG versions 2 and later are ..."
> "SWIG version 2 or later is ..."
>
> If there are no concerns about the substance of the patch, I suggest a
> good course of action is to commit it ASAP, then just make your edits in
> follow-up commits, and optionally mention in email that you did so.
> Even when some of the language is a bit unclear, most of this sort of
> thing doesn't need pre-commit review.
I agree.
Yasuhito, if possible could you please commit your patch?
Then, I will work on the description in the INSTALL file and I'll
incorporate the good input from Daniel, Julian, and Brane... :-)
Thanks!
Nathan
Re: [Patch] Supported SWIG version(Re: Supported SWIG version on
swig-py3)
Posted by Julian Foad <ju...@apache.org>.
Nathan Hartman wrote:
> What about my "SWIG version 2.0.0 and later" singular / plural question
> and mix of treating it as singular in some places and plural in others?
"SWIG versions 2 and later are ..."
"SWIG version 2 or later is ..."
If there are no concerns about the substance of the patch, I suggest a
good course of action is to commit it ASAP, then just make your edits in
follow-up commits, and optionally mention in email that you did so.
Even when some of the language is a bit unclear, most of this sort of
thing doesn't need pre-commit review.
- Julian
Re: [Patch] Supported SWIG version(Re: Supported SWIG version on swig-py3)
Posted by Nathan Hartman <ha...@gmail.com>.
On Tue, Nov 12, 2019 at 7:26 AM Branko Čibej <br...@apache.org> wrote:
> On 12.11.2019 13:08, Julian Foad wrote:
> > Daniel Shahaf wrote:
> >> There should be a comma after "e.g.".
> >
> > In my experience (in England)[,] a comma can be used[,] but is not
> > often used. I appreciate [that] we like to get grammar
> > {right|"Right"}, but ... :-)
>
>
> In MY experience, my English teacher (i.e., someone from England, not
> just someone who taught me English, in a British school) hammered the
> comma after 'e.g.' and 'i.e.' into my brainstem in no uncertain terms. :)
>
Thank you for that. :-)
What about my "SWIG version 2.0.0 and later" singular / plural question
and mix of treating it as singular in some places and plural in others?
Nathan
Re: [Patch] Supported SWIG version(Re: Supported SWIG version on
swig-py3)
Posted by Branko Čibej <br...@apache.org>.
On 12.11.2019 13:08, Julian Foad wrote:
> Daniel Shahaf wrote:
>> There should be a comma after "e.g.".
>
> In my experience (in England)[,] a comma can be used[,] but is not
> often used. I appreciate [that] we like to get grammar
> {right|"Right"}, but ... :-)
In MY experience, my English teacher (i.e., someone from England, not
just someone who taught me English, in a British school) hammered the
comma after 'e.g.' and 'i.e.' into my brainstem in no uncertain terms. :)
-- Brane
Re: [Patch] Supported SWIG version(Re: Supported SWIG version on
swig-py3)
Posted by Julian Foad <ju...@apache.org>.
Daniel Shahaf wrote:
> There should be a comma after "e.g.".
In my experience (in England)[,] a comma can be used[,] but is not often
used. I appreciate [that] we like to get grammar {right|"Right"}, but
... :-)
- Julian
Re: [Patch] Supported SWIG version(Re: Supported SWIG version on
swig-py3)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Nathan Hartman wrote on Mon, Nov 11, 2019 at 21:37:44 -0500:
> * In the swig-x.y.z, directory, run ./configure (where x.y.z is
> the SWIG version, e.g. 3.0.12).
There should be a comma after "e.g.".
Cheers,
Daniel
Re: [Patch] Supported SWIG version(Re: Supported SWIG version on swig-py3)
Posted by Nathan Hartman <ha...@gmail.com>.
On Fri, Nov 8, 2019 at 9:59 AM Yasuhito FUTATSUKI <fu...@poem.co.jp>
wrote:
> Hi, I updated the patch to restrict supported SWIG version
> posted before, <4e...@poem.co.jp>.
>
...
> Please review, especially I don't have any confidence in description
> in INSTALL file.
Hello Yasuhito,
I have reviewed your patch and made some edits, which I will describe
in detail here...
I hope it is helpful. :-)
I think that the SWIG bindings should be considered plural for
linguistic purposes (i.e., bindings, rather than binding).
I am not sure whether a phrase like "SWIG version 2.0.0 and later"
should be considered singular or plural. Is it plural because we are
potentially discussing multiple SWIG versions? Or is it singular
because the subject is "SWIG version 2.0.0" with any later version(s)
being incidental? I made some edits based on what "sounds" correct to
me. This results in some cases being singular and others plural.
Please let me know if this does not seem correct.
With the above points in mind, I suggest the following changes. To
save you some work, I have updated your patch with these changes
(attached):
In build/ac-macros/swig.m4:
AC_MSG_WARN([Subversion Python bindings for Python 3 requires 3.0.10 <=
SWIG < 4.0.0])
^^^^^^^^
(require)
AC_MSG_WARN([Subversion Python bindings for Python 2 requires 1.3.24 <=
SWIG < 4.0.0])
^^^^^^^^
(require)
In subversion/bindings/swig/INSTALL:
This bullet point:
+ * SWIG installation is optional if you are using distribution tarball,
+ because it contains source fils generated by SWIG. If you are using
+ a source working copy checked out from Subversion source repository,
+ or you want to regenerate SWIG language bindings source files from
+ SWIG source files for some reason, you need suitable version of
+ SWIG installation.
may I suggest:
* SWIG installation is optional if you are using a Subversion
distribution tarball, because it contains the source files generated
by SWIG. If you are using a working copy checked out from
Subversion's source repository, or you want to regenerate the SWIG
language bindings source files for some reason, you need a suitable
version of SWIG installed.
This bullet point:
+ * Currently supported SWIG version is 2.0.0 and later, however,
+ - SWIG 1.3.24 and later in 1.3.x may work, but we didn't test on
+ latest our source code.
+ - Python bindings for Python 2 doesn't supported SWIG 4.0.0 and
later.
+ - Python 3 support for Python bindings requires SWIG 3.0.10 and
later,
+ but SWIG 4.0.0 and later is not supported (yet).
+ - Ruby bindings doesn't support SWIG 3.0.8.
+ - Make sure language specific supported version of SWIG itself, e.g.
+ - Perl 5.16 and later requires SWIG 2.0.8 and later.
+ - SWIG 3.0.9 has some trouble with Python support.
+ (see
https://sourceforge.net/p/swig/news/2016/06/swig-3010-released/)
may I suggest:
* Currently, SWIG versions 2.0.0 and later are supported, with the
following notes:
- SWIG 1.3.24 and later 1.3.x may work, but we do not test these
versions on our latest source code.
- For Python 2 bindings, SWIG 4.0.0 or later is not supported.
- For Python 3 bindings, SWIG 3.0.10 or later is required, but
SWIG 4.0.0 and later is not supported (yet).
- Note that SWIG 3.0.9 has some trouble with Python support.
(See https://sourceforge.net/p/swig/news/2016/06/swig-3010-released/
)
- For Perl 5.16 and later, SWIG 2.0.8 or later is required.
- For Ruby bindings, SWIG 3.0.8 is not supported.
This was not part of your patch, but:
> * Perhaps your distribution packages a suitable version - if it does
> install it, and skip to the last bullet point in this section.
I suggest changing the punctuation:
* Perhaps your distribution packages a suitable version. If it does,
install it and skip to the last bullet point in this section.
This bullet point:
+ * In the swig-x.y.z, directory, run ./configure (where x.y.z is
+ SWIG version, e.g. 3.0.12).
I suggest adding "the":
* In the swig-x.y.z, directory, run ./configure (where x.y.z is
the SWIG version, e.g. 3.0.12).
This bullet point:
* To verify you have SWIG installed correctly, run "swig -version"
+ from the command line. SWIG should report that it is suitable version
+ mentioned above.
I suggest adding "one of the":
* To verify you have SWIG installed correctly, run "swig -version"
from the command line. SWIG should report that it is one of the
suitable versions mentioned above.
I hope this is helpful. Updated patch attached...
Kind regards,
Nathan
[Patch] Supported SWIG version(Re: Supported SWIG version on
swig-py3)
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
Hi, I updated the patch to restrict supported SWIG version
posted before, <4e...@poem.co.jp>.
In build system:
* Make symbolic link for *.so files in build tree on the copy-swig-py
target, for the check-swig-py target (for SWIG >= 4.0.0). this symbolic
links are cleared on the clean-swig-py target
* Stop build with Python 3 + SWIG < 3.0.10 (or SWIG >= 4.0.0, yet)
* Stop build with Python 2 + SWIG >= 4.0.0
In subversion/bindings/swig/INSTALL:
* Add description that SWIG is optional for using distribution tarball.
* Update description of supported SWIG version.
Please review, especially I don't have any confidence in description
in INSTALL file.
Thanks,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Supported SWIG version on swig-py3
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/10/29 19:37, Branko Čibej wrote:
> If I understand your patch correctly, those symbolic links would be
> created in the build directory, not the source directory? In any case,
> if you commit that patch, you'll also have to fix the "clean-swig-py"
> target to remove the symlinks.
Yes, the symbolic links are created in $(SWIG_PY_DIR)/libsvn in the
build directory, built by the "$(SWIG_PY_DIR)/libsvn" target which is
also source of the "copy-swig-py" target. $(SWIG_PY_DIR)/libsvn
directory and its contents are removed by the first action
"rm -rf $(SWIG_PY_DIR}/libsvn" in the rule of "clean-swig-py" target.
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Supported SWIG version on swig-py3
Posted by Branko Čibej <br...@apache.org>.
On 29.10.2019 01:47, Yasuhito FUTATSUKI wrote:
> On 2019/10/29 2:45, Branko Čibej wrote:
>> Because if that's the case, I would
>> prefer
>> making 3.0.10 our required minimum version. Remember that users who
>> build
>> from our tarballs do not need Swig on Unix, it's only needed by
>> developers
>> and the RM.
>
> I see. I missunderstood it because ome downstream packager attempt to use
> their own platform SWIG, at least on FreeBSD and Arch Linux make
> dependecy
> to SWIG for their package building....
>
> https://www.freebsd.org/cgi/ports.cgi?query=py27-subversion&stype=all&sektion=all
>
> https://www.archlinux.org/packages/extra/x86_64/subversion/
>
> Then, I think we should update subversion/bindings/INSTALL to mension
> it is optional even for building swig language bindings.
Our .tar.bz2 and .tar.gz packages contain source files generated by
Swig. A packager /can/ decide to regenerate them; I don't know why they
would want to, but they can.
If I understand your patch correctly, those symbolic links would be
created in the build directory, not the source directory? In any case,
if you commit that patch, you'll also have to fix the "clean-swig-py"
target to remove the symlinks.
-- Brane
Re: Supported SWIG version on swig-py3
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/10/29 2:45, Branko Čibej wrote:
> On Mon 28 Oct 2019, 15:11 Yasuhito FUTATSUKI, <fu...@poem.co.jp> wrote:
>
>> (I sent this message 40 hours ago, however it is not derivered yet,
>> so I send again ....)
>>
>> On 2019/10/23 6:36, Yasuhito FUTATSUKI wrote:
>>> I ran check-swig-{py|pl|rb} with some SWIG versions on swig-py3 branch.
>>>
>>> Environment:
>>> OS: FreeBSD 11.2
>>> Python 2: 2.7.16
>>> Python 3: 3.7.3
>>> Perl: 5.28.2
>>> Ruby: 2.5.5p157
>>>
>>> Results are below.
>>
>> <snip>
>>
>>> SWIG 3.0.9:
>>> Python 2 ... can't import modules
>>> (regression, fixed in SWIG 3.0.10)[2]
>>> Python 3 ... can't import modules
>>> (regression, fixed in SWIG 3.0.10)[2]
>>> Perl ... PASS
>>> Ruby ... 100% passed
>>
>>
>> With patch against Makefile.in below, which makes install time module
>> hierarchy in build/test directory by using symbolic link in
>> copy-swig-py target, 'make check-swig-py' passed with SWIG 3.0.9,
>> both with Python 2 and with Python 3.
>>
>> [[[
>> Index: Makefile.in
>> ===================================================================
>> --- Makefile.in (revision 1868723)
>> +++ Makefile.in (working copy)
>> @@ -934,6 +934,7 @@
>> @for f in $(SWIG_PY_SRC_DIR)/*.py $(SWIG_PY_DIR)/*.py; do \
>> ! [ -f "$$f" ] || cp -pf $$f $(SWIG_PY_DIR)/libsvn; \
>> done
>> + @cd $(SWIG_PY_DIR)/libsvn;ln -sf ../.libs/*.so .
>> @touch $(SWIG_PY_DIR)/libsvn/__init__.py
>>
>> swig-py: autogen-swig-py copy-swig-py
>> ]]]
>>
>
>
> Do I understand correctly that this is a workaround for a bug in Swig
> 3.0.9, that was fixed in 3.0.10?
Yes, with Python >= 2.7 + SWIG 3.0.9, libsvn.foo try to import C extension
module _foo from its relative path only, doesn't fall back to normal module
search path. This only affects on test. Also, the code of libsvn.foo
produced by SWIG 4.0 attempts relative import only (as far as it is imported
as `libsvn.foo', not as `foo'). So This patch also address to SWIG 4.0
(as there are other issues with SWIG 4.0, our code doesn't work yet, though).
> Because if that's the case, I would prefer
> making 3.0.10 our required minimum version. Remember that users who build
> from our tarballs do not need Swig on Unix, it's only needed by developers
> and the RM.
I see. I missunderstood it because ome downstream packager attempt to use
their own platform SWIG, at least on FreeBSD and Arch Linux make dependecy
to SWIG for their package building....
https://www.freebsd.org/cgi/ports.cgi?query=py27-subversion&stype=all&sektion=all
https://www.archlinux.org/packages/extra/x86_64/subversion/
Then, I think we should update subversion/bindings/INSTALL to mension
it is optional even for building swig language bindings.
> (N.b., we may have to include generated sources for both Python 2 and 3 in
> our tarballs, but that's a separate issue.)
>
>
>
> (On Windows, as far as I read win_tests.py, it copies modules
>>
> for test with same hierarchy as install time, so it doesn't
>> affect, I guess.)
>>
>
>
> Yes, Windows shouldn't be affected.
Ah, thanks for clearify it.
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Supported SWIG version on swig-py3
Posted by Branko Čibej <br...@apache.org>.
On Mon 28 Oct 2019, 15:11 Yasuhito FUTATSUKI, <fu...@poem.co.jp> wrote:
> (I sent this message 40 hours ago, however it is not derivered yet,
> so I send again ....)
>
> On 2019/10/23 6:36, Yasuhito FUTATSUKI wrote:
> > I ran check-swig-{py|pl|rb} with some SWIG versions on swig-py3 branch.
> >
> > Environment:
> > OS: FreeBSD 11.2
> > Python 2: 2.7.16
> > Python 3: 3.7.3
> > Perl: 5.28.2
> > Ruby: 2.5.5p157
> >
> > Results are below.
>
> <snip>
>
> > SWIG 3.0.9:
> > Python 2 ... can't import modules
> > (regression, fixed in SWIG 3.0.10)[2]
> > Python 3 ... can't import modules
> > (regression, fixed in SWIG 3.0.10)[2]
> > Perl ... PASS
> > Ruby ... 100% passed
>
>
> With patch against Makefile.in below, which makes install time module
> hierarchy in build/test directory by using symbolic link in
> copy-swig-py target, 'make check-swig-py' passed with SWIG 3.0.9,
> both with Python 2 and with Python 3.
>
> [[[
> Index: Makefile.in
> ===================================================================
> --- Makefile.in (revision 1868723)
> +++ Makefile.in (working copy)
> @@ -934,6 +934,7 @@
> @for f in $(SWIG_PY_SRC_DIR)/*.py $(SWIG_PY_DIR)/*.py; do \
> ! [ -f "$$f" ] || cp -pf $$f $(SWIG_PY_DIR)/libsvn; \
> done
> + @cd $(SWIG_PY_DIR)/libsvn;ln -sf ../.libs/*.so .
> @touch $(SWIG_PY_DIR)/libsvn/__init__.py
>
> swig-py: autogen-swig-py copy-swig-py
> ]]]
>
Do I understand correctly that this is a workaround for a bug in Swig
3.0.9, that was fixed in 3.0.10? Because if that's the case, I would prefer
making 3.0.10 our required minimum version. Remember that users who build
from our tarballs do not need Swig on Unix, it's only needed by developers
and the RM.
(N.b., we may have to include generated sources for both Python 2 and 3 in
our tarballs, but that's a separate issue.)
(On Windows, as far as I read win_tests.py, it copies modules
>
for test with same hierarchy as install time, so it doesn't
> affect, I guess.)
>
Yes, Windows shouldn't be affected.
-- Brane
>
Re: Supported SWIG version on swig-py3
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
(I sent this message 40 hours ago, however it is not derivered yet,
so I send again ....)
On 2019/10/23 6:36, Yasuhito FUTATSUKI wrote:
> I ran check-swig-{py|pl|rb} with some SWIG versions on swig-py3 branch.
>
> Environment:
> OS: FreeBSD 11.2
> Python 2: 2.7.16
> Python 3: 3.7.3
> Perl: 5.28.2
> Ruby: 2.5.5p157
>
> Results are below.
<snip>
> SWIG 3.0.9:
> Python 2 ... can't import modules
> (regression, fixed in SWIG 3.0.10)[2]
> Python 3 ... can't import modules
> (regression, fixed in SWIG 3.0.10)[2]
> Perl ... PASS
> Ruby ... 100% passed
With patch against Makefile.in below, which makes install time module
hierarchy in build/test directory by using symbolic link in
copy-swig-py target, 'make check-swig-py' passed with SWIG 3.0.9,
both with Python 2 and with Python 3.
[[[
Index: Makefile.in
===================================================================
--- Makefile.in (revision 1868723)
+++ Makefile.in (working copy)
@@ -934,6 +934,7 @@
@for f in $(SWIG_PY_SRC_DIR)/*.py $(SWIG_PY_DIR)/*.py; do \
! [ -f "$$f" ] || cp -pf $$f $(SWIG_PY_DIR)/libsvn; \
done
+ @cd $(SWIG_PY_DIR)/libsvn;ln -sf ../.libs/*.so .
@touch $(SWIG_PY_DIR)/libsvn/__init__.py
swig-py: autogen-swig-py copy-swig-py
]]]
(On Windows, as far as I read win_tests.py, it copies modules
for test with same hierarchy as install time, so it doesn't
affect, I guess.)
So I think swig Python bindings has no problem with SWIG 3.0.9.
Cheers,
--
Yasuhito FUTATSUKI <fu...@yf.bsdclub.org>
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Supported SWIG version on swig-py3
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
I ran check-swig-{py|pl|rb} with some SWIG versions on swig-py3 branch.
Environment:
OS: FreeBSD 11.2
Python 2: 2.7.16
Python 3: 3.7.3
Perl: 5.28.2
Ruby: 2.5.5p157
Results are below.
SWIG 1.3.24:
I couldn't build swig itself.
SWIG 2.0.0:
Python 2 ... OK (skipped=1)
Python 3 ... can't build modules
Perl ... can't build modules (Perhaps due to issue on build
with Perl >= 5.16, fixed in SWIG 2.0.8)[1]
Ruby ... 100% passed
SWIG 2.0.12:
Python 2 ... OK (skipped=1)
Python 3 ... FAILED (failures=16, errors=22)
Perl ... PASS
Ruby ... 100% passed
SWIG 3.0.0:
Python 2 ... OK (skipped=1)
Python 3 ... FAILED (failures=16, errors=22)
Perl ... PASS
Ruby ... 100% passed
SWIG 3.0.9:
Python 2 ... can't import modules
(regression, fixed in SWIG 3.0.10)[2]
Python 3 ... can't import modules
(regression, fixed in SWIG 3.0.10)[2]
Perl ... PASS
Ruby ... 100% passed
SWIG 3.0.10:
Python 2 ... OK (skipped=1)
Python 3 ... OK
Perl ... PASS
Ruby ... 100% passed
SWIG 3.0.12:
Python 2 ... OK (skipped=1)
Python 3 ... OK
Perl ... PASS
Ruby ... 100% passed
SWIG 4.0.1:
Python 2 ... can't build modules correctly (SVN-4818)
Python 3 ... can't build modules correctly (SVN-4818)
Perl ... PASS
Ruby ... 100% passed
[1] https://github.com/swig/swig/blob/rel-2.0.8/CHANGES.current#L30
[2] https://github.com/swig/swig/blob/rel-3.0.10/RELEASENOTES#L11
I also make a patch to update subversion/binding/swig/INSTALL,
however I don't touch build/ac-macros/swig.m4.
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Supported SWIG version on swig-py3
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/10/22 7:25, Yasuhito FUTATSUKI wrote:
> On 2019/10/21 19:55, Jun Omae wrote:
>> Hi,
>>
>> I'm trying to build and test swig-py3 branch (r1868677) on Ubuntu 16.04 with
>> Python 3.7, however I get FAILED(failures=16, errors=22) from the unit tests.
>>
>> Investigating the issue with helps of Yasuhito, that is caused by using old
>> SWIG version with no SWIG_PYTHON_STRICT_BYTE_CHAR feature. The
>> SWIG_PYTHON_STRICT_BYTE_CHAR feature is available since SWIG 3.0.9 but SWIG is
>> 3.0.8 in Ubuntu 16.04.
>>
>> I consider that we should warn the required SWIG version at least.
>>
>>
>> [1] https://github.com/swig/swig/blob/rel-3.0.10/CHANGES#L160
>
> Thank you for your report. I think if that feature or some other changes on
> swig-py3 breaks Python 2 (or accidentally Ruby and/or Perl bindings),
> we should bump required SWIG verersion or resolve issues with older SWIGs.
> However if it affect only with Python 3, we only need to restrict
> per Language bindings requirement.
>
> Anyway, I think we need more test with older SWIG (or restrict required
> SWIG version if we can't test).
With SWIG 3.0.0/2.0.12 + Python 2.7, make check-py passes all tests.
[[[
$ head -2 subversion/bindings/swig/python/core.py && make check-swig-py
# This file was automatically generated by SWIG (http://www.swig.org).
# Version 2.0.12
if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; export PYTHONLIB=/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/.libs:/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python2.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py
..............s..........................................................................................................................................
----------------------------------------------------------------------
Ran 153 tests in 13.568s
OK (skipped=1)
]]]
[[[
$ head -2 subversion/bindings/swig/python/core.py && make check-swig-py
# This file was automatically generated by SWIG (http://www.swig.org).
# Version 3.0.0
if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; export PYTHONLIB=/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/.libs:/home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python2.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py
..............s..........................................................................................................................................
----------------------------------------------------------------------
Ran 153 tests in 14.341s
OK (skipped=1)
]]]
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Supported SWIG version on swig-py3 (was: Re: Test failures with
Python 3 (Re: PMCs: any Hackathon requests? (deadline 11 October)))
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/10/21 19:55, Jun Omae wrote:
> Hi,
>
> I'm trying to build and test swig-py3 branch (r1868677) on Ubuntu 16.04 with
> Python 3.7, however I get FAILED(failures=16, errors=22) from the unit tests.
>
> Investigating the issue with helps of Yasuhito, that is caused by using old
> SWIG version with no SWIG_PYTHON_STRICT_BYTE_CHAR feature. The
> SWIG_PYTHON_STRICT_BYTE_CHAR feature is available since SWIG 3.0.9 but SWIG is
> 3.0.8 in Ubuntu 16.04.
>
> I consider that we should warn the required SWIG version at least.
>
>
> [1] https://github.com/swig/swig/blob/rel-3.0.10/CHANGES#L160
Thank you for your report. I think if that feature or some other changes on
swig-py3 breaks Python 2 (or accidentally Ruby and/or Perl bindings),
we should bump required SWIG verersion or resolve issues with older SWIGs.
However if it affect only with Python 3, we only need to restrict
per Language bindings requirement.
Anyway, I think we need more test with older SWIG (or restrict required
SWIG version if we can't test).
Thanks,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Jun Omae <ju...@gmail.com>.
Hi,
I'm trying to build and test swig-py3 branch (r1868677) on Ubuntu 16.04 with
Python 3.7, however I get FAILED(failures=16, errors=22) from the unit tests.
Investigating the issue with helps of Yasuhito, that is caused by using old
SWIG version with no SWIG_PYTHON_STRICT_BYTE_CHAR feature. The
SWIG_PYTHON_STRICT_BYTE_CHAR feature is available since SWIG 3.0.9 but SWIG is
3.0.8 in Ubuntu 16.04.
I consider that we should warn the required SWIG version at least.
[1] https://github.com/swig/swig/blob/rel-3.0.10/CHANGES#L160
[[[
Index: build/ac-macros/swig.m4
===================================================================
--- build/ac-macros/swig.m4 (revision 1868677)
+++ build/ac-macros/swig.m4 (working copy)
@@ -91,12 +91,12 @@
AC_MSG_RESULT([$SWIG_VERSION_RAW])
# If you change the required swig version number, don't forget to update:
# subversion/bindings/swig/INSTALL
- if test -n "$SWIG_VERSION" && test "$SWIG_VERSION" -ge "103024"; then
+ if test -n "$SWIG_VERSION" && test "$SWIG_VERSION" -ge "300009"; then
SWIG_SUITABLE=yes
else
SWIG_SUITABLE=no
AC_MSG_WARN([Detected SWIG version $SWIG_VERSION_RAW])
- AC_MSG_WARN([Subversion requires SWIG >= 1.3.24])
+ AC_MSG_WARN([Subversion requires SWIG >= 3.0.9])
fi
fi
Index: subversion/bindings/swig/INSTALL
===================================================================
--- subversion/bindings/swig/INSTALL (revision 1868677)
+++ subversion/bindings/swig/INSTALL (working copy)
@@ -65,7 +65,7 @@
Step 1: Install a suitable version of SWIG (which is
- currently SWIG version 1.3.24 or later).
+ currently SWIG version 3.0.9 or later).
* Perhaps your distribution packages a suitable version - if it does
install it, and skip to the last bullet point in this section.
@@ -72,7 +72,7 @@
* Go to http://www.swig.org/, download the source tarball, and unpack.
- * In the SWIG-1.3.xx directory, run ./configure.
+ * In the SWIG-3.0.xx directory, run ./configure.
If you plan to build the Python bindings, and have a system
with more than one version of Python installed, you may need
@@ -95,7 +95,7 @@
Run 'make && make install'
* To verify you have SWIG installed correctly, run "swig -version"
- from the command line. SWIG should report that it is version 1.3.24
+ from the command line. SWIG should report that it is version 3.0.9
or newer.
Step 1a: Install py3c library if building Python SWIG bindings.
]]]
--
Jun Omae <ju...@gmail.com> (大前 潤)
Test failures with Python 3 (Re: PMCs: any Hackathon requests?
(deadline 11 October))
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019-10-11 06:45, Daniel Shahaf wrote:
...
> From autogen.sh to 'make check', all steps pass with PYTHON=/usr/bin/python3.
There were two failures in 1.11 on tests.
https://lists.apache.org/thread.html/366e77c7443a0c575b47de150a2c394e1e685fde8887c805a656a33d@%3Cusers.subversion.apache.org%3E
and it seems they still exist in trunk.
on FreeBSD 11.1, Python 3.7.0, Subversion trunk@1868235:
[[[
W: CWD: /home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline
Traceback (most recent call last):
File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/main.py", line 1931, in run
rc = self.pred.run(sandbox)
File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/testcase.py", line 178, in run
result = self.func(sandbox)
File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svnadmin_tests.py", line 3862, in dump_no_canonicalize_svndate
assert propval + '\n' in dump_lines
AssertionError
FAIL: svnadmin_tests.py 69: svnadmin dump shouldn't canonicalize svn:date
]]]
[[[
W: A1/B/lambda has unexpectected contents
W: EXPECTED svn-test-work/working_copies/tree_conflict_tests-26/A1/B/lambda (match_all=True):
W: | This is the file 'lambda'.
W: | This is more content.
W: ACTUAL svn-test-work/working_copies/tree_conflict_tests-26/A1/B/lambda:
W: CWD: /home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline
Traceback (most recent call last):
File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/main.py", line 1931, in run
rc = self.pred.run(sandbox)
File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/testcase.py", line 178, in run
result = self.func(sandbox)
File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/tree_conflict_tests.py", line 1547, in local_missing_dir_endless_loop
[ "This is the file 'lambda'.\n", "This is more content.\n"], contents)
File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/verify.py", line 503, in compare_and_display_lines
expected.display_differences(message, label, actual)
File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/verify.py", line 154, in display_differences
display_lines(message, self.expected, actual, e_label, label)
File "/home/futatuki/work/subversion/vwc/trunk/subversion/tests/cmdline/svntest/verify.py", line 478, in display_lines
logger.warn('| ' + x.rstrip())
TypeError: can only concatenate str (not "bytes") to str
FAIL: tree_conflict_tests.py 26: endless loop when resolving local-missing dir
]]]
The former failure can be fixed by attached patch, fix_svnadmin_tests_patch.txt
The latter also can be fixed by fix_tree_conflict_tests_patch.txt
at least so that the test can be passed on Python 3. However, from above
stack trace, I think it is incomplete because there still exists some sort of
confusion between bytes and str in subversion/tests/cmdline/svntest/*.py
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Nathan Hartman wrote on Thu, 10 Oct 2019 21:12 +00:00:
> On Thu, Oct 10, 2019 at 4:54 PM Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> >
> > Nathan Hartman wrote on Thu, Oct 10, 2019 at 16:28:21 -0400:
> > > If we have to pull that item out (not saying we do yet, just IF), then that
> > > leaves us with no Python tasks. I'd like to have something Python-related
> > > on the list because there's a good chance that a 700+ member university
> > > hackathon will have some Python programmers. Any other suggested
> > > Python tasks we could add?
> >
> > Something in the test harness. For example, make it easier to run «make check»
> > with client version ≠ server version, to actually test on-the-wire
> > compatibility. Or use trunk client and trunk server, but make them both expose
> > fewer capabilities, to test that compatibility codepaths.
>
> Has any work been done on making the test harness Py3 compatible?
From autogen.sh to 'make check', all steps pass with PYTHON=/usr/bin/python3.
Julian, do you have any release.py tasks to suggest?
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Nathan Hartman <ha...@gmail.com>.
Hi all,
I'd like to thank everyone who participated in this weeklong debate and
for being patient with my incessant nagging to get this done. :-)
I replied to Sally's hackathon request (CC'd to this list a little while
ago). Hopefully everyone is happy with it.
It's not perfect. Sadly, we didn't end up listing any Python work. But
this has been a good learning experience.
My hope is to attract more developers to this great project. I know
that the chances of a hackathon participant becoming a long term
developer are slim, but nevertheless that's still within the realm of
possibility. The more we put ourselves out there, the better the
chances.
What I learned:
This time around we were passive. We didn't act until the request came
to us. In the future, I hope to actively search for these kinds of
opportunities.
Also, this time we weren't quite ready. Our issue tracker contains
items marked "bite size" that aren't and needs some housecleaning in
general. I like Julian's suggestion of bringing one item to the list
each week and gradually getting it under control (and I'll make sure that
it happens). Also, I'd like to get "bite size" items properly labeled,
so that the next time we need to provide a list of hackathon-ready work
items, we'll be able to do it much more quickly and easily.
But for a first effort, we put something out there, and maybe we'll
get a few of those items worked on. Whatever happens, we're better
off for trying.
Again, thank you to everyone for your input (and your patience!!).
Your efforts are much appreciated.
Nathan
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Branko Čibej <br...@apache.org>.
On 10.10.2019 23:12, Nathan Hartman wrote:
> On Thu, Oct 10, 2019 at 4:54 PM Daniel Shahaf <d....@daniel.shahaf.name> wrote:
>> Nathan Hartman wrote on Thu, Oct 10, 2019 at 16:28:21 -0400:
>>> If we have to pull that item out (not saying we do yet, just IF), then that
>>> leaves us with no Python tasks. I'd like to have something Python-related
>>> on the list because there's a good chance that a 700+ member university
>>> hackathon will have some Python programmers. Any other suggested
>>> Python tasks we could add?
>> Something in the test harness. For example, make it easier to run «make check»
>> with client version ≠ server version, to actually test on-the-wire
>> compatibility. Or use trunk client and trunk server, but make them both expose
>> fewer capabilities, to test that compatibility codepaths.
> Has any work been done on making the test harness Py3 compatible?
> If not, then perhaps a Py2 -> Py3 port is something that should be on the
> list. Even if a 2 day hackathon is not enough time, even making a dent in
> that will be a helpful thing.
The test harness (and other build-related stuff) has been compatible
with Python3 for quite a few years now.
-- Brane
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Nathan Hartman <ha...@gmail.com>.
On Thu, Oct 10, 2019 at 4:54 PM Daniel Shahaf <d....@daniel.shahaf.name> wrote:
>
> Nathan Hartman wrote on Thu, Oct 10, 2019 at 16:28:21 -0400:
> > If we have to pull that item out (not saying we do yet, just IF), then that
> > leaves us with no Python tasks. I'd like to have something Python-related
> > on the list because there's a good chance that a 700+ member university
> > hackathon will have some Python programmers. Any other suggested
> > Python tasks we could add?
>
> Something in the test harness. For example, make it easier to run «make check»
> with client version ≠ server version, to actually test on-the-wire
> compatibility. Or use trunk client and trunk server, but make them both expose
> fewer capabilities, to test that compatibility codepaths.
Has any work been done on making the test harness Py3 compatible?
If not, then perhaps a Py2 -> Py3 port is something that should be on the
list. Even if a 2 day hackathon is not enough time, even making a dent in
that will be a helpful thing.
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Nathan Hartman wrote on Thu, Oct 10, 2019 at 16:28:21 -0400:
> If we have to pull that item out (not saying we do yet, just IF), then that
> leaves us with no Python tasks. I'd like to have something Python-related
> on the list because there's a good chance that a 700+ member university
> hackathon will have some Python programmers. Any other suggested
> Python tasks we could add?
Something in the test harness. For example, make it easier to run «make check»
with client version ≠ server version, to actually test on-the-wire
compatibility. Or use trunk client and trunk server, but make them both expose
fewer capabilities, to test that compatibility codepaths.
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Branko Čibej <br...@apache.org>.
On 10.10.2019 22:12, Nathan Hartman wrote:
> On Thu, Oct 10, 2019 at 1:48 PM Daniel Shahaf <d....@daniel.shahaf.name> wrote:
>>> Should these be re-merged into one item, as you had it before?
>> That depends. _Do_ the Windows build scripts support libmagic?
> I think they don't. Unless I missed something.
It does not because getting libmagic built on Windows is an extremely
huge pain. No-one has ever bothered to try yet.
>
> It looks as though the Windows build uses the makefile at
> tools/dev/windows-build/Makefile.
Nope. We have a build generator that creates a Visual Studio solution
and project files.
-- Brane
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Nathan Hartman <ha...@gmail.com>.
On Thu, Oct 10, 2019 at 4:18 PM Johan Corveleyn <jc...@gmail.com> wrote:
>
> On Thu, Oct 10, 2019 at 10:12 PM Nathan Hartman
> <ha...@gmail.com> wrote:
> > I'll merge the two items into one:
> >
> > * Python programming:
> >
> > - SVN-3914 - Implement building with libmagic on Windows in the build
> > scripts and document it in INSTALL.
> > https://issues.apache.org/jira/browse/SVN-3914?issueNumber=3914
>
> I build SVN on Windows regularly. It's not for the faint of heart --
> it might take a hackathon participant the full two days just to get it
> to build. (of course, it doesn't help that I'm not very experienced in
> C and Visual Studio ... but anyway. The main problem is all the
> dependencies that you need to get built first)
>
> Indeed, I don't think the build scripts for Windows currently support
> libmagic. The first question is actually: where can I download
> libmagic "for windows"? I don't know whether libmagic itself is easily
> available / buildable on Windows ...
>
> I'm putting Stefan Sperling in cc, because he filed SVN-3914. Perhaps
> he knows more about it?
>
> All in all, I think it's a bit of a risky issue to put on the list,
> because: who will guide / mentor that one? It requires at least good
> knowledge of the Windows build system, and I, for one, do not have
> that knowledge. I just use the build scripts, I don't really
> understand them :-).
Well that sounds scary. :-)
If we have to pull that item out (not saying we do yet, just IF), then that
leaves us with no Python tasks. I'd like to have something Python-related
on the list because there's a good chance that a 700+ member university
hackathon will have some Python programmers. Any other suggested
Python tasks we could add?
> BTW: thanks a lot for driving this, Nathan.
My pleasure.
Nathan
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Johan Corveleyn <jc...@gmail.com>.
On Thu, Oct 10, 2019 at 10:12 PM Nathan Hartman
<ha...@gmail.com> wrote:
>
> On Thu, Oct 10, 2019 at 1:48 PM Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> > > Should these be re-merged into one item, as you had it before?
> >
> > That depends. _Do_ the Windows build scripts support libmagic?
>
> I think they don't. Unless I missed something.
>
> It looks as though the Windows build uses the makefile at
> tools/dev/windows-build/Makefile. I am guessing that packagers for the
> Windows platform are using this as their starting point? Anyway, this
> makefile makes absolutely no mention of libmagic.
>
> I'll merge the two items into one:
>
> * Python programming:
>
> - SVN-3914 - Implement building with libmagic on Windows in the build
> scripts and document it in INSTALL.
> https://issues.apache.org/jira/browse/SVN-3914?issueNumber=3914
I build SVN on Windows regularly. It's not for the faint of heart --
it might take a hackathon participant the full two days just to get it
to build. (of course, it doesn't help that I'm not very experienced in
C and Visual Studio ... but anyway. The main problem is all the
dependencies that you need to get built first)
Indeed, I don't think the build scripts for Windows currently support
libmagic. The first question is actually: where can I download
libmagic "for windows"? I don't know whether libmagic itself is easily
available / buildable on Windows ...
I'm putting Stefan Sperling in cc, because he filed SVN-3914. Perhaps
he knows more about it?
All in all, I think it's a bit of a risky issue to put on the list,
because: who will guide / mentor that one? It requires at least good
knowledge of the Windows build system, and I, for one, do not have
that knowledge. I just use the build scripts, I don't really
understand them :-).
BTW: thanks a lot for driving this, Nathan.
--
Johan
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Nathan Hartman <ha...@gmail.com>.
On Thu, Oct 10, 2019 at 1:48 PM Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> > Should these be re-merged into one item, as you had it before?
>
> That depends. _Do_ the Windows build scripts support libmagic?
I think they don't. Unless I missed something.
It looks as though the Windows build uses the makefile at
tools/dev/windows-build/Makefile. I am guessing that packagers for the
Windows platform are using this as their starting point? Anyway, this
makefile makes absolutely no mention of libmagic.
I'll merge the two items into one:
* Python programming:
- SVN-3914 - Implement building with libmagic on Windows in the build
scripts and document it in INSTALL.
https://issues.apache.org/jira/browse/SVN-3914?issueNumber=3914
Nathan
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Nathan Hartman <ha...@gmail.com>.
On Thu, Oct 10, 2019 at 1:48 PM Daniel Shahaf <d....@daniel.shahaf.name>
wrote:
> Nathan Hartman wrote on Thu, 10 Oct 2019 17:36 +00:00:
> > On Thu, Oct 10, 2019 at 1:15 PM Daniel Shahaf <d....@daniel.shahaf.name>
> wrote:
> > >
> > > Nathan Hartman wrote on Thu, Oct 10, 2019 at 10:24:28 -0400:
> > > > * Documentation:
> > > >
> > > > - SVN-3914 - INSTALL: document how to compile with libmagic on
> > > > Windows.
> > > > https://issues.apache.org/jira/browse/SVN-3914?issueNumber=3914
> > >
> > > This is a subtask of the next one, isn't it?
> >
> > I separated them because one is documentation, the other is Python
> > programming. I tried to keep things organized by skill set. Also SVN-3914
> > didn't actually include updating the scripts (unless I missed something).
> >
> > Should these be re-merged into one item, as you had it before?
>
> That depends. _Do_ the Windows build scripts support libmagic?
>
Good question. I've never built SVN on Windows, only Unix, so I don't
actually know. I can look at the build scripts but if someone knows the
answer to this please chime in!!
More below
> We can include the github link but there was a whole discussion here
> > about how pull requests were sitting there untouched, and there was no
> > clean way to accept them and have them marked as such. Has that been
> > fixed? If not, I'd rather avoid the confusion that would cause. Let me
> > know...
>
> Sure, there's no point in recommending a patch submission channel that's
> not easy for us to work with. However, there's a third option: post the
> github link but ask patches to be emailed. (We should probably just add
> the link to HACKING, though, in that case; it's not specific to this
> event.)
We could do that and monitor github during the event in case some
participants don't notice the request to mail patches.
But we should eventually figure out a better solution for that submission
channel.
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Nathan Hartman wrote on Thu, 10 Oct 2019 17:36 +00:00:
> On Thu, Oct 10, 2019 at 1:15 PM Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> >
> > Nathan Hartman wrote on Thu, Oct 10, 2019 at 10:24:28 -0400:
> > > * Documentation:
> > >
> > > - SVN-3914 - INSTALL: document how to compile with libmagic on
> > > Windows.
> > > https://issues.apache.org/jira/browse/SVN-3914?issueNumber=3914
> >
> > This is a subtask of the next one, isn't it?
>
> I separated them because one is documentation, the other is Python
> programming. I tried to keep things organized by skill set. Also SVN-3914
> didn't actually include updating the scripts (unless I missed something).
>
> Should these be re-merged into one item, as you had it before?
That depends. _Do_ the Windows build scripts support libmagic?
If they do, then it's a documentation task and the Python task should be
deleted (as already done).
If they don't, then it's a Python task and the documentation task should
be merged into it; whoever implements it will be expected to document it
as well.
> > > We'll be happy to provide whatever guidance we can, as well as review
> > > and (hopefully) apply some quality patches!
> > >
> >
> > I wonder how many participants we're excluding here just by using a dated
> > expression such as "apply patches", as opposed to "merge a pull request"… ;-)
>
> Is applying a patch really such a strange concept these days? I
> participate in some projects that use Git and take as many patches as
> they do pull requests.
>
> We can include the github link but there was a whole discussion here
> about how pull requests were sitting there untouched, and there was no
> clean way to accept them and have them marked as such. Has that been
> fixed? If not, I'd rather avoid the confusion that would cause. Let me
> know...
Sure, there's no point in recommending a patch submission channel that's
not easy for us to work with. However, there's a third option: post the
github link but ask patches to be emailed. (We should probably just add
the link to HACKING, though, in that case; it's not specific to this event.)
Cheers,
Daniel
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Nathan Hartman <ha...@gmail.com>.
On Thu, Oct 10, 2019 at 1:15 PM Daniel Shahaf <d....@daniel.shahaf.name> wrote:
>
> Nathan Hartman wrote on Thu, Oct 10, 2019 at 10:24:28 -0400:
> > * Documentation:
> >
> > - SVN-3914 - INSTALL: document how to compile with libmagic on
> > Windows.
> > https://issues.apache.org/jira/browse/SVN-3914?issueNumber=3914
>
> This is a subtask of the next one, isn't it?
I separated them because one is documentation, the other is Python
programming. I tried to keep things organized by skill set. Also SVN-3914
didn't actually include updating the scripts (unless I missed something).
Should these be re-merged into one item, as you had it before?
> > We'll be happy to provide whatever guidance we can, as well as review
> > and (hopefully) apply some quality patches!
> >
>
> I wonder how many participants we're excluding here just by using a dated
> expression such as "apply patches", as opposed to "merge a pull request"… ;-)
Is applying a patch really such a strange concept these days? I
participate in some projects that use Git and take as many patches as
they do pull requests.
We can include the github link but there was a whole discussion here
about how pull requests were sitting there untouched, and there was no
clean way to accept them and have them marked as such. Has that been
fixed? If not, I'd rather avoid the confusion that would cause. Let me
know...
Thanks,
Nathan Hartman
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Nathan Hartman wrote on Thu, Oct 10, 2019 at 10:24:28 -0400:
> * Documentation:
>
> - SVN-3914 - INSTALL: document how to compile with libmagic on
> Windows.
> https://issues.apache.org/jira/browse/SVN-3914?issueNumber=3914
This is a subtask of the next one, isn't it?
> * Python programming:
>
> - Implement linking with libmagic on Windows in the build scripts.
> Hackathon participants should join our 'dev' mailing list and introduce
> themselves. To join, email dev-subscribe@subversion.apache.org -- see
> https://subversion.apache.org/mailing-lists.html for details.
>
Some more useful links (up to you which ones, if any, to include):
https://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/structure
https://svn.apache.org/repos/asf/subversion/site/
https://svn.apache.org/repos/asf/subversion/trunk/subversion/bindings/swig/INSTALL
https://ci.apache.org/projects/subversion/nightlies/index.html
https://subversion.apache.org/patches
https://subversion.apache.org/docs/community-guide/
https://subversion.apache.org/docs/api/latest/
https://github.com/apache/subversion/
> We'll be happy to provide whatever guidance we can, as well as review
> and (hopefully) apply some quality patches!
>
I wonder how many participants we're excluding here just by using a dated
expression such as "apply patches", as opposed to "merge a pull request"… ;-)
> If you have any questions, please let us know by emailing
> dev@subversion.apache.org!
>
> Kind regards,
> The Apache Subversion PMC
>
> ]]]
> If I hear nothing to the contrary, I'll assume silence = agreement
> and reply to Sally (with CC to this list) at 12:00 AM GMT.
+1 and thanks!
Re: SVN-4065 - server should enforce LF normalization for
svn:eol-style=native
Posted by Branko Čibej <br...@apache.org>.
On 09.10.2019 19:14, Daniel Shahaf wrote:
> The FS layer exposes
> a filesystem that's a 0-indexed array of trees [implemented by the DAG layer].
> The repos layer does… what, exactly, on top of that?
I've been asking that from day one. :) All that API duplication, which
is incomplete to boot, is confusing.
Well, the repos layer is responsible for authorization checks, too.
-- Brane
Re: SVN-4065 - server should enforce LF normalization for svn:eol-style=native
Posted by Nathan Hartman <ha...@gmail.com>.
On Wed, Oct 9, 2019 at 7:12 PM Johan Corveleyn <jc...@gmail.com> wrote:
> But I think this kind of content validation is different from
> "application-level content validation", like "follow coding style X",
> or "commit message should contain an issue key", or "I want to enforce
> that .c files have svn:eol-style=native". Such application-level
> content rules have no effect whatsoever on the workings of SVN itself.
> SVN clients don't care about those, only the applications (IDE, users,
> ...) do.
>
> In contrast, having a "non-LF-normalized with svn:eol-style=native" in
> the repository breaks "svn diff" (all lines shown as different) and
> "svn status" (if timestamps mismatch, contents are checksummed, and
> the file shows up as modified while it isn't -- causing confusion and
> even worse, spurious tree conflicts).
>
> So in my eyes this is more of an obligatory validation, to preserve
> the sanity of your "svn ecosystem".
I don't (yet) have an opinion whether this should be a hook or implemented
internally. But when you explain it this way, I suppose I might gravitate
towards obligatory validation. I'm undecided so far because that seems to
cause some headaches in terms of legacy data and a path forward. I need to
learn more about it.
On the separate question of a standardized C-coded binary svnhooks program,
I like this idea. I see it as a "busybox"[1] of svn hooks. I like this
because many (perhaps most?) installations could use it rather than coding
up their own custom scripts, reducing setup difficulty on admins and
hopefully reducing errors and misunderstandings. I like standardization.
But I'm probably repeating everything that was already said. :-)
It is important to let admins continue to write their own scripts when they
have custom needs but I'm sure that's the intention anyway.
[1] Busybox is a single executable that implements several standard UNIX
utilities:
https://en.wikipedia.org/wiki/BusyBox
I see a "svnhooks" program as being an analogue of that because it could
consolidate all the existing commonly used hook scripts into one program.
Nathan
Re: SVN-4065 - server should enforce LF normalization for svn:eol-style=native
Posted by Johan Corveleyn <jc...@gmail.com>.
On Wed, Oct 9, 2019 at 7:14 PM Daniel Shahaf <d....@daniel.shahaf.name> wrote:
>
> Branko Čibej wrote on Wed, Oct 09, 2019 at 15:45:32 +0200:
> > On 09.10.2019 15:00, Johan Corveleyn wrote:
> > > During that entire discussion thread the only objections raised were
> > > about "enforcing it in the repos layer / server directly". No-one
> > > objected to the proposal(s) to solve the issue via pre-commit hooks.
> >
> > Validating contents (such as line endings based on svn:eol-style) is a
> > perfect fit for hooks. Not normalising of course. :)
>
> The repos layer validates svn:* revprops in svn_repos__validate_prop().
>
> The FS layer doesn't do that validation.
>
> The result of that is that old repositories with invalid svn:* properties can
> be dump/load'd but can't be svnsync'.d
>
> ---
>
> As to separation of concerns, that's a valid point, but we should be consistent
> about what concerns belong where. The validation of svn:* props and of
> contents of files with svn:eol-style set belongs in the same layer, doesn't it?
>
> Therefore, I think there are two options:
>
> - These validation belongs in the repos layer. The simpler/lower layer that
> doesn't do such validation is the FS layer. svn:eol-style validation belongs
> in the repos layer. (And if svnsync needs to bypass it, we'll design a way
> for it to do so.)
>
> - These validations belong $elsewhere. svn:eol-style validation will be added
> $elsewhere and svn_repos__validate_prop() will be moved $elsewhere.
>
> My interpretation: I don't have an opinion yet, except that if we move the
> validation out of the repos layer, I won't be quite as sure any more what the
> difference between the FS layer and the repos layer _is_. The FS layer exposes
> a filesystem that's a 0-indexed array of trees [implemented by the DAG layer].
> The repos layer does… what, exactly, on top of that?
Thinking about it again some more, I guess I agree that this really
belongs in the same place as svn_repos__validate_prop(), and really
should be in the repos layer (or in any case, more widely /
automatically / standardly available than just a hook script, where it
depends too much on the individual sysadmin).
In fact, back in 2012 I wasn't happy with the consensus gravitating
towards "solve this in a pre-commit hook" (except as a stop-gap
measure). But the "standard pre-commit utility" seemed to be the best
attainable idea that would receive support from most devs, at the
time.
Earlier in the current thread Brane said:
"Validating contents (such as line endings based on svn:eol-style) is
a perfect fit for hooks."
But I think this kind of content validation is different from
"application-level content validation", like "follow coding style X",
or "commit message should contain an issue key", or "I want to enforce
that .c files have svn:eol-style=native". Such application-level
content rules have no effect whatsoever on the workings of SVN itself.
SVN clients don't care about those, only the applications (IDE, users,
...) do.
In contrast, having a "non-LF-normalized with svn:eol-style=native" in
the repository breaks "svn diff" (all lines shown as different) and
"svn status" (if timestamps mismatch, contents are checksummed, and
the file shows up as modified while it isn't -- causing confusion and
even worse, spurious tree conflicts).
So in my eyes this is more of an obligatory validation, to preserve
the sanity of your "svn ecosystem". A bit like the sha1 collision
protection (yes, the repository can store sha1 collisions perfectly
fine, but they wreak havoc amongst your clients / working copies if
you let them enter your repository). The "havoc" caused by an SVN-4065
file is orders of magnitude less severe than what a sha1 collision
causes (entire working copy hosed), but still, it's not nice.
(Julian: sorry for my frustrated reaction earlier, and thanks for
staying constructive, as always ... you were right, the discussion was
clearly not over yet :-))
--
Johan
Re: SVN-4065 - server should enforce LF normalization for
svn:eol-style=native
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Branko Čibej wrote on Wed, Oct 09, 2019 at 15:45:32 +0200:
> On 09.10.2019 15:00, Johan Corveleyn wrote:
> > During that entire discussion thread the only objections raised were
> > about "enforcing it in the repos layer / server directly". No-one
> > objected to the proposal(s) to solve the issue via pre-commit hooks.
>
> Validating contents (such as line endings based on svn:eol-style) is a
> perfect fit for hooks. Not normalising of course. :)
The repos layer validates svn:* revprops in svn_repos__validate_prop().
The FS layer doesn't do that validation.
The result of that is that old repositories with invalid svn:* properties can
be dump/load'd but can't be svnsync'.d
---
As to separation of concerns, that's a valid point, but we should be consistent
about what concerns belong where. The validation of svn:* props and of
contents of files with svn:eol-style set belongs in the same layer, doesn't it?
Therefore, I think there are two options:
- These validation belongs in the repos layer. The simpler/lower layer that
doesn't do such validation is the FS layer. svn:eol-style validation belongs
in the repos layer. (And if svnsync needs to bypass it, we'll design a way
for it to do so.)
- These validations belong $elsewhere. svn:eol-style validation will be added
$elsewhere and svn_repos__validate_prop() will be moved $elsewhere.
My interpretation: I don't have an opinion yet, except that if we move the
validation out of the repos layer, I won't be quite as sure any more what the
difference between the FS layer and the repos layer _is_. The FS layer exposes
a filesystem that's a 0-indexed array of trees [implemented by the DAG layer].
The repos layer does… what, exactly, on top of that?
Cheers,
Daniel
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Nathan Hartman <ha...@gmail.com>.
On Wed, Oct 2, 2019 at 5:43 PM Sally Khudairi <sk...@apache.org> wrote:
> Hello PMCs --I hope you are well.
>
> We received an email from hackCBS 2.0, a hackathon that will be taking place 19-20 October at the University of Delhi with 700+ students.
>
> They are interested in our participation by providing a list of tasks from various Apache projects for them to work on.
>
> If you would like to submit a list of work areas, please let me know your interest and forward your list(s) no later than 11 October.
>
> Many thanks,
> Sally
>
> - - -
> Vice President Marketing & Publicity
> Vice President Sponsor Relations
> The Apache Software Foundation
>
> Tel +1 617 921 8656 | sk@apache.org
We are basically out of time.
Unless there are any reasonable objections, I propose to reply to
Sally's email with the following.
I've taken into account everything that was discussed previously,
incorporated Johan's and Daniel's suggested tasks, removed the ones
that Julian pointed out are a bad idea, need review, or are pending a
design, etc.
Please speak up with any improvements as soon as possible!
If I hear nothing to the contrary, I'll assume silence = agreement
and reply to Sally (with CC to this list) at 12:00 AM GMT.
Proposed message follows:
[[[
Dear Sally,
We have opportunities for the 2-day hackathon across several skill set
areas including: web design, documentation, swig bindings, Python, and
C programming:
* Web design on https://subversion.apache.org:
- Incorporate normalize.css and main.css from html5boilerplate.com.
- Improve the navigation bar to make the static pages mobile
friendly, while keeping the site in a form that can be
hand-edited.
* Documentation:
- SVN-3914 - INSTALL: document how to compile with libmagic on
Windows.
https://issues.apache.org/jira/browse/SVN-3914?issueNumber=3914
* Python programming:
- Implement linking with libmagic on Windows in the build scripts.
* Swig bindings:
- SVN-4781 - expose svn_fs_change_rev_prop2() to swig.
https://issues.apache.org/jira/browse/SVN-4781?issueNumber=4781
- SVN-4568 - expose svn_fs_set_warning_func() to swig.
https://issues.apache.org/jira/browse/SVN-4568?issueNumber=4568
* C programming:
- SVN-4343 - FSFS backend work: "svnadmin verify" should verify
changed-paths list.
https://issues.apache.org/jira/browse/SVN-4343?issueNumber=4343
- SVN-4605 - "svnadmin verify" doesn't verify locks.
https://issues.apache.org/jira/browse/SVN-4605?issueNumber=4605
Hackathon participants should join our 'dev' mailing list and introduce
themselves. To join, email dev-subscribe@subversion.apache.org -- see
https://subversion.apache.org/mailing-lists.html for details.
We'll be happy to provide whatever guidance we can, as well as review
and (hopefully) apply some quality patches!
If you have any questions, please let us know by emailing
dev@subversion.apache.org!
Kind regards,
The Apache Subversion PMC
]]]
Re: SVN-4065 - server should enforce LF normalization for
svn:eol-style=native
Posted by Julian Foad <ju...@apache.org>.
Branko Čibej wrote:
>> The repos layer to a large extent transparent to properties and their
>> values, though not so much: it has some validation and even
>> "normalization" of "svn:" property names and values. I feel this is
>> generally Bad; there is some room for repos-layer knowledge of
>> properties but we should have separated the concerns better.
>
> Does the repos layer actually normalize svn: properties? I know
> 'svnadmin load' can, but I don't believe the repos API does that?
AFAICT the RA parts of the libsvn_repos API never modify node props,
only revision props.
The normalization of node props done by 'svnadmin load' is implemented
in libsvn_repos through svn_repos_get_fs_build_parser6():
http://svn.apache.org/viewvc/subversion/tags/1.12.0/subversion/include/svn_repos.h?view=markup#l3863
That is yet another case where it's implemented at the wrong level.
Here, "svnrdump load" can't share it. The dumpstream loader should be
refactored so svnadmin and svnrdump share code:
https://subversion.apache.org/issue/4780 "Factor out the dumpstream
loader editor driver".
- Julian
Re: SVN-4065 - server should enforce LF normalization for
svn:eol-style=native
Posted by Branko Čibej <br...@apache.org>.
On 09.10.2019 11:50, Julian Foad wrote:
> Julian Foad wrote:
>> If we want to change this to a repository-level rule, then that
>> implies we are changing the repository semantics in a
>> backward-incompatible way and we would need to address that (using a
>> format number bump, and an upgrade/migration path). We could discuss
>> that path but I don't think anyone's currently pushing for that and I
>> don't see good reason to do so, so let's leave that aside.
>
> I want to add something stronger. It would be a mistake to push
> validation of file content against property values into the repos
> layer as a new rule about the allowed semantics of repository contents.
>
> It's important to have different levels in the architecture with
> strongly defined characteristics, generally with lower layers having
> simpler semantics. Currently the repos layer does not interfere with
> file content at all. That is Good.
Couldn't agree more. There's a rather large can of really ugly worms
hiding in there and we do not want to look for it, let alone open it.
> The repos layer to a large extent transparent to properties and their
> values, though not so much: it has some validation and even
> "normalization" of "svn:" property names and values. I feel this is
> generally Bad; there is some room for repos-layer knowledge of
> properties but we should have separated the concerns better.
Does the repos layer actually normalize svn: properties? I know
'svnadmin load' can, but I don't believe the repos API does that?
-- Brane
Re: SVN-4065 - server should enforce LF normalization for
svn:eol-style=native
Posted by Julian Foad <ju...@apache.org>.
Julian Foad wrote:
> If we want to change this to a repository-level rule, then that implies
> we are changing the repository semantics in a backward-incompatible way
> and we would need to address that (using a format number bump, and an
> upgrade/migration path). We could discuss that path but I don't think
> anyone's currently pushing for that and I don't see good reason to do
> so, so let's leave that aside.
I want to add something stronger. It would be a mistake to push
validation of file content against property values into the repos layer
as a new rule about the allowed semantics of repository contents.
It's important to have different levels in the architecture with
strongly defined characteristics, generally with lower layers having
simpler semantics. Currently the repos layer does not interfere with
file content at all. That is Good.
The repos layer to a large extent transparent to properties and their
values, though not so much: it has some validation and even
"normalization" of "svn:" property names and values. I feel this is
generally Bad; there is some room for repos-layer knowledge of
properties but we should have separated the concerns better.
- Julian
Re: SVN-4065 - server should enforce LF normalization for
svn:eol-style=native
Posted by Branko Čibej <br...@apache.org>.
On 09.10.2019 15:00, Johan Corveleyn wrote:
> During that entire discussion thread the only objections raised were
> about "enforcing it in the repos layer / server directly". No-one
> objected to the proposal(s) to solve the issue via pre-commit hooks.
Validating contents (such as line endings based on svn:eol-style) is a
perfect fit for hooks. Not normalising of course. :)
-- Brane
Re: SVN-4065 - server should enforce LF normalization for svn:eol-style=native
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Julian Foad wrote on Wed, 09 Oct 2019 15:13 +00:00:
> * [SVN-4065] Redefine as:
> "Write a pre-commit hook to reject non-conforming eol-style=native."
> Put it in 'tools/hook-scripts/'.
...
> The redefined SVN-4065 could be a suitable hackathon task.
Isn't writing `svnlook changed -t $TXN | cut -c 5- | while read -r line;
do if svnlook propget $REPOS svn:eol-style $line | grep -qx native &&
svnlook cat -t $TXN $REPOS $line | xxd -p -c 1 | grep -qxi 0d; then
exit 1; fi; done` is too small a task for a hackathon?
Re: SVN-4065 - server should enforce LF normalization for
svn:eol-style=native
Posted by Julian Foad <ju...@apache.org>.
I suggest there are three issues mixed up here:
* [SVN-4065] Redefine as:
"Write a pre-commit hook to reject non-conforming eol-style=native."
Put it in 'tools/hook-scripts/'.
* [new issue]
Review/organize the set of recommended/suggested hooks.
See 'tools/hook-scripts/'. See what categories there are, e.g.
enforcing standard svn client behaviours (such as eol-style=native).
What can we do to make them easier to choose, configure, deploy?
Should some be withdrawn?
* [new issue]
Reconsider the idea of a configurable multi-hooks program.
I think that is not as obvious as it might look in that
old email thread and needs wider discussion.
The redefined SVN-4065 could be a suitable hackathon task.
- Julian
Re: SVN-4065 - server should enforce LF normalization for
svn:eol-style=native
Posted by Julian Foad <ju...@apache.org>.
Johan Corveleyn wrote:
> Julian Foad wrote:
>> Johan Corveleyn wrote:
>>> I think that was the conclusion from those threads. I.e. it would be
>>> best if we developed a standard "svnhooks" program that can be invoked
>>> from the pre-commit hook (and not try to implement this directly in
>>> the repos layer). At least, after those svnhooks suggestions no-one
>>> objected, so I assumed silent consensus about that way forward :-) ...
>>
>> Silence meant "errm... what?"
>
> I disagree. On this list silence means "no objections". If someone
> wanted to say "errm... what?", they should have mailed it.
Sorry, I was too harsh there.
Nevertheless, there are hundreds of issues and mail threads that are
incomplete but don't end with someone saying so.
My reading of the issue ended with the impression that there should be a
standard supplied "hooks program", but with no definition of what its
scope/purpose/role should be, so unclear what we should add to it, in
what contexts the svn server would execute it (like, svnadmin load?),
why it should be an external program, which server operators would
reasonably want to replace this program with a different one in what
kinds of situation, etc.
> In that very thread, 7 years ago, it seemed quite clear that consensus
> was gravitating towards "don't solve it in the repos layer, but in a
> pre-commit hook":
> * Daniel first posted a simple script for pre-commit hook validation:
> https://svn.haxx.se/dev/archive-2012-12/0180.shtml
[...]
> * Ivan suggested to create a separate program for it "svnhooks", with
> this concrete "rule" as a first use case for a more general tool:
> https://svn.haxx.se/dev/archive-2012-12/0202.shtml
> * Ivan specified his idea a bit further here:
> https://svn.haxx.se/dev/archive-2012-12/0217.shtml
>
> During that entire discussion thread the only objections raised were
> about "enforcing it in the repos layer / server directly". No-one
> objected to the proposal(s) to solve the issue via pre-commit hooks.
>
> Sound pretty consensus-y to me.
Thanks for summarizing the thread. I had only skimmed through bits of
it. In particular I hadn't seen where Ivan said the program would
implement hooks for "standard recommended polices", and I hadn't picked
up on it looking so consensus-y.
>> That means we have to design it in such a way that only "client commits"
>> are affected, and "server tools" such as sump/load are not, because we
>> can't have them suddenly start failing to load existing repository data.
>
> Yes, and hooks can do that. You're arriving at the same conclusion as
> the thread 7 years ago.
>
>> Are "svnrdump" and "svnsync" client-layer or repos-layer tools, for
>> these purposes? We don't yet have a formal way to distinguish and use
>> "repos-layer" tools through our client-server (RA) interface. So at the
>> moment I suppose we might say that ("de facto") all interactions through
>> the RA interface are deemed to be client-layer interactions. We could
>> consider an enhancement to enable "repos-layer" interactions to be
>> performed over RA with suitable authorization. We probably ought to
>> file that as a future enhancement issue.
>>
>> Currently we have "svn commit" and other client-layer operations, vs.
>> "svnadmin load" as the main repos-layer (server side) interaction.
>>
>> "svnadmin load" has these options:
>> --use-pre-commit-hook
>> --use-post-commit-hook
>> --normalize-props
>> --bypass-prop-validation
>>
>> To me, this looks like a crude attempt to allow it to support both
>> client-layer and repos-layer modes, but with no overall mode switch so
>> the user has to think about the implications of each individual sub-switch.
>>
>> We never spelled out the role of commit hooks. Therefore presumably
>> some commit hooks are used for client-side purposes (enforcing rules
>> like LF normalization and log message formats, etc.) and some for
>> server-side purposes (logging, backups, mirroring to svnsync, etc.).
>> The option to enable or disable all commit hooks seems misguided:
>> instead it would seem more appropriate to categorize hooks into client
>> and server roles, and have "svnadmin load" apply only those in the
>> server role.
>>
>> Is two roles of hooks something it makes sense to introduce? Or could
>> we clarify the current situation, document it?
>>
>> It looks to me like "enforcement of the standard svn client's rules" is
>> an option we would ideally like to provide to server operators. How
>> would this be defined more precisely? How should we design it to cope
>> with different client versions, whose rules have changed a few times?
>
> You're pulling this way out of scope.
I'm unpicking a load of unspoken assumptions.
> Issue SVN-4065 is very concrete
> and specific, and a concrete proposal was made on how best to solve
> it.
>
> You're using it to open a much broader architectural discussion. Which
> is fine of course, and I think quite interesting. But that shouldn't
> block progress for SVN-4065 in the way which was proposed 7 years ago,
> and which still seems the best way (via a utility program that can be
> invoked from hooks, where our hooks still have the same semantics /
> confusions / shortcomings / ... as today).
OK, so a valid option is to treat this as "just another pre-commit
hook". If the suggestion was just to write and supply another hook
script, it would be reasonable to ignore all that: we don't need to
think about the system semantics, as it's just like any other pre-commit
hook.
The talk about building the required hook functionality into a new
compiled program led to me wondering more widely about the design. If
we're going to combine multiple "standard" hooks into one configurable
program, that sort of thing does need to be discussed.
It seems to me that combining multiple hooks into one configurable
program is orthogonal to the actual issue. The only reason it came up,
AFAICT, is because of possibly premature concern about efficiency.
> That being said, maybe that svnhooks utility program (Ivan's proposal)
> could be used to introduce a way to separate those different roles
> that hooks serve. Ivan hinted a bit in that direction in his post
> there. From https://svn.haxx.se/dev/archive-2012-12/0217.shtml:
>
>> svnhooks configuration file has separate section for each policy. For example:
>> [[[
>> [check-eol-normalization]
>> enable = yes
>>
>> [rev-propchange]
>> enable = yes
>> users = svnsync
>>
>> [edit-log-message]
>> enable = yes
>> users = admin, @author
>> ]]]
>
> I agree this should be thought through, with a good design (a "users"
> field might not cut it -- perhaps something else). It's clear that the
> proposal needs further design work.
Thanks.
I didn't mean to prevent a simple task from being done.
- Julian
Re: SVN-4065 - server should enforce LF normalization for svn:eol-style=native
Posted by Johan Corveleyn <jc...@gmail.com>.
On Wed, Oct 9, 2019 at 11:41 AM Julian Foad <ju...@apache.org> wrote:
>
> Johan Corveleyn wrote:
> > I think that was the conclusion from those threads. I.e. it would be
> > best if we developed a standard "svnhooks" program that can be invoked
> > from the pre-commit hook (and not try to implement this directly in
> > the repos layer). At least, after those svnhooks suggestions no-one
> > objected, so I assumed silent consensus about that way forward :-) ...
>
> Silence meant "errm... what?"
I disagree. On this list silence means "no objections". If someone
wanted to say "errm... what?", they should have mailed it.
In that very thread, 7 years ago, it seemed quite clear that consensus
was gravitating towards "don't solve it in the repos layer, but in a
pre-commit hook":
* Daniel first posted a simple script for pre-commit hook validation:
https://svn.haxx.se/dev/archive-2012-12/0180.shtml
* Brane said the only sane place to do it was in pre-commit hook:
https://svn.haxx.se/dev/archive-2012-12/0182.shtml
* Brane also suggested to write it in C:
https://svn.haxx.se/dev/archive-2012-12/0191.shtml
* Ivan suggested to create a separate program for it "svnhooks", with
this concrete "rule" as a first use case for a more general tool:
https://svn.haxx.se/dev/archive-2012-12/0202.shtml
* Ivan specified his idea a bit further here:
https://svn.haxx.se/dev/archive-2012-12/0217.shtml
During that entire discussion thread the only objections raised were
about "enforcing it in the repos layer / server directly". No-one
objected to the proposal(s) to solve the issue via pre-commit hooks.
Sound pretty consensus-y to me.
...
> That means we have to design it in such a way that only "client commits"
> are affected, and "server tools" such as sump/load are not, because we
> can't have them suddenly start failing to load existing repository data.
Yes, and hooks can do that. You're arriving at the same conclusion as
the thread 7 years ago.
> Are "svnrdump" and "svnsync" client-layer or repos-layer tools, for
> these purposes? We don't yet have a formal way to distinguish and use
> "repos-layer" tools through our client-server (RA) interface. So at the
> moment I suppose we might say that ("de facto") all interactions through
> the RA interface are deemed to be client-layer interactions. We could
> consider an enhancement to enable "repos-layer" interactions to be
> performed over RA with suitable authorization. We probably ought to
> file that as a future enhancement issue.
>
> Currently we have "svn commit" and other client-layer operations, vs.
> "svnadmin load" as the main repos-layer (server side) interaction.
>
> "svnadmin load" has these options:
> --use-pre-commit-hook
> --use-post-commit-hook
> --normalize-props
> --bypass-prop-validation
>
> To me, this looks like a crude attempt to allow it to support both
> client-layer and repos-layer modes, but with no overall mode switch so
> the user has to think about the implications of each individual sub-switch.
>
> We never spelled out the role of commit hooks. Therefore presumably
> some commit hooks are used for client-side purposes (enforcing rules
> like LF normalization and log message formats, etc.) and some for
> server-side purposes (logging, backups, mirroring to svnsync, etc.).
> The option to enable or disable all commit hooks seems misguided:
> instead it would seem more appropriate to categorize hooks into client
> and server roles, and have "svnadmin load" apply only those in the
> server role.
>
> Is two roles of hooks something it makes sense to introduce? Or could
> we clarify the current situation, document it?
>
> It looks to me like "enforcement of the standard svn client's rules" is
> an option we would ideally like to provide to server operators. How
> would this be defined more precisely? How should we design it to cope
> with different client versions, whose rules have changed a few times?
You're pulling this way out of scope. Issue SVN-4065 is very concrete
and specific, and a concrete proposal was made on how best to solve
it.
You're using it to open a much broader architectural discussion. Which
is fine of course, and I think quite interesting. But that shouldn't
block progress for SVN-4065 in the way which was proposed 7 years ago,
and which still seems the best way (via a utility program that can be
invoked from hooks, where our hooks still have the same semantics /
confusions / shortcomings / ... as today).
That being said, maybe that svnhooks utility program (Ivan's proposal)
could be used to introduce a way to separate those different roles
that hooks serve. Ivan hinted a bit in that direction in his post
there. From https://svn.haxx.se/dev/archive-2012-12/0217.shtml:
> svnhooks configuration file has separate section for each policy. For example:
> [[[
> [check-eol-normalization]
> enable = yes
>
> [rev-propchange]
> enable = yes
> users = svnsync
>
> [edit-log-message]
> enable = yes
> users = admin, @author
> ]]]
I agree this should be thought through, with a good design (a "users"
field might not cut it -- perhaps something else). It's clear that the
proposal needs further design work.
--
Johan
Re: SVN-4065 - server should enforce LF normalization for
svn:eol-style=native
Posted by Julian Foad <ju...@apache.org>.
Johan Corveleyn wrote:
> I think that was the conclusion from those threads. I.e. it would be
> best if we developed a standard "svnhooks" program that can be invoked
> from the pre-commit hook (and not try to implement this directly in
> the repos layer). At least, after those svnhooks suggestions no-one
> objected, so I assumed silent consensus about that way forward :-) ...
Silence meant "errm... what?"
> Not sure if this is a good bite-sized task for interested hackers though ...
> Though it's quite well-contained (develop a separate (small) program
> with a configuration file, depending on existing server-side API's),
> and we have a clear use case to start with.
Glad we're reviewing this old issue.
What do you envision the purpose/scope/role of this program should be?
It's easy to suggest a facetious answer: "a collecting place for random
bug fixes". Of course not.
Seriously, the question that needs addressing for this issue is at what
level the LF normalization is to be enforced in a Subversion system
(deployment) in general. So far it has been client-side, with the
implication that different clients are expected but not forcibly
required to co-operate.
The first implication of that is that clients should handle gracefully
the case where repository data is not in fact normalized how they
expected it. That's one actionable conclusion.
If we want to change this to a repository-level rule, then that implies
we are changing the repository semantics in a backward-incompatible way
and we would need to address that (using a format number bump, and an
upgrade/migration path). We could discuss that path but I don't think
anyone's currently pushing for that and I don't see good reason to do
so, so let's leave that aside.
It seems we want to keep the design where this normalization is a
client-side rule, but now we want to provide additional server-side
enforcement of the client-side rule. We must still in odd cases allow
server tools (like dump/load) to continue working with repository
content that doesn't obey that rule.
That means we have to design it in such a way that only "client commits"
are affected, and "server tools" such as sump/load are not, because we
can't have them suddenly start failing to load existing repository data.
Are "svnrdump" and "svnsync" client-layer or repos-layer tools, for
these purposes? We don't yet have a formal way to distinguish and use
"repos-layer" tools through our client-server (RA) interface. So at the
moment I suppose we might say that ("de facto") all interactions through
the RA interface are deemed to be client-layer interactions. We could
consider an enhancement to enable "repos-layer" interactions to be
performed over RA with suitable authorization. We probably ought to
file that as a future enhancement issue.
Currently we have "svn commit" and other client-layer operations, vs.
"svnadmin load" as the main repos-layer (server side) interaction.
"svnadmin load" has these options:
--use-pre-commit-hook
--use-post-commit-hook
--normalize-props
--bypass-prop-validation
To me, this looks like a crude attempt to allow it to support both
client-layer and repos-layer modes, but with no overall mode switch so
the user has to think about the implications of each individual sub-switch.
We never spelled out the role of commit hooks. Therefore presumably
some commit hooks are used for client-side purposes (enforcing rules
like LF normalization and log message formats, etc.) and some for
server-side purposes (logging, backups, mirroring to svnsync, etc.).
The option to enable or disable all commit hooks seems misguided:
instead it would seem more appropriate to categorize hooks into client
and server roles, and have "svnadmin load" apply only those in the
server role.
Is two roles of hooks something it makes sense to introduce? Or could
we clarify the current situation, document it?
It looks to me like "enforcement of the standard svn client's rules" is
an option we would ideally like to provide to server operators. How
would this be defined more precisely? How should we design it to cope
with different client versions, whose rules have changed a few times?
- Julian
SVN-4065 - server should enforce LF normalization for
svn:eol-style=native (was: PMCs: any Hackathon requests? (deadline 11 October))
Posted by Johan Corveleyn <jc...@gmail.com>.
On Tue, Oct 8, 2019 at 6:13 PM Julian Foad <ju...@apache.org> wrote:
> Daniel Shahaf wrote:
> > SVN-4065 - server should enforce LF normalization for svn:eol-
> > style=native files. (Not just a patch is needed, but also a FAQ entry
> > that explains how to handle existing repositories that have this
> > problem, once the patch is released. If it'll be like when
> > svn_repos__validate_prop() was added, then dump/load will still work,
> > but svnsync not.)
>
> That issue appears to need review. It's not clear to me whether the
> server should really enforce LF normalization. I rather think not.
> There's a discussion thread linked from it, with no final conclusion.
My last comment in the issue [1] summarizes the discussion(s) as follows:
[[[
Some recent discussion on dev@:
http://svn.haxx.se/dev/archive-2012-12/0179.shtml
The current thinking from that thread is that this should (or can only) be
enforced through a pre-commit hook. It was suggested that this would ideally be
some standard, supported (and efficient) pre-commit hook. In a follow-up thread,
Ivan suggested to create a new program "svnhooks" for standardizing such hooks
(and go together with a configuration file to enable/disable certain behaviors):
http://svn.haxx.se/dev/archive-2012-12/0217.shtml
]]]
I think that was the conclusion from those threads. I.e. it would be
best if we developed a standard "svnhooks" program that can be invoked
from the pre-commit hook (and not try to implement this directly in
the repos layer). At least, after those svnhooks suggestions no-one
objected, so I assumed silent consensus about that way forward :-) ...
Not sure if this is a good bite-sized task for interested hackers though ...
Though it's quite well-contained (develop a separate (small) program
with a configuration file, depending on existing server-side API's),
and we have a clear use case to start with.
[1] https://issues.apache.org/jira/browse/SVN-4065?focusedCommentId=14931167&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14931167
--
Johan
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Julian Foad <ju...@apache.org>.
Daniel Shahaf wrote:
> SVN-4065 - server should enforce LF normalization for svn:eol-
> style=native files. (Not just a patch is needed, but also a FAQ entry
> that explains how to handle existing repositories that have this
> problem, once the patch is released. If it'll be like when
> svn_repos__validate_prop() was added, then dump/load will still work,
> but svnsync not.)
That issue appears to need review. It's not clear to me whether the
server should really enforce LF normalization. I rather think not.
There's a discussion thread linked from it, with no final conclusion.
- Julian
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Nathan Hartman wrote on Tue, 08 Oct 2019 15:09 +00:00:
> Now can we please have some more suggestions for a "byte size"
> issue and a Python-related task?
SVN-4781 - expose svn_fs_change_rev_prop2() to swig.
SVN-4568 - expose svn_fs_set_warning_func() to swig.
SVN-4343 - "svnadmin verify" should verify changed-paths list. (FSFS
backend work; not hard, but need to make sure the solution scales)
SVN-4605 - "svnadmin verify" doesn't verify locks.
SVN-3914 - INSTALL: document how to compile with libmagic on Windows.
(Is this also a Python task, to _implement_ linking with libmagic on
windows in the build scripts?)
SVN-4065 - server should enforce LF normalization for svn:eol-
style=native files. (Not just a patch is needed, but also a FAQ entry
that explains how to handle existing repositories that have this
problem, once the patch is released. If it'll be like when
svn_repos__validate_prop() was added, then dump/load will still work,
but svnsync not.)
> Reminder: We must reply to the hackathon request this week!
>
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Nathan Hartman <ha...@gmail.com>.
On Mon, Oct 7, 2019 at 10:49 AM Nathan Hartman <ha...@gmail.com> wrote:
> Since there will be 700+ university participants, there may be a
> variety of different skill sets, but we don't know what they are.
>
> Let's make a list of 2 or 3 items, each from a different skill area.
>
> Perhaps:
> * 1 "byte size" code quality issue from the issue tracker
> * 1 website task
> * 1 Py3 support task
I think we have our website task:
> to incorporate normalize.css and main.css from html5boilerplate.com
> and improve the navigation bar to make the static pages mobile
> friendly.
Now can we please have some more suggestions for a "byte size"
issue and a Python-related task?
Reminder: We must reply to the hackathon request this week!
Re: PMCs: any Hackathon requests? (deadline 11 October)
Posted by Nathan Hartman <ha...@gmail.com>.
On Mon, Oct 7, 2019 at 5:13 AM Julian Foad <ju...@apache.org> wrote:
> Moving this thread to dev@ as it should be public [1].
Sally Khudairi <sk...@apache.org> wrote:
> We received an email from hackCBS 2.0, a hackathon that will be
> taking place 19-20 October at the University of Delhi with 700+
> students.
>
> They are interested in our participation by providing a list of
> tasks from various Apache projects for them to work on.
>
> If you would like to submit a list of work areas, please let me
> know your interest and forward your list(s) no later than 11
> October.
We should take advantage of this hackathon request. We should do the
same for any opportunity to increase participation!
Since there will be 700+ university participants, there may be a
variety of different skill sets, but we don't know what they are.
Let's make a list of 2 or 3 items, each from a different skill area.
Perhaps:
* 1 "byte size" code quality issue from the issue tracker
* 1 website task
* 1 Py3 support task
Byte size issue tracker task: Some suggestions were already made but
are not a good fit for several reasons. Let's suggest other ones. I'll
go first again: https://issues.apache.org/jira/browse/SVN-1722 "svn
diff may missreport a revision as the working copy."
Website task: Johan suggests the website overhaul and Julian suggests
a subtask of that. I think that's a good idea. The task I suggest is
to incorporate normalize.css and main.css from html5boilerplate.com
and improve the navigation bar to make the static pages mobile
friendly.
Python 3 support task: I think something Python-related should be on
the list. How about https://issues.apache.org/jira/browse/SVN-2079
"utf8_tests.py should be made non-iso8859-1 specific"
Thoughts? Other items we should consider?
Nathan