You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Troy Curtis Jr <tr...@gmail.com> on 2019/09/02 01:41:47 UTC
Re: The run up to Subversion 1.13.0
On Fri, Aug 30, 2019, 7:13 PM James McCoy <ja...@jamessan.com> wrote:
> On Fri, Aug 30, 2019 at 05:12:21PM +0100, Julian Foad wrote:
> > 2. As 1.13 is only weeks away, there is little point in making any
> further
> > 1.12.x patch releases from now on, so I will not plan to do so.
> >
> > Any other thoughts on anything we ought to include, or to do?
>
> Finishing(?) and merging the Python3 support would be ideal. That would
> give one release for broader feedback before being in an LTS release.
>
> I haven't kept up with the status of that branch, nor can I guarantee
> any time to helping with this, but since you asked... :)
>
It needs to get reintegrated with trunk again with the latest changes, but
the linux side of the house was looking good. It was in trying to get my
Windows Dev environment back up and running again that got me frustrated
and gave me an excuse to wander off doing other things... IIRC, there is a
build issue with the Py2 bindings in that branch on Windows, but I never
got far enough to check it out.
If there was someone who could help out on the Windows side, I can jump in
and get the branch up to trunk and retested in order to get this thing over
the finish line.
> Cheers,
> --
> James
> GPG Key: 4096R/91BF BF4D 6956 BD5D F7B7 2D23 DFE6 91AE 331B A3DB
>
Troy
>
Re: The run up to Subversion 1.13.0
Posted by Stefan Sperling <st...@elego.de>.
On Sun, Sep 01, 2019 at 09:41:47PM -0400, Troy Curtis Jr wrote:
> It needs to get reintegrated with trunk again with the latest changes, but
> the linux side of the house was looking good. It was in trying to get my
> Windows Dev environment back up and running again that got me frustrated
> and gave me an excuse to wander off doing other things... IIRC, there is a
> build issue with the Py2 bindings in that branch on Windows, but I never
> got far enough to check it out.
If it helps, we could require Py3-only on Windows, going forward.
That particular ecosystem has been dragging its feet for long enough ;)
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Johan Corveleyn wrote on Wed, 18 Sep 2019 07:44 +00:00:
> On Wed, Sep 18, 2019 at 9:31 AM Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> >
> > Johan Corveleyn wrote on Wed, 18 Sep 2019 00:48 +00:00:
> > > No problem, after downloading py3c from Github, and adding
> > > --with-py3c, I can start building it.
> >
> > What version of py3c did you download? HEAD or a release?
>
> Hm, I downloaded from master at https://github.com/encukou/py3c. Maybe
> not such a good choice?
> Should I go for 0.9 from https://github.com/encukou/py3c/releases instead?
I didn't intend to imply that; I merely wanted to know which version to
use for investigations and replication attempts.
Note that on the /releases page there's a v1.0 dated three months after
the v0.9, if you click "Show newer tags" at the top. I'm not sure why github
doesn't offer that one.
Cheers,
Daniel
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Johan Corveleyn <jc...@gmail.com>.
On Wed, Sep 18, 2019 at 9:31 AM Daniel Shahaf <d....@daniel.shahaf.name> wrote:
>
> Johan Corveleyn wrote on Wed, 18 Sep 2019 00:48 +00:00:
> > No problem, after downloading py3c from Github, and adding
> > --with-py3c, I can start building it.
>
> What version of py3c did you download? HEAD or a release?
Hm, I downloaded from master at https://github.com/encukou/py3c. Maybe
not such a good choice?
Should I go for 0.9 from https://github.com/encukou/py3c/releases instead?
--
Johan
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Johan Corveleyn wrote on Wed, 18 Sep 2019 00:48 +00:00:
> No problem, after downloading py3c from Github, and adding
> --with-py3c, I can start building it.
What version of py3c did you download? HEAD or a release?
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Branko Čibej <br...@apache.org>.
On 20.09.2019 11:53, Paul Hammant wrote:
> One more thing re Python3 - there's Svn Homebrew fu for Mac folks -
> https://github.com/Homebrew/homebrew-core/blob/master/Formula/subversion.rb
> - that work admirably for Python 2.7.x but has no code for Python 3.
> I don't think this brew formula is maintained in the svn dev team. Of
> course, pull-requests are processed as you'd expected for a project
> hosted on GitHub.
It's not maintained by us. I've send a number of pull requests for new
versions, but whoever the maintainers are on the Homebrew team have some
very specific ideas about how packages should be produced -- obviously. :)
I think, if we get the Python 3 support in tell HB about it (via a PR of
course), they won't mind switching the dependency to Python 3, or maybe
even both.
(Homebrew Py3 is more reliable in my experience than their Py2, which
has strange interactions with the system default.)
-- Brane
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Paul Hammant <pa...@hammant.org>.
One more thing re Python3 - there's Svn Homebrew fu for Mac folks -
https://github.com/Homebrew/homebrew-core/blob/master/Formula/subversion.rb
- that work admirably for Python 2.7.x but has no code for Python 3. I
don't think this brew formula is maintained in the svn dev team. Of course,
pull-requests are processed as you'd expected for a project hosted on
GitHub.
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/09/28 22:36, Branko Čibej wrote:
> On 28.09.2019 11:20, Yasuhito FUTATSUKI wrote:
>> On 2019/09/24 16:34, Branko Čibej wrote:
>>> On 23.09.2019 22:48, Johan Corveleyn wrote:
>>>> On Mon, Sep 23, 2019 at 1:53 AM Yasuhito FUTATSUKI
>>>> <fu...@poem.co.jp> wrote:
>>>>> On 2019/09/23 6:16, Johan Corveleyn wrote:
>>
>>>>>> Building with Python 3.7.4 still fails with the same error though (no
>>>>>> problem, I know your patch wasn't addressing that, just mentioning it
>>>>>> here for completeness). Consequently I haven't been able to run the
>>>>>> swig-python tests with python 3.7 yet.
>>>>>>
>>>>>> [[[
>>>>>> c:\python37\include\pytime.h(123): error C4115: 'timeval': named type
>>>>>> definition in parentheses
>>>>>> [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj
>>>>>>
>>>>>> ]]]
>>>>> Here is a patch not to treat C4115 as error, globally. If this makes
>>>>> it possible to build with Python 3.7 on Windows, then next step can be
>>>>> to limit to apply this relaxation of compile option to files which
>>>>> contains "#include <Python.h>" (most of them are generated by
>>>>> swig...).
>>>> Yes, that makes the build succeed, thanks. I can't comment on whether
>>>> or not it's good to change this error into a warning overall, or just
>>>> for a limited set of files. I'll leave that discussion to others :-).
>>>
>>> It should not be an error because the source is valid C, regardless of
>>> what Microsoft's compiler thinks about it. :)
>>
>> However it is ourselves to decide treat it as an error, on r876281.
>>
>> https://svn.apache.org/viewvc?view=revision&revision=876281
>>
>> Unfortunately I can't find any reason or discussion about this
>> decision, yet.
>> That's why I try to treat it carefully. (Of course there might be no
>> serious reason, though)
>
>
> The most likely reason is that /our/ code should not contain a use of
> 'struct foo*' without at least a forward declaration or a typedef.
> That's a coding style decision that we shouldn't enforce on generated
> code (but it should be a warning, not an error). If you can turn off
> that compiler option only for the files generated by Swig, that would be
> great, but just turning it into a warning is good enough, IMO.
I agree your opinion. I also think a warning is enough for this purpose.
And, I found that C4055 is also once treated as an error on r876281,
but not to be treated as error again on 876286 for generated swig python
code, just same reason :)
(Though C4055 seems obsolete on Visual Studio 2017 and later)
https://svn.apache.org/viewvc?view=revision&revision=876286
(Daniel, thank you for your suggestion about revision number offset)
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Branko Čibej <br...@apache.org>.
On 28.09.2019 11:20, Yasuhito FUTATSUKI wrote:
> On 2019/09/24 16:34, Branko Čibej wrote:
>> On 23.09.2019 22:48, Johan Corveleyn wrote:
>>> On Mon, Sep 23, 2019 at 1:53 AM Yasuhito FUTATSUKI
>>> <fu...@poem.co.jp> wrote:
>>>> On 2019/09/23 6:16, Johan Corveleyn wrote:
>
>>>>> Building with Python 3.7.4 still fails with the same error though (no
>>>>> problem, I know your patch wasn't addressing that, just mentioning it
>>>>> here for completeness). Consequently I haven't been able to run the
>>>>> swig-python tests with python 3.7 yet.
>>>>>
>>>>> [[[
>>>>> c:\python37\include\pytime.h(123): error C4115: 'timeval': named type
>>>>> definition in parentheses
>>>>> [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj
>>>>>
>>>>> ]]]
>>>> Here is a patch not to treat C4115 as error, globally. If this makes
>>>> it possible to build with Python 3.7 on Windows, then next step can be
>>>> to limit to apply this relaxation of compile option to files which
>>>> contains "#include <Python.h>" (most of them are generated by
>>>> swig...).
>>> Yes, that makes the build succeed, thanks. I can't comment on whether
>>> or not it's good to change this error into a warning overall, or just
>>> for a limited set of files. I'll leave that discussion to others :-).
>>
>> It should not be an error because the source is valid C, regardless of
>> what Microsoft's compiler thinks about it. :)
>
> However it is ourselves to decide treat it as an error, on r876281.
>
> https://svn.apache.org/viewvc?view=revision&revision=876281
>
> Unfortunately I can't find any reason or discussion about this
> decision, yet.
> That's why I try to treat it carefully. (Of course there might be no
> serious reason, though)
The most likely reason is that /our/ code should not contain a use of
'struct foo*' without at least a forward declaration or a typedef.
That's a coding style decision that we shouldn't enforce on generated
code (but it should be a warning, not an error). If you can turn off
that compiler option only for the files generated by Swig, that would be
great, but just turning it into a warning is good enough, IMO.
-- Brane
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Yasuhito FUTATSUKI wrote on Sat, 28 Sep 2019 09:20 +00:00:
> https://svn.apache.org/viewvc?view=revision&revision=876281
>
> Unfortunately I can't find any reason or discussion about this decision, yet.
Are you aware of the "+ 840074" revision number offset that needs to be
applied to old revision numbers (those < 50000) in our history?
See https://svn.apache.org/repos/asf/subversion/README (sic) for details. In
this case, it means that where r876281 (= r36207) says "r36160", "r876234"
should be substituted:
https://svn.apache.org/r876234
However, that patch doesn't seem related to the compiler diagnostic at hand.
Bert, do you remember the rationale behind including C4115 in r876281?
Cheers,
Daniel
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/09/24 16:34, Branko Čibej wrote:
> On 23.09.2019 22:48, Johan Corveleyn wrote:
>> On Mon, Sep 23, 2019 at 1:53 AM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
>>> On 2019/09/23 6:16, Johan Corveleyn wrote:
>>>> Building with Python 3.7.4 still fails with the same error though (no
>>>> problem, I know your patch wasn't addressing that, just mentioning it
>>>> here for completeness). Consequently I haven't been able to run the
>>>> swig-python tests with python 3.7 yet.
>>>>
>>>> [[[
>>>> c:\python37\include\pytime.h(123): error C4115: 'timeval': named type
>>>> definition in parentheses
>>>> [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj
>>>> ]]]
>>> Here is a patch not to treat C4115 as error, globally. If this makes
>>> it possible to build with Python 3.7 on Windows, then next step can be
>>> to limit to apply this relaxation of compile option to files which
>>> contains "#include <Python.h>" (most of them are generated by swig...).
>> Yes, that makes the build succeed, thanks. I can't comment on whether
>> or not it's good to change this error into a warning overall, or just
>> for a limited set of files. I'll leave that discussion to others :-).
>
> It should not be an error because the source is valid C, regardless of
> what Microsoft's compiler thinks about it. :)
However it is ourselves to decide treat it as an error, on r876281.
https://svn.apache.org/viewvc?view=revision&revision=876281
Unfortunately I can't find any reason or discussion about this decision, yet.
That's why I try to treat it carefully. (Of course there might be no
serious reason, though)
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Branko Čibej <br...@apache.org>.
On 23.09.2019 22:48, Johan Corveleyn wrote:
> On Mon, Sep 23, 2019 at 1:53 AM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
>> On 2019/09/23 6:16, Johan Corveleyn wrote:
>>> On Sat, Sep 21, 2019 at 10:28 AM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
>>>> On 2019/09/20 0:44, Daniel Shahaf wrote:
>>>>> Yasuhito FUTATSUKI wrote on Thu, 19 Sep 2019 06:41 +00:00:
>>>>>> The FAIL on test_merge_peg3 is a newline style issue on reading file
>>>>>> in raw mode, which is introduced by my patch.
>>>>>> (https://svn.apache.org/viewvc/subversion/branches/swig-py3/subversion/bindings/swig/python/tests/client.py?view=annotate#l438)
>>>>>>
>>>>>> Rest of those errors and failures are in the tests that were added only
>>>>>> to swig-py, and it seems most of the causes of them are confusion of
>>>>>> paths in platform specific format and Subversion's canoical format,
>>>>>> as the comment in SubversionClientTestCase.test_update4() says.
>>>>>>
>>>>>> (I'll try to fix those issues on check-swig-py later, hopefully
>>>>>> next weekend.)
>>>>> Thanks! No rush.
>>>> Now I re-sync swig-py3 branch with trunk and make a patch addressing
>>>> issues on testing on other than Unix/Linux platform.
>>>>
>>>> I've tested on FreeBSD 11.2, with Python 2.7 and Python 3.6.
>>>>
>>>> Could anyone please test it on Windows ?
>>> Tested on Windows 7, and yes, this fixes the previous test failures
>>> with Python 2.7.16. Thanks!
>>>
>>> Test output:
>>> [[[
>>> Testing Release configuration on local repository.
>>> -- Running Swig Python tests --
>>> ..............s.........................................................................................................
>>> .................................
>>> ----------------------------------------------------------------------
>>> Ran 153 tests in 63.214s
>>>
>>> OK (skipped=1)
>>> ]]]
>>>
>> I've commited it, as r1866362. Thanks!
>>
>>> Building with Python 3.7.4 still fails with the same error though (no
>>> problem, I know your patch wasn't addressing that, just mentioning it
>>> here for completeness). Consequently I haven't been able to run the
>>> swig-python tests with python 3.7 yet.
>>>
>>> [[[
>>> c:\python37\include\pytime.h(123): error C4115: 'timeval': named type
>>> definition in parentheses
>>> [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj
>>> ]]]
>> Here is a patch not to treat C4115 as error, globally. If this makes
>> it possible to build with Python 3.7 on Windows, then next step can be
>> to limit to apply this relaxation of compile option to files which
>> contains "#include <Python.h>" (most of them are generated by swig...).
> Yes, that makes the build succeed, thanks. I can't comment on whether
> or not it's good to change this error into a warning overall, or just
> for a limited set of files. I'll leave that discussion to others :-).
It should not be an error because the source is valid C, regardless of
what Microsoft's compiler thinks about it. :)
-- Brane
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Johan Corveleyn <jc...@gmail.com>.
On Fri, Oct 11, 2019 at 2:38 PM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
>
> On 2019-09-30 06:32, Johan Corveleyn wrote:
> ...
> > At this point, I think the only thing blocking reintegration of this
> > branch into trunk seems to be the C4115 warning / error. As I said, I
> > have no informed opinion either way (and Brane was okay with turning
> > it into a warning). So if Bert doesn't reply within a couple of days,
> > I'd say we are good to go.
>
> ... and more than a week has past. So I've commited the patch address
> to it on swig-py3 branch, as r1868290.
Great. I've retested it all on Windows (once with Python 2.7.16, and
once with Python 3.7.4) and it looks good to me.
+1 to reintegrate the branch into trunk.
--
Johan
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019-09-30 06:32, Johan Corveleyn wrote:
...
> At this point, I think the only thing blocking reintegration of this
> branch into trunk seems to be the C4115 warning / error. As I said, I
> have no informed opinion either way (and Brane was okay with turning
> it into a warning). So if Bert doesn't reply within a couple of days,
> I'd say we are good to go.
... and more than a week has past. So I've commited the patch address
to it on swig-py3 branch, as r1868290.
Thanks,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Branko Čibej <br...@apache.org>.
On 29.09.2019 23:32, Johan Corveleyn wrote:
> BTW, I have one more question: is it expected that "swig-python
> bindings compiled with python 3" cannot be used by python 2? Because
> if I run the testsuite with Python 2.7.16, after I have compiled them
> with Python 3.7.4, I get the following error:
>
> [[[
> C:\research\svn\dev\swig-py3>python --version
> Python 2.7.16
>
> C:\research\svn\dev\swig-py3>runtest.bat --swig=python
> Testing Release configuration on local repository.
> -- Running Swig Python tests --
> Traceback (most recent call last):
> File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\run_all.py",
> line 23, in <module>
> import mergeinfo, core, client, delta, checksum, pool, fs, ra, wc,
> repository, \
> File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\mergeinfo.py",
> line 22, in <module>
> from svn import core, repos, fs
> File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests/..\svn\core.py",
> line 26, in <module>
> from libsvn.core import *
> File "R:\test_release--swigpython\swig\pylib\libsvn\core.py", line 137
> def apr_initialize() -> "apr_status_t":
> ^
> SyntaxError: invalid syntax
> [Test runner reported failure]
> ]]]
>
> So distributors / packagers that want to support both py2 and py3
> users, will have to create two sets of bindings.
Yes, they will. Python 2 and 3 libraries have to be separate anyway, and
the native bits of our bindings use different runtime libs.
Interesting and also very nice that Swig uses PEP 484 type annotations.
The Python 2 parser doesn't recognise those.
-- Brane
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Johan Corveleyn <jc...@gmail.com>.
On Sun, Sep 29, 2019 at 8:42 PM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
...
> Hmm.. It seems the diffrence comes from difference of implementation
> of class subprocess.Popen.
>
> I tweaked again clean up code for class svn.fs.FileDiff. Only if
> Python >= 3.3, it wait for maximum 10 sec to terminate subprocess.
> 'timeout' parameter for subprocess.Popen.wait() is added on Python 3.3.
> and not back ported to Python 2, so if we want to use wait() on
> earlier Python, it may wait forever.... (It can be possible to implement
> timeout by using poll() and sleep loop, but is it needed here?)
Okay, with that latest patch the testsuite runs fine now. Perfect.
[[[
C:\research\svn\dev\swig-py3>runtest.bat --swig=python
'ruby' is not recognized as an internal or external command,
operable program or batch file.
Testing Release configuration on local repository.
-- Running Swig Python tests --
........................................................................................................................
.................................
----------------------------------------------------------------------
Ran 153 tests in 63.938s
OK
]]]
> > When I treat ResourceWarning as an error, I get a lot more output,
> > especially before the test runs start. I suppose those are unrelated
> > to that one RW above.
>
> Yes, ... and ResourceWarnings are usually hided by default warning filter.
>
> https://docs.python.org/3/library/warnings.html#default-warning-filter
>
> Only while executing unittest.main(), warning filter is overrided by
> its optional parameter `warning'. Its default is specified filter on
> Python start up by environment variable PYTHONWARNINGS or -W option
> if it is specified, otherwise "default" action for all warning
> categories (i.e. show all warnings).
>
> > [[[
> > C:\research\svn\dev\swig-py3>set PYTHONWARNINGS=error::ResourceWarning::
> >
> > C:\research\svn\dev\swig-py3>runtest.bat --swig=python
> > Exception ignored in: <_io.FileIO name='build.conf' mode='rb' closefd=True>
> > ResourceWarning: unclosed file <_io.TextIOWrapper name='build.conf'
> > mode='r' encoding='cp1252'>
> > Exception ignored in: <_io.FileIO
> > name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr\\include\\apr_version.h'
> > mode='rb' closefd=True>
> > ResourceWarning: unclosed file <_io.TextIOWrapper
> > name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr\\include
> > \\apr_version.h' mode='r' encoding='cp1252'>
> > Exception ignored in: <_io.FileIO
> > name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\include\\apu_versi
> > on.h' mode='rb' closefd=True>
> > ResourceWarning: unclosed file <_io.TextIOWrapper
> > name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\in
> > clude\\apu_version.h' mode='r' encoding='cp1252'>
> > Exception ignored in: <_io.FileIO
> > name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\xml/expat/lib\\exp
> > at.h' mode='rb' closefd=True>
> > ResourceWarning: unclosed file <_io.TextIOWrapper
> > name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\xm
> > l/expat/lib\\expat.h' mode='r' encoding='cp1252'>
> > Exception ignored in: <_io.FileIO
> > name='C:\\research\\svn\\dev\\deps\\zlib-1.2.11\\zlib.h' mode='rb'
> > closefd=True>
> > ResourceWarning: unclosed file <_io.TextIOWrapper
> > name='C:\\research\\svn\\dev\\deps\\zlib-1.2.11\\zlib.h' mode='r' enco
> > ding='cp1252'>
> > Exception ignored in: <_io.FileIO
> > name='C:\\research\\svn\\dev\\deps\\sqlite-amalgamation-3.22.0.0\\sqlite3.h'
> > mode='rb'
> > closefd=True>
> > ResourceWarning: unclosed file <_io.TextIOWrapper
> > name='C:\\research\\svn\\dev\\deps\\sqlite-amalgamation-3.22.0.0\\sqli
> > te3.h' mode='r' encoding='cp1252'>
> > Exception ignored in: <_io.FileIO
> > name='subversion\\libsvn_subr\\lz4\\lz4internal.h' mode='rb'
> > closefd=True>
> > ResourceWarning: unclosed file <_io.TextIOWrapper
> > name='subversion\\libsvn_subr\\lz4\\lz4internal.h' mode='r' encoding='
> > cp1252'>
> > Exception ignored in: <_io.FileIO
> > name='subversion\\libsvn_subr\\utf8proc\\utf8proc_internal.h'
> > mode='rb' closefd=True>
> > ResourceWarning: unclosed file <_io.TextIOWrapper
> > name='subversion\\libsvn_subr\\utf8proc\\utf8proc_internal.h' mode='r'
> > encoding='cp1252'>
> > Exception ignored in: <_io.FileIO
> > name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\include\\ap_release.h'
> > mode='rb' clo
> > sefd=True>
> > ResourceWarning: unclosed file <_io.TextIOWrapper
> > name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\include\\ap_release.
> > h' mode='r' encoding='cp1252'>
> > Exception ignored in: <_io.FileIO
> > name='C:\\research\\svn\\dev\\deps\\openssl-1.0.2n\\inc32/openssl/opensslv.h'
> > mode='rb
> > ' closefd=True>
> > ResourceWarning: unclosed file <_io.TextIOWrapper
> > name='C:\\research\\svn\\dev\\deps\\openssl-1.0.2n\\inc32/openssl/open
> > sslv.h' mode='r' encoding='cp1252'>
> > Exception ignored in: <_io.FileIO
> > name='C:\\research\\svn\\dev\\deps\\serf-1.3.9\\serf.h' mode='rb'
> > closefd=True>
> > ResourceWarning: unclosed file <_io.TextIOWrapper
> > name='C:\\research\\svn\\dev\\deps\\serf-1.3.9\\serf.h' mode='r' encod
> > ing='cp1252'>
> > Exception ignored in: <function Popen.__del__ at 0x01043420>
> > Traceback (most recent call last):
> > File "C:\Python37\lib\subprocess.py", line 858, in __del__
> > ResourceWarning, source=self)
> > ResourceWarning: subprocess 3716 is still running
> > Exception ignored in: <function Popen.__del__ at 0x01043420>
> > Traceback (most recent call last):
> > File "C:\Python37\lib\subprocess.py", line 858, in __del__
> > ResourceWarning, source=self)
> > ResourceWarning: subprocess 1356 is still running
> > 'ruby' is not recognized as an internal or external command,
> > operable program or batch file.
> > Testing Release configuration on local repository.
> > -- Running Swig Python tests --
> > ..........................................................Exception
> > ignored in: <function Popen.__del__ at 0x02D4B030>
> > Traceback (most recent call last):
> > File "C:\Python37\lib\subprocess.py", line 858, in __del__
> > ResourceWarning, source=self)
> > ResourceWarning: subprocess 2972 is still running
> > ...............................................................................................
> > ----------------------------------------------------------------------
> > Ran 153 tests in 93.884s
> >
> > OK
> > ]]]
> >
>
> These ResourceWarning except last one are raised in test driver
> win_test.py itself and in gen_win_dependencies module which is
> used by win_test.py, I guess.
Okay, I'll ignore those. I have unset the PYTHONWARNINGS env var again.
At this point, I think the only thing blocking reintegration of this
branch into trunk seems to be the C4115 warning / error. As I said, I
have no informed opinion either way (and Brane was okay with turning
it into a warning). So if Bert doesn't reply within a couple of days,
I'd say we are good to go.
BTW, I have one more question: is it expected that "swig-python
bindings compiled with python 3" cannot be used by python 2? Because
if I run the testsuite with Python 2.7.16, after I have compiled them
with Python 3.7.4, I get the following error:
[[[
C:\research\svn\dev\swig-py3>python --version
Python 2.7.16
C:\research\svn\dev\swig-py3>runtest.bat --swig=python
Testing Release configuration on local repository.
-- Running Swig Python tests --
Traceback (most recent call last):
File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\run_all.py",
line 23, in <module>
import mergeinfo, core, client, delta, checksum, pool, fs, ra, wc,
repository, \
File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\mergeinfo.py",
line 22, in <module>
from svn import core, repos, fs
File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests/..\svn\core.py",
line 26, in <module>
from libsvn.core import *
File "R:\test_release--swigpython\swig\pylib\libsvn\core.py", line 137
def apr_initialize() -> "apr_status_t":
^
SyntaxError: invalid syntax
[Test runner reported failure]
]]]
So distributors / packagers that want to support both py2 and py3
users, will have to create two sets of bindings.
--
Johan
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/09/29 23:36, Johan Corveleyn wrote:
> On Sun, Sep 29, 2019 at 2:00 PM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
>>
>> On 2019/09/28 17:17, Yasuhito FUTATSUKI wrote:
>>> On 2019/09/28 5:51, Johan Corveleyn wrote:
>>
>>>> [[[
>>>> Testing Release configuration on local repository.
>>>> -- Running Swig Python tests --
>>>> ..........................................................C:\Python37\lib\subprocess.py:858:
>>>> ResourceWarning: subprocess
>>>> 8548 is still running
>>>> ResourceWarning, source=self)
>>>> ResourceWarning: Enable tracemalloc to get the object allocation traceback
>>>> C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file
>>>> <_io.BufferedReader name=3>
>>>> testMethod()
>>>> ResourceWarning: Enable tracemalloc to get the object allocation traceback
>>>> .C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file
>>>> <_io.BufferedReader name='R:\\temp\\tmpryeb61g1'>
>>>> testMethod()
>>>> ResourceWarning: Enable tracemalloc to get the object allocation traceback
>>>> ..............................................................................................
>>>> ----------------------------------------------------------------------
>>>> Ran 153 tests in 66.182s
>>>>
>>>> OK
>>>> ]]]
>>>
>>> On FreeBSD those warning are also produced.
>>>
>>> [[[
>>> futatuki@retina-alpha[15] make check-swig-py
>>> if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py
>>> ........................................................../usr/local/lib/python3.7/subprocess.py:858: ResourceWarning: subprocess 81136 is still running
>>> ResourceWarning, source=self)
>>> ResourceWarning: Enable tracemalloc to get the object allocation traceback
>>> /usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed file <_io.BufferedReader name=4>
>>> testMethod()
>>> ResourceWarning: Enable tracemalloc to get the object allocation traceback
>>> ./usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/tmporh5038d'>
>>> testMethod()
>>> ResourceWarning: Enable tracemalloc to get the object allocation traceback
>>> ..............................................................................................
>>> ----------------------------------------------------------------------
>>> Ran 153 tests in 10.773s
>>>
>>> OK
>>> ]]]
>>>
>>> I think these warnings comes from difference of warning between Python 2
>>> and Python 3, and not affect quality of swig Python binding itself but
>>> affect quality of its test code.
>>
>> This was partly wrong. The warning caused by subprocess comes from
>> class svn.fs.FileDiff.get_pipe(), not in test code but in support class.
>>
>>>
>>> ResourceWarning itself appeared in Python 3.2
>>>
>>> https://docs.python.org/3/whatsnew/3.2.html
>>> (section "Other Language Changes")
>>>
>>> If we can allow this warning and want to hide it, we can use
>>> environment variable PYTHONWARNINGS='ignore::ResourceWarning::',
>>> as the document above says.
>>>
>>> [[[
>>> futatuki@retina-alpha[17] env PYTHONWARNINGS='ignore::ResourceWarning::' make check-swig-py
>>> if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py
>>> .........................................................................................................................................................
>>> ----------------------------------------------------------------------
>>> Ran 153 tests in 10.757s
>>>
>>> OK
>>> ]]]
>>>
>>> Of course, we can also resolve those warnings by fixing clean up code
>>> on tests.
>>
>> The patch attached is expected to fix these warnings. I already tested
>> on FreeBSD with Python 3.7 (treat ResourceWarning as an error).
>>
>> [[[
>> futatuki@retina-alpha[113] env PYTHONWARNINGS='error::ResourceWarning::' make check-swig-py
>> if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py
>> .........................................................................................................................................................
>> ----------------------------------------------------------------------
>> Ran 153 tests in 11.040s
>>
>> OK
>> ]]]
>
> On my Windows setup, there is still 1 ResourceWarning (before your
> patch there were 3, now only 1 remains).
>
> [[[
> C:\research\svn\dev\swig-py3>runtest.bat --swig=python
> 'ruby' is not recognized as an internal or external command,
> operable program or batch file.
> Testing Release configuration on local repository.
> -- Running Swig Python tests --
> ..........................................................C:\Python37\lib\subprocess.py:858:
> ResourceWarning: subprocess
> 1408 is still running
> ResourceWarning, source=self)
> ResourceWarning: Enable tracemalloc to get the object allocation traceback
> ...............................................................................................
> ----------------------------------------------------------------------
> Ran 153 tests in 63.597s
>
> OK
> ]]]
Hmm.. It seems the diffrence comes from difference of implementation
of class subprocess.Popen.
I tweaked again clean up code for class svn.fs.FileDiff. Only if
Python >= 3.3, it wait for maximum 10 sec to terminate subprocess.
'timeout' parameter for subprocess.Popen.wait() is added on Python 3.3.
and not back ported to Python 2, so if we want to use wait() on
earlier Python, it may wait forever.... (It can be possible to implement
timeout by using poll() and sleep loop, but is it needed here?)
> When I treat ResourceWarning as an error, I get a lot more output,
> especially before the test runs start. I suppose those are unrelated
> to that one RW above.
Yes, ... and ResourceWarnings are usually hided by default warning filter.
https://docs.python.org/3/library/warnings.html#default-warning-filter
Only while executing unittest.main(), warning filter is overrided by
its optional parameter `warning'. Its default is specified filter on
Python start up by environment variable PYTHONWARNINGS or -W option
if it is specified, otherwise "default" action for all warning
categories (i.e. show all warnings).
> [[[
> C:\research\svn\dev\swig-py3>set PYTHONWARNINGS=error::ResourceWarning::
>
> C:\research\svn\dev\swig-py3>runtest.bat --swig=python
> Exception ignored in: <_io.FileIO name='build.conf' mode='rb' closefd=True>
> ResourceWarning: unclosed file <_io.TextIOWrapper name='build.conf'
> mode='r' encoding='cp1252'>
> Exception ignored in: <_io.FileIO
> name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr\\include\\apr_version.h'
> mode='rb' closefd=True>
> ResourceWarning: unclosed file <_io.TextIOWrapper
> name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr\\include
> \\apr_version.h' mode='r' encoding='cp1252'>
> Exception ignored in: <_io.FileIO
> name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\include\\apu_versi
> on.h' mode='rb' closefd=True>
> ResourceWarning: unclosed file <_io.TextIOWrapper
> name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\in
> clude\\apu_version.h' mode='r' encoding='cp1252'>
> Exception ignored in: <_io.FileIO
> name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\xml/expat/lib\\exp
> at.h' mode='rb' closefd=True>
> ResourceWarning: unclosed file <_io.TextIOWrapper
> name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\xm
> l/expat/lib\\expat.h' mode='r' encoding='cp1252'>
> Exception ignored in: <_io.FileIO
> name='C:\\research\\svn\\dev\\deps\\zlib-1.2.11\\zlib.h' mode='rb'
> closefd=True>
> ResourceWarning: unclosed file <_io.TextIOWrapper
> name='C:\\research\\svn\\dev\\deps\\zlib-1.2.11\\zlib.h' mode='r' enco
> ding='cp1252'>
> Exception ignored in: <_io.FileIO
> name='C:\\research\\svn\\dev\\deps\\sqlite-amalgamation-3.22.0.0\\sqlite3.h'
> mode='rb'
> closefd=True>
> ResourceWarning: unclosed file <_io.TextIOWrapper
> name='C:\\research\\svn\\dev\\deps\\sqlite-amalgamation-3.22.0.0\\sqli
> te3.h' mode='r' encoding='cp1252'>
> Exception ignored in: <_io.FileIO
> name='subversion\\libsvn_subr\\lz4\\lz4internal.h' mode='rb'
> closefd=True>
> ResourceWarning: unclosed file <_io.TextIOWrapper
> name='subversion\\libsvn_subr\\lz4\\lz4internal.h' mode='r' encoding='
> cp1252'>
> Exception ignored in: <_io.FileIO
> name='subversion\\libsvn_subr\\utf8proc\\utf8proc_internal.h'
> mode='rb' closefd=True>
> ResourceWarning: unclosed file <_io.TextIOWrapper
> name='subversion\\libsvn_subr\\utf8proc\\utf8proc_internal.h' mode='r'
> encoding='cp1252'>
> Exception ignored in: <_io.FileIO
> name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\include\\ap_release.h'
> mode='rb' clo
> sefd=True>
> ResourceWarning: unclosed file <_io.TextIOWrapper
> name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\include\\ap_release.
> h' mode='r' encoding='cp1252'>
> Exception ignored in: <_io.FileIO
> name='C:\\research\\svn\\dev\\deps\\openssl-1.0.2n\\inc32/openssl/opensslv.h'
> mode='rb
> ' closefd=True>
> ResourceWarning: unclosed file <_io.TextIOWrapper
> name='C:\\research\\svn\\dev\\deps\\openssl-1.0.2n\\inc32/openssl/open
> sslv.h' mode='r' encoding='cp1252'>
> Exception ignored in: <_io.FileIO
> name='C:\\research\\svn\\dev\\deps\\serf-1.3.9\\serf.h' mode='rb'
> closefd=True>
> ResourceWarning: unclosed file <_io.TextIOWrapper
> name='C:\\research\\svn\\dev\\deps\\serf-1.3.9\\serf.h' mode='r' encod
> ing='cp1252'>
> Exception ignored in: <function Popen.__del__ at 0x01043420>
> Traceback (most recent call last):
> File "C:\Python37\lib\subprocess.py", line 858, in __del__
> ResourceWarning, source=self)
> ResourceWarning: subprocess 3716 is still running
> Exception ignored in: <function Popen.__del__ at 0x01043420>
> Traceback (most recent call last):
> File "C:\Python37\lib\subprocess.py", line 858, in __del__
> ResourceWarning, source=self)
> ResourceWarning: subprocess 1356 is still running
> 'ruby' is not recognized as an internal or external command,
> operable program or batch file.
> Testing Release configuration on local repository.
> -- Running Swig Python tests --
> ..........................................................Exception
> ignored in: <function Popen.__del__ at 0x02D4B030>
> Traceback (most recent call last):
> File "C:\Python37\lib\subprocess.py", line 858, in __del__
> ResourceWarning, source=self)
> ResourceWarning: subprocess 2972 is still running
> ...............................................................................................
> ----------------------------------------------------------------------
> Ran 153 tests in 93.884s
>
> OK
> ]]]
>
These ResourceWarning except last one are raised in test driver
win_test.py itself and in gen_win_dependencies module which is
used by win_test.py, I guess.
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Johan Corveleyn <jc...@gmail.com>.
On Sun, Sep 29, 2019 at 2:00 PM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
>
> On 2019/09/28 17:17, Yasuhito FUTATSUKI wrote:
> > On 2019/09/28 5:51, Johan Corveleyn wrote:
>
> >> [[[
> >> Testing Release configuration on local repository.
> >> -- Running Swig Python tests --
> >> ..........................................................C:\Python37\lib\subprocess.py:858:
> >> ResourceWarning: subprocess
> >> 8548 is still running
> >> ResourceWarning, source=self)
> >> ResourceWarning: Enable tracemalloc to get the object allocation traceback
> >> C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file
> >> <_io.BufferedReader name=3>
> >> testMethod()
> >> ResourceWarning: Enable tracemalloc to get the object allocation traceback
> >> .C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file
> >> <_io.BufferedReader name='R:\\temp\\tmpryeb61g1'>
> >> testMethod()
> >> ResourceWarning: Enable tracemalloc to get the object allocation traceback
> >> ..............................................................................................
> >> ----------------------------------------------------------------------
> >> Ran 153 tests in 66.182s
> >>
> >> OK
> >> ]]]
> >
> > On FreeBSD those warning are also produced.
> >
> > [[[
> > futatuki@retina-alpha[15] make check-swig-py
> > if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py
> > ........................................................../usr/local/lib/python3.7/subprocess.py:858: ResourceWarning: subprocess 81136 is still running
> > ResourceWarning, source=self)
> > ResourceWarning: Enable tracemalloc to get the object allocation traceback
> > /usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed file <_io.BufferedReader name=4>
> > testMethod()
> > ResourceWarning: Enable tracemalloc to get the object allocation traceback
> > ./usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/tmporh5038d'>
> > testMethod()
> > ResourceWarning: Enable tracemalloc to get the object allocation traceback
> > ..............................................................................................
> > ----------------------------------------------------------------------
> > Ran 153 tests in 10.773s
> >
> > OK
> > ]]]
> >
> > I think these warnings comes from difference of warning between Python 2
> > and Python 3, and not affect quality of swig Python binding itself but
> > affect quality of its test code.
>
> This was partly wrong. The warning caused by subprocess comes from
> class svn.fs.FileDiff.get_pipe(), not in test code but in support class.
>
> >
> > ResourceWarning itself appeared in Python 3.2
> >
> > https://docs.python.org/3/whatsnew/3.2.html
> > (section "Other Language Changes")
> >
> > If we can allow this warning and want to hide it, we can use
> > environment variable PYTHONWARNINGS='ignore::ResourceWarning::',
> > as the document above says.
> >
> > [[[
> > futatuki@retina-alpha[17] env PYTHONWARNINGS='ignore::ResourceWarning::' make check-swig-py
> > if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py
> > .........................................................................................................................................................
> > ----------------------------------------------------------------------
> > Ran 153 tests in 10.757s
> >
> > OK
> > ]]]
> >
> > Of course, we can also resolve those warnings by fixing clean up code
> > on tests.
>
> The patch attached is expected to fix these warnings. I already tested
> on FreeBSD with Python 3.7 (treat ResourceWarning as an error).
>
> [[[
> futatuki@retina-alpha[113] env PYTHONWARNINGS='error::ResourceWarning::' make check-swig-py
> if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py
> .........................................................................................................................................................
> ----------------------------------------------------------------------
> Ran 153 tests in 11.040s
>
> OK
> ]]]
On my Windows setup, there is still 1 ResourceWarning (before your
patch there were 3, now only 1 remains).
[[[
C:\research\svn\dev\swig-py3>runtest.bat --swig=python
'ruby' is not recognized as an internal or external command,
operable program or batch file.
Testing Release configuration on local repository.
-- Running Swig Python tests --
..........................................................C:\Python37\lib\subprocess.py:858:
ResourceWarning: subprocess
1408 is still running
ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
...............................................................................................
----------------------------------------------------------------------
Ran 153 tests in 63.597s
OK
]]]
When I treat ResourceWarning as an error, I get a lot more output,
especially before the test runs start. I suppose those are unrelated
to that one RW above.
[[[
C:\research\svn\dev\swig-py3>set PYTHONWARNINGS=error::ResourceWarning::
C:\research\svn\dev\swig-py3>runtest.bat --swig=python
Exception ignored in: <_io.FileIO name='build.conf' mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name='build.conf'
mode='r' encoding='cp1252'>
Exception ignored in: <_io.FileIO
name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr\\include\\apr_version.h'
mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr\\include
\\apr_version.h' mode='r' encoding='cp1252'>
Exception ignored in: <_io.FileIO
name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\include\\apu_versi
on.h' mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\in
clude\\apu_version.h' mode='r' encoding='cp1252'>
Exception ignored in: <_io.FileIO
name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\xml/expat/lib\\exp
at.h' mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\srclib\\apr-util\\xm
l/expat/lib\\expat.h' mode='r' encoding='cp1252'>
Exception ignored in: <_io.FileIO
name='C:\\research\\svn\\dev\\deps\\zlib-1.2.11\\zlib.h' mode='rb'
closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='C:\\research\\svn\\dev\\deps\\zlib-1.2.11\\zlib.h' mode='r' enco
ding='cp1252'>
Exception ignored in: <_io.FileIO
name='C:\\research\\svn\\dev\\deps\\sqlite-amalgamation-3.22.0.0\\sqlite3.h'
mode='rb'
closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='C:\\research\\svn\\dev\\deps\\sqlite-amalgamation-3.22.0.0\\sqli
te3.h' mode='r' encoding='cp1252'>
Exception ignored in: <_io.FileIO
name='subversion\\libsvn_subr\\lz4\\lz4internal.h' mode='rb'
closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='subversion\\libsvn_subr\\lz4\\lz4internal.h' mode='r' encoding='
cp1252'>
Exception ignored in: <_io.FileIO
name='subversion\\libsvn_subr\\utf8proc\\utf8proc_internal.h'
mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='subversion\\libsvn_subr\\utf8proc\\utf8proc_internal.h' mode='r'
encoding='cp1252'>
Exception ignored in: <_io.FileIO
name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\include\\ap_release.h'
mode='rb' clo
sefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='C:\\research\\svn\\dev\\deps\\httpd-2.4.29\\include\\ap_release.
h' mode='r' encoding='cp1252'>
Exception ignored in: <_io.FileIO
name='C:\\research\\svn\\dev\\deps\\openssl-1.0.2n\\inc32/openssl/opensslv.h'
mode='rb
' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='C:\\research\\svn\\dev\\deps\\openssl-1.0.2n\\inc32/openssl/open
sslv.h' mode='r' encoding='cp1252'>
Exception ignored in: <_io.FileIO
name='C:\\research\\svn\\dev\\deps\\serf-1.3.9\\serf.h' mode='rb'
closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper
name='C:\\research\\svn\\dev\\deps\\serf-1.3.9\\serf.h' mode='r' encod
ing='cp1252'>
Exception ignored in: <function Popen.__del__ at 0x01043420>
Traceback (most recent call last):
File "C:\Python37\lib\subprocess.py", line 858, in __del__
ResourceWarning, source=self)
ResourceWarning: subprocess 3716 is still running
Exception ignored in: <function Popen.__del__ at 0x01043420>
Traceback (most recent call last):
File "C:\Python37\lib\subprocess.py", line 858, in __del__
ResourceWarning, source=self)
ResourceWarning: subprocess 1356 is still running
'ruby' is not recognized as an internal or external command,
operable program or batch file.
Testing Release configuration on local repository.
-- Running Swig Python tests --
..........................................................Exception
ignored in: <function Popen.__del__ at 0x02D4B030>
Traceback (most recent call last):
File "C:\Python37\lib\subprocess.py", line 858, in __del__
ResourceWarning, source=self)
ResourceWarning: subprocess 2972 is still running
...............................................................................................
----------------------------------------------------------------------
Ran 153 tests in 93.884s
OK
]]]
--
Johan
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/09/28 17:17, Yasuhito FUTATSUKI wrote:
> On 2019/09/28 5:51, Johan Corveleyn wrote:
>> [[[
>> Testing Release configuration on local repository.
>> -- Running Swig Python tests --
>> ..........................................................C:\Python37\lib\subprocess.py:858:
>> ResourceWarning: subprocess
>> 8548 is still running
>> ResourceWarning, source=self)
>> ResourceWarning: Enable tracemalloc to get the object allocation traceback
>> C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file
>> <_io.BufferedReader name=3>
>> testMethod()
>> ResourceWarning: Enable tracemalloc to get the object allocation traceback
>> .C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file
>> <_io.BufferedReader name='R:\\temp\\tmpryeb61g1'>
>> testMethod()
>> ResourceWarning: Enable tracemalloc to get the object allocation traceback
>> ..............................................................................................
>> ----------------------------------------------------------------------
>> Ran 153 tests in 66.182s
>>
>> OK
>> ]]]
>
> On FreeBSD those warning are also produced.
>
> [[[
> futatuki@retina-alpha[15] make check-swig-py
> if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py
> ........................................................../usr/local/lib/python3.7/subprocess.py:858: ResourceWarning: subprocess 81136 is still running
> ResourceWarning, source=self)
> ResourceWarning: Enable tracemalloc to get the object allocation traceback
> /usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed file <_io.BufferedReader name=4>
> testMethod()
> ResourceWarning: Enable tracemalloc to get the object allocation traceback
> ./usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/tmporh5038d'>
> testMethod()
> ResourceWarning: Enable tracemalloc to get the object allocation traceback
> ..............................................................................................
> ----------------------------------------------------------------------
> Ran 153 tests in 10.773s
>
> OK
> ]]]
>
> I think these warnings comes from difference of warning between Python 2
> and Python 3, and not affect quality of swig Python binding itself but
> affect quality of its test code.
This was partly wrong. The warning caused by subprocess comes from
class svn.fs.FileDiff.get_pipe(), not in test code but in support class.
>
> ResourceWarning itself appeared in Python 3.2
>
> https://docs.python.org/3/whatsnew/3.2.html
> (section "Other Language Changes")
>
> If we can allow this warning and want to hide it, we can use
> environment variable PYTHONWARNINGS='ignore::ResourceWarning::',
> as the document above says.
>
> [[[
> futatuki@retina-alpha[17] env PYTHONWARNINGS='ignore::ResourceWarning::' make check-swig-py
> if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py
> .........................................................................................................................................................
> ----------------------------------------------------------------------
> Ran 153 tests in 10.757s
>
> OK
> ]]]
>
> Of course, we can also resolve those warnings by fixing clean up code
> on tests.
The patch attached is expected to fix these warnings. I already tested
on FreeBSD with Python 3.7 (treat ResourceWarning as an error).
[[[
futatuki@retina-alpha[113] env PYTHONWARNINGS='error::ResourceWarning::' make check-swig-py
if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py
.........................................................................................................................................................
----------------------------------------------------------------------
Ran 153 tests in 11.040s
OK
]]]
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/09/28 5:51, Johan Corveleyn wrote:
> Great! That fixes this particular problem, and the test suite now runs
> successfully ... almost. There seems to be one more problem:
>
> [[[
> Testing Release configuration on local repository.
> -- Running Swig Python tests --
> ..........................................................C:\Python37\lib\subprocess.py:858:
> ResourceWarning: subprocess
> 8548 is still running
> ResourceWarning, source=self)
> ResourceWarning: Enable tracemalloc to get the object allocation traceback
> C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file
> <_io.BufferedReader name=3>
> testMethod()
> ResourceWarning: Enable tracemalloc to get the object allocation traceback
> .C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file
> <_io.BufferedReader name='R:\\temp\\tmpryeb61g1'>
> testMethod()
> ResourceWarning: Enable tracemalloc to get the object allocation traceback
> ..............................................................................................
> ----------------------------------------------------------------------
> Ran 153 tests in 66.182s
>
> OK
> ]]]
On FreeBSD those warning are also produced.
[[[
futatuki@retina-alpha[15] make check-swig-py
if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py
........................................................../usr/local/lib/python3.7/subprocess.py:858: ResourceWarning: subprocess 81136 is still running
ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed file <_io.BufferedReader name=4>
testMethod()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
./usr/local/lib/python3.7/unittest/case.py:615: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/tmporh5038d'>
testMethod()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
..............................................................................................
----------------------------------------------------------------------
Ran 153 tests in 10.773s
OK
]]]
I think these warnings comes from difference of warning between Python 2
and Python 3, and not affect quality of swig Python binding itself but
affect quality of its test code.
ResourceWarning itself appeared in Python 3.2
https://docs.python.org/3/whatsnew/3.2.html
(section "Other Language Changes")
If we can allow this warning and want to hide it, we can use
environment variable PYTHONWARNINGS='ignore::ResourceWarning::',
as the document above says.
[[[
futatuki@retina-alpha[17] env PYTHONWARNINGS='ignore::ResourceWarning::' make check-swig-py
if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/libsvn_swig_py /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/../../../libsvn_*; do if [ -n "$DYLD_LIBRARY_PATH" ]; then LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; cd /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python; /usr/local/bin/python3.7 /home/futatuki/work/subversion/vwc/branches/swig-py3/subversion/bindings/swig/python/tests/run_all.py
.........................................................................................................................................................
----------------------------------------------------------------------
Ran 153 tests in 10.757s
OK
]]]
Of course, we can also resolve those warnings by fixing clean up code
on tests.
>
>>> (The warning about 'ruby' is not a big deal I suppose, but it's also
>>> something I saw when running gen-make.py with python 3.7 -- not when
>>> I'm running it with python 2.7)
>>
>> I think that message comes from build/generator/gen_win_dependencies.py,
>> GenDependenciesBase._find_ruby() line 967 (message from shell program
>> on Windows).
>>
>> https://svn.apache.org/viewvc/subversion/branches/swig-py3/build/generator/gen_win_dependencies.py?revision=1862754&view=markup#l967
>>
>> This may be also caused with Python 2, if 'ruby' is not in command
>> search path.
>
> Thanks. I'll try to take a closer look, but it's not a priority. It's
> still a bit strange to me that Python 2 doesn't show this warning
> while Python 3 does (environment is the same, PATH is the same, except
> for Python itself).
I suspect this may come from difference of implementation of os.popen()
and unittest library between Python 2 and Python 3, especially handling
of stderr of child process.
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Johan Corveleyn <jc...@gmail.com>.
On Fri, Sep 27, 2019 at 8:14 PM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
> Thank you for testing.
And thank you for hanging in there for the py3 work :-).
> It seems that urllib.request.pathname2url() on Windows doesn't accept
> bytes on Python 3, while on Linux/Unix it accepts both of bytes and str.
>
> The patch attached may fix it.
Great! That fixes this particular problem, and the test suite now runs
successfully ... almost. There seems to be one more problem:
[[[
Testing Release configuration on local repository.
-- Running Swig Python tests --
..........................................................C:\Python37\lib\subprocess.py:858:
ResourceWarning: subprocess
8548 is still running
ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file
<_io.BufferedReader name=3>
testMethod()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
.C:\Python37\lib\unittest\case.py:628: ResourceWarning: unclosed file
<_io.BufferedReader name='R:\\temp\\tmpryeb61g1'>
testMethod()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
..............................................................................................
----------------------------------------------------------------------
Ran 153 tests in 66.182s
OK
]]]
> > (The warning about 'ruby' is not a big deal I suppose, but it's also
> > something I saw when running gen-make.py with python 3.7 -- not when
> > I'm running it with python 2.7)
>
> I think that message comes from build/generator/gen_win_dependencies.py,
> GenDependenciesBase._find_ruby() line 967 (message from shell program
> on Windows).
>
> https://svn.apache.org/viewvc/subversion/branches/swig-py3/build/generator/gen_win_dependencies.py?revision=1862754&view=markup#l967
>
> This may be also caused with Python 2, if 'ruby' is not in command
> search path.
Thanks. I'll try to take a closer look, but it's not a priority. It's
still a bit strange to me that Python 2 doesn't show this warning
while Python 3 does (environment is the same, PATH is the same, except
for Python itself).
--
Johan
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
Thank you for testing.
On 2019/09/24 5:48, Johan Corveleyn wrote:
> So, on to the next problem. When testing with Python 3.7.4 on Windows,
> I get the following error:
>
> [[[
> C:\research\svn\dev\swig-py3>python win-tests.py -c --log-level=DEBUG
> --release --swig=python R:\test_py
> 'ruby' is not recognized as an internal or external command,
> operable program or batch file.
> Testing Release configuration on local repository.
> -- Running Swig Python tests --
> Traceback (most recent call last):
> File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\run_all.py",
> line 23, in <module>
> import mergeinfo, core, client, delta, checksum, pool, fs, ra, wc,
> repository, \
> File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\trac\versioncontrol\tests\__init__.py",
> line 23, in <module>
> from trac.versioncontrol.tests import svn_fs
> File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\trac\versioncontrol\tests\svn_fs.py",
> line 73, in <module>
> REPOS_URL = pathname2url(temp_path).encode('UTF-8')
> File "C:\Python37\lib\nturl2path.py", line 53, in pathname2url
> if not ':' in p:
> TypeError: a bytes-like object is required, not 'str'
> [Test runner reported failure]
> ]]]
It seems that urllib.request.pathname2url() on Windows doesn't accept
bytes on Python 3, while on Linux/Unix it accepts both of bytes and str.
The patch attached may fix it.
> (The warning about 'ruby' is not a big deal I suppose, but it's also
> something I saw when running gen-make.py with python 3.7 -- not when
> I'm running it with python 2.7)
I think that message comes from build/generator/gen_win_dependencies.py,
GenDependenciesBase._find_ruby() line 967 (message from shell program
on Windows).
https://svn.apache.org/viewvc/subversion/branches/swig-py3/build/generator/gen_win_dependencies.py?revision=1862754&view=markup#l967
This may be also caused with Python 2, if 'ruby' is not in command
search path.
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Johan Corveleyn <jc...@gmail.com>.
On Mon, Sep 23, 2019 at 1:53 AM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
>
> On 2019/09/23 6:16, Johan Corveleyn wrote:
> > On Sat, Sep 21, 2019 at 10:28 AM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
> >>
> >> On 2019/09/20 0:44, Daniel Shahaf wrote:
> >>> Yasuhito FUTATSUKI wrote on Thu, 19 Sep 2019 06:41 +00:00:
> >>
> >>>> The FAIL on test_merge_peg3 is a newline style issue on reading file
> >>>> in raw mode, which is introduced by my patch.
> >>>> (https://svn.apache.org/viewvc/subversion/branches/swig-py3/subversion/bindings/swig/python/tests/client.py?view=annotate#l438)
> >>>>
> >>>> Rest of those errors and failures are in the tests that were added only
> >>>> to swig-py, and it seems most of the causes of them are confusion of
> >>>> paths in platform specific format and Subversion's canoical format,
> >>>> as the comment in SubversionClientTestCase.test_update4() says.
> >>>>
> >>>> (I'll try to fix those issues on check-swig-py later, hopefully
> >>>> next weekend.)
> >>>
> >>> Thanks! No rush.
> >>
> >> Now I re-sync swig-py3 branch with trunk and make a patch addressing
> >> issues on testing on other than Unix/Linux platform.
> >>
> >> I've tested on FreeBSD 11.2, with Python 2.7 and Python 3.6.
> >>
> >> Could anyone please test it on Windows ?
> >
> > Tested on Windows 7, and yes, this fixes the previous test failures
> > with Python 2.7.16. Thanks!
> >
> > Test output:
> > [[[
> > Testing Release configuration on local repository.
> > -- Running Swig Python tests --
> > ..............s.........................................................................................................
> > .................................
> > ----------------------------------------------------------------------
> > Ran 153 tests in 63.214s
> >
> > OK (skipped=1)
> > ]]]
> >
>
> I've commited it, as r1866362. Thanks!
>
> >
> > Building with Python 3.7.4 still fails with the same error though (no
> > problem, I know your patch wasn't addressing that, just mentioning it
> > here for completeness). Consequently I haven't been able to run the
> > swig-python tests with python 3.7 yet.
> >
> > [[[
> > c:\python37\include\pytime.h(123): error C4115: 'timeval': named type
> > definition in parentheses
> > [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj
> > ]]]
>
> Here is a patch not to treat C4115 as error, globally. If this makes
> it possible to build with Python 3.7 on Windows, then next step can be
> to limit to apply this relaxation of compile option to files which
> contains "#include <Python.h>" (most of them are generated by swig...).
Yes, that makes the build succeed, thanks. I can't comment on whether
or not it's good to change this error into a warning overall, or just
for a limited set of files. I'll leave that discussion to others :-).
So, on to the next problem. When testing with Python 3.7.4 on Windows,
I get the following error:
[[[
C:\research\svn\dev\swig-py3>python win-tests.py -c --log-level=DEBUG
--release --swig=python R:\test_py
'ruby' is not recognized as an internal or external command,
operable program or batch file.
Testing Release configuration on local repository.
-- Running Swig Python tests --
Traceback (most recent call last):
File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\run_all.py",
line 23, in <module>
import mergeinfo, core, client, delta, checksum, pool, fs, ra, wc,
repository, \
File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\trac\versioncontrol\tests\__init__.py",
line 23, in <module>
from trac.versioncontrol.tests import svn_fs
File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\trac\versioncontrol\tests\svn_fs.py",
line 73, in <module>
REPOS_URL = pathname2url(temp_path).encode('UTF-8')
File "C:\Python37\lib\nturl2path.py", line 53, in pathname2url
if not ':' in p:
TypeError: a bytes-like object is required, not 'str'
[Test runner reported failure]
]]]
(The warning about 'ruby' is not a big deal I suppose, but it's also
something I saw when running gen-make.py with python 3.7 -- not when
I'm running it with python 2.7)
--
Johan
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/09/23 6:16, Johan Corveleyn wrote:
> On Sat, Sep 21, 2019 at 10:28 AM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
>>
>> On 2019/09/20 0:44, Daniel Shahaf wrote:
>>> Yasuhito FUTATSUKI wrote on Thu, 19 Sep 2019 06:41 +00:00:
>>
>>>> The FAIL on test_merge_peg3 is a newline style issue on reading file
>>>> in raw mode, which is introduced by my patch.
>>>> (https://svn.apache.org/viewvc/subversion/branches/swig-py3/subversion/bindings/swig/python/tests/client.py?view=annotate#l438)
>>>>
>>>> Rest of those errors and failures are in the tests that were added only
>>>> to swig-py, and it seems most of the causes of them are confusion of
>>>> paths in platform specific format and Subversion's canoical format,
>>>> as the comment in SubversionClientTestCase.test_update4() says.
>>>>
>>>> (I'll try to fix those issues on check-swig-py later, hopefully
>>>> next weekend.)
>>>
>>> Thanks! No rush.
>>
>> Now I re-sync swig-py3 branch with trunk and make a patch addressing
>> issues on testing on other than Unix/Linux platform.
>>
>> I've tested on FreeBSD 11.2, with Python 2.7 and Python 3.6.
>>
>> Could anyone please test it on Windows ?
>
> Tested on Windows 7, and yes, this fixes the previous test failures
> with Python 2.7.16. Thanks!
>
> Test output:
> [[[
> Testing Release configuration on local repository.
> -- Running Swig Python tests --
> ..............s.........................................................................................................
> .................................
> ----------------------------------------------------------------------
> Ran 153 tests in 63.214s
>
> OK (skipped=1)
> ]]]
>
I've commited it, as r1866362. Thanks!
>
> Building with Python 3.7.4 still fails with the same error though (no
> problem, I know your patch wasn't addressing that, just mentioning it
> here for completeness). Consequently I haven't been able to run the
> swig-python tests with python 3.7 yet.
>
> [[[
> c:\python37\include\pytime.h(123): error C4115: 'timeval': named type
> definition in parentheses
> [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj
> ]]]
Here is a patch not to treat C4115 as error, globally. If this makes
it possible to build with Python 3.7 on Windows, then next step can be
to limit to apply this relaxation of compile option to files which
contains "#include <Python.h>" (most of them are generated by swig...).
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Johan Corveleyn <jc...@gmail.com>.
On Sat, Sep 21, 2019 at 10:28 AM Yasuhito FUTATSUKI <fu...@poem.co.jp> wrote:
>
> On 2019/09/20 0:44, Daniel Shahaf wrote:
> > Yasuhito FUTATSUKI wrote on Thu, 19 Sep 2019 06:41 +00:00:
>
> >> The FAIL on test_merge_peg3 is a newline style issue on reading file
> >> in raw mode, which is introduced by my patch.
> >> (https://svn.apache.org/viewvc/subversion/branches/swig-py3/subversion/bindings/swig/python/tests/client.py?view=annotate#l438)
> >>
> >> Rest of those errors and failures are in the tests that were added only
> >> to swig-py, and it seems most of the causes of them are confusion of
> >> paths in platform specific format and Subversion's canoical format,
> >> as the comment in SubversionClientTestCase.test_update4() says.
> >>
> >> (I'll try to fix those issues on check-swig-py later, hopefully
> >> next weekend.)
> >
> > Thanks! No rush.
>
> Now I re-sync swig-py3 branch with trunk and make a patch addressing
> issues on testing on other than Unix/Linux platform.
>
> I've tested on FreeBSD 11.2, with Python 2.7 and Python 3.6.
>
> Could anyone please test it on Windows ?
Tested on Windows 7, and yes, this fixes the previous test failures
with Python 2.7.16. Thanks!
Test output:
[[[
Testing Release configuration on local repository.
-- Running Swig Python tests --
..............s.........................................................................................................
.................................
----------------------------------------------------------------------
Ran 153 tests in 63.214s
OK (skipped=1)
]]]
Building with Python 3.7.4 still fails with the same error though (no
problem, I know your patch wasn't addressing that, just mentioning it
here for completeness). Consequently I haven't been able to run the
swig-python tests with python 3.7 yet.
[[[
c:\python37\include\pytime.h(123): error C4115: 'timeval': named type
definition in parentheses
[C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj
]]]
--
Johan
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/09/20 0:44, Daniel Shahaf wrote:
> Yasuhito FUTATSUKI wrote on Thu, 19 Sep 2019 06:41 +00:00:
>> The FAIL on test_merge_peg3 is a newline style issue on reading file
>> in raw mode, which is introduced by my patch.
>> (https://svn.apache.org/viewvc/subversion/branches/swig-py3/subversion/bindings/swig/python/tests/client.py?view=annotate#l438)
>>
>> Rest of those errors and failures are in the tests that were added only
>> to swig-py, and it seems most of the causes of them are confusion of
>> paths in platform specific format and Subversion's canoical format,
>> as the comment in SubversionClientTestCase.test_update4() says.
>>
>> (I'll try to fix those issues on check-swig-py later, hopefully
>> next weekend.)
>
> Thanks! No rush.
Now I re-sync swig-py3 branch with trunk and make a patch addressing
issues on testing on other than Unix/Linux platform.
I've tested on FreeBSD 11.2, with Python 2.7 and Python 3.6.
Could anyone please test it on Windows ?
Cheers,
--
Yasuhito FUTATSUKI <fu...@poem.co.jp>
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Yasuhito FUTATSUKI wrote on Thu, 19 Sep 2019 06:41 +00:00:
> On 2019/09/19 8:21, Johan Corveleyn wrote:
> > On Wed, Sep 18, 2019 at 12:08 PM Branko Čibej <br...@apache.org> wrote:
> >> You do *not* need a definition of 'struct foo' in order to use a 'struct
> >> foo*' pointer. So, no, that's not the problem.
>
> Actually C4115 is only a warning.
>
> https://docs.microsoft.com/cpp/error-messages/compiler-warnings/compiler-warning-levels-1-and-4-c4115?view=vs-2019
>
> However, it seems we are treating it as an error by specifyinging build
> option in build/generator/templates/vcnet_vcxproj.ezt .
>
> (Is it related to https://bugs.python.org/issue25878, especially
> msg256496 ?)
>
That certainly sounds similar.
For comparison, on a toy example, I get a warning if I use a pointer to
an undeclared struct type:
[[[
% cat foo.c
void f(struct foo *);
% cc -c -Wall foo.c
foo.c:1:15: warning: declaration of 'struct foo' will not be visible outside of
this function [-Wvisibility]
void f(struct foo *);
^
1 warning generated.
%
]]]
That doesn't happen if the struct type has been declared:
[[[
% cat bar.c
struct bar;
void g(struct bar *);
% cc -c -Wall bar.c
%
]]]
I think that -Wvisibility warning is basically what that C4115 is about?
pytime.h doesn't declare 'struct timeval', neither directly nor via
an #include.
I haven't checked whether my build warns about -Wvisibility there.
> The FAIL on test_merge_peg3 is a newline style issue on reading file
> in raw mode, which is introduced by my patch.
> (https://svn.apache.org/viewvc/subversion/branches/swig-py3/subversion/bindings/swig/python/tests/client.py?view=annotate#l438)
>
> Rest of those errors and failures are in the tests that were added only
> to swig-py, and it seems most of the causes of them are confusion of
> paths in platform specific format and Subversion's canoical format,
> as the comment in SubversionClientTestCase.test_update4() says.
>
> (I'll try to fix those issues on check-swig-py later, hopefully
> next weekend.)
Thanks! No rush.
Cheers,
Daniel
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Yasuhito FUTATSUKI <fu...@poem.co.jp>.
On 2019/09/19 8:21, Johan Corveleyn wrote:
> On Wed, Sep 18, 2019 at 12:08 PM Branko Čibej <br...@apache.org> wrote:
>>
>> On 18.09.2019 09:20, Johan Corveleyn wrote:
>>> On Wed, Sep 18, 2019 at 5:36 AM Branko Čibej <br...@apache.org> wrote:
>>>> On 18.09.2019 02:48, Johan Corveleyn wrote:
<snip>
>>>>> [[[
>>>>> c:\python37\include\pytime.h(123): error C4115: 'timeval': named type
>>>>> definition in parentheses
>>>>> [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj]
>>>>> ]]]
>>>> That makes no sense, my copy of pytime.h on Windows for Python 3.7.4 has
>>>> this on line 123:
>>>>
>>>> PyAPI_FUNC(int) _PyTime_FromTimeval(_PyTime_t *tp, struct timeval *tv);
>>>>
>>>>
>>>> Looks like perfectly valid C to me. So we'll need a bit more context to
>>>> see where the actual error is coming from.
>>> Not at my pc right now (I'll check again tonight), but from memory: I
>>> think my copy of pytime.h looks the same. Line 123 seems to be the
>>> first usage of 'timeval' though ... Is it possible that some include
>>> is missing, so there is no declaration of the timeval struct type?
>>
>>
>> You do *not* need a definition of 'struct foo' in order to use a 'struct
>> foo*' pointer. So, no, that's not the problem.
Actually C4115 is only a warning.
https://docs.microsoft.com/cpp/error-messages/compiler-warnings/compiler-warning-levels-1-and-4-c4115?view=vs-2019
However, it seems we are treating it as an error by specifyinging build
option in build/generator/templates/vcnet_vcxproj.ezt .
(Is it related to https://bugs.python.org/issue25878, especially
msg256496 ?)
<snip>
> In other news, building the swig-py3 branch (after sync-merge from
> trunk) with Python 2.7.16 (and swig 3.0.12) was successful.
> Running the testsuite gives some failures though:
>
> [[[
> Testing Release configuration on local repository.
> -- Running Swig Python tests --
> ..............s..............EE........F....F...........................................................................
> ..................................
> ======================================================================
> ERROR: test_conflict (client.SubversionClientTestCase)
> Test conflict api.
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\client.py",
> line 593, in test_conflict
> conflict = client.conflict_get(trunk_path, self.client_ctx, pool)
> File "R:\test_release--swigpython\swig\pylib\libsvn\client.py", line
> 1642, in svn_client_conflict_get
> return _client.svn_client_conflict_get(*args)
> SubversionException: 155007 - 'R:\temp\tmpyeizeh-conflict\trunk' is
> not a working copy
>
> ======================================================================
> ERROR: test_conflict (client.SubversionClientTestCase)
> Test conflict api.
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\client.py",
> line 94, in tearDown
> self.temper.cleanup()
> File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\utils.py",
> line 44, in cleanup
> clean_func(target)
> File "R:\test_release--swigpython\swig\pylib\libsvn\core.py", line
> 8453, in svn_io_remove_dir
> return _core.svn_io_remove_dir(*args)
> SubversionException: 720032 - Can't remove
> 'R:\temp\tmpyeizeh-conflict\.svn\wc.db'
> 720032 - Can't remove file 'R:\temp\tmpyeizeh-conflict\.svn\wc.db':
> The process cannot access the file because it is being used by another
> process.
>
> ======================================================================
> FAIL: test_merge_peg3 (client.SubversionClientTestCase)
> Test svn_client_merge_peg3.
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\client.py",
> line 442, in test_merge_peg3
> self.assertEqual(readme_text, b'This is a test.\n')
> AssertionError: 'This is a test.\r\n' != 'This is a test.\n'
>
> ======================================================================
> FAIL: test_shelf (client.SubversionClientTestCase)
> Test shelf api.
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\client.py",
> line 644, in test_shelf
> self.assertIn(new_subpath, statused_paths)
> AssertionError: 'trunk\\new-shelf-test.txt' not found in
> ['trunk/new-shelf-test.txt', 'trunk/new-shelf-test.txt', None]
>
> ----------------------------------------------------------------------
> Ran 153 tests in 38.984s
>
> FAILED (failures=2, errors=2, skipped=1)
> Exception svn.core.SubversionException: SubversionException("Can't
> open file 'R:\\temp\\tmp9ubdal-client\\db\\fs-type':
> The system cannot find the path specified. ", 720003) in <bound
> method Temper.__del__ of <utils.Temper object at 0x02B8B4F0>> ignored
> [Test runner reported failure]
> ]]]
>
> (Running the swig-python tests on trunk, with Python 2.7.16, was fully
> successful)
>
The FAIL on test_merge_peg3 is a newline style issue on reading file
in raw mode, which is introduced by my patch.
(https://svn.apache.org/viewvc/subversion/branches/swig-py3/subversion/bindings/swig/python/tests/client.py?view=annotate#l438)
Rest of those errors and failures are in the tests that were added only
to swig-py, and it seems most of the causes of them are confusion of
paths in platform specific format and Subversion's canoical format,
as the comment in SubversionClientTestCase.test_update4() says.
(I'll try to fix those issues on check-swig-py later, hopefully
next weekend.)
--
Yasuhito FUTATSUKI
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Johan Corveleyn <jc...@gmail.com>.
On Wed, Sep 18, 2019 at 12:08 PM Branko Čibej <br...@apache.org> wrote:
>
> On 18.09.2019 09:20, Johan Corveleyn wrote:
> > On Wed, Sep 18, 2019 at 5:36 AM Branko Čibej <br...@apache.org> wrote:
> >> On 18.09.2019 02:48, Johan Corveleyn wrote:
> >>> On Mon, Sep 2, 2019 at 11:45 AM Julian Foad <ju...@apache.org> wrote:
> >>>>> Troy Curtis Jr wrote:
> >>>>>> James McCoy wrote:
> >>>>>>> Finishing(?) and merging the Python3 support would be ideal. That would
> >>>>>>> give one release for broader feedback before being in an LTS release.
> >>>>>> It needs to get reintegrated with trunk again with the latest changes, but the linux side of the house was looking good. It was in trying to get my Windows Dev environment back up and running again that got me frustrated and gave me an excuse to wander off doing other things... IIRC, there is a build issue with the Py2 bindings in that branch on Windows, but I never got far enough to check it out.
> >>>>>> If there was someone who could help out on the Windows side, I can jump in and get the branch up to trunk and retested in order to get this thing over the finish line.
> >>>> Johan Corveleyn wrote:
> >>>>> I might be able to devote some time to this in the coming week(s), if
> >>>>> you can tell me what I need to do / test / validate / ... :-).
> >>>> Until Troy says something more specific, if you could switch to the
> >>>> branch, catch-up merge from trunk, and see how much works on Py3 and on
> >>>> Py2, that would be helpful.
> >>> I'm sorry I missed the 1.13.x-branch deadline, but I finally got
> >>> around to playing with the swig-py3 branch on Windows.
> >>> Downloaded the latest Python release: 3.7.4. And using swig 3.0.12.
> >>>
> >>> Of course I didn't reread INSTALL, so I first ran into:
> >>>
> >>> [[[
> >>> WARNING: "C:\research\svn\dev\swig-py3\py3c\include\py3c.h" not found
> >>> Use "--with-py3c" to configure py3c location.
> >>> ]]]
> >>>
> >>> No problem, after downloading py3c from Github, and adding
> >>> --with-py3c, I can start building it.
> >>>
> >>> However, I ran into the following error:
> >>>
> >>> [[[
> >>> c:\python37\include\pytime.h(123): error C4115: 'timeval': named type
> >>> definition in parentheses
> >>> [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj]
> >>> ]]]
> >> That makes no sense, my copy of pytime.h on Windows for Python 3.7.4 has
> >> this on line 123:
> >>
> >> PyAPI_FUNC(int) _PyTime_FromTimeval(_PyTime_t *tp, struct timeval *tv);
> >>
> >>
> >> Looks like perfectly valid C to me. So we'll need a bit more context to
> >> see where the actual error is coming from.
> > Not at my pc right now (I'll check again tonight), but from memory: I
> > think my copy of pytime.h looks the same. Line 123 seems to be the
> > first usage of 'timeval' though ... Is it possible that some include
> > is missing, so there is no declaration of the timeval struct type?
>
>
> You do *not* need a definition of 'struct foo' in order to use a 'struct
> foo*' pointer. So, no, that's not the problem.
Hmm, okay, just tried again ... same problem. Here is some more output:
[[[
C:\research\svn\dev\swig-py3> msbuild subversion_vcnet.sln /nologo
/v:q /p:Configuration=release /t:__SWIG_PYTHON__
c:\python37\include\pytime.h(123): error C4115: 'timeval': named type
definition in parentheses
[C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj]
C:\research\svn\dev\swig-py3\subversion\bindings\swig\proxy\swig_python_external_runtime.swg(2459):
warning C4459: declaration of 'swig_this' hides global declaration
[C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj]
C:\research\svn\dev\swig-py3\subversion\bindings\swig\proxy\swig_python_external_runtime.swg(2534):
warning C4459: declaration of 'swig_this' hides global declaration
[C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj]
..\..\..\subversion\bindings\swig\python\libsvn_swig_py\swigutil_py.c(2363):
warning C4101: 'tmp': unreferenced local variable
[C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj]
..\..\..\subversion\bindings\swig\python\libsvn_swig_py\swigutil_py.c(2772):
warning C4018: '>': signed/unsigned mismatch
[C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj]
]]]
I tried with both the x86-64 and the x86 builds of Python 3.7.4 ...
same result. Weird. I'm continuing to investigate.
In other news, building the swig-py3 branch (after sync-merge from
trunk) with Python 2.7.16 (and swig 3.0.12) was successful.
Running the testsuite gives some failures though:
[[[
Testing Release configuration on local repository.
-- Running Swig Python tests --
..............s..............EE........F....F...........................................................................
..................................
======================================================================
ERROR: test_conflict (client.SubversionClientTestCase)
Test conflict api.
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\client.py",
line 593, in test_conflict
conflict = client.conflict_get(trunk_path, self.client_ctx, pool)
File "R:\test_release--swigpython\swig\pylib\libsvn\client.py", line
1642, in svn_client_conflict_get
return _client.svn_client_conflict_get(*args)
SubversionException: 155007 - 'R:\temp\tmpyeizeh-conflict\trunk' is
not a working copy
======================================================================
ERROR: test_conflict (client.SubversionClientTestCase)
Test conflict api.
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\client.py",
line 94, in tearDown
self.temper.cleanup()
File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\utils.py",
line 44, in cleanup
clean_func(target)
File "R:\test_release--swigpython\swig\pylib\libsvn\core.py", line
8453, in svn_io_remove_dir
return _core.svn_io_remove_dir(*args)
SubversionException: 720032 - Can't remove
'R:\temp\tmpyeizeh-conflict\.svn\wc.db'
720032 - Can't remove file 'R:\temp\tmpyeizeh-conflict\.svn\wc.db':
The process cannot access the file because it is being used by another
process.
======================================================================
FAIL: test_merge_peg3 (client.SubversionClientTestCase)
Test svn_client_merge_peg3.
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\client.py",
line 442, in test_merge_peg3
self.assertEqual(readme_text, b'This is a test.\n')
AssertionError: 'This is a test.\r\n' != 'This is a test.\n'
======================================================================
FAIL: test_shelf (client.SubversionClientTestCase)
Test shelf api.
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\research\svn\dev\swig-py3\subversion\bindings\swig\python\tests\client.py",
line 644, in test_shelf
self.assertIn(new_subpath, statused_paths)
AssertionError: 'trunk\\new-shelf-test.txt' not found in
['trunk/new-shelf-test.txt', 'trunk/new-shelf-test.txt', None]
----------------------------------------------------------------------
Ran 153 tests in 38.984s
FAILED (failures=2, errors=2, skipped=1)
Exception svn.core.SubversionException: SubversionException("Can't
open file 'R:\\temp\\tmp9ubdal-client\\db\\fs-type':
The system cannot find the path specified. ", 720003) in <bound
method Temper.__del__ of <utils.Temper object at 0x02B8B4F0>> ignored
[Test runner reported failure]
]]]
(Running the swig-python tests on trunk, with Python 2.7.16, was fully
successful)
--
Johan
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Branko Čibej <br...@apache.org>.
On 18.09.2019 09:20, Johan Corveleyn wrote:
> On Wed, Sep 18, 2019 at 5:36 AM Branko Čibej <br...@apache.org> wrote:
>> On 18.09.2019 02:48, Johan Corveleyn wrote:
>>> On Mon, Sep 2, 2019 at 11:45 AM Julian Foad <ju...@apache.org> wrote:
>>>>> Troy Curtis Jr wrote:
>>>>>> James McCoy wrote:
>>>>>>> Finishing(?) and merging the Python3 support would be ideal. That would
>>>>>>> give one release for broader feedback before being in an LTS release.
>>>>>> It needs to get reintegrated with trunk again with the latest changes, but the linux side of the house was looking good. It was in trying to get my Windows Dev environment back up and running again that got me frustrated and gave me an excuse to wander off doing other things... IIRC, there is a build issue with the Py2 bindings in that branch on Windows, but I never got far enough to check it out.
>>>>>> If there was someone who could help out on the Windows side, I can jump in and get the branch up to trunk and retested in order to get this thing over the finish line.
>>>> Johan Corveleyn wrote:
>>>>> I might be able to devote some time to this in the coming week(s), if
>>>>> you can tell me what I need to do / test / validate / ... :-).
>>>> Until Troy says something more specific, if you could switch to the
>>>> branch, catch-up merge from trunk, and see how much works on Py3 and on
>>>> Py2, that would be helpful.
>>> I'm sorry I missed the 1.13.x-branch deadline, but I finally got
>>> around to playing with the swig-py3 branch on Windows.
>>> Downloaded the latest Python release: 3.7.4. And using swig 3.0.12.
>>>
>>> Of course I didn't reread INSTALL, so I first ran into:
>>>
>>> [[[
>>> WARNING: "C:\research\svn\dev\swig-py3\py3c\include\py3c.h" not found
>>> Use "--with-py3c" to configure py3c location.
>>> ]]]
>>>
>>> No problem, after downloading py3c from Github, and adding
>>> --with-py3c, I can start building it.
>>>
>>> However, I ran into the following error:
>>>
>>> [[[
>>> c:\python37\include\pytime.h(123): error C4115: 'timeval': named type
>>> definition in parentheses
>>> [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj]
>>> ]]]
>> That makes no sense, my copy of pytime.h on Windows for Python 3.7.4 has
>> this on line 123:
>>
>> PyAPI_FUNC(int) _PyTime_FromTimeval(_PyTime_t *tp, struct timeval *tv);
>>
>>
>> Looks like perfectly valid C to me. So we'll need a bit more context to
>> see where the actual error is coming from.
> Not at my pc right now (I'll check again tonight), but from memory: I
> think my copy of pytime.h looks the same. Line 123 seems to be the
> first usage of 'timeval' though ... Is it possible that some include
> is missing, so there is no declaration of the timeval struct type?
You do *not* need a definition of 'struct foo' in order to use a 'struct
foo*' pointer. So, no, that's not the problem.
-- Brane
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Johan Corveleyn <jc...@gmail.com>.
On Wed, Sep 18, 2019 at 5:36 AM Branko Čibej <br...@apache.org> wrote:
>
> On 18.09.2019 02:48, Johan Corveleyn wrote:
> > On Mon, Sep 2, 2019 at 11:45 AM Julian Foad <ju...@apache.org> wrote:
> >>> Troy Curtis Jr wrote:
> >>>> James McCoy wrote:
> >>>>> Finishing(?) and merging the Python3 support would be ideal. That would
> >>>>> give one release for broader feedback before being in an LTS release.
> >>>> It needs to get reintegrated with trunk again with the latest changes, but the linux side of the house was looking good. It was in trying to get my Windows Dev environment back up and running again that got me frustrated and gave me an excuse to wander off doing other things... IIRC, there is a build issue with the Py2 bindings in that branch on Windows, but I never got far enough to check it out.
> >>>> If there was someone who could help out on the Windows side, I can jump in and get the branch up to trunk and retested in order to get this thing over the finish line.
> >> Johan Corveleyn wrote:
> >>> I might be able to devote some time to this in the coming week(s), if
> >>> you can tell me what I need to do / test / validate / ... :-).
> >> Until Troy says something more specific, if you could switch to the
> >> branch, catch-up merge from trunk, and see how much works on Py3 and on
> >> Py2, that would be helpful.
> > I'm sorry I missed the 1.13.x-branch deadline, but I finally got
> > around to playing with the swig-py3 branch on Windows.
> > Downloaded the latest Python release: 3.7.4. And using swig 3.0.12.
> >
> > Of course I didn't reread INSTALL, so I first ran into:
> >
> > [[[
> > WARNING: "C:\research\svn\dev\swig-py3\py3c\include\py3c.h" not found
> > Use "--with-py3c" to configure py3c location.
> > ]]]
> >
> > No problem, after downloading py3c from Github, and adding
> > --with-py3c, I can start building it.
> >
> > However, I ran into the following error:
> >
> > [[[
> > c:\python37\include\pytime.h(123): error C4115: 'timeval': named type
> > definition in parentheses
> > [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj]
> > ]]]
>
> That makes no sense, my copy of pytime.h on Windows for Python 3.7.4 has
> this on line 123:
>
> PyAPI_FUNC(int) _PyTime_FromTimeval(_PyTime_t *tp, struct timeval *tv);
>
>
> Looks like perfectly valid C to me. So we'll need a bit more context to
> see where the actual error is coming from.
Not at my pc right now (I'll check again tonight), but from memory: I
think my copy of pytime.h looks the same. Line 123 seems to be the
first usage of 'timeval' though ... Is it possible that some include
is missing, so there is no declaration of the timeval struct type?
--
Johan
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Branko Čibej <br...@apache.org>.
On 18.09.2019 02:48, Johan Corveleyn wrote:
> On Mon, Sep 2, 2019 at 11:45 AM Julian Foad <ju...@apache.org> wrote:
>>> Troy Curtis Jr wrote:
>>>> James McCoy wrote:
>>>>> Finishing(?) and merging the Python3 support would be ideal. That would
>>>>> give one release for broader feedback before being in an LTS release.
>>>> It needs to get reintegrated with trunk again with the latest changes, but the linux side of the house was looking good. It was in trying to get my Windows Dev environment back up and running again that got me frustrated and gave me an excuse to wander off doing other things... IIRC, there is a build issue with the Py2 bindings in that branch on Windows, but I never got far enough to check it out.
>>>> If there was someone who could help out on the Windows side, I can jump in and get the branch up to trunk and retested in order to get this thing over the finish line.
>> Johan Corveleyn wrote:
>>> I might be able to devote some time to this in the coming week(s), if
>>> you can tell me what I need to do / test / validate / ... :-).
>> Until Troy says something more specific, if you could switch to the
>> branch, catch-up merge from trunk, and see how much works on Py3 and on
>> Py2, that would be helpful.
> I'm sorry I missed the 1.13.x-branch deadline, but I finally got
> around to playing with the swig-py3 branch on Windows.
> Downloaded the latest Python release: 3.7.4. And using swig 3.0.12.
>
> Of course I didn't reread INSTALL, so I first ran into:
>
> [[[
> WARNING: "C:\research\svn\dev\swig-py3\py3c\include\py3c.h" not found
> Use "--with-py3c" to configure py3c location.
> ]]]
>
> No problem, after downloading py3c from Github, and adding
> --with-py3c, I can start building it.
>
> However, I ran into the following error:
>
> [[[
> c:\python37\include\pytime.h(123): error C4115: 'timeval': named type
> definition in parentheses
> [C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj]
> ]]]
That makes no sense, my copy of pytime.h on Windows for Python 3.7.4 has
this on line 123:
PyAPI_FUNC(int) _PyTime_FromTimeval(_PyTime_t *tp, struct timeval *tv);
Looks like perfectly valid C to me. So we'll need a bit more context to
see where the actual error is coming from.
-- Brane
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Johan Corveleyn <jc...@gmail.com>.
On Mon, Sep 2, 2019 at 11:45 AM Julian Foad <ju...@apache.org> wrote:
> > Troy Curtis Jr wrote:
> >> James McCoy wrote:
> >>> Finishing(?) and merging the Python3 support would be ideal. That would
> >>> give one release for broader feedback before being in an LTS release.
> >>
> >> It needs to get reintegrated with trunk again with the latest changes, but the linux side of the house was looking good. It was in trying to get my Windows Dev environment back up and running again that got me frustrated and gave me an excuse to wander off doing other things... IIRC, there is a build issue with the Py2 bindings in that branch on Windows, but I never got far enough to check it out.
>
> >> If there was someone who could help out on the Windows side, I can jump in and get the branch up to trunk and retested in order to get this thing over the finish line.
>
> Johan Corveleyn wrote:
> > I might be able to devote some time to this in the coming week(s), if
> > you can tell me what I need to do / test / validate / ... :-).
>
> Until Troy says something more specific, if you could switch to the
> branch, catch-up merge from trunk, and see how much works on Py3 and on
> Py2, that would be helpful.
I'm sorry I missed the 1.13.x-branch deadline, but I finally got
around to playing with the swig-py3 branch on Windows.
Downloaded the latest Python release: 3.7.4. And using swig 3.0.12.
Of course I didn't reread INSTALL, so I first ran into:
[[[
WARNING: "C:\research\svn\dev\swig-py3\py3c\include\py3c.h" not found
Use "--with-py3c" to configure py3c location.
]]]
No problem, after downloading py3c from Github, and adding
--with-py3c, I can start building it.
However, I ran into the following error:
[[[
c:\python37\include\pytime.h(123): error C4115: 'timeval': named type
definition in parentheses
[C:\research\svn\dev\swig-py3\build\win32\vcnet-vcproj\libsvn_swig_py.vcxproj]
]]]
(PS: before firing up the swig-py3 branch, I double-checked that I
could successfully build the swig-python bindings on trunk with Python
2.7.16 and swig 3.0.12, and ran the swig-python tests successfully)
--
Johan
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Nathan Hartman <ha...@gmail.com>.
On Mon, Sep 2, 2019 at 8:23 AM Julian Foad <ju...@apache.org> wrote:
> Thanks. I see no reason to make a special effort to have swig-py
> bindings still support Python 2 for Subversion 1.13 and 1.14-LTS. In
> fact, it's probably best if we explicitly remove Py2 support for 1.14,
> to avoid getting trapped in maintenance difficulties.
Py2 is EOL 1/1/2020. Keeping it in 1.14 which implies support until 2024
would be a mistake because support will only get harder. Imagine what
happens when your OS vendor removes Py2 packages. Even if Py2 still
technically "works" with 1.14 (because that makes it easier to support Py3
for the remainder of 1.10), I suggest to document that Py2 is explicitly
NOT supported in the 1.14 release notes. The internal policy should be that
once 1.10 support ends, changes are allowed to break Py2 support and/or not
be tested with Py2.
I have (among other things) a Windows 10 machine available to me with
Python 2 and 3. I'll be glad to help, with a little guidance. At the very
least I can help with testing.
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Julian Foad <ju...@apache.org>.
Branko Čibej wrote:
>> What Py3 support are we talking about anyway? svn build? svn test
>> suite? swig-py bindings?
>
> This is about the work that's happening on the swig-py3 branch. So: that
> would be support for building the swig-py bindings for both Python3 and
> Python2. Trunk only support Python2 which has reached end of life.
>
> Our build and test scripts have already been modified to work with both
> variants of Python, or rather, with Python 2.7.x and 3.something, for
> some unknown value of "something."
Thanks. I see no reason to make a special effort to have swig-py
bindings still support Python 2 for Subversion 1.13 and 1.14-LTS. In
fact, it's probably best if we explicitly remove Py2 support for 1.14,
to avoid getting trapped in maintenance difficulties.
Py2 support is needed if we want to backport Py3 support to 1.10.x,
which would be a good thing to aim for; and given that dual-Py-version
capability is mostly in place but unfinished, that sounds like we are on
the right track to be confident that if we put it on trunk now we would
be able to fix up the Py2 support for backport to 1.10.x later, when
it's ready.
It may be tempting to "just" try to fix the remaining Py2 issues right
now before merge to trunk, but I need to prepare a 1.13 branch within
the next 10 days to stay on schedule, so please consider prioritising
merge to trunk with Py3 working, leaving Py2 to be fixed up after that.
- Julian
Re: Python3 work [was: The run up to Subversion 1.13.0]
Posted by Branko Čibej <br...@apache.org>.
On 02.09.2019 11:45, Julian Foad wrote:
>> Troy Curtis Jr wrote:
>>> James McCoy wrote:
>>>> Finishing(?) and merging the Python3 support would be ideal. That
>>>> would
>>>> give one release for broader feedback before being in an LTS release.
>>>
>>> It needs to get reintegrated with trunk again with the latest
>>> changes, but the linux side of the house was looking good. It was in
>>> trying to get my Windows Dev environment back up and running again
>>> that got me frustrated and gave me an excuse to wander off doing
>>> other things... IIRC, there is a build issue with the Py2 bindings
>>> in that branch on Windows, but I never got far enough to check it out.
>
> Stefan Sperling wrote:
>> If it helps, we could require Py3-only on Windows, going forward.
>> That particular ecosystem has been dragging its feet for long enough ;)
>
> What Py3 support are we talking about anyway? svn build? svn test
> suite? swig-py bindings?
This is about the work that's happening on the swig-py3 branch. So: that
would be support for building the swig-py bindings for both Python3 and
Python2. Trunk only support Python2 which has reached end of life.
Our build and test scripts have already been modified to work with both
variants of Python, or rather, with Python 2.7.x and 3.something, for
some unknown value of "something."
-- Brane
Python3 work [was: The run up to Subversion 1.13.0]
Posted by Julian Foad <ju...@apache.org>.
> Troy Curtis Jr wrote:
>> James McCoy wrote:
>>> Finishing(?) and merging the Python3 support would be ideal. That would
>>> give one release for broader feedback before being in an LTS release.
>>
>> It needs to get reintegrated with trunk again with the latest changes, but the linux side of the house was looking good. It was in trying to get my Windows Dev environment back up and running again that got me frustrated and gave me an excuse to wander off doing other things... IIRC, there is a build issue with the Py2 bindings in that branch on Windows, but I never got far enough to check it out.
Stefan Sperling wrote:
> If it helps, we could require Py3-only on Windows, going forward.
> That particular ecosystem has been dragging its feet for long enough ;)
What Py3 support are we talking about anyway? svn build? svn test suite?
swig-py bindings?
Stefan Kueng, how would that be for you and TortoiseSVN? If it's fine
for you, I would be very happy to release Subversion 1.13.0 with Py3 the
only supported version on Windows, if we can get it integrated to trunk
in the next few days.
>> If there was someone who could help out on the Windows side, I can jump in and get the branch up to trunk and retested in order to get this thing over the finish line.
Johan Corveleyn wrote:
> I might be able to devote some time to this in the coming week(s), if
> you can tell me what I need to do / test / validate / ... :-).
Until Troy says something more specific, if you could switch to the
branch, catch-up merge from trunk, and see how much works on Py3 and on
Py2, that would be helpful.
Can someone else check the same on *nix please?
1.13 is a "regular release" with emphasis on new features etc. rather
than stability. Of course we always like to have as much stability as
possible as well, but please let's take this opportunity to get
something of this important work integrated right now rather than
waiting until it's absolutely finished.
- Julian
Re: The run up to Subversion 1.13.0
Posted by Johan Corveleyn <jc...@gmail.com>.
On Mon, Sep 2, 2019 at 3:42 AM Troy Curtis Jr <tr...@gmail.com> wrote:
>
>
>
> On Fri, Aug 30, 2019, 7:13 PM James McCoy <ja...@jamessan.com> wrote:
>>
>> On Fri, Aug 30, 2019 at 05:12:21PM +0100, Julian Foad wrote:
>> > 2. As 1.13 is only weeks away, there is little point in making any further
>> > 1.12.x patch releases from now on, so I will not plan to do so.
>> >
>> > Any other thoughts on anything we ought to include, or to do?
>>
>> Finishing(?) and merging the Python3 support would be ideal. That would
>> give one release for broader feedback before being in an LTS release.
>>
>> I haven't kept up with the status of that branch, nor can I guarantee
>> any time to helping with this, but since you asked... :)
>
>
> It needs to get reintegrated with trunk again with the latest changes, but the linux side of the house was looking good. It was in trying to get my Windows Dev environment back up and running again that got me frustrated and gave me an excuse to wander off doing other things... IIRC, there is a build issue with the Py2 bindings in that branch on Windows, but I never got far enough to check it out.
>
> If there was someone who could help out on the Windows side, I can jump in and get the branch up to trunk and retested in order to get this thing over the finish line.
>
I might be able to devote some time to this in the coming week(s), if
you can tell me what I need to do / test / validate / ... :-).
--
Johan