You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Stefan Hett <st...@egosoft.com> on 2015/04/22 15:58:40 UTC
building SVN trunk on Windows fails on python gen-make.py
Hi,
I just tried to build
https://svn.apache.org/repos/asf/subversion/branches/svn-mergeinfo-normalizer
on Windows following the INSTALL file documentation up to the point
where the make file is being generated.
Using this call: python gen-make.py -t vcproj --with-zlib=..\zlib
--with-apr=..\apr --with-apr-util=..\apr-util --vsnet-version=2010
I'm now getting the following error:
Traceback (most recent call last):
File "gen-make.py", line 325, in <module> main(conf, gentype,
skip_depends=skip, other_options=rest.list)
File "gen-make.py", line 62, in main generator =
gen_module.Generator(fname, verfname, other_options)
File "build\generator\gen_vcnet_vcproj.py", line 36, in __init__
'vcnet-vcproj')
File "build\generator\gen_win.py", line 83, in __init__
self.find_libraries(True)
File "build\generator\gen_win_dependencies.py", line 286, in
find_libraries self._find_apr()
File "build\generator\gen_win_dependencies.py", line 395, in
_find_apr bin_files = os.listdir(dll_dir)
FileNotFoundError: [WinError 3] The system cannot find the path
specified: '..\\apr\\bin'
Looking at the apr-directory I downloaded/extracted, there's no
apr/bin-directory.
I downloaded the following APR-source package (1.5.1):
http://mirror.serversupportforum.de/apache//apr/apr-1.5.1-win32-src.zip
Did I get the wrong one? Or did I miss some step?
Regards,
Stefan
[PATCH] APR install/configuration issue error handling (was: building
SVN trunk on Windows fails on python gen-make.py)
Posted by Stefan Hett <st...@egosoft.com>.
>> -----Original Message-----
>> From: Stefan Hett [mailto:stefan@egosoft.com]
>> Sent: woensdag 22 april 2015 15:59
>> To: 'subversion'
>> Subject: building SVN trunk on Windows fails on python gen-make.py
>>
>> Hi,
>>
>> I just tried to build
>> https://svn.apache.org/repos/asf/subversion/branches/svn-mergeinfo-
>> normalizer
>> on Windows following the INSTALL file documentation up to the point
>> where the make file is being generated.
>>
>> Using this call: python gen-make.py -t vcproj --with-zlib=..\zlib
>> --with-apr=..\apr --with-apr-util=..\apr-util --vsnet-version=2010
>> I'm now getting the following error:
>>
>> Traceback (most recent call last):
>> File "gen-make.py", line 325, in <module> main(conf, gentype,
>> skip_depends=skip, other_options=rest.list)
>> File "gen-make.py", line 62, in main generator =
>> gen_module.Generator(fname, verfname, other_options)
>> File "build\generator\gen_vcnet_vcproj.py", line 36, in __init__
>> 'vcnet-vcproj')
>> File "build\generator\gen_win.py", line 83, in __init__
>> self.find_libraries(True)
>> File "build\generator\gen_win_dependencies.py", line 286, in
>> find_libraries self._find_apr()
>> File "build\generator\gen_win_dependencies.py", line 395, in
>> _find_apr bin_files = os.listdir(dll_dir)
>> FileNotFoundError: [WinError 3] The system cannot find the path
>> specified: '..\\apr\\bin'
>>
>> Looking at the apr-directory I downloaded/extracted, there's no
>> apr/bin-directory.
>> I downloaded the following APR-source package (1.5.1):
>> http://mirror.serversupportforum.de/apache//apr/apr-1.5.1-win32-src.zip
>>
>> Did I get the wrong one? Or did I miss some step?
> Subversion expects that you already build apr before using it as a source of binaries to link against. It is not going to build apr for you.
>
> Building APR would have created this bin directory. (A patch to produce a nicer error message would be welcome :))
>
>
> There are other scripts that may help you building Subversion and all dependencies. I maintain one such script as part of SharpSvn (Repository: https://ctf.open.collab.net/svn/repos/sharpsvn/trunk/imports ; username guest; no password). This script uses NAnt, and is used to produce the publicly available binaries for SharpSvn and SlikSvn.
>
> Somewhere in the Subversion repository there is a perl script that should also be able to help with building on Windows, but I don't have experience with that script.
>
> Bert
As the saying goes "better later than never", as per ur suggestion,
here's a patch which adds some better handling to detect user-errors
when putting just the downloaded source of apr (and/or apr-util) into
the specified folder. Suggested patch for trunk and nomination for 1.9 I
guess.
[[[
Produce human readable error when APR/APR-util wasn't
installed/configured correctly.
* build/generator/gen_win_dependencies.py
(_find_apr,
_find_apr_util): Add detection for missing lib/dll directories,
pointing to
an unconfigured/uninstalled source directory.
]]]
Regards,
Stefan
Re: 1.9.0 minimal Python version
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Philip Martin wrote on Fri, Jul 24, 2015 at 14:13:04 +0100:
> Philip Martin <ph...@wandisco.com> writes:
>
> > Philip Martin <ph...@wandisco.com> writes:
> >
> >> RHEL 6 is on Python 2.6 and is unlikely to change. Python 2.7 is
> >> available as a Red Hat Software Collection component.
> >
> > The Software Collection python27 is sufficient to build Subversion and
> > run the regression tests but some of the tests FAIL. The tests that
> > fail all install a python hook script and the hook script fails to run.
> > The reason is that the Software Collection relies on setting environment
> > variables to make the Python 2.7 binary and libraries available but
> > Subversion cleans the environment before running hooks.
>
> If tried using a script
>
> #!/bin/sh
> LD_LIBRARY_PATH=/opt/rh/python27/root/usr/lib64:$LD_LIBRARY_PATH /opt/rh/python27/root/usr/bin/python "$@"
>
> but it doesn't fix the FAILs as the hook scripts have an explict
>
> #!/opt/rh/python27/root/usr/bin/python
>
> The PYTHON setting in the Makefile is the path to the script and this is
> what we want in the hook.
You could fix that by making your wrapper script set Python's argv[0]
(at the C level) to the absolute path to the wrapper script. That'll
cause Python's sys.executable to be the wrapper script, which will make
the test harness write the path to the wrapper script in the #! line.
> We have a related, but sort of opposite, problem with parallel tests. I
> can use the script even if it is not found in PATH by doing:
>
> PYTHON=/path/to/script ./autogen.sh
> PYTHON=/path/to/script ./configure
> make check
>
> and the tests run (with the hook tests failing). If I attempt to run
> the tests in parallel
>
> make check PARALLEL=1
>
> then all the python tests SKIP because when the parallel code spawns
> additional python it uses the python found in PATH rather than the
> Makefile PYTHON.
IMO that's a bug, we shouldn't use the PATH python for subprocesses. We
should probably make the PARALLEL codepath use imp.load_module() like
run_tests() uses.
>
Cheers,
Daniel
Re: 1.9.0 minimal Python version
Posted by Philip Martin <ph...@wandisco.com>.
Philip Martin <ph...@wandisco.com> writes:
> Philip Martin <ph...@wandisco.com> writes:
>
>> RHEL 6 is on Python 2.6 and is unlikely to change. Python 2.7 is
>> available as a Red Hat Software Collection component.
>
> The Software Collection python27 is sufficient to build Subversion and
> run the regression tests but some of the tests FAIL. The tests that
> fail all install a python hook script and the hook script fails to run.
> The reason is that the Software Collection relies on setting environment
> variables to make the Python 2.7 binary and libraries available but
> Subversion cleans the environment before running hooks.
If tried using a script
#!/bin/sh
LD_LIBRARY_PATH=/opt/rh/python27/root/usr/lib64:$LD_LIBRARY_PATH /opt/rh/python27/root/usr/bin/python "$@"
but it doesn't fix the FAILs as the hook scripts have an explict
#!/opt/rh/python27/root/usr/bin/python
The PYTHON setting in the Makefile is the path to the script and this is
what we want in the hook.
We have a related, but sort of opposite, problem with parallel tests. I
can use the script even if it is not found in PATH by doing:
PYTHON=/path/to/script ./autogen.sh
PYTHON=/path/to/script ./configure
make check
and the tests run (with the hook tests failing). If I attempt to run
the tests in parallel
make check PARALLEL=1
then all the python tests SKIP because when the parallel code spawns
additional python it uses the python found in PATH rather than the
Makefile PYTHON.
--
Philip Martin
WANdisco
Re: 1.9.0 minimal Python version
Posted by Philip Martin <ph...@wandisco.com>.
Philip Martin <ph...@wandisco.com> writes:
> RHEL 6 is on Python 2.6 and is unlikely to change. Python 2.7 is
> available as a Red Hat Software Collection component.
The Software Collection python27 is sufficient to build Subversion and
run the regression tests but some of the tests FAIL. The tests that
fail all install a python hook script and the hook script fails to run.
The reason is that the Software Collection relies on setting environment
variables to make the Python 2.7 binary and libraries available but
Subversion cleans the environment before running hooks.
--
Philip Martin
WANdisco
Re: 1.9.0 minimal Python version
Posted by Philip Martin <ph...@wandisco.com>.
Branko Čibej <br...@wandisco.com> writes:
> [please do not top-post]
>
> On 24.07.2015 11:00, Stefan Hett wrote:
>> In case that helps with the decision making process:
>> Debian 5.0 (lenny - no longer supported): py 2.5
>> Debian 6.0 (squeeze - current oldoldstable): py2.6
>> Debian 7.0/8.0/9.0 (wheezy/jessie/stretch - currently
>> oldstable/stable/testing): py2.7
>
> The "very old enterprise versions" are probably RHEL 6 and SuSE 10; both
> are still in wide use.
RHEL 6 is on Python 2.6 and is unlikely to change. Python 2.7 is
available as a Red Hat Software Collection component.
--
Philip Martin
WANdisco
Re: 1.9.0 minimal Python version
Posted by Branko Čibej <br...@wandisco.com>.
[please do not top-post]
On 24.07.2015 11:00, Stefan Hett wrote:
> In case that helps with the decision making process:
> Debian 5.0 (lenny - no longer supported): py 2.5
> Debian 6.0 (squeeze - current oldoldstable): py2.6
> Debian 7.0/8.0/9.0 (wheezy/jessie/stretch - currently
> oldstable/stable/testing): py2.7
The "very old enterprise versions" are probably RHEL 6 and SuSE 10; both
are still in wide use.
> So even this very conservative Linux distribution is supporting py2.7
> for the past two years (and with the backporting branch I assume even
> squeeze supports 2.7 which would make it a supported platform for the
> pat 4 years on that distribution).
>
> Source:
> https://packages.debian.org/search?keywords=python&searchon=names&suite=all§ion=all
>> On 23.07.2015 17:01, Bert Huijben wrote:
>>> I like this proposal... but I'm wondering in what case Linux build
>>> environments need python. Those very old enterprise versions are
>>> unlikely to have python 2.7 or newer.
>>
>> When building from a tarball, you only need Python to run the test suite.
>>
>> There are certain edge cases in the Python syntax where it's almost
>> impossible to be compatible with both 2.5/2.6 and 3.x. Because of
>> that, and because 2.6 is no longer supporter by the Python devs, it
>> makes sense to stop supporting those versions in 1.9/trunk.
>>
>> Those very old enterprise versions will just have to be upgraded, IMO.
>>
>>
>>> ------------------------------------------------------------------------
>>> From: Branko Čibej <ma...@wandisco.com>
>>> Sent: 23-7-2015 11:02
>>> To: dev@subversion.apache.org
>>> Subject: Re: 1.9.0 minimal Python version
>>>
>>> On 23.07.2015 10:30, Stefan Hett wrote:
>>> > Hi,
>>> >
>>> >>>> - For 1.9, it's a little late to make any changes, but I would
>>> >>>> consider
>>> >>>> dropping py2.5 support (and converting to the 'except' 'as'
>>> >>>> syntax),
>>> >>>> since for 1.9 py3 support is more important than py2.5 support.
>>> >>>>
>>> >>>> Thoughts?
>>> >> I'd rather not mess with the 1.9 branch at this point ... we're
>>> so close
>>> >> to the release (I hope).
>>> >>
>>> >> -- Brane
>>> > as a thought for the 1.9 case:
>>> > In case you do not want to drop py2.5 support for 1.9 because it's
>>> > released stating that minimum support already, would it be an option
>>> > to still support py3 in a following patch (1.9.1)? For instance by
>>> > providing py-script versions for older as well as later versions?
>>> >
>>> > Reasoning would be that given that SVN-versions have a lifetime of
>>> > roughly 2 years before the successive version is released it'd be
>>> > quite a limitation if that'd only work with a very old python
>>> version...
>>> >
>>> > As an alternative approach: You'd also consider mentioning a minimum
>>> > requirement of python 2.6 just in the docs (no code changes yet) and
>>> > then release 1.9.1 with the actual "fixes". So technically then even
>>> > post release you would not change the minimal system requirements.
>>> >
>>> > (just some thoughts from a user's point of view, if that input would
>>> > be of any benefit)
>>>
>>> Daniel and I have just been chatting about this on IRC, and we agreed
>>> that it makes sense to just drop 2.5/2.6 support for 1.9 and trunk; the
>>> proposal is that we declare support for 2.7, and at some point make
>>> 1.9.x and trunk compatible with both 2.7 and 3.x.
>>>
>>> -- Brane
>>
>
Re: 1.9.0 minimal Python version
Posted by Stefan Hett <st...@egosoft.com>.
In case that helps with the decision making process:
Debian 5.0 (lenny - no longer supported): py 2.5
Debian 6.0 (squeeze - current oldoldstable): py2.6
Debian 7.0/8.0/9.0 (wheezy/jessie/stretch - currently
oldstable/stable/testing): py2.7
So even this very conservative Linux distribution is supporting py2.7
for the past two years (and with the backporting branch I assume even
squeeze supports 2.7 which would make it a supported platform for the
pat 4 years on that distribution).
Source:
https://packages.debian.org/search?keywords=python&searchon=names&suite=all§ion=all
> On 23.07.2015 17:01, Bert Huijben wrote:
>> I like this proposal... but I'm wondering in what case Linux build
>> environments need python. Those very old enterprise versions are
>> unlikely to have python 2.7 or newer.
>
> When building from a tarball, you only need Python to run the test suite.
>
> There are certain edge cases in the Python syntax where it's almost
> impossible to be compatible with both 2.5/2.6 and 3.x. Because of
> that, and because 2.6 is no longer supporter by the Python devs, it
> makes sense to stop supporting those versions in 1.9/trunk.
>
> Those very old enterprise versions will just have to be upgraded, IMO.
>
>
>> ------------------------------------------------------------------------
>> From: Branko Čibej <ma...@wandisco.com>
>> Sent: 23-7-2015 11:02
>> To: dev@subversion.apache.org <ma...@subversion.apache.org>
>> Subject: Re: 1.9.0 minimal Python version
>>
>> On 23.07.2015 10:30, Stefan Hett wrote:
>> > Hi,
>> >
>> >>>> - For 1.9, it's a little late to make any changes, but I would
>> >>>> consider
>> >>>> dropping py2.5 support (and converting to the 'except' 'as'
>> >>>> syntax),
>> >>>> since for 1.9 py3 support is more important than py2.5 support.
>> >>>>
>> >>>> Thoughts?
>> >> I'd rather not mess with the 1.9 branch at this point ... we're so
>> close
>> >> to the release (I hope).
>> >>
>> >> -- Brane
>> > as a thought for the 1.9 case:
>> > In case you do not want to drop py2.5 support for 1.9 because it's
>> > released stating that minimum support already, would it be an option
>> > to still support py3 in a following patch (1.9.1)? For instance by
>> > providing py-script versions for older as well as later versions?
>> >
>> > Reasoning would be that given that SVN-versions have a lifetime of
>> > roughly 2 years before the successive version is released it'd be
>> > quite a limitation if that'd only work with a very old python
>> version...
>> >
>> > As an alternative approach: You'd also consider mentioning a minimum
>> > requirement of python 2.6 just in the docs (no code changes yet) and
>> > then release 1.9.1 with the actual "fixes". So technically then even
>> > post release you would not change the minimal system requirements.
>> >
>> > (just some thoughts from a user's point of view, if that input would
>> > be of any benefit)
>>
>> Daniel and I have just been chatting about this on IRC, and we agreed
>> that it makes sense to just drop 2.5/2.6 support for 1.9 and trunk; the
>> proposal is that we declare support for 2.7, and at some point make
>> 1.9.x and trunk compatible with both 2.7 and 3.x.
>>
>> -- Brane
>
Re: 1.9.0 minimal Python version
Posted by Branko Čibej <br...@wandisco.com>.
On 23.07.2015 17:01, Bert Huijben wrote:
> I like this proposal... but I'm wondering in what case Linux build
> environments need python. Those very old enterprise versions are
> unlikely to have python 2.7 or newer.
When building from a tarball, you only need Python to run the test suite.
There are certain edge cases in the Python syntax where it's almost
impossible to be compatible with both 2.5/2.6 and 3.x. Because of that,
and because 2.6 is no longer supporter by the Python devs, it makes
sense to stop supporting those versions in 1.9/trunk.
Those very old enterprise versions will just have to be upgraded, IMO.
> ------------------------------------------------------------------------
> From: Branko Čibej <ma...@wandisco.com>
> Sent: 23-7-2015 11:02
> To: dev@subversion.apache.org <ma...@subversion.apache.org>
> Subject: Re: 1.9.0 minimal Python version
>
> On 23.07.2015 10:30, Stefan Hett wrote:
> > Hi,
> >
> >>>> - For 1.9, it's a little late to make any changes, but I would
> >>>> consider
> >>>> dropping py2.5 support (and converting to the 'except' 'as'
> >>>> syntax),
> >>>> since for 1.9 py3 support is more important than py2.5 support.
> >>>>
> >>>> Thoughts?
> >> I'd rather not mess with the 1.9 branch at this point ... we're so
> close
> >> to the release (I hope).
> >>
> >> -- Brane
> > as a thought for the 1.9 case:
> > In case you do not want to drop py2.5 support for 1.9 because it's
> > released stating that minimum support already, would it be an option
> > to still support py3 in a following patch (1.9.1)? For instance by
> > providing py-script versions for older as well as later versions?
> >
> > Reasoning would be that given that SVN-versions have a lifetime of
> > roughly 2 years before the successive version is released it'd be
> > quite a limitation if that'd only work with a very old python version...
> >
> > As an alternative approach: You'd also consider mentioning a minimum
> > requirement of python 2.6 just in the docs (no code changes yet) and
> > then release 1.9.1 with the actual "fixes". So technically then even
> > post release you would not change the minimal system requirements.
> >
> > (just some thoughts from a user's point of view, if that input would
> > be of any benefit)
>
> Daniel and I have just been chatting about this on IRC, and we agreed
> that it makes sense to just drop 2.5/2.6 support for 1.9 and trunk; the
> proposal is that we declare support for 2.7, and at some point make
> 1.9.x and trunk compatible with both 2.7 and 3.x.
>
> -- Brane
RE: 1.9.0 minimal Python version
Posted by Bert Huijben <be...@qqmail.nl>.
I like this proposal... but I'm wondering in what case Linux build environments need python. Those very old enterprise versions are unlikely to have python 2.7 or newer.
Bert
-----Original Message-----
From: "Branko Čibej" <br...@wandisco.com>
Sent: 23-7-2015 11:02
To: "dev@subversion.apache.org" <de...@subversion.apache.org>
Subject: Re: 1.9.0 minimal Python version
On 23.07.2015 10:30, Stefan Hett wrote:
> Hi,
>
>>>> - For 1.9, it's a little late to make any changes, but I would
>>>> consider
>>>> dropping py2.5 support (and converting to the 'except' 'as'
>>>> syntax),
>>>> since for 1.9 py3 support is more important than py2.5 support.
>>>>
>>>> Thoughts?
>> I'd rather not mess with the 1.9 branch at this point ... we're so close
>> to the release (I hope).
>>
>> -- Brane
> as a thought for the 1.9 case:
> In case you do not want to drop py2.5 support for 1.9 because it's
> released stating that minimum support already, would it be an option
> to still support py3 in a following patch (1.9.1)? For instance by
> providing py-script versions for older as well as later versions?
>
> Reasoning would be that given that SVN-versions have a lifetime of
> roughly 2 years before the successive version is released it'd be
> quite a limitation if that'd only work with a very old python version...
>
> As an alternative approach: You'd also consider mentioning a minimum
> requirement of python 2.6 just in the docs (no code changes yet) and
> then release 1.9.1 with the actual "fixes". So technically then even
> post release you would not change the minimal system requirements.
>
> (just some thoughts from a user's point of view, if that input would
> be of any benefit)
Daniel and I have just been chatting about this on IRC, and we agreed
that it makes sense to just drop 2.5/2.6 support for 1.9 and trunk; the
proposal is that we declare support for 2.7, and at some point make
1.9.x and trunk compatible with both 2.7 and 3.x.
-- Brane
Re: 1.9.0 minimal Python version
Posted by Branko Čibej <br...@wandisco.com>.
On 23.07.2015 10:30, Stefan Hett wrote:
> Hi,
>
>>>> - For 1.9, it's a little late to make any changes, but I would
>>>> consider
>>>> dropping py2.5 support (and converting to the 'except' 'as'
>>>> syntax),
>>>> since for 1.9 py3 support is more important than py2.5 support.
>>>>
>>>> Thoughts?
>> I'd rather not mess with the 1.9 branch at this point ... we're so close
>> to the release (I hope).
>>
>> -- Brane
> as a thought for the 1.9 case:
> In case you do not want to drop py2.5 support for 1.9 because it's
> released stating that minimum support already, would it be an option
> to still support py3 in a following patch (1.9.1)? For instance by
> providing py-script versions for older as well as later versions?
>
> Reasoning would be that given that SVN-versions have a lifetime of
> roughly 2 years before the successive version is released it'd be
> quite a limitation if that'd only work with a very old python version...
>
> As an alternative approach: You'd also consider mentioning a minimum
> requirement of python 2.6 just in the docs (no code changes yet) and
> then release 1.9.1 with the actual "fixes". So technically then even
> post release you would not change the minimal system requirements.
>
> (just some thoughts from a user's point of view, if that input would
> be of any benefit)
Daniel and I have just been chatting about this on IRC, and we agreed
that it makes sense to just drop 2.5/2.6 support for 1.9 and trunk; the
proposal is that we declare support for 2.7, and at some point make
1.9.x and trunk compatible with both 2.7 and 3.x.
-- Brane
Re: 1.9.0 minimal Python version
Posted by Stefan Hett <st...@egosoft.com>.
Hi,
>>> - For 1.9, it's a little late to make any changes, but I would consider
>>> dropping py2.5 support (and converting to the 'except' 'as' syntax),
>>> since for 1.9 py3 support is more important than py2.5 support.
>>>
>>> Thoughts?
> I'd rather not mess with the 1.9 branch at this point ... we're so close
> to the release (I hope).
>
> -- Brane
as a thought for the 1.9 case:
In case you do not want to drop py2.5 support for 1.9 because it's
released stating that minimum support already, would it be an option to
still support py3 in a following patch (1.9.1)? For instance by
providing py-script versions for older as well as later versions?
Reasoning would be that given that SVN-versions have a lifetime of
roughly 2 years before the successive version is released it'd be quite
a limitation if that'd only work with a very old python version...
As an alternative approach: You'd also consider mentioning a minimum
requirement of python 2.6 just in the docs (no code changes yet) and
then release 1.9.1 with the actual "fixes". So technically then even
post release you would not change the minimal system requirements.
(just some thoughts from a user's point of view, if that input would be
of any benefit)
Regards,
Stefan
Re: 1.9.0 minimal Python version (was: Re: [PATCH] Error out with
proper error message when using Python >= 3.0
Posted by Branko Čibej <br...@wandisco.com>.
On 22.07.2015 13:46, Daniel Shahaf wrote:
> [Re-sending with a new subject to draw attention to the 1.9 question.]
>
> Daniel Shahaf wrote on Wed, Jul 22, 2015 at 11:31:31 +0000:
>> Stefan Hett wrote on Mon, Jul 20, 2015 at 16:59:08 +0200:
>>> Please note that while testing I also found 1.7/1.8 not working with
>>> Python 3.4.3 as well. Error (in both cases):
>>> File "gen-make.py", line 271
>>> except getopt.GetoptError, e:
>>> ^
>>> SyntaxError: invalid syntax
>>>
>> The 'except' comma syntax is supported by Python through 2.7 (inclusive)
>> and the 'except' 'as' syntax is supported by Python 2.6 and newer.
>> Currently, trunk advertises support for Python 2.5 and newer (so, in
>> particular, so do the older branches).
>>
>> So, I think we should:
>>
>> - For 1.7/1.8, if we expect people will try to build them with py3,
>> we can apply the patch. (The patch is correct; we must remain
>> compatible with py2.5 on those branches; and we can't easily be
>> compatible with py3 at the same time.)
Ack.
>> - For trunk, I think we should convert to the 'except' 'as' syntax and
>> drop py2.5 support. (Actually, we could drop py2.6 support as well,
>> as py2.6 has been EOL for nearly two years now.)
Ack.
>> - For 1.9, it's a little late to make any changes, but I would consider
>> dropping py2.5 support (and converting to the 'except' 'as' syntax),
>> since for 1.9 py3 support is more important than py2.5 support.
>>
>> Thoughts?
I'd rather not mess with the 1.9 branch at this point ... we're so close
to the release (I hope).
-- Brane
>> References:
>>
>> [py2.5 'except' syntax] https://docs.python.org/2.5/ref/try.html
>> [py2.6 'except' syntax] https://docs.python.org/2.6/reference/compound_stmts.html#the-try-statement
>> [py2.6 EOL] https://www.python.org/dev/peps/pep-0361/
>>
>>> So I guess it might be worthwhile adding some check there too?
>>>
>>> [[[
>>> Detect Python >= 3.0 and error out in gen-make.py (rather than
>>> producing some cryptic error/callstack).
>>>
>>> * gen-make.py
>>> (): Add python version check >= 3.0 and error out.
>>> ]]]
1.9.0 minimal Python version (was: Re: [PATCH] Error out with proper
error message when using Python >= 3.0 (was: building SVN trunk on Windows
fails on python gen-make.py))
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
[Re-sending with a new subject to draw attention to the 1.9 question.]
Daniel Shahaf wrote on Wed, Jul 22, 2015 at 11:31:31 +0000:
> Stefan Hett wrote on Mon, Jul 20, 2015 at 16:59:08 +0200:
> > Please note that while testing I also found 1.7/1.8 not working with
> > Python 3.4.3 as well. Error (in both cases):
> > File "gen-make.py", line 271
> > except getopt.GetoptError, e:
> > ^
> > SyntaxError: invalid syntax
> >
>
> The 'except' comma syntax is supported by Python through 2.7 (inclusive)
> and the 'except' 'as' syntax is supported by Python 2.6 and newer.
> Currently, trunk advertises support for Python 2.5 and newer (so, in
> particular, so do the older branches).
>
> So, I think we should:
>
> - For 1.7/1.8, if we expect people will try to build them with py3,
> we can apply the patch. (The patch is correct; we must remain
> compatible with py2.5 on those branches; and we can't easily be
> compatible with py3 at the same time.)
>
> - For trunk, I think we should convert to the 'except' 'as' syntax and
> drop py2.5 support. (Actually, we could drop py2.6 support as well,
> as py2.6 has been EOL for nearly two years now.)
>
> - For 1.9, it's a little late to make any changes, but I would consider
> dropping py2.5 support (and converting to the 'except' 'as' syntax),
> since for 1.9 py3 support is more important than py2.5 support.
>
> Thoughts?
>
> Daniel
>
> --
>
> References:
>
> [py2.5 'except' syntax] https://docs.python.org/2.5/ref/try.html
> [py2.6 'except' syntax] https://docs.python.org/2.6/reference/compound_stmts.html#the-try-statement
> [py2.6 EOL] https://www.python.org/dev/peps/pep-0361/
>
> > So I guess it might be worthwhile adding some check there too?
> >
> > [[[
> > Detect Python >= 3.0 and error out in gen-make.py (rather than
> > producing some cryptic error/callstack).
> >
> > * gen-make.py
> > (): Add python version check >= 3.0 and error out.
> > ]]]
Re: [PATCH] Error out with proper error message when using Python >=
3.0 (was: building SVN trunk on Windows fails on python gen-make.py)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Stefan Hett wrote on Mon, Jul 20, 2015 at 16:59:08 +0200:
> Please note that while testing I also found 1.7/1.8 not working with
> Python 3.4.3 as well. Error (in both cases):
> File "gen-make.py", line 271
> except getopt.GetoptError, e:
> ^
> SyntaxError: invalid syntax
>
The 'except' comma syntax is supported by Python through 2.7 (inclusive)
and the 'except' 'as' syntax is supported by Python 2.6 and newer.
Currently, trunk advertises support for Python 2.5 and newer (so, in
particular, so do the older branches).
So, I think we should:
- For 1.7/1.8, if we expect people will try to build them with py3,
we can apply the patch. (The patch is correct; we must remain
compatible with py2.5 on those branches; and we can't easily be
compatible with py3 at the same time.)
- For trunk, I think we should convert to the 'except' 'as' syntax and
drop py2.5 support. (Actually, we could drop py2.6 support as well,
as py2.6 has been EOL for nearly two years now.)
- For 1.9, it's a little late to make any changes, but I would consider
dropping py2.5 support (and converting to the 'except' 'as' syntax),
since for 1.9 py3 support is more important than py2.5 support.
Thoughts?
Daniel
--
References:
[py2.5 'except' syntax] https://docs.python.org/2.5/ref/try.html
[py2.6 'except' syntax] https://docs.python.org/2.6/reference/compound_stmts.html#the-try-statement
[py2.6 EOL] https://www.python.org/dev/peps/pep-0361/
> So I guess it might be worthwhile adding some check there too?
>
> [[[
> Detect Python >= 3.0 and error out in gen-make.py (rather than
> producing some cryptic error/callstack).
>
> * gen-make.py
> (): Add python version check >= 3.0 and error out.
> ]]]
>
> Regards,
> Stefan
> Index: gen-make.py
> ===================================================================
> --- gen-make.py (revision 1691913)
> +++ gen-make.py (working copy)
> @@ -28,6 +28,10 @@
> import traceback
> import sys
>
> +if sys.hexversion >= 0x03000000:
> + print("Python >= 3.0 not supported. Please use Python >= 2.5 and < 3.0")
> + sys.exit(2)
> +
> import getopt
> try:
> my_getopt = getopt.gnu_getopt
[PATCH] Error out with proper error message when using Python >= 3.0
(was: building SVN trunk on Windows fails on python gen-make.py)
Posted by Stefan Hett <st...@egosoft.com>.
> [ moved from users@ to dev@; tldr: trunk gen-make.py fails with py3.4/windows ]
>
> Stefan Hett wrote on Mon, Jun 22, 2015 at 15:12:31 +0200:
>>> On Mon, Jun 22, 2015 at 12:45 PM, Stefan Hett <st...@egosoft.com> wrote:
>>>> File "E:\Python34\lib\encodings\cp1252.py", line 23, in decode return
>>>> codecs.charmap_decode(input,self.errors,decoding_table)[0]
>>>> UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 2116:
>>>> character maps to <undefined>
>>> Quick shot: perhaps it's related to the Python version. I'm using
>>> Python 2.7. Can you try that?
>>>
>> Hi Johan,
>>
>> thanks for the hint. That indeed seemed to have been the problem.
>> Passes through the python script now when using Python 2.7.10.
>>
>> Regards,
>> Stefan
> Two things about that error:
>
> 1. The file that triggers it is probably
> subversion/tests/cmdline/upgrade_tests_data/upgrade_with_externals.tar.bz2
> (it has a 0x81 byte at the given offset). It's a binary file so it
> shouldn't be fed to a unicode decoder in the first place. It's only
> used by some unit tests.
>
> 2. I went through autogen.sh/configure on Python 3 on Linux a few weeks
> ago and got it to pass. (I don't recall what's the status of the test
> suite under py3.) So either this issue is specific to Py3 on Windows,
> or I missed something, or the gen-make.py code has changed.
>
> Anyway, gen-make.py shouldn't fail like that. I'd rather it just worked
> under py3, but if we can't make it work, we should error out up front if
> Python is newer than we support.
Given there's been not much progress on the issue, I quickly
put-together (and tested) a patch to detect Python >= 3 and issue a
user-readable error. Maybe it'd be worth including this in 1.9, so
people trying to build 1.9 don't have to go through the hazzle of
figuring out that the problem is their current python-version rather
than some other problem?
Please note that while testing I also found 1.7/1.8 not working with
Python 3.4.3 as well. Error (in both cases):
File "gen-make.py", line 271
except getopt.GetoptError, e:
^
SyntaxError: invalid syntax
So I guess it might be worthwhile adding some check there too?
[[[
Detect Python >= 3.0 and error out in gen-make.py (rather than
producing some cryptic error/callstack).
* gen-make.py
(): Add python version check >= 3.0 and error out.
]]]
Regards,
Stefan
Re: building SVN trunk on Windows fails on python gen-make.py
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
[ moved from users@ to dev@; tldr: trunk gen-make.py fails with py3.4/windows ]
Stefan Hett wrote on Mon, Jun 22, 2015 at 15:12:31 +0200:
> >On Mon, Jun 22, 2015 at 12:45 PM, Stefan Hett <st...@egosoft.com> wrote:
> >> File "E:\Python34\lib\encodings\cp1252.py", line 23, in decode return
> >>codecs.charmap_decode(input,self.errors,decoding_table)[0]
> >>UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 2116:
> >>character maps to <undefined>
> >Quick shot: perhaps it's related to the Python version. I'm using
> >Python 2.7. Can you try that?
> >
>
> Hi Johan,
>
> thanks for the hint. That indeed seemed to have been the problem.
> Passes through the python script now when using Python 2.7.10.
>
> Regards,
> Stefan
Two things about that error:
1. The file that triggers it is probably
subversion/tests/cmdline/upgrade_tests_data/upgrade_with_externals.tar.bz2
(it has a 0x81 byte at the given offset). It's a binary file so it
shouldn't be fed to a unicode decoder in the first place. It's only
used by some unit tests.
2. I went through autogen.sh/configure on Python 3 on Linux a few weeks
ago and got it to pass. (I don't recall what's the status of the test
suite under py3.) So either this issue is specific to Py3 on Windows,
or I missed something, or the gen-make.py code has changed.
Anyway, gen-make.py shouldn't fail like that. I'd rather it just worked
under py3, but if we can't make it work, we should error out up front if
Python is newer than we support.
Cheers,
Daniel
P.S. If anyone wonders, the command I used was 'xxd -s 2114 -l 5
-g 2 | grep -qw 81'.
Re: building SVN trunk on Windows fails on python gen-make.py
Posted by Stefan Hett <st...@egosoft.com>.
Hi Johan,
thanks for the hint. That indeed seemed to have been the problem. Passes
through the python script now when using Python 2.7.10.
Regards,
Stefan
> On Mon, Jun 22, 2015 at 12:45 PM, Stefan Hett <st...@egosoft.com> wrote:
>> Sorry for the late response. Had to shift priorities the last month here,
>> before I could get back to this issue.
>>
>> Thanks for the additional information on ways to build SVN. I quickly
>> skimmed through the scripts/steps and might use them later when I can't get
>> the current steps to run...
>>
>> After build APR myself, it got passed the previous error. However I'm now
>> stuck with another one and can't make-up what's wrong now... Any hint
>> (couldn't find any reference on google or in the mail archive):
>> swig not found; skipping SWIG file generation...
>> Traceback (most recent call last):
>> File "gen-make.py", line 325, in <module> main(conf, gentype,
>> skip_depends=skip, other_options=rest.list)
>> File "gen-make.py", line 65, in main generator.compute_hdr_deps()
>> File "build\generator\gen_base.py", line 212, in compute_hdr_deps for
>> include_file in include_deps.query(native_path(source.filename)):
>> File "build\generator\gen_base.py", line 1167, in query c_includes,
>> swig_includes = self.query_swig(fname)
>> File "build\generator\gen_base.py", line 1150, in query_swig hdrs =
>> self._scan_for_includes(fname)
>> File "build\generator\gen_base.py", line 1203, in _scan_for_includes
>> for line in fileinput.input(fname):
>> File "E:\Python34\lib\fileinput.py", line 263, in __next__ line =
>> self.readline()
>> File "E:\Python34\lib\fileinput.py", line 363, in readline self._buffer =
>> self._file.readlines(self._bufsize)
>> File "E:\Python34\lib\encodings\cp1252.py", line 23, in decode return
>> codecs.charmap_decode(input,self.errors,decoding_table)[0]
>> UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 2116:
>> character maps to <undefined>
> Quick shot: perhaps it's related to the Python version. I'm using
> Python 2.7. Can you try that?
>
Re: building SVN trunk on Windows fails on python gen-make.py
Posted by Johan Corveleyn <jc...@gmail.com>.
On Mon, Jun 22, 2015 at 12:45 PM, Stefan Hett <st...@egosoft.com> wrote:
> Sorry for the late response. Had to shift priorities the last month here,
> before I could get back to this issue.
>
> Thanks for the additional information on ways to build SVN. I quickly
> skimmed through the scripts/steps and might use them later when I can't get
> the current steps to run...
>
> After build APR myself, it got passed the previous error. However I'm now
> stuck with another one and can't make-up what's wrong now... Any hint
> (couldn't find any reference on google or in the mail archive):
> swig not found; skipping SWIG file generation...
> Traceback (most recent call last):
> File "gen-make.py", line 325, in <module> main(conf, gentype,
> skip_depends=skip, other_options=rest.list)
> File "gen-make.py", line 65, in main generator.compute_hdr_deps()
> File "build\generator\gen_base.py", line 212, in compute_hdr_deps for
> include_file in include_deps.query(native_path(source.filename)):
> File "build\generator\gen_base.py", line 1167, in query c_includes,
> swig_includes = self.query_swig(fname)
> File "build\generator\gen_base.py", line 1150, in query_swig hdrs =
> self._scan_for_includes(fname)
> File "build\generator\gen_base.py", line 1203, in _scan_for_includes
> for line in fileinput.input(fname):
> File "E:\Python34\lib\fileinput.py", line 263, in __next__ line =
> self.readline()
> File "E:\Python34\lib\fileinput.py", line 363, in readline self._buffer =
> self._file.readlines(self._bufsize)
> File "E:\Python34\lib\encodings\cp1252.py", line 23, in decode return
> codecs.charmap_decode(input,self.errors,decoding_table)[0]
> UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 2116:
> character maps to <undefined>
Quick shot: perhaps it's related to the Python version. I'm using
Python 2.7. Can you try that?
--
Johan
RE: building SVN trunk on Windows fails on python gen-make.py
Posted by Bert Huijben <be...@qqmail.nl>.
> -----Original Message-----
> From: Cooke, Mark [mailto:mark.cooke@siemens.com]
> Sent: maandag 22 juni 2015 13:46
> To: Stefan Hett; users@subversion.apache.org
> Subject: RE: building SVN trunk on Windows fails on python gen-make.py
>
> Sorry I cannot help with the current issue but I wanted to add a comment about
> the content of INSTALL (below)...
> While I agree in principle, I need to build httpd and subversion to work with
> Trac (trac.edgewall.org) which is currently limited to python < 3.0 (I use 2.7
> also). For this to work properly my understanding is that httpd and python (and
> svn) all need to use the same compiler libraries and Python 2.7 is built using
> Visual C++ 2008...
>
> So the 2008 stuff is not completely irrelevant (I got it compiling and making the
> python bindings). I have some notes but they are not really fit for publication
> and are for VC2008 but I can share them if you think they would be useful.
To build AnkhSVN for Visual Studio 2005 and 2008 I continue to build Subversion using VS 2008 at least every week, so I would consider VS 2005 and 2008 as fully supported for Subversion.
But I don't think anybody working on Subversion really uses VS 6.0 (released +- 18 years ago) or 2002 and 2003, so I don't consider our support for these versions as really supported. I'm guessing that the VS 6.0 support might still work with a minor tweak, while I think the 2002/2003 support would need at least some patches to make it work again.
If nobody is going to comment on this after the release of 1.9.0, I think we will drop support for these 12+ year old versions with 1.10+. With VS 2013 Community Edition now free to use for everybody working on open source projects, there is no good reason to continue to support these very old versions, that target platforms that are no longer supported anyway.
One interesting point for some usecases is that the VS 2010 tooling allows building with the VS2008 compiler by passing a few specific flags to msbuild. This is how the first version of XP compatibility in VS2012+ was implemented. (And this is also how VC 2010+'s C++/CLI can target .Net 2.0-3.5)
Bert
RE: building SVN trunk on Windows fails on python gen-make.py
Posted by "Cooke, Mark" <ma...@siemens.com>.
Sorry I cannot help with the current issue but I wanted to add a comment about the content of INSTALL (below)...
> -----Original Message-----
> From: Stefan Hett [mailto:stefan@egosoft.com]
> Sent: 22 June 2015 11:46
> To: users@subversion.apache.org
> Subject: Re: building SVN trunk on Windows fails on python gen-make.py
>
> Sorry for the late response. Had to shift priorities the last month
> here, before I could get back to this issue.
>
> Thanks for the additional information on ways to build SVN. I quickly
> skimmed through the scripts/steps and might use them later when I can't
> get the current steps to run...
>
> After build APR myself, it got passed the previous error. However I'm
> now stuck with another one and can't make-up what's wrong now... Any
> hint (couldn't find any reference on google or in the mail archive):
> swig not found; skipping SWIG file generation...
> Traceback (most recent call last):
> File "gen-make.py", line 325, in <module> main(conf, gentype,
> skip_depends=skip, other_options=rest.list)
> File "gen-make.py", line 65, in main generator.compute_hdr_deps()
> File "build\generator\gen_base.py", line 212, in compute_hdr_deps
> for include_file in include_deps.query(native_path(source.filename)):
> File "build\generator\gen_base.py", line 1167, in query c_includes,
> swig_includes = self.query_swig(fname)
> File "build\generator\gen_base.py", line 1150, in query_swig hdrs =
> self._scan_for_includes(fname)
> File "build\generator\gen_base.py", line 1203, in
> _scan_for_includes for line in fileinput.input(fname):
> File "E:\Python34\lib\fileinput.py", line 263, in __next__ line =
> self.readline()
> File "E:\Python34\lib\fileinput.py", line 363, in readline
> self._buffer = self._file.readlines(self._bufsize)
> File "E:\Python34\lib\encodings\cp1252.py", line 23, in decode
> return codecs.charmap_decode(input,self.errors,decoding_table)[0]
> UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position
> 2116: character maps to <undefined>
>
> Regards,
> Stefan
>
> > On Wed, Apr 22, 2015 at 6:13 PM, Bert Huijben <be...@qqmail.nl> wrote:
> >>
> >>> -----Original Message-----
> >>> From: Stefan Hett [mailto:stefan@egosoft.com]
> >>> Sent: woensdag 22 april 2015 15:59
> >>> To: 'subversion'
> >>> Subject: building SVN trunk on Windows fails on python gen-make.py
> >>>
> >>> Hi,
> >>>
> >>> I just tried to build
> >>> https://svn.apache.org/repos/asf/subversion/branches/svn-mergeinfo-
> >>> normalizer
> >>> on Windows following the INSTALL file documentation up to the point
> >>> where the make file is being generated.
> >>>
> >>> Using this call: python gen-make.py -t vcproj --with-zlib=..\zlib
> >>> --with-apr=..\apr --with-apr-util=..\apr-util --vsnet-version=2010
> >>> I'm now getting the following error:
> >>>
> >>> Traceback (most recent call last):
> >>> File "gen-make.py", line 325, in <module> main(conf, gentype,
> >>> skip_depends=skip, other_options=rest.list)
> >>> File "gen-make.py", line 62, in main generator =
> >>> gen_module.Generator(fname, verfname, other_options)
> >>> File "build\generator\gen_vcnet_vcproj.py", line 36, in __init__
> >>> 'vcnet-vcproj')
> >>> File "build\generator\gen_win.py", line 83, in __init__
> >>> self.find_libraries(True)
> >>> File "build\generator\gen_win_dependencies.py", line 286, in
> >>> find_libraries self._find_apr()
> >>> File "build\generator\gen_win_dependencies.py", line 395, in
> >>> _find_apr bin_files = os.listdir(dll_dir)
> >>> FileNotFoundError: [WinError 3] The system cannot find the path
> >>> specified: '..\\apr\\bin'
> >>>
> >>> Looking at the apr-directory I downloaded/extracted, there's no
> >>> apr/bin-directory.
> >>> I downloaded the following APR-source package (1.5.1):
> >>> http://mirror.serversupportforum.de/apache//apr/apr-1.5.1-win32-src.zip
> >>>
> >>> Did I get the wrong one? Or did I miss some step?
> >> Subversion expects that you already build apr before using it as a source
> of binaries to link against. It is not going to build apr for you.
> >>
> >> Building APR would have created this bin directory. (A patch to produce a
> nicer error message would be welcome :))
> >>
> >>
> >> There are other scripts that may help you building Subversion and all
> dependencies. I maintain one such script as part of SharpSvn (Repository:
> https://ctf.open.collab.net/svn/repos/sharpsvn/trunk/imports ; username
> guest; no password). This script uses NAnt, and is used to produce the
> publicly available binaries for SharpSvn and SlikSvn.
> >>
> >> Somewhere in the Subversion repository there is a perl script that should
> also be able to help with building on Windows, but I don't have experience
> with that script.
> >>
> > Indeed, that script is tools/dev/build-svn-deps-win.pl. But it's a bit
> > outdated now. I recently used bits and pieces of it to get myself a
> > new working build environment for svn on Windows. In general our tools
> > and documentation for building svn on Windows are quite outdated and
> > could really use some love. For instance, INSTALL still contains a lot
> > of old irrelevant information, it should primarily focus on current
> > tools that people install these days (e.g. focusing on VS 2012, 2013
> > and 2015).
While I agree in principle, I need to build httpd and subversion to work with Trac (trac.edgewall.org) which is currently limited to python < 3.0 (I use 2.7 also). For this to work properly my understanding is that httpd and python (and svn) all need to use the same compiler libraries and Python 2.7 is built using Visual C++ 2008...
So the 2008 stuff is not completely irrelevant (I got it compiling and making the python bindings). I have some notes but they are not really fit for publication and are for VC2008 but I can share them if you think they would be useful.
~ Mark C
> > With my last build-setup attempt, I kept a short log of what I did
> > (based on things from build-svn-deps-win.pl, and further trial and
> > error). Perhaps this is useful:
> >
> > [[[
> > download awk95.exe and rename it to awk.exe and put somewhere in PATH
> > (as documented in INSTALL)
> > download and install:
> > - CMake
> > - SCons
> > - Python
> > - Perl
> >
> > Install Visual Studio (I used 2013).
> >
> > Download latest (or recent enough stable) sources of:
> > - zlib
> > - openssl
> > - sqlite
> > - serf
> > - httpd
> > - apr
> > - apr-util
> > - apr-iconv
> > - pcre
> >
> > Extract these in a "deps" directory (I usually name the individual
> > dirs with the version number, so I have C:\svn\deps\httpd-2.4.12,
> > C:\svn\deps\zlib-1.2.8, ...).
> > Except: put apr, apr-util, apr-iconv and pcre as subdirectories
> > (without version number in the name) inside $HTTPD_SOURCE\srclib.
> >
> > Then start building. Do all the following from inside a Visual Studio
> > Command Prompt:
> >
> > zlib:
> > For Debug build:
> > set CC_OPTS=/MDd /Gm /ZI /Od /GZ /D_DEBUG
> > set COMMON_CC_OPTS=/nologo /W3 /DWIN32 /D_WINDOWS
> >
> > For Release build:
> > set CC_OPTS=/MD /02 /Zi
> > set COMMON_CC_OPTS=/nologo /W3 /DWIN32 /D_WINDOWS
> >
> > Somehow, you need to make both a static build (for svn itself) and a
> > dynamic build (for serf):
> > Static:
> > nmake /nologo -f win32/Makefile.msc STATICLIB=zlibstat.lib all
> > And for serf:
> > nmake /nologo -f win32/Makefile.msc all
> >
> > unset CC_OPTS
> > unset COMMON_CC_OPTS
> >
> > PCRE:
> > (replace RelWithDebInfo with Debug in the line below to make a debug
> build)
> > cmake -G "NMake Makefiles" -DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo
> > -DBUILD_SHARED_LIBS:BOOL=ON
> > -DCMAKE_INSTALL_PREFIX:PATH=C:\svn\deps\install
> > -DPCRE_NO_RECURSE:BOOL=ON .
> > nmake
> > nmake test
> > nmake install
> >
> > openssl:
> > (replace VC-WIN32 with debug-VC-WIN32 to make a debug build)
> > perl Configure no-asm --prefix=C:\svn\deps\install VC-WIN32
> > call ms\do_ms
> > nmake -f ms\ntdll.mak
> > nmake -f ms\ntdll.mak test
> > nmake -f ms\ntdll.mak install
> >
> > httpd (+ apr, apr-util, apr-iconv):
> > perl srclib\apr\build\lineends.pl
> > perl srclib\apr\build\cvtdsp.pl -2005
> > Strip cases of "/YuXXXX" en "/YcXXXX" from
> > srclib/apr-iconv/build/modules.mk.win
> > Insert
> "<PropertyGroup><GenerateManifest>false</GenerateManifest></PropertyGroup>"
> > in support/win32/ApacheMonitor.vcxproj
> > Insert "<PropertyGroup><TargetName>libapr-
> 1</TargetName></PropertyGroup>"
> > in srclib/apr/libapr.vcxproj
> > Insert "<PropertyGroup><TargetName>apr_ldap-
> 1</TargetName></PropertyGroup>"
> > in srclib/aprutil/ldap/apr_ldap.vcxproj
> > Insert "<PropertyGroup><TargetName>libapriconv-
> 1</TargetName></PropertyGroup>"
> > in srclib/apr-iconv/libapriconv.vcxproj
> > Insert "<PropertyGroup><TargetName>libaprutil-
> 1</TargetName></PropertyGroup>"
> > in srclib/aprutil/libaprutil.vcxproj
> > Applied fix from http://marc.info/?l=apr-dev&m=138386228311375&w=2
> > (see also https://www.apachelounge.com/viewtopic.php?t=6174) --
> > apparently otherwise there is a problem building httpd 2.4.10+ on
> > Windows 64-bit.
> >
> > For a debug build:
> > nmake /f Makefile.win installd INSTDIR=C:\Apache24
> >
> > For a release build:
> > nmake /f Makefile.win installr INSTDIR=C:\Apache24
> > (for some reason, the build sometimes fails ... rerunning it usually
> > succeeds, don't know why)
> >
> > serf:
> > scons APR=..\httpd-2.4.12\srclib\apr
> > APU=..\httpd-2.4.12\srclib\apr-util OPENSSL=..\openssl-1.0.2
> > ZLIB=..\zlib-1.2.8
> > ]]]
> >
> > HTH,
Re: building SVN trunk on Windows fails on python gen-make.py
Posted by Stefan Hett <st...@egosoft.com>.
Sorry for the late response. Had to shift priorities the last month
here, before I could get back to this issue.
Thanks for the additional information on ways to build SVN. I quickly
skimmed through the scripts/steps and might use them later when I can't
get the current steps to run...
After build APR myself, it got passed the previous error. However I'm
now stuck with another one and can't make-up what's wrong now... Any
hint (couldn't find any reference on google or in the mail archive):
swig not found; skipping SWIG file generation...
Traceback (most recent call last):
File "gen-make.py", line 325, in <module> main(conf, gentype,
skip_depends=skip, other_options=rest.list)
File "gen-make.py", line 65, in main generator.compute_hdr_deps()
File "build\generator\gen_base.py", line 212, in compute_hdr_deps
for include_file in include_deps.query(native_path(source.filename)):
File "build\generator\gen_base.py", line 1167, in query c_includes,
swig_includes = self.query_swig(fname)
File "build\generator\gen_base.py", line 1150, in query_swig hdrs =
self._scan_for_includes(fname)
File "build\generator\gen_base.py", line 1203, in
_scan_for_includes for line in fileinput.input(fname):
File "E:\Python34\lib\fileinput.py", line 263, in __next__ line =
self.readline()
File "E:\Python34\lib\fileinput.py", line 363, in readline
self._buffer = self._file.readlines(self._bufsize)
File "E:\Python34\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position
2116: character maps to <undefined>
Regards,
Stefan
> On Wed, Apr 22, 2015 at 6:13 PM, Bert Huijben <be...@qqmail.nl> wrote:
>>
>>> -----Original Message-----
>>> From: Stefan Hett [mailto:stefan@egosoft.com]
>>> Sent: woensdag 22 april 2015 15:59
>>> To: 'subversion'
>>> Subject: building SVN trunk on Windows fails on python gen-make.py
>>>
>>> Hi,
>>>
>>> I just tried to build
>>> https://svn.apache.org/repos/asf/subversion/branches/svn-mergeinfo-
>>> normalizer
>>> on Windows following the INSTALL file documentation up to the point
>>> where the make file is being generated.
>>>
>>> Using this call: python gen-make.py -t vcproj --with-zlib=..\zlib
>>> --with-apr=..\apr --with-apr-util=..\apr-util --vsnet-version=2010
>>> I'm now getting the following error:
>>>
>>> Traceback (most recent call last):
>>> File "gen-make.py", line 325, in <module> main(conf, gentype,
>>> skip_depends=skip, other_options=rest.list)
>>> File "gen-make.py", line 62, in main generator =
>>> gen_module.Generator(fname, verfname, other_options)
>>> File "build\generator\gen_vcnet_vcproj.py", line 36, in __init__
>>> 'vcnet-vcproj')
>>> File "build\generator\gen_win.py", line 83, in __init__
>>> self.find_libraries(True)
>>> File "build\generator\gen_win_dependencies.py", line 286, in
>>> find_libraries self._find_apr()
>>> File "build\generator\gen_win_dependencies.py", line 395, in
>>> _find_apr bin_files = os.listdir(dll_dir)
>>> FileNotFoundError: [WinError 3] The system cannot find the path
>>> specified: '..\\apr\\bin'
>>>
>>> Looking at the apr-directory I downloaded/extracted, there's no
>>> apr/bin-directory.
>>> I downloaded the following APR-source package (1.5.1):
>>> http://mirror.serversupportforum.de/apache//apr/apr-1.5.1-win32-src.zip
>>>
>>> Did I get the wrong one? Or did I miss some step?
>> Subversion expects that you already build apr before using it as a source of binaries to link against. It is not going to build apr for you.
>>
>> Building APR would have created this bin directory. (A patch to produce a nicer error message would be welcome :))
>>
>>
>> There are other scripts that may help you building Subversion and all dependencies. I maintain one such script as part of SharpSvn (Repository: https://ctf.open.collab.net/svn/repos/sharpsvn/trunk/imports ; username guest; no password). This script uses NAnt, and is used to produce the publicly available binaries for SharpSvn and SlikSvn.
>>
>> Somewhere in the Subversion repository there is a perl script that should also be able to help with building on Windows, but I don't have experience with that script.
>>
> Indeed, that script is tools/dev/build-svn-deps-win.pl. But it's a bit
> outdated now. I recently used bits and pieces of it to get myself a
> new working build environment for svn on Windows. In general our tools
> and documentation for building svn on Windows are quite outdated and
> could really use some love. For instance, INSTALL still contains a lot
> of old irrelevant information, it should primarily focus on current
> tools that people install these days (e.g. focusing on VS 2012, 2013
> and 2015).
>
> With my last build-setup attempt, I kept a short log of what I did
> (based on things from build-svn-deps-win.pl, and further trial and
> error). Perhaps this is useful:
>
> [[[
> download awk95.exe and rename it to awk.exe and put somewhere in PATH
> (as documented in INSTALL)
> download and install:
> - CMake
> - SCons
> - Python
> - Perl
>
> Install Visual Studio (I used 2013).
>
> Download latest (or recent enough stable) sources of:
> - zlib
> - openssl
> - sqlite
> - serf
> - httpd
> - apr
> - apr-util
> - apr-iconv
> - pcre
>
> Extract these in a "deps" directory (I usually name the individual
> dirs with the version number, so I have C:\svn\deps\httpd-2.4.12,
> C:\svn\deps\zlib-1.2.8, ...).
> Except: put apr, apr-util, apr-iconv and pcre as subdirectories
> (without version number in the name) inside $HTTPD_SOURCE\srclib.
>
> Then start building. Do all the following from inside a Visual Studio
> Command Prompt:
>
> zlib:
> For Debug build:
> set CC_OPTS=/MDd /Gm /ZI /Od /GZ /D_DEBUG
> set COMMON_CC_OPTS=/nologo /W3 /DWIN32 /D_WINDOWS
>
> For Release build:
> set CC_OPTS=/MD /02 /Zi
> set COMMON_CC_OPTS=/nologo /W3 /DWIN32 /D_WINDOWS
>
> Somehow, you need to make both a static build (for svn itself) and a
> dynamic build (for serf):
> Static:
> nmake /nologo -f win32/Makefile.msc STATICLIB=zlibstat.lib all
> And for serf:
> nmake /nologo -f win32/Makefile.msc all
>
> unset CC_OPTS
> unset COMMON_CC_OPTS
>
> PCRE:
> (replace RelWithDebInfo with Debug in the line below to make a debug build)
> cmake -G "NMake Makefiles" -DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo
> -DBUILD_SHARED_LIBS:BOOL=ON
> -DCMAKE_INSTALL_PREFIX:PATH=C:\svn\deps\install
> -DPCRE_NO_RECURSE:BOOL=ON .
> nmake
> nmake test
> nmake install
>
> openssl:
> (replace VC-WIN32 with debug-VC-WIN32 to make a debug build)
> perl Configure no-asm --prefix=C:\svn\deps\install VC-WIN32
> call ms\do_ms
> nmake -f ms\ntdll.mak
> nmake -f ms\ntdll.mak test
> nmake -f ms\ntdll.mak install
>
> httpd (+ apr, apr-util, apr-iconv):
> perl srclib\apr\build\lineends.pl
> perl srclib\apr\build\cvtdsp.pl -2005
> Strip cases of "/YuXXXX" en "/YcXXXX" from
> srclib/apr-iconv/build/modules.mk.win
> Insert "<PropertyGroup><GenerateManifest>false</GenerateManifest></PropertyGroup>"
> in support/win32/ApacheMonitor.vcxproj
> Insert "<PropertyGroup><TargetName>libapr-1</TargetName></PropertyGroup>"
> in srclib/apr/libapr.vcxproj
> Insert "<PropertyGroup><TargetName>apr_ldap-1</TargetName></PropertyGroup>"
> in srclib/aprutil/ldap/apr_ldap.vcxproj
> Insert "<PropertyGroup><TargetName>libapriconv-1</TargetName></PropertyGroup>"
> in srclib/apr-iconv/libapriconv.vcxproj
> Insert "<PropertyGroup><TargetName>libaprutil-1</TargetName></PropertyGroup>"
> in srclib/aprutil/libaprutil.vcxproj
> Applied fix from http://marc.info/?l=apr-dev&m=138386228311375&w=2
> (see also https://www.apachelounge.com/viewtopic.php?t=6174) --
> apparently otherwise there is a problem building httpd 2.4.10+ on
> Windows 64-bit.
>
> For a debug build:
> nmake /f Makefile.win installd INSTDIR=C:\Apache24
>
> For a release build:
> nmake /f Makefile.win installr INSTDIR=C:\Apache24
> (for some reason, the build sometimes fails ... rerunning it usually
> succeeds, don't know why)
>
> serf:
> scons APR=..\httpd-2.4.12\srclib\apr
> APU=..\httpd-2.4.12\srclib\apr-util OPENSSL=..\openssl-1.0.2
> ZLIB=..\zlib-1.2.8
> ]]]
>
> HTH,
Re: building SVN trunk on Windows fails on python gen-make.py
Posted by Johan Corveleyn <jc...@gmail.com>.
On Wed, Apr 22, 2015 at 6:13 PM, Bert Huijben <be...@qqmail.nl> wrote:
>
>
>> -----Original Message-----
>> From: Stefan Hett [mailto:stefan@egosoft.com]
>> Sent: woensdag 22 april 2015 15:59
>> To: 'subversion'
>> Subject: building SVN trunk on Windows fails on python gen-make.py
>>
>> Hi,
>>
>> I just tried to build
>> https://svn.apache.org/repos/asf/subversion/branches/svn-mergeinfo-
>> normalizer
>> on Windows following the INSTALL file documentation up to the point
>> where the make file is being generated.
>>
>> Using this call: python gen-make.py -t vcproj --with-zlib=..\zlib
>> --with-apr=..\apr --with-apr-util=..\apr-util --vsnet-version=2010
>> I'm now getting the following error:
>>
>> Traceback (most recent call last):
>> File "gen-make.py", line 325, in <module> main(conf, gentype,
>> skip_depends=skip, other_options=rest.list)
>> File "gen-make.py", line 62, in main generator =
>> gen_module.Generator(fname, verfname, other_options)
>> File "build\generator\gen_vcnet_vcproj.py", line 36, in __init__
>> 'vcnet-vcproj')
>> File "build\generator\gen_win.py", line 83, in __init__
>> self.find_libraries(True)
>> File "build\generator\gen_win_dependencies.py", line 286, in
>> find_libraries self._find_apr()
>> File "build\generator\gen_win_dependencies.py", line 395, in
>> _find_apr bin_files = os.listdir(dll_dir)
>> FileNotFoundError: [WinError 3] The system cannot find the path
>> specified: '..\\apr\\bin'
>>
>> Looking at the apr-directory I downloaded/extracted, there's no
>> apr/bin-directory.
>> I downloaded the following APR-source package (1.5.1):
>> http://mirror.serversupportforum.de/apache//apr/apr-1.5.1-win32-src.zip
>>
>> Did I get the wrong one? Or did I miss some step?
>
> Subversion expects that you already build apr before using it as a source of binaries to link against. It is not going to build apr for you.
>
> Building APR would have created this bin directory. (A patch to produce a nicer error message would be welcome :))
>
>
> There are other scripts that may help you building Subversion and all dependencies. I maintain one such script as part of SharpSvn (Repository: https://ctf.open.collab.net/svn/repos/sharpsvn/trunk/imports ; username guest; no password). This script uses NAnt, and is used to produce the publicly available binaries for SharpSvn and SlikSvn.
>
> Somewhere in the Subversion repository there is a perl script that should also be able to help with building on Windows, but I don't have experience with that script.
>
Indeed, that script is tools/dev/build-svn-deps-win.pl. But it's a bit
outdated now. I recently used bits and pieces of it to get myself a
new working build environment for svn on Windows. In general our tools
and documentation for building svn on Windows are quite outdated and
could really use some love. For instance, INSTALL still contains a lot
of old irrelevant information, it should primarily focus on current
tools that people install these days (e.g. focusing on VS 2012, 2013
and 2015).
With my last build-setup attempt, I kept a short log of what I did
(based on things from build-svn-deps-win.pl, and further trial and
error). Perhaps this is useful:
[[[
download awk95.exe and rename it to awk.exe and put somewhere in PATH
(as documented in INSTALL)
download and install:
- CMake
- SCons
- Python
- Perl
Install Visual Studio (I used 2013).
Download latest (or recent enough stable) sources of:
- zlib
- openssl
- sqlite
- serf
- httpd
- apr
- apr-util
- apr-iconv
- pcre
Extract these in a "deps" directory (I usually name the individual
dirs with the version number, so I have C:\svn\deps\httpd-2.4.12,
C:\svn\deps\zlib-1.2.8, ...).
Except: put apr, apr-util, apr-iconv and pcre as subdirectories
(without version number in the name) inside $HTTPD_SOURCE\srclib.
Then start building. Do all the following from inside a Visual Studio
Command Prompt:
zlib:
For Debug build:
set CC_OPTS=/MDd /Gm /ZI /Od /GZ /D_DEBUG
set COMMON_CC_OPTS=/nologo /W3 /DWIN32 /D_WINDOWS
For Release build:
set CC_OPTS=/MD /02 /Zi
set COMMON_CC_OPTS=/nologo /W3 /DWIN32 /D_WINDOWS
Somehow, you need to make both a static build (for svn itself) and a
dynamic build (for serf):
Static:
nmake /nologo -f win32/Makefile.msc STATICLIB=zlibstat.lib all
And for serf:
nmake /nologo -f win32/Makefile.msc all
unset CC_OPTS
unset COMMON_CC_OPTS
PCRE:
(replace RelWithDebInfo with Debug in the line below to make a debug build)
cmake -G "NMake Makefiles" -DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo
-DBUILD_SHARED_LIBS:BOOL=ON
-DCMAKE_INSTALL_PREFIX:PATH=C:\svn\deps\install
-DPCRE_NO_RECURSE:BOOL=ON .
nmake
nmake test
nmake install
openssl:
(replace VC-WIN32 with debug-VC-WIN32 to make a debug build)
perl Configure no-asm --prefix=C:\svn\deps\install VC-WIN32
call ms\do_ms
nmake -f ms\ntdll.mak
nmake -f ms\ntdll.mak test
nmake -f ms\ntdll.mak install
httpd (+ apr, apr-util, apr-iconv):
perl srclib\apr\build\lineends.pl
perl srclib\apr\build\cvtdsp.pl -2005
Strip cases of "/YuXXXX" en "/YcXXXX" from
srclib/apr-iconv/build/modules.mk.win
Insert "<PropertyGroup><GenerateManifest>false</GenerateManifest></PropertyGroup>"
in support/win32/ApacheMonitor.vcxproj
Insert "<PropertyGroup><TargetName>libapr-1</TargetName></PropertyGroup>"
in srclib/apr/libapr.vcxproj
Insert "<PropertyGroup><TargetName>apr_ldap-1</TargetName></PropertyGroup>"
in srclib/aprutil/ldap/apr_ldap.vcxproj
Insert "<PropertyGroup><TargetName>libapriconv-1</TargetName></PropertyGroup>"
in srclib/apr-iconv/libapriconv.vcxproj
Insert "<PropertyGroup><TargetName>libaprutil-1</TargetName></PropertyGroup>"
in srclib/aprutil/libaprutil.vcxproj
Applied fix from http://marc.info/?l=apr-dev&m=138386228311375&w=2
(see also https://www.apachelounge.com/viewtopic.php?t=6174) --
apparently otherwise there is a problem building httpd 2.4.10+ on
Windows 64-bit.
For a debug build:
nmake /f Makefile.win installd INSTDIR=C:\Apache24
For a release build:
nmake /f Makefile.win installr INSTDIR=C:\Apache24
(for some reason, the build sometimes fails ... rerunning it usually
succeeds, don't know why)
serf:
scons APR=..\httpd-2.4.12\srclib\apr
APU=..\httpd-2.4.12\srclib\apr-util OPENSSL=..\openssl-1.0.2
ZLIB=..\zlib-1.2.8
]]]
HTH,
--
Johan
RE: building SVN trunk on Windows fails on python gen-make.py
Posted by Bert Huijben <be...@qqmail.nl>.
> -----Original Message-----
> From: Stefan Hett [mailto:stefan@egosoft.com]
> Sent: woensdag 22 april 2015 15:59
> To: 'subversion'
> Subject: building SVN trunk on Windows fails on python gen-make.py
>
> Hi,
>
> I just tried to build
> https://svn.apache.org/repos/asf/subversion/branches/svn-mergeinfo-
> normalizer
> on Windows following the INSTALL file documentation up to the point
> where the make file is being generated.
>
> Using this call: python gen-make.py -t vcproj --with-zlib=..\zlib
> --with-apr=..\apr --with-apr-util=..\apr-util --vsnet-version=2010
> I'm now getting the following error:
>
> Traceback (most recent call last):
> File "gen-make.py", line 325, in <module> main(conf, gentype,
> skip_depends=skip, other_options=rest.list)
> File "gen-make.py", line 62, in main generator =
> gen_module.Generator(fname, verfname, other_options)
> File "build\generator\gen_vcnet_vcproj.py", line 36, in __init__
> 'vcnet-vcproj')
> File "build\generator\gen_win.py", line 83, in __init__
> self.find_libraries(True)
> File "build\generator\gen_win_dependencies.py", line 286, in
> find_libraries self._find_apr()
> File "build\generator\gen_win_dependencies.py", line 395, in
> _find_apr bin_files = os.listdir(dll_dir)
> FileNotFoundError: [WinError 3] The system cannot find the path
> specified: '..\\apr\\bin'
>
> Looking at the apr-directory I downloaded/extracted, there's no
> apr/bin-directory.
> I downloaded the following APR-source package (1.5.1):
> http://mirror.serversupportforum.de/apache//apr/apr-1.5.1-win32-src.zip
>
> Did I get the wrong one? Or did I miss some step?
Subversion expects that you already build apr before using it as a source of binaries to link against. It is not going to build apr for you.
Building APR would have created this bin directory. (A patch to produce a nicer error message would be welcome :))
There are other scripts that may help you building Subversion and all dependencies. I maintain one such script as part of SharpSvn (Repository: https://ctf.open.collab.net/svn/repos/sharpsvn/trunk/imports ; username guest; no password). This script uses NAnt, and is used to produce the publicly available binaries for SharpSvn and SlikSvn.
Somewhere in the Subversion repository there is a perl script that should also be able to help with building on Windows, but I don't have experience with that script.
Bert
Re: building SVN trunk on Windows fails on python gen-make.py
Posted by Stefan Hett <st...@egosoft.com>.
The branch is based on the latest trunk. The only difference in
gen_win.py is in line 70, where on trunk there's the call to
generator.write_config_keys() which is missing on the branch-version.
I didn't try adding this call tough, since I found it kinda unlikely
that this is the cause of the problem (I rather suspect some set-up
issue on my side, which I can't just see atm).
> Hi,
>
> I just tried to build
> https://svn.apache.org/repos/asf/subversion/branches/svn-mergeinfo-normalizer
> on Windows following the INSTALL file documentation up to the point
> where the make file is being generated.
>
> Using this call: python gen-make.py -t vcproj --with-zlib=..\zlib
> --with-apr=..\apr --with-apr-util=..\apr-util --vsnet-version=2010
> I'm now getting the following error:
>
> Traceback (most recent call last):
> File "gen-make.py", line 325, in <module> main(conf, gentype,
> skip_depends=skip, other_options=rest.list)
> File "gen-make.py", line 62, in main generator =
> gen_module.Generator(fname, verfname, other_options)
> File "build\generator\gen_vcnet_vcproj.py", line 36, in __init__
> 'vcnet-vcproj')
> File "build\generator\gen_win.py", line 83, in __init__
> self.find_libraries(True)
> File "build\generator\gen_win_dependencies.py", line 286, in
> find_libraries self._find_apr()
> File "build\generator\gen_win_dependencies.py", line 395, in
> _find_apr bin_files = os.listdir(dll_dir)
> FileNotFoundError: [WinError 3] The system cannot find the path
> specified: '..\\apr\\bin'
>
> Looking at the apr-directory I downloaded/extracted, there's no
> apr/bin-directory.
> I downloaded the following APR-source package (1.5.1):
> http://mirror.serversupportforum.de/apache//apr/apr-1.5.1-win32-src.zip
>
> Did I get the wrong one? Or did I miss some step?
>
> Regards,
> Stefan
>