You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@serf.apache.org by Johan Corveleyn <jc...@gmail.com> on 2020/04/19 13:47:31 UTC
Serf build problems on Windows
Hi serf devs,
I've been building Subversion 1.14.0-rc2 on Windows last week, and for
doing that I needed to build serf again (1.3.9, but I also tried from
branch 1.4.x). I ran into three different problems during that build.
I worked around them, but wanted to report them here so perhaps the
serf build scripts can be improved.
I have scons 3.1.2 and Visual Studio 2019 Community Edition (Version 16.5.3).
I'm running this command from within the serf-1.3.9 directory, after I
built apr (1.6.5), apr-util (1.6.1), openssl (1.1.1f) and zlib
(1.2.11):
scons APR=..\apr-1.6.5 APU=..\apr-util-1.6.1
OPENSSL=..\openssl-1.1.1f ZLIB=..\zlib-1.2.11
1) Running this with Python 3.8.2 gives this error:
[[[
scons: Reading SConscript files ...
File "C:\research\svn\dev\deps\serf-1.3.9\SConstruct", line 186
print 'Warning: Used unknown variables:', ', '.join(unknown.keys())
^
SyntaxError: invalid syntax
]]]
Putting Python 2.7 in front in my PATH allows it to continue.
2) Now I get this error:
[[[
scons: Reading SConscript files ...
scons: *** Invalid value for option MSVC_VERSION: 14.2. Valid values
are: ('14.0', '12.0', '11.0', '10.0', '9.0', '8.0', '6.0')
File "C:\research\svn\dev\deps\serf-1.3.9\SConstruct", line 157, in <module>
]]]
Interestingly, if I delete the file .saved_config (which actually
contains the value "MSVC_VERSION = '14.2'"), scons will continue (and
save a new .saved_config file ... i.e. I can always run it if I just
remember to delete that file first).
3) Now, linking with openssl 1.1.1f fails:
[[[
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
cl /Fobuckets\ssl_buckets.obj /c buckets\ssl_buckets.c /nologo /W4
/wd4100 /O2 /MD /DNDEBUG /DWIN32 /DWIN32_LEAN_AND_MEAN /DNOUSER
/DNOGDI /DNONLS /DNOCRYPT /DSERF_HAVE_SSPI /I.
/IC:\research\svn\dev\deps\httpd-2.4.43\srclib\apr\include
/IC:\research\svn\dev\deps\httpd-2.4.43\srclib\apr-util\include
/IC:\research\svn\dev\deps\zlib-1.2.11
/IC:\research\svn\dev\deps\openssl-1.1.1f\inc32 /Z7
ssl_buckets.c
buckets\ssl_buckets.c(37): fatal error C1083: Cannot open include
file: 'openssl/bio.h': No such file or directory
scons: *** [buckets\ssl_buckets.obj] Error 2
scons: building terminated because of errors.
]]]
The reason is that, as of openssl 1.1.x:
* headers are in 'include' rather than 'inc32'
* libs come in the root folder of the openssl folder instead of 'out32'
* libs are now named libcrypto.lib and libssl.lib instead of
libeay32.lib and ssleay32.lib
If I patch SConstruct as follows then I can link successfully with
openssl 1.1.1.f:
[[[
--- SConstruct.orig 2020-04-19 15:36:16.257450600 +0200
+++ SConstruct 2020-04-19 15:36:51.855740800 +0200
@@ -335,7 +335,7 @@
LIBPATH=['$ZLIB'])
# openssl
- env.Append(LIBS=['libeay32.lib', 'ssleay32.lib'])
+ env.Append(LIBS=['libcrypto.lib', 'libssl.lib'])
if not env.get('SOURCE_LAYOUT', None):
env.Append(CPPPATH=['$OPENSSL/include/openssl'],
LIBPATH=['$OPENSSL/lib'])
@@ -343,8 +343,8 @@
env.Append(CPPPATH=['$OPENSSL/inc32'],
LIBPATH=['$OPENSSL/out32'])
else:
- env.Append(CPPPATH=['$OPENSSL/inc32'],
- LIBPATH=['$OPENSSL/out32dll'])
+ env.Append(CPPPATH=['$OPENSSL/include'],
+ LIBPATH=['$OPENSSL'])
else:
if os.path.isdir(apr):
apr = os.path.join(apr, 'bin', 'apr-1-config')
]]]
But then it won't work with openssl 1.0.x anymore. So I'm not sure,
perhaps the openssl version should be detected and the appropriate
configuration should be set accordingly.
For some inspiration:
- the httpd build requires one to explicitly run:
perl srclib\apr\build\cvtdsp.pl -ossl11
to build with openssl 1.1.x. So not auto-adapting, but at least it's
documented and there is an easy script.
- the subversion build auto-detects the openssl version and adapts the
necessary include paths and library filenames accordingly. See
function _find_openssl() in
https://svn.apache.org/repos/asf/subversion/trunk/build/generator/gen_win_dependencies.py
After fixing / working around the above three problems I can
successfully build serf 1.3.9, phew :-).
Thanks,
--
Johan
Re: Serf build problems on Windows
Posted by Branko Čibej <br...@apache.org>.
On 20.04.2020 15:44, Johan Corveleyn wrote:
> On Mon, Apr 20, 2020 at 2:10 PM Justin Erenkrantz <ju...@erenkrantz.com> wrote:
>> Great. I'll stay the course on prepping 1.4.x...if anyone pops up here asking for 1.3.x and VS 2019, we'll just point them at your email. =)
> Perfect :-).
>
>> I know APR, APR-util, and Expat all have CMake files already - I haven't seen anything yet for SVN...am I just missing that?
> Ah, I don't think there is anything CMake-y in SVN yet.
There is not. Converting to CMake is less than trivial, especially since
there are a *lot* of dependencies that either do not use CMake at all,
or use it incorrectly (e.g., the FindApr macros is currently in the Serf
repo, not the APR repo ...). There's a reason why the Serf CMake build
doesn't support BROTLI and GSSAPI. :(
Personally I hope that a better build system comes along (and people
recognize CMake for the monstrosity it is) before we start converting
SVN away from autotools. :)
-- Brane
P.S.: Wrong list, couldn't resist.
Re: Serf build problems on Windows
Posted by Johan Corveleyn <jc...@gmail.com>.
On Mon, Apr 20, 2020 at 2:10 PM Justin Erenkrantz <ju...@erenkrantz.com> wrote:
>
> Great. I'll stay the course on prepping 1.4.x...if anyone pops up here asking for 1.3.x and VS 2019, we'll just point them at your email. =)
Perfect :-).
> I know APR, APR-util, and Expat all have CMake files already - I haven't seen anything yet for SVN...am I just missing that?
Ah, I don't think there is anything CMake-y in SVN yet.
> I've gotten as far as building APR, APR-util, Expat, and OpenSSL with VS 2019 on my Windows box; but, I haven't sat down yet and gotten Serf itself to build. Ideally, after Serf, it'd be good for SVN to pick up CMake too...would also like to pick up httpd too if I could. But, toddler steps... =)
>
That would be great. Whatever helps making these builds (on all
platforms) easier.
FWIW, I started a thread on dev@subversion about "Building SVN
(dependencies) on Windows" (with a dump of my "notes" during the
process, and a pointer to this very thread on dev@serf):
https://lists.apache.org/thread.html/r59a30aabaab7bf69effa909b331eaa177418325280ea25859e8fa294%40%3Cdev.subversion.apache.org%3E
Feel free to float the CMake idea there (or in a separate thread if you prefer).
Thanks,
--
Johan
Re: Serf build problems on Windows
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
Great. I'll stay the course on prepping 1.4.x...if anyone pops up here
asking for 1.3.x and VS 2019, we'll just point them at your email. =)
I know APR, APR-util, and Expat all have CMake files already - I haven't
seen anything yet for SVN...am I just missing that? I've gotten as far as
building APR, APR-util, Expat, and OpenSSL with VS 2019 on my Windows box;
but, I haven't sat down yet and gotten Serf itself to build. Ideally,
after Serf, it'd be good for SVN to pick up CMake too...would also like to
pick up httpd too if I could. But, toddler steps... =)
Cheers. -- justin
On Mon, Apr 20, 2020 at 7:59 AM Johan Corveleyn <jc...@gmail.com> wrote:
> Hi Justin,
>
> Ah, I hadn't checked the 1.3.x branch. Good to know that Python3 for
> the build is supported there. :-). And I didn't see the things in
> 1.3.x/STATUS.
>
> I can certainly sympathize with trying to find the right focus while
> having (very) limited resources. So I think getting 1.4 out the door
> is more important. And if it doesn't have those problems (and the
> CMake build works on Windows), great!
> For me personally the problem is already solved (I found the necessary
> workarounds, and got SVN built with serf 1.3.9 combined with openssl
> 1.1.1f). If you (or anyone in dev@serf) feel like it, you might
> perform some last fixes or document the kwown issues / workarounds in
> README on the 1.3.x branch. That might still be a useful improvement,
> even if you don't cut a new 1.3.10 anymore. But if you don't get
> around to it, no problem ... 1.4.x is probably the way forward in any
> case.
>
> As soon as you cut a 1.4 release, with a CMake build that would work
> on Windows (with VS 2019), I will certainly try it for my next SVN
> build. I'm subscribed to this list now, so I should be able to see the
> announcement :-).
>
> Thanks for your efforts,
> --
> Johan
>
> On Mon, Apr 20, 2020 at 1:34 PM Justin Erenkrantz <ju...@erenkrantz.com>
> wrote:
> >
> > Johan,
> >
> > Thanks for the note!
> >
> > The Python3 patches should already be in 1.3.x branch ready for our next
> release. We've also got 2 +1s in STATUS for VS 2017 support for 1.3.x, but
> I've yet to be able to confirm it for 2017 myself and also there's now 2019
> as well...so, I've held off merging that change. And, I haven't yet gotten
> as far as trying to compile with OpenSSL 1.1.x either on Windows; so, I
> expect if I got to that, your patch is very helpful for SCons.
> >
> > However, I posted a few weeks ago on-list and there wasn't a lot of
> interest in making a new 1.3.x release; so, I've been focusing on the 1.4.x
> branch which switches over to CMake as that seems to be where the world is
> going now. Most of the issues that we have right now seem to stem from
> Python and our creaky build system - not actual serf bugs. Though it is a
> bit of a slog for me to get CMake up, so I don't have a firm ETA on a
> release there.
> >
> > If we can muster two other interested folks for a 1.3.10 release, I
> think that I can shift gears a bit and focus on completing the 1.3.x stuff
> which should include most of the relevant build fixes for modern
> toolchains...but, it's probably not the right focus long-term as there's a
> bunch of cool features in 1.4.x that would love to see the light of day.
> >
> > Cheers. -- justin
> >
> > On Sun, Apr 19, 2020 at 9:47 AM Johan Corveleyn <jc...@gmail.com>
> wrote:
> >>
> >> Hi serf devs,
> >>
> >> I've been building Subversion 1.14.0-rc2 on Windows last week, and for
> >> doing that I needed to build serf again (1.3.9, but I also tried from
> >> branch 1.4.x). I ran into three different problems during that build.
> >> I worked around them, but wanted to report them here so perhaps the
> >> serf build scripts can be improved.
> >>
> >> I have scons 3.1.2 and Visual Studio 2019 Community Edition (Version
> 16.5.3).
> >>
> >> I'm running this command from within the serf-1.3.9 directory, after I
> >> built apr (1.6.5), apr-util (1.6.1), openssl (1.1.1f) and zlib
> >> (1.2.11):
> >>
> >> scons APR=..\apr-1.6.5 APU=..\apr-util-1.6.1
> >> OPENSSL=..\openssl-1.1.1f ZLIB=..\zlib-1.2.11
> >>
> >> 1) Running this with Python 3.8.2 gives this error:
> >>
> >> [[[
> >> scons: Reading SConscript files ...
> >> File "C:\research\svn\dev\deps\serf-1.3.9\SConstruct", line 186
> >>
> >> print 'Warning: Used unknown variables:', ', '.join(unknown.keys())
> >>
> >> ^
> >>
> >> SyntaxError: invalid syntax
> >> ]]]
> >>
> >> Putting Python 2.7 in front in my PATH allows it to continue.
> >>
> >>
> >> 2) Now I get this error:
> >>
> >> [[[
> >> scons: Reading SConscript files ...
> >>
> >> scons: *** Invalid value for option MSVC_VERSION: 14.2. Valid values
> >> are: ('14.0', '12.0', '11.0', '10.0', '9.0', '8.0', '6.0')
> >> File "C:\research\svn\dev\deps\serf-1.3.9\SConstruct", line 157, in
> <module>
> >> ]]]
> >>
> >> Interestingly, if I delete the file .saved_config (which actually
> >> contains the value "MSVC_VERSION = '14.2'"), scons will continue (and
> >> save a new .saved_config file ... i.e. I can always run it if I just
> >> remember to delete that file first).
> >>
> >>
> >> 3) Now, linking with openssl 1.1.1f fails:
> >>
> >> [[[
> >> scons: Reading SConscript files ...
> >> scons: done reading SConscript files.
> >> scons: Building targets ...
> >> cl /Fobuckets\ssl_buckets.obj /c buckets\ssl_buckets.c /nologo /W4
> >> /wd4100 /O2 /MD /DNDEBUG /DWIN32 /DWIN32_LEAN_AND_MEAN /DNOUSER
> >> /DNOGDI /DNONLS /DNOCRYPT /DSERF_HAVE_SSPI /I.
> >> /IC:\research\svn\dev\deps\httpd-2.4.43\srclib\apr\include
> >> /IC:\research\svn\dev\deps\httpd-2.4.43\srclib\apr-util\include
> >> /IC:\research\svn\dev\deps\zlib-1.2.11
> >> /IC:\research\svn\dev\deps\openssl-1.1.1f\inc32 /Z7
> >> ssl_buckets.c
> >> buckets\ssl_buckets.c(37): fatal error C1083: Cannot open include
> >> file: 'openssl/bio.h': No such file or directory
> >> scons: *** [buckets\ssl_buckets.obj] Error 2
> >> scons: building terminated because of errors.
> >> ]]]
> >>
> >> The reason is that, as of openssl 1.1.x:
> >> * headers are in 'include' rather than 'inc32'
> >> * libs come in the root folder of the openssl folder instead of 'out32'
> >> * libs are now named libcrypto.lib and libssl.lib instead of
> >> libeay32.lib and ssleay32.lib
> >>
> >> If I patch SConstruct as follows then I can link successfully with
> >> openssl 1.1.1.f:
> >>
> >> [[[
> >> --- SConstruct.orig 2020-04-19 15:36:16.257450600 +0200
> >> +++ SConstruct 2020-04-19 15:36:51.855740800 +0200
> >> @@ -335,7 +335,7 @@
> >> LIBPATH=['$ZLIB'])
> >>
> >> # openssl
> >> - env.Append(LIBS=['libeay32.lib', 'ssleay32.lib'])
> >> + env.Append(LIBS=['libcrypto.lib', 'libssl.lib'])
> >> if not env.get('SOURCE_LAYOUT', None):
> >> env.Append(CPPPATH=['$OPENSSL/include/openssl'],
> >> LIBPATH=['$OPENSSL/lib'])
> >> @@ -343,8 +343,8 @@
> >> env.Append(CPPPATH=['$OPENSSL/inc32'],
> >> LIBPATH=['$OPENSSL/out32'])
> >> else:
> >> - env.Append(CPPPATH=['$OPENSSL/inc32'],
> >> - LIBPATH=['$OPENSSL/out32dll'])
> >> + env.Append(CPPPATH=['$OPENSSL/include'],
> >> + LIBPATH=['$OPENSSL'])
> >> else:
> >> if os.path.isdir(apr):
> >> apr = os.path.join(apr, 'bin', 'apr-1-config')
> >> ]]]
> >>
> >> But then it won't work with openssl 1.0.x anymore. So I'm not sure,
> >> perhaps the openssl version should be detected and the appropriate
> >> configuration should be set accordingly.
> >>
> >> For some inspiration:
> >>
> >> - the httpd build requires one to explicitly run:
> >> perl srclib\apr\build\cvtdsp.pl -ossl11
> >> to build with openssl 1.1.x. So not auto-adapting, but at least it's
> >> documented and there is an easy script.
> >>
> >> - the subversion build auto-detects the openssl version and adapts the
> >> necessary include paths and library filenames accordingly. See
> >> function _find_openssl() in
> >>
> https://svn.apache.org/repos/asf/subversion/trunk/build/generator/gen_win_dependencies.py
> >>
> >>
> >> After fixing / working around the above three problems I can
> >> successfully build serf 1.3.9, phew :-).
> >>
> >> Thanks,
> >> --
> >> Johan
>
Re: Serf build problems on Windows
Posted by Johan Corveleyn <jc...@gmail.com>.
Hi Justin,
Ah, I hadn't checked the 1.3.x branch. Good to know that Python3 for
the build is supported there. :-). And I didn't see the things in
1.3.x/STATUS.
I can certainly sympathize with trying to find the right focus while
having (very) limited resources. So I think getting 1.4 out the door
is more important. And if it doesn't have those problems (and the
CMake build works on Windows), great!
For me personally the problem is already solved (I found the necessary
workarounds, and got SVN built with serf 1.3.9 combined with openssl
1.1.1f). If you (or anyone in dev@serf) feel like it, you might
perform some last fixes or document the kwown issues / workarounds in
README on the 1.3.x branch. That might still be a useful improvement,
even if you don't cut a new 1.3.10 anymore. But if you don't get
around to it, no problem ... 1.4.x is probably the way forward in any
case.
As soon as you cut a 1.4 release, with a CMake build that would work
on Windows (with VS 2019), I will certainly try it for my next SVN
build. I'm subscribed to this list now, so I should be able to see the
announcement :-).
Thanks for your efforts,
--
Johan
On Mon, Apr 20, 2020 at 1:34 PM Justin Erenkrantz <ju...@erenkrantz.com> wrote:
>
> Johan,
>
> Thanks for the note!
>
> The Python3 patches should already be in 1.3.x branch ready for our next release. We've also got 2 +1s in STATUS for VS 2017 support for 1.3.x, but I've yet to be able to confirm it for 2017 myself and also there's now 2019 as well...so, I've held off merging that change. And, I haven't yet gotten as far as trying to compile with OpenSSL 1.1.x either on Windows; so, I expect if I got to that, your patch is very helpful for SCons.
>
> However, I posted a few weeks ago on-list and there wasn't a lot of interest in making a new 1.3.x release; so, I've been focusing on the 1.4.x branch which switches over to CMake as that seems to be where the world is going now. Most of the issues that we have right now seem to stem from Python and our creaky build system - not actual serf bugs. Though it is a bit of a slog for me to get CMake up, so I don't have a firm ETA on a release there.
>
> If we can muster two other interested folks for a 1.3.10 release, I think that I can shift gears a bit and focus on completing the 1.3.x stuff which should include most of the relevant build fixes for modern toolchains...but, it's probably not the right focus long-term as there's a bunch of cool features in 1.4.x that would love to see the light of day.
>
> Cheers. -- justin
>
> On Sun, Apr 19, 2020 at 9:47 AM Johan Corveleyn <jc...@gmail.com> wrote:
>>
>> Hi serf devs,
>>
>> I've been building Subversion 1.14.0-rc2 on Windows last week, and for
>> doing that I needed to build serf again (1.3.9, but I also tried from
>> branch 1.4.x). I ran into three different problems during that build.
>> I worked around them, but wanted to report them here so perhaps the
>> serf build scripts can be improved.
>>
>> I have scons 3.1.2 and Visual Studio 2019 Community Edition (Version 16.5.3).
>>
>> I'm running this command from within the serf-1.3.9 directory, after I
>> built apr (1.6.5), apr-util (1.6.1), openssl (1.1.1f) and zlib
>> (1.2.11):
>>
>> scons APR=..\apr-1.6.5 APU=..\apr-util-1.6.1
>> OPENSSL=..\openssl-1.1.1f ZLIB=..\zlib-1.2.11
>>
>> 1) Running this with Python 3.8.2 gives this error:
>>
>> [[[
>> scons: Reading SConscript files ...
>> File "C:\research\svn\dev\deps\serf-1.3.9\SConstruct", line 186
>>
>> print 'Warning: Used unknown variables:', ', '.join(unknown.keys())
>>
>> ^
>>
>> SyntaxError: invalid syntax
>> ]]]
>>
>> Putting Python 2.7 in front in my PATH allows it to continue.
>>
>>
>> 2) Now I get this error:
>>
>> [[[
>> scons: Reading SConscript files ...
>>
>> scons: *** Invalid value for option MSVC_VERSION: 14.2. Valid values
>> are: ('14.0', '12.0', '11.0', '10.0', '9.0', '8.0', '6.0')
>> File "C:\research\svn\dev\deps\serf-1.3.9\SConstruct", line 157, in <module>
>> ]]]
>>
>> Interestingly, if I delete the file .saved_config (which actually
>> contains the value "MSVC_VERSION = '14.2'"), scons will continue (and
>> save a new .saved_config file ... i.e. I can always run it if I just
>> remember to delete that file first).
>>
>>
>> 3) Now, linking with openssl 1.1.1f fails:
>>
>> [[[
>> scons: Reading SConscript files ...
>> scons: done reading SConscript files.
>> scons: Building targets ...
>> cl /Fobuckets\ssl_buckets.obj /c buckets\ssl_buckets.c /nologo /W4
>> /wd4100 /O2 /MD /DNDEBUG /DWIN32 /DWIN32_LEAN_AND_MEAN /DNOUSER
>> /DNOGDI /DNONLS /DNOCRYPT /DSERF_HAVE_SSPI /I.
>> /IC:\research\svn\dev\deps\httpd-2.4.43\srclib\apr\include
>> /IC:\research\svn\dev\deps\httpd-2.4.43\srclib\apr-util\include
>> /IC:\research\svn\dev\deps\zlib-1.2.11
>> /IC:\research\svn\dev\deps\openssl-1.1.1f\inc32 /Z7
>> ssl_buckets.c
>> buckets\ssl_buckets.c(37): fatal error C1083: Cannot open include
>> file: 'openssl/bio.h': No such file or directory
>> scons: *** [buckets\ssl_buckets.obj] Error 2
>> scons: building terminated because of errors.
>> ]]]
>>
>> The reason is that, as of openssl 1.1.x:
>> * headers are in 'include' rather than 'inc32'
>> * libs come in the root folder of the openssl folder instead of 'out32'
>> * libs are now named libcrypto.lib and libssl.lib instead of
>> libeay32.lib and ssleay32.lib
>>
>> If I patch SConstruct as follows then I can link successfully with
>> openssl 1.1.1.f:
>>
>> [[[
>> --- SConstruct.orig 2020-04-19 15:36:16.257450600 +0200
>> +++ SConstruct 2020-04-19 15:36:51.855740800 +0200
>> @@ -335,7 +335,7 @@
>> LIBPATH=['$ZLIB'])
>>
>> # openssl
>> - env.Append(LIBS=['libeay32.lib', 'ssleay32.lib'])
>> + env.Append(LIBS=['libcrypto.lib', 'libssl.lib'])
>> if not env.get('SOURCE_LAYOUT', None):
>> env.Append(CPPPATH=['$OPENSSL/include/openssl'],
>> LIBPATH=['$OPENSSL/lib'])
>> @@ -343,8 +343,8 @@
>> env.Append(CPPPATH=['$OPENSSL/inc32'],
>> LIBPATH=['$OPENSSL/out32'])
>> else:
>> - env.Append(CPPPATH=['$OPENSSL/inc32'],
>> - LIBPATH=['$OPENSSL/out32dll'])
>> + env.Append(CPPPATH=['$OPENSSL/include'],
>> + LIBPATH=['$OPENSSL'])
>> else:
>> if os.path.isdir(apr):
>> apr = os.path.join(apr, 'bin', 'apr-1-config')
>> ]]]
>>
>> But then it won't work with openssl 1.0.x anymore. So I'm not sure,
>> perhaps the openssl version should be detected and the appropriate
>> configuration should be set accordingly.
>>
>> For some inspiration:
>>
>> - the httpd build requires one to explicitly run:
>> perl srclib\apr\build\cvtdsp.pl -ossl11
>> to build with openssl 1.1.x. So not auto-adapting, but at least it's
>> documented and there is an easy script.
>>
>> - the subversion build auto-detects the openssl version and adapts the
>> necessary include paths and library filenames accordingly. See
>> function _find_openssl() in
>> https://svn.apache.org/repos/asf/subversion/trunk/build/generator/gen_win_dependencies.py
>>
>>
>> After fixing / working around the above three problems I can
>> successfully build serf 1.3.9, phew :-).
>>
>> Thanks,
>> --
>> Johan
Re: Serf build problems on Windows
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
Johan,
Thanks for the note!
The Python3 patches should already be in 1.3.x branch ready for our next
release. We've also got 2 +1s in STATUS for VS 2017 support for 1.3.x, but
I've yet to be able to confirm it for 2017 myself and also there's now 2019
as well...so, I've held off merging that change. And, I haven't yet gotten
as far as trying to compile with OpenSSL 1.1.x either on Windows; so, I
expect if I got to that, your patch is very helpful for SCons.
However, I posted a few weeks ago on-list and there wasn't a lot of
interest in making a new 1.3.x release; so, I've been focusing on the 1.4.x
branch which switches over to CMake as that seems to be where the world is
going now. Most of the issues that we have right now seem to stem from
Python and our creaky build system - not actual serf bugs. Though it is a
bit of a slog for me to get CMake up, so I don't have a firm ETA on a
release there.
If we can muster two other interested folks for a 1.3.10 release, I think
that I can shift gears a bit and focus on completing the 1.3.x stuff which
should include most of the relevant build fixes for modern
toolchains...but, it's probably not the right focus long-term as there's a
bunch of cool features in 1.4.x that would love to see the light of day.
Cheers. -- justin
On Sun, Apr 19, 2020 at 9:47 AM Johan Corveleyn <jc...@gmail.com> wrote:
> Hi serf devs,
>
> I've been building Subversion 1.14.0-rc2 on Windows last week, and for
> doing that I needed to build serf again (1.3.9, but I also tried from
> branch 1.4.x). I ran into three different problems during that build.
> I worked around them, but wanted to report them here so perhaps the
> serf build scripts can be improved.
>
> I have scons 3.1.2 and Visual Studio 2019 Community Edition (Version
> 16.5.3).
>
> I'm running this command from within the serf-1.3.9 directory, after I
> built apr (1.6.5), apr-util (1.6.1), openssl (1.1.1f) and zlib
> (1.2.11):
>
> scons APR=..\apr-1.6.5 APU=..\apr-util-1.6.1
> OPENSSL=..\openssl-1.1.1f ZLIB=..\zlib-1.2.11
>
> 1) Running this with Python 3.8.2 gives this error:
>
> [[[
> scons: Reading SConscript files ...
> File "C:\research\svn\dev\deps\serf-1.3.9\SConstruct", line 186
>
> print 'Warning: Used unknown variables:', ', '.join(unknown.keys())
>
> ^
>
> SyntaxError: invalid syntax
> ]]]
>
> Putting Python 2.7 in front in my PATH allows it to continue.
>
>
> 2) Now I get this error:
>
> [[[
> scons: Reading SConscript files ...
>
> scons: *** Invalid value for option MSVC_VERSION: 14.2. Valid values
> are: ('14.0', '12.0', '11.0', '10.0', '9.0', '8.0', '6.0')
> File "C:\research\svn\dev\deps\serf-1.3.9\SConstruct", line 157, in
> <module>
> ]]]
>
> Interestingly, if I delete the file .saved_config (which actually
> contains the value "MSVC_VERSION = '14.2'"), scons will continue (and
> save a new .saved_config file ... i.e. I can always run it if I just
> remember to delete that file first).
>
>
> 3) Now, linking with openssl 1.1.1f fails:
>
> [[[
> scons: Reading SConscript files ...
> scons: done reading SConscript files.
> scons: Building targets ...
> cl /Fobuckets\ssl_buckets.obj /c buckets\ssl_buckets.c /nologo /W4
> /wd4100 /O2 /MD /DNDEBUG /DWIN32 /DWIN32_LEAN_AND_MEAN /DNOUSER
> /DNOGDI /DNONLS /DNOCRYPT /DSERF_HAVE_SSPI /I.
> /IC:\research\svn\dev\deps\httpd-2.4.43\srclib\apr\include
> /IC:\research\svn\dev\deps\httpd-2.4.43\srclib\apr-util\include
> /IC:\research\svn\dev\deps\zlib-1.2.11
> /IC:\research\svn\dev\deps\openssl-1.1.1f\inc32 /Z7
> ssl_buckets.c
> buckets\ssl_buckets.c(37): fatal error C1083: Cannot open include
> file: 'openssl/bio.h': No such file or directory
> scons: *** [buckets\ssl_buckets.obj] Error 2
> scons: building terminated because of errors.
> ]]]
>
> The reason is that, as of openssl 1.1.x:
> * headers are in 'include' rather than 'inc32'
> * libs come in the root folder of the openssl folder instead of 'out32'
> * libs are now named libcrypto.lib and libssl.lib instead of
> libeay32.lib and ssleay32.lib
>
> If I patch SConstruct as follows then I can link successfully with
> openssl 1.1.1.f:
>
> [[[
> --- SConstruct.orig 2020-04-19 15:36:16.257450600 +0200
> +++ SConstruct 2020-04-19 15:36:51.855740800 +0200
> @@ -335,7 +335,7 @@
> LIBPATH=['$ZLIB'])
>
> # openssl
> - env.Append(LIBS=['libeay32.lib', 'ssleay32.lib'])
> + env.Append(LIBS=['libcrypto.lib', 'libssl.lib'])
> if not env.get('SOURCE_LAYOUT', None):
> env.Append(CPPPATH=['$OPENSSL/include/openssl'],
> LIBPATH=['$OPENSSL/lib'])
> @@ -343,8 +343,8 @@
> env.Append(CPPPATH=['$OPENSSL/inc32'],
> LIBPATH=['$OPENSSL/out32'])
> else:
> - env.Append(CPPPATH=['$OPENSSL/inc32'],
> - LIBPATH=['$OPENSSL/out32dll'])
> + env.Append(CPPPATH=['$OPENSSL/include'],
> + LIBPATH=['$OPENSSL'])
> else:
> if os.path.isdir(apr):
> apr = os.path.join(apr, 'bin', 'apr-1-config')
> ]]]
>
> But then it won't work with openssl 1.0.x anymore. So I'm not sure,
> perhaps the openssl version should be detected and the appropriate
> configuration should be set accordingly.
>
> For some inspiration:
>
> - the httpd build requires one to explicitly run:
> perl srclib\apr\build\cvtdsp.pl -ossl11
> to build with openssl 1.1.x. So not auto-adapting, but at least it's
> documented and there is an easy script.
>
> - the subversion build auto-detects the openssl version and adapts the
> necessary include paths and library filenames accordingly. See
> function _find_openssl() in
>
> https://svn.apache.org/repos/asf/subversion/trunk/build/generator/gen_win_dependencies.py
>
>
> After fixing / working around the above three problems I can
> successfully build serf 1.3.9, phew :-).
>
> Thanks,
> --
> Johan
>