You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Ben Reser <be...@reser.org> on 2013/04/14 02:46:12 UTC
Making the Windows Build Easier
As we've discussed previously a big part of the problem in getting the
Windows build working has been getting the dependencies built.
Everyone I've talked to manages to get this done, then has no idea how
to duplicate what they've done.
This past week I wanted to use some static analysis tools that were
only available on Windows. So I set out to build Subversion on
Windows. I spent about 2 days fighting with it, taking notes the
whole way so I could try and reproduce it. On the 3rd day I realized
that taking notes wasn't going to work and that by the time I finished
I'd have some instructions that left some important detail out or
didn't explain it fully. So I started writing a script to automate
this annoying process.
The result is the build-svn-deps-win.pl script that I've commited in r1467714.
This script is far from perfect. It still needs some work on it, but
it's probably about 90% of the way there to turning setting up a
Windows development environment into about an hour effort from a
couple day slog that you have no hope of being able to reproduce.
A lot of credit goes to pburba for his blog post here:
http://blogs.collab.net/subversion/building-subversion-on-windows-a-walk-through
Without it I would have spent even more time doing this than I did.
So with all that said, here are some details.
It downloads the following things:
bdb
zlib
pcre
httpd
apr
apr-util
apr-iconv
sqlite-amalgamation
serf
It builds all of the above expect for sqlite and serf which get built
as part of the Subversion build itself (zlib could be built there as
well but I built it so that mod_deflate would work in httpd).
The script is built to use an entirely modern tool chain and
dependencies. Everything is the current released version. That means
httpd-2.4.4, Visual Studio 2012, etc... It does not build or deal
with neon, though that could be trivially added.
Unfortunately, it does not work with Visual Studio 2012 Windows
Desktop Express due to the lack of devenv.(com|exe). However, the
script is written to produce a the binaries in a way that they can be
packaged. There's still some work to do here in making the package as
minimal as possible (some of it possibly in our build system).
At current the script only builds a Win32 Release build. But I expect
to change that in the near future.
So without further ado, here are some instructions:
1) Get the dependencies. If you have a full version of Visual Studio
(not an Express version) you can follow a) or b) depending on your
preference. For Visual Studio Express you have to follow b).
a) Pick a location to do the build and make a directory to build the
deps, in my case I put it in C:\Users\breser\svn-trunk-deps. Download
the script at http://svn.apache.org/repos/asf/subversion/trunk/tools/dev/build-svn-deps-win.pl
and put it in this directory. Open the script in a text editor and
make sure you install the few dependencies you'll need (Perl, Python,
7-Zip and CMake). Python actually may not be needed for this but it's
needed for the Subversion build itself. If presented the option to
add the commands to your path accept it (Python and Perl will do this
by default, CMake you have to explicitly decide to do this). 7-zip
doesn't have one so make a note of where you installed it for later.
Once you have the dependencies downloaded open the "VS2012 x86 Native
Tools Command Prompt" (usually found in the Visual Studio Tools
group). cd to your directory and run build-svn-deps-win.pl. If
7z.exe is not in "C:\Program Files\7-Zip\7z.exe" you can override this
by passing "SEVEN_ZIP=C:\Path\To\7z.exe" to the script. The script
will then download and build the dependencies for you. For me on my
VM setup it takes about 45 minutes for this to run.
b) Download http://ben.reser.org/svn-windows-deps/svn-trunk-deps-win32-release-20130413.7z
and extract it. You can rename the folder but it should be named
svn-trunk-deps. In my case I put it in
C:\Users\breser\svn-trunk-deps. The file is 116MB and it's compressed
with 7-zip (sorry but it was less than half the size using 7-zip over
normal zip). You'll need at least Python which you can find a URL to
at the top of the build-svn-deps-win.pl script in the svn-trunk-deps.
You may need Perl but I'm not sure, I had it on my setup since I was
originally trying to do step a with Express. You won't need CMake for
sure though. There is a GPG signature of the file for the
appropriately cautious:
http://ben.reser.org/svn-windows-deps/svn-trunk-deps-win32-release-20130413.7z.asc
2) Get Subversion's source. At this point I haven't tested this with
anything other than trunk, so you'll need a Subversion client to do a
checkout.
3) Configure Subversion. In the VS2012 Command Prompt cd into the
Subversion source you checked out and run the following (adjusting the
paths so that C:\Users\breser\svn-trunk-deps is wherever you put the
dependencies:
gen-make.py -t vcproj --vsnet-version=2012
--with-berkeley-db=c:\Users\breser\svn-trunk-deps
--with-httpd=c:\Users\breser\svn-trunk-deps\build\httpd
--with-serf=c:\Users\breser\svn-trunk-deps\serf
--with-sqlite=c:\Users\breser\svn-trunk-deps\sqlite-amalgamation
--with-zlib=c:\Users\breser\svn-trunk-deps\build\httpd\srclib\zlib
--with-openssl=c:\Users\breser\svn-trunk-deps\build\httpd\srclib\openssl
4) Build Subversion. Once that finishes run the following in the came
command prompt:
msbuild subversion_vcnet.sln /t:__ALL_TESTS__ /p:Configuration=Release
5) Test Subversion.
Before testing Subversion you need to set the PATH to include the bin
dir in the deps directory (otherwise tests will fail due to missing
libapriconv-1.dll):
set PATH=c:\Users\breser\svn-trunk-deps\bin;%PATH%
The following command can be used to test Subversion with ra_local:
win-tests.py --release --parallel
The following command can test it with ra_serf (and will start httpd for you):
win-tests.py --release --httpd-dir=C:\Users\breser\svn-trunk-deps --httpd-daemon
[For some reason for me ra_serf fails with parallel, which speeds up the tests]
The following command can test it with ra_svn (and will start svnserve for you):
win-tests.py --release --parallel --url=svn://localhost
Re: Making the Windows Build Easier
Posted by Ben Reser <be...@reser.org>.
On Tue, May 7, 2013 at 11:50 AM, Mark Phippard <ma...@gmail.com> wrote:
> Yes, started it over from scratch as I was not sure if just re-running
> would be the right thing to do.
>
> In general, what is the process in the future when there are new
> versions of components. Should I be able to just update the version
> number and re-run the script or should I always start from scratch?
For now probably best to start from scratch. However, I should be
able to make it able to deal with that.
Re: Making the Windows Build Easier
Posted by Mark Phippard <ma...@gmail.com>.
On Tue, May 7, 2013 at 2:50 PM, Mark Phippard <ma...@gmail.com> wrote:
> On Tue, May 7, 2013 at 2:47 PM, Ben Reser <be...@reser.org> wrote:
>> On Tue, May 7, 2013 at 10:43 AM, Mark Phippard <ma...@gmail.com> wrote:
>>> I am attaching the log file it found. I also searched the
>>> httxt2dbm.vcxproj file, which seems to be the relevant file, and it
>>> does NOT seem to have that property you set about TrackFiles. I note
>>> this project file is in the support folder. Does your script touch
>>> those projects too?
>>
>> I think we're dealing with a race condition that's just not happening
>> on my setup. I noticed that the modules that failed for me were
>> somewhat random. So I guess due to differences in systems I'm just
>> not seeing the failures in the support tree.
>>
>> I've added the support tree to the tree of files that I turn off
>> tracking of file accesses in the projects in the latest commit.
>>
>> Can you give that a try?
>>
>> Thanks for your patience in dealing with this.
>
> Yes, started it over from scratch as I was not sure if just re-running
> would be the right thing to do.
>
> In general, what is the process in the future when there are new
> versions of components. Should I be able to just update the version
> number and re-run the script or should I always start from scratch?
Looks like that latest commit solved it! Thanks.
Now on to the next steps where I actually build SVN to use the dependencies.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Making the Windows Build Easier
Posted by Mark Phippard <ma...@gmail.com>.
On Tue, May 7, 2013 at 2:47 PM, Ben Reser <be...@reser.org> wrote:
> On Tue, May 7, 2013 at 10:43 AM, Mark Phippard <ma...@gmail.com> wrote:
>> I am attaching the log file it found. I also searched the
>> httxt2dbm.vcxproj file, which seems to be the relevant file, and it
>> does NOT seem to have that property you set about TrackFiles. I note
>> this project file is in the support folder. Does your script touch
>> those projects too?
>
> I think we're dealing with a race condition that's just not happening
> on my setup. I noticed that the modules that failed for me were
> somewhat random. So I guess due to differences in systems I'm just
> not seeing the failures in the support tree.
>
> I've added the support tree to the tree of files that I turn off
> tracking of file accesses in the projects in the latest commit.
>
> Can you give that a try?
>
> Thanks for your patience in dealing with this.
Yes, started it over from scratch as I was not sure if just re-running
would be the right thing to do.
In general, what is the process in the future when there are new
versions of components. Should I be able to just update the version
number and re-run the script or should I always start from scratch?
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Making the Windows Build Easier
Posted by Ben Reser <be...@reser.org>.
On Tue, May 7, 2013 at 10:43 AM, Mark Phippard <ma...@gmail.com> wrote:
> I am attaching the log file it found. I also searched the
> httxt2dbm.vcxproj file, which seems to be the relevant file, and it
> does NOT seem to have that property you set about TrackFiles. I note
> this project file is in the support folder. Does your script touch
> those projects too?
I think we're dealing with a race condition that's just not happening
on my setup. I noticed that the modules that failed for me were
somewhat random. So I guess due to differences in systems I'm just
not seeing the failures in the support tree.
I've added the support tree to the tree of files that I turn off
tracking of file accesses in the projects in the latest commit.
Can you give that a try?
Thanks for your patience in dealing with this.
Re: Making the Windows Build Easier
Posted by Mark Phippard <ma...@gmail.com>.
On Tue, May 7, 2013 at 12:46 PM, Ben Reser <be...@reser.org> wrote:
> On Tue, May 7, 2013 at 6:14 AM, Mark Phippard <ma...@gmail.com> wrote:
>> On Tue, May 7, 2013 at 2:06 AM, Ben Reser <be...@reser.org> wrote:
>>> Can you look to see that there is a line like this in all the vcxproj
>>> files under modules\proxy\balancers:
>>> <PropertyGroup><TrackFileAccess>false</TrackFileAccess></PropertyGroup>
>>
>> Yes, next to last line right before the closing Project tag.
>
> Now I'm wondering if it isn't recognizing this property setting. Can
> you check your path to make sure you don't have some other tools on
> the path? If you run devenv from the terminal you were running this
> in does it start Visual Studio 2012 or some previous version?
I am using the VS 2012 command prompt and it opens VS 2012.
>>> The scripting that adds that bit is not really the greatest. Perhaps
>>> it's failing for you for some reason. One possibility here is a
>>> different version of Perl. I'm using Perl 5.16.3.1603 from:
>>> http://www.activestate.com/activeperl/downloads
>>
>> I am also using ActiveState. 5.12.2 in my case.
>
> Yeah well if the lines are there it shouldn't be an issue.
>
>>> If the above doesn't help, can you try the attached script (adjusting
>>> paths). I know it's ugly it's basically just a find2perl script that
>>> I threw together to find failed builds while I was using this. Just
>>> run it while the cd is the trunk and it should point out which bits of
>>> the build have Build FAILED in them. You can then look at those logs
>>> and you should get a more detailed error. You very well may only find
>>> what you've already found.
>>
>> There were no attachments.
>
> Oops. Attached this time.
>
>>> The other comment I have is I did this on Windows 8 x64 (though I only
>>> built 32-bit binaries). I'm not sure what version of Windows you're
>>> using, but this might be another possible source of issues.
>>
>> Windows 7 x64, doing a 32-bit build.
>
> Okay I'll try it on Windows 7 x64 if I get a chance to later today.
I am attaching the log file it found. I also searched the
httxt2dbm.vcxproj file, which seems to be the relevant file, and it
does NOT seem to have that property you set about TrackFiles. I note
this project file is in the support folder. Does your script touch
those projects too?
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Making the Windows Build Easier
Posted by Ben Reser <be...@reser.org>.
On Tue, May 7, 2013 at 6:14 AM, Mark Phippard <ma...@gmail.com> wrote:
> On Tue, May 7, 2013 at 2:06 AM, Ben Reser <be...@reser.org> wrote:
>> Can you look to see that there is a line like this in all the vcxproj
>> files under modules\proxy\balancers:
>> <PropertyGroup><TrackFileAccess>false</TrackFileAccess></PropertyGroup>
>
> Yes, next to last line right before the closing Project tag.
Now I'm wondering if it isn't recognizing this property setting. Can
you check your path to make sure you don't have some other tools on
the path? If you run devenv from the terminal you were running this
in does it start Visual Studio 2012 or some previous version?
>> The scripting that adds that bit is not really the greatest. Perhaps
>> it's failing for you for some reason. One possibility here is a
>> different version of Perl. I'm using Perl 5.16.3.1603 from:
>> http://www.activestate.com/activeperl/downloads
>
> I am also using ActiveState. 5.12.2 in my case.
Yeah well if the lines are there it shouldn't be an issue.
>> If the above doesn't help, can you try the attached script (adjusting
>> paths). I know it's ugly it's basically just a find2perl script that
>> I threw together to find failed builds while I was using this. Just
>> run it while the cd is the trunk and it should point out which bits of
>> the build have Build FAILED in them. You can then look at those logs
>> and you should get a more detailed error. You very well may only find
>> what you've already found.
>
> There were no attachments.
Oops. Attached this time.
>> The other comment I have is I did this on Windows 8 x64 (though I only
>> built 32-bit binaries). I'm not sure what version of Windows you're
>> using, but this might be another possible source of issues.
>
> Windows 7 x64, doing a 32-bit build.
Okay I'll try it on Windows 7 x64 if I get a chance to later today.
Re: Making the Windows Build Easier
Posted by Mark Phippard <ma...@gmail.com>.
On Tue, May 7, 2013 at 2:06 AM, Ben Reser <be...@reser.org> wrote:
> On Mon, May 6, 2013 at 4:47 PM, Mark Phippard <ma...@gmail.com> wrote:
>> I have Update 2 installed.
>> http://www.microsoft.com/en-us/download/details.aspx?id=38188
>
> I've installed update 2, still get a completely finished build.
I started over fresh overnight and got the same problem.
> Can you look to see that there is a line like this in all the vcxproj
> files under modules\proxy\balancers:
> <PropertyGroup><TrackFileAccess>false</TrackFileAccess></PropertyGroup>
Yes, next to last line right before the closing Project tag.
> The scripting that adds that bit is not really the greatest. Perhaps
> it's failing for you for some reason. One possibility here is a
> different version of Perl. I'm using Perl 5.16.3.1603 from:
> http://www.activestate.com/activeperl/downloads
I am also using ActiveState. 5.12.2 in my case.
> If the above doesn't help, can you try the attached script (adjusting
> paths). I know it's ugly it's basically just a find2perl script that
> I threw together to find failed builds while I was using this. Just
> run it while the cd is the trunk and it should point out which bits of
> the build have Build FAILED in them. You can then look at those logs
> and you should get a more detailed error. You very well may only find
> what you've already found.
There were no attachments.
> The other comment I have is I did this on Windows 8 x64 (though I only
> built 32-bit binaries). I'm not sure what version of Windows you're
> using, but this might be another possible source of issues.
Windows 7 x64, doing a 32-bit build.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Making the Windows Build Easier
Posted by Ben Reser <be...@reser.org>.
On Mon, May 6, 2013 at 4:47 PM, Mark Phippard <ma...@gmail.com> wrote:
> I have Update 2 installed.
> http://www.microsoft.com/en-us/download/details.aspx?id=38188
I've installed update 2, still get a completely finished build.
Can you look to see that there is a line like this in all the vcxproj
files under modules\proxy\balancers:
<PropertyGroup><TrackFileAccess>false</TrackFileAccess></PropertyGroup>
The scripting that adds that bit is not really the greatest. Perhaps
it's failing for you for some reason. One possibility here is a
different version of Perl. I'm using Perl 5.16.3.1603 from:
http://www.activestate.com/activeperl/downloads
If the above doesn't help, can you try the attached script (adjusting
paths). I know it's ugly it's basically just a find2perl script that
I threw together to find failed builds while I was using this. Just
run it while the cd is the trunk and it should point out which bits of
the build have Build FAILED in them. You can then look at those logs
and you should get a more detailed error. You very well may only find
what you've already found.
The other comment I have is I did this on Windows 8 x64 (though I only
built 32-bit binaries). I'm not sure what version of Windows you're
using, but this might be another possible source of issues.
Re: Making the Windows Build Easier
Posted by Mark Phippard <ma...@gmail.com>.
I have Update 2 installed. http://www.microsoft.com/en-us/download/details.aspx?id=38188
Sent from my iPad
On May 6, 2013, at 7:29 PM, Ben Reser <be...@reser.org> wrote:
> On Mon, May 6, 2013 at 3:40 PM, Mark Phippard <ma...@gmail.com> wrote:
>> I like that the script can be re-run.
>
> Yes, I tried to make it smart enough to avoid redoing work that was
> already done.
>
>> The script then started chugging along and building. It looks like it
>> failed at the end of building httpd. Not really sure why or what is
>> wrong:
>>
>> ========== Build: 7 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========
>> ========== Build: 118 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========
>> NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 11.0
>> \Common7\IDE\devenv.COM"' : return code '0x1'
>> Stop.
>> NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 11.0
>> \VC\BIN\nmake.exe"' : return code '0x2'
>> Stop.
>> Failed building/installing httpd/apr/apu/api (exit code: 512) at build-svn-deps-
>> win.pl line 245.
>>
>> Any ideas? I searched the output for what Failed and do not see
>> anything referenced.
>>
>> Searching for all instanced of "Error". It might be this:
>>
>> C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V110\Microsoft.CppCommon.ta
>> rgets(347,5): error MSB6003: The specified task executable "CL.exe" could not be
>> run. The process cannot access the file 'C:\SVN\deps-1.8.x\build\httpd\modules\
>> proxy\balancers\Release\cl.write.1.tlog' because it is being used by another pro
>> cess.
>
> The above error is a bug in the httpd build process related to newer
> versions of Visual Studio. Since everything shares the same
> intermediate directories concurrent builds start blowing up over this
> log file. The correct fix for this is to use different intermediate
> build directories for each module. This is a major change to the
> httpd build system so I didn't do it. Rather I applied a workaround
> by setting a flag that tells Visual Studio not to use the files that
> create the problem.
>
> Let me see if I can reproduce this on my end.
>
> What's the exact version of Visual Studio 2012 that you're using? I'm
> on 11.0.50727.1. I haven't bothered to apply the updates to it.
Re: Making the Windows Build Easier
Posted by Ben Reser <be...@reser.org>.
On Mon, May 6, 2013 at 3:40 PM, Mark Phippard <ma...@gmail.com> wrote:
> I like that the script can be re-run.
Yes, I tried to make it smart enough to avoid redoing work that was
already done.
> The script then started chugging along and building. It looks like it
> failed at the end of building httpd. Not really sure why or what is
> wrong:
>
> ========== Build: 7 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========
> ========== Build: 118 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========
> NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 11.0
> \Common7\IDE\devenv.COM"' : return code '0x1'
> Stop.
> NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 11.0
> \VC\BIN\nmake.exe"' : return code '0x2'
> Stop.
> Failed building/installing httpd/apr/apu/api (exit code: 512) at build-svn-deps-
> win.pl line 245.
>
> Any ideas? I searched the output for what Failed and do not see
> anything referenced.
>
> Searching for all instanced of "Error". It might be this:
>
> C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V110\Microsoft.CppCommon.ta
> rgets(347,5): error MSB6003: The specified task executable "CL.exe" could not be
> run. The process cannot access the file 'C:\SVN\deps-1.8.x\build\httpd\modules\
> proxy\balancers\Release\cl.write.1.tlog' because it is being used by another pro
> cess.
The above error is a bug in the httpd build process related to newer
versions of Visual Studio. Since everything shares the same
intermediate directories concurrent builds start blowing up over this
log file. The correct fix for this is to use different intermediate
build directories for each module. This is a major change to the
httpd build system so I didn't do it. Rather I applied a workaround
by setting a flag that tells Visual Studio not to use the files that
create the problem.
Let me see if I can reproduce this on my end.
What's the exact version of Visual Studio 2012 that you're using? I'm
on 11.0.50727.1. I haven't bothered to apply the updates to it.
Re: Making the Windows Build Easier
Posted by Mark Phippard <ma...@gmail.com>.
On Sat, Apr 13, 2013 at 8:46 PM, Ben Reser <be...@reser.org> wrote:
> The result is the build-svn-deps-win.pl script that I've commited in r1467714.
>
> This script is far from perfect. It still needs some work on it, but
> it's probably about 90% of the way there to turning setting up a
> Windows development environment into about an hour effort from a
> couple day slog that you have no hope of being able to reproduce.
Giving this a try with 1.8.x RC1
> a) Pick a location to do the build and make a directory to build the
> deps, in my case I put it in C:\Users\breser\svn-trunk-deps. Download
> the script at http://svn.apache.org/repos/asf/subversion/trunk/tools/dev/build-svn-deps-win.pl
> and put it in this directory. Open the script in a text editor and
> make sure you install the few dependencies you'll need (Perl, Python,
> 7-Zip and CMake). Python actually may not be needed for this but it's
> needed for the Subversion build itself. If presented the option to
> add the commands to your path accept it (Python and Perl will do this
> by default, CMake you have to explicitly decide to do this). 7-zip
> doesn't have one so make a note of where you installed it for later.
> Once you have the dependencies downloaded open the "VS2012 x86 Native
> Tools Command Prompt" (usually found in the Visual Studio Tools
> group). cd to your directory and run build-svn-deps-win.pl. If
> 7z.exe is not in "C:\Program Files\7-Zip\7z.exe" you can override this
> by passing "SEVEN_ZIP=C:\Path\To\7z.exe" to the script. The script
> will then download and build the dependencies for you. For me on my
> VM setup it takes about 45 minutes for this to run.
I used this option. Ran into two quick problems that were easy to solve:
1) Script got a 404 when downloading zlib 1.2.7. I recall seeing
messages about 1.2.8 on the list here so I changed the version in the
script and that solved that problem.
2) Got an error about invalid command line option when it tried to use
7-zip to unpack the tarballs. Turned out I had an old version
installed. I installed the latest and that resolved it.
I like that the script can be re-run.
The script then started chugging along and building. It looks like it
failed at the end of building httpd. Not really sure why or what is
wrong:
========== Build: 7 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========
========== Build: 118 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 11.0
\Common7\IDE\devenv.COM"' : return code '0x1'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 11.0
\VC\BIN\nmake.exe"' : return code '0x2'
Stop.
Failed building/installing httpd/apr/apu/api (exit code: 512) at build-svn-deps-
win.pl line 245.
Any ideas? I searched the output for what Failed and do not see
anything referenced.
Searching for all instanced of "Error". It might be this:
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V110\Microsoft.CppCommon.ta
rgets(347,5): error MSB6003: The specified task executable "CL.exe" could not be
run. The process cannot access the file 'C:\SVN\deps-1.8.x\build\httpd\modules\
proxy\balancers\Release\cl.write.1.tlog' because it is being used by another pro
cess.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Making the Windows Build Easier
Posted by Mario Brandt <jb...@gmail.com>.
On 14 April 2013 02:46, Ben Reser <be...@reser.org> wrote:
> Unfortunately, it does not work with Visual Studio 2012 Windows
> Desktop Express due to the lack of devenv.(com|exe). However, the
> script is written to produce a the binaries in a way that they can be
> packaged. There's still some work to do here in making the package as
> minimal as possible (some of it possibly in our build system).
I'm not sure, but maybe it is worth to take a look at
https://github.com/winlibs/apache how it is done.
Re: Making the Windows Build Easier
Posted by Ben Reser <be...@reser.org>.
On Sat, Apr 13, 2013 at 5:46 PM, Ben Reser <be...@reser.org> wrote:
> The result is the build-svn-deps-win.pl script that I've commited in r1467714.
>
> This script is far from perfect. It still needs some work on it, but
> it's probably about 90% of the way there to turning setting up a
> Windows development environment into about an hour effort from a
> couple day slog that you have no hope of being able to reproduce.
For what it's worth I forgot to mention that the script and the
instructions are only for trunk/1.8.0. At first it looked like it
would work since the Subversion build was just unhappy about neon
(which I gave it) and then it didn't know about 2012 in gen-make.py so
I told it 2010 and converted the project. But when you try to build
it, it ends up with a ton of errors about not being able to open
source files. Maybe we should back port the 2012 generator to 1.7.x.
AW: Making the Windows Build Easier
Posted by Markus Schaber <m....@codesys.com>.
Hi,
Berts SharpSVN build also includes a way to pull all the dependencies, using msbuild.
However, it does not work with the express editions yet, AFAICS, so your script clearly has at least one advantage. :-)
Will it work using the VS 2010 command prompt?
Best regards
Markus Schaber
CODESYS® a trademark of 3S-Smart Software Solutions GmbH
Inspiring Automation Solutions
3S-Smart Software Solutions GmbH
Dipl.-Inf. Markus Schaber | Product Development Core Technology
Memminger Str. 151 | 87439 Kempten | Germany
Tel. +49-831-54031-979 | Fax +49-831-54031-50
E-Mail: m.schaber@codesys.com | Web: http://www.codesys.com | CODESYS store: http://store.codesys.com
CODESYS forum: http://forum.codesys.com
Managing Directors: Dipl.Inf. Dieter Hess, Dipl.Inf. Manfred Werner | Trade register: Kempten HRB 6186 | Tax ID No.: DE 167014915
> -----Ursprüngliche Nachricht-----
> Von: Ben Reser [mailto:ben@reser.org]
> Gesendet: Sonntag, 14. April 2013 02:46
> An: Subversion Development
> Betreff: Making the Windows Build Easier
>
> As we've discussed previously a big part of the problem in getting the
> Windows build working has been getting the dependencies built.
> Everyone I've talked to manages to get this done, then has no idea how
> to duplicate what they've done.
>
> This past week I wanted to use some static analysis tools that were only
> available on Windows. So I set out to build Subversion on Windows. I
> spent about 2 days fighting with it, taking notes the whole way so I
> could try and reproduce it. On the 3rd day I realized that taking notes
> wasn't going to work and that by the time I finished I'd have some
> instructions that left some important detail out or didn't explain it
> fully. So I started writing a script to automate this annoying process.
>
> The result is the build-svn-deps-win.pl script that I've commited in
> r1467714.
>
> This script is far from perfect. It still needs some work on it, but
> it's probably about 90% of the way there to turning setting up a Windows
> development environment into about an hour effort from a couple day slog
> that you have no hope of being able to reproduce.
>
> A lot of credit goes to pburba for his blog post here:
> http://blogs.collab.net/subversion/building-subversion-on-windows-a-
> walk-through
>
> Without it I would have spent even more time doing this than I did.
>
> So with all that said, here are some details.
>
> It downloads the following things:
> bdb
> zlib
> pcre
> httpd
> apr
> apr-util
> apr-iconv
> sqlite-amalgamation
> serf
>
> It builds all of the above expect for sqlite and serf which get built as
> part of the Subversion build itself (zlib could be built there as well
> but I built it so that mod_deflate would work in httpd).
>
> The script is built to use an entirely modern tool chain and
> dependencies. Everything is the current released version. That means
> httpd-2.4.4, Visual Studio 2012, etc... It does not build or deal with
> neon, though that could be trivially added.
>
> Unfortunately, it does not work with Visual Studio 2012 Windows Desktop
> Express due to the lack of devenv.(com|exe). However, the script is
> written to produce a the binaries in a way that they can be packaged.
> There's still some work to do here in making the package as minimal as
> possible (some of it possibly in our build system).
>
> At current the script only builds a Win32 Release build. But I expect
> to change that in the near future.
>
> So without further ado, here are some instructions:
>
> 1) Get the dependencies. If you have a full version of Visual Studio
> (not an Express version) you can follow a) or b) depending on your
> preference. For Visual Studio Express you have to follow b).
>
> a) Pick a location to do the build and make a directory to build the
> deps, in my case I put it in C:\Users\breser\svn-trunk-deps. Download
> the script at
> http://svn.apache.org/repos/asf/subversion/trunk/tools/dev/build-svn-
> deps-win.pl
> and put it in this directory. Open the script in a text editor and make
> sure you install the few dependencies you'll need (Perl, Python, 7-Zip
> and CMake). Python actually may not be needed for this but it's needed
> for the Subversion build itself. If presented the option to add the
> commands to your path accept it (Python and Perl will do this by default,
> CMake you have to explicitly decide to do this). 7-zip doesn't have one
> so make a note of where you installed it for later.
> Once you have the dependencies downloaded open the "VS2012 x86 Native
> Tools Command Prompt" (usually found in the Visual Studio Tools group).
> cd to your directory and run build-svn-deps-win.pl. If 7z.exe is not in
> "C:\Program Files\7-Zip\7z.exe" you can override this by passing
> "SEVEN_ZIP=C:\Path\To\7z.exe" to the script. The script will then
> download and build the dependencies for you. For me on my VM setup it
> takes about 45 minutes for this to run.
>
> b) Download http://ben.reser.org/svn-windows-deps/svn-trunk-deps-win32-
> release-20130413.7z
> and extract it. You can rename the folder but it should be named svn-
> trunk-deps. In my case I put it in C:\Users\breser\svn-trunk-deps. The
> file is 116MB and it's compressed with 7-zip (sorry but it was less than
> half the size using 7-zip over normal zip). You'll need at least Python
> which you can find a URL to at the top of the build-svn-deps-win.pl
> script in the svn-trunk-deps.
> You may need Perl but I'm not sure, I had it on my setup since I was
> originally trying to do step a with Express. You won't need CMake for
> sure though. There is a GPG signature of the file for the appropriately
> cautious:
> http://ben.reser.org/svn-windows-deps/svn-trunk-deps-win32-release-
> 20130413.7z.asc
>
> 2) Get Subversion's source. At this point I haven't tested this with
> anything other than trunk, so you'll need a Subversion client to do a
> checkout.
>
> 3) Configure Subversion. In the VS2012 Command Prompt cd into the
> Subversion source you checked out and run the following (adjusting the
> paths so that C:\Users\breser\svn-trunk-deps is wherever you put the
> dependencies:
> gen-make.py -t vcproj --vsnet-version=2012 --with-berkeley-
> db=c:\Users\breser\svn-trunk-deps
> --with-httpd=c:\Users\breser\svn-trunk-deps\build\httpd
> --with-serf=c:\Users\breser\svn-trunk-deps\serf
> --with-sqlite=c:\Users\breser\svn-trunk-deps\sqlite-amalgamation
> --with-zlib=c:\Users\breser\svn-trunk-deps\build\httpd\srclib\zlib
> --with-openssl=c:\Users\breser\svn-trunk-deps\build\httpd\srclib\openssl
>
> 4) Build Subversion. Once that finishes run the following in the came
> command prompt:
> msbuild subversion_vcnet.sln /t:__ALL_TESTS__ /p:Configuration=Release
>
> 5) Test Subversion.
>
> Before testing Subversion you need to set the PATH to include the bin
> dir in the deps directory (otherwise tests will fail due to missing
> libapriconv-1.dll):
> set PATH=c:\Users\breser\svn-trunk-deps\bin;%PATH%
>
> The following command can be used to test Subversion with ra_local:
> win-tests.py --release --parallel
>
> The following command can test it with ra_serf (and will start httpd for
> you):
> win-tests.py --release --httpd-dir=C:\Users\breser\svn-trunk-deps --
> httpd-daemon [For some reason for me ra_serf fails with parallel, which
> speeds up the tests]
>
> The following command can test it with ra_svn (and will start svnserve
> for you):
> win-tests.py --release --parallel --url=svn://localhost
Re: Making the Windows Build Easier
Posted by Branko Čibej <br...@wandisco.com>.
On 14.04.2013 20:18, Mark Phippard wrote:
> On Sun, Apr 14, 2013 at 1:48 PM, Ben Reser <be...@reser.org> wrote:
>> On Sun, Apr 14, 2013 at 10:24 AM, Branko Čibej <br...@wandisco.com> wrote:
>>> Have you seen:
>>>
>>> https://github.com/brainy/subversion-windeps
>> No I had not. I'd known that you'd done some work but didn't realize
>> you'd gotten so far.
>>
>> I'm not sure which approach is better. I see that you've given
>> yourself standard Unix tools. I went with Perl since I could get away
>> without having them. I was also trying to keep the required
>> dependencies to use it down to a minimum.
> I am not sure where he keeps it, but Bert has an msbuild script that
> does everything as well. I believe the only thing you needed to have
> installed was a recent Windows SDK and it used the compiler from that.
> I do not recall how he handled different bindings, but I recall the
> script could even build Python from source so you did not need any of
> it installed.
I've seen the MSBuild script, but I found it way too complicated to
support in our code base. A matter of taste, I suppose.
-- Brane
--
Branko Čibej
Director of Subversion | WANdisco | www.wandisco.com
Re: Making the Windows Build Easier
Posted by Mark Phippard <ma...@gmail.com>.
On Sun, Apr 14, 2013 at 1:48 PM, Ben Reser <be...@reser.org> wrote:
> On Sun, Apr 14, 2013 at 10:24 AM, Branko Čibej <br...@wandisco.com> wrote:
>> Have you seen:
>>
>> https://github.com/brainy/subversion-windeps
>
> No I had not. I'd known that you'd done some work but didn't realize
> you'd gotten so far.
>
> I'm not sure which approach is better. I see that you've given
> yourself standard Unix tools. I went with Perl since I could get away
> without having them. I was also trying to keep the required
> dependencies to use it down to a minimum.
I am not sure where he keeps it, but Bert has an msbuild script that
does everything as well. I believe the only thing you needed to have
installed was a recent Windows SDK and it used the compiler from that.
I do not recall how he handled different bindings, but I recall the
script could even build Python from source so you did not need any of
it installed.
For SVN Edge we used his script as a starting point and then modified
it so that we could use pre-built zips of things we do not want to
rebuild every time like Perl/Python:
https://ctf.open.collab.net/integration/viewvc/viewvc.cgi/trunk/svn-server/windows/?root=svnedge&system=exsy1005
I know we are using VS 2008 and not the SDK. I do not recall if that
is because we ran into an issue where we needed it or not.
I think these scripts are useful for people doing packaging, but I am
not sure how good they are if you want to hack on Subversion using
Visual Studio where you want to use the debugger and have incremental
rebuilds etc. I think those are the kinds of docs that would be
needed.
TortoiseSVN build also uses a lot of the same concepts:
https://code.google.com/p/tortoisesvn/source/browse/trunk/build.txt
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Making the Windows Build Easier
Posted by Ben Reser <be...@reser.org>.
On Sun, Apr 14, 2013 at 10:24 AM, Branko Čibej <br...@wandisco.com> wrote:
> Have you seen:
>
> https://github.com/brainy/subversion-windeps
No I had not. I'd known that you'd done some work but didn't realize
you'd gotten so far.
I'm not sure which approach is better. I see that you've given
yourself standard Unix tools. I went with Perl since I could get away
without having them. I was also trying to keep the required
dependencies to use it down to a minimum.
Both of us appear to want to be able to look for things in bin,
include and lib dirs, which the build system doesn't really want to do
at current.
> I haven't had time to expand on that but I certainly intend to make time
> during 1.9 development. It'll mean making considerable changes to our
> Windows build generator (mainly removing a bunch of assumptions), but
> that's all for the better.
Yup, part of the reason my packaged up output is so big is I'm
shipping a lot of crap I don't want to because of the assumptions of
the build system.
Re: Making the Windows Build Easier
Posted by Branko Čibej <br...@wandisco.com>.
On 14.04.2013 02:46, Ben Reser wrote:
> As we've discussed previously a big part of the problem in getting the
> Windows build working has been getting the dependencies built.
> Everyone I've talked to manages to get this done, then has no idea how
> to duplicate what they've done.
>
> This past week I wanted to use some static analysis tools that were
> only available on Windows. So I set out to build Subversion on
> Windows. I spent about 2 days fighting with it, taking notes the
> whole way so I could try and reproduce it. On the 3rd day I realized
> that taking notes wasn't going to work and that by the time I finished
> I'd have some instructions that left some important detail out or
> didn't explain it fully. So I started writing a script to automate
> this annoying process.
>
> The result is the build-svn-deps-win.pl script that I've commited in r1467714.
Have you seen:
https://github.com/brainy/subversion-windeps
I haven't had time to expand on that but I certainly intend to make time
during 1.9 development. It'll mean making considerable changes to our
Windows build generator (mainly removing a bunch of assumptions), but
that's all for the better.
-- Brane
-- Brane
--
Branko Čibej
Director of Subversion | WANdisco | www.wandisco.com
Re: Making the Windows Build Easier
Posted by Ben Reser <be...@reser.org>.
On Tue, May 7, 2013 at 12:38 PM, Mark Phippard <ma...@gmail.com> wrote:
> Is this right? Specifying the root folder? I think it is because you
> have a lib and include in it. However, it tells me that BDB is not
> found and the fs will not be built when I run this. Everything else
> seems OK.
It is but it looks like the windows build generator only knows about
berkeley db versions up to 4.8.
See _find_bdb() in build/generators/gen_win.py.
Adding "53" to that list of version numbers should fix it. I'm
testing it right now.
Re: Making the Windows Build Easier
Posted by Mark Phippard <ma...@gmail.com>.
On Sat, Apr 13, 2013 at 8:46 PM, Ben Reser <be...@reser.org> wrote:
> gen-make.py -t vcproj --vsnet-version=2012
> --with-berkeley-db=c:\Users\breser\svn-trunk-deps
Is this right? Specifying the root folder? I think it is because you
have a lib and include in it. However, it tells me that BDB is not
found and the fs will not be built when I run this. Everything else
seems OK.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/