You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by "Cooke, Mark" <ma...@siemens.com> on 2016/06/22 10:46:21 UTC
Compiling svn + httpd for windows python 2.x
Folks,
We use subversion with Trac behind httpd on Windows Server. As Trac is written in "old" Python (2.x), I have had to resort to building everything from source. This is not simple and so I thought I would publish my notes here in case it helps anyone else and in the hope that if I have made any mistakes, someone will be kind enough to point them out to me!
~ Mark C
These are my notes for building Apache httpd and subversion for use with Trac [1]. All of the components need to be built using the same compiler to avoid run-time issues and, since Trac currently relies on Python 2.x, that means Visual Studio 2008.
[1] http://trac.edgewall.org
Pre-requisites
--------------
All of the following assumes a suitable build environment, which for me is:-
- Windows 7 Enterprise (virtual machine)
- Visual Studio Professional 2008 (plus SP1) == VC9
- Python 2.x (currently 2.7.11) [2]
- PyWin32 [3]
- StrawberryPerl [4]
- "awk" [5] (can't remember why)
- SCons [6] for building the Serf library
- CMake [7] for building the PCRE library
- SWIG [8] for the Python bindings
- 7-zip [9] for unpacking all the downloads!
[2] https://www.python.org/downloads/
[3] http://sourceforge.net/projects/pywin32/files/pywin32/
[4] http://strawberryperl.com/
[5] https://www.cs.princeton.edu/~bwk/btl.mirror/
[6] http://www.scons.org/
[7] http://www.cmake.org/download/
[8] http://www.swig.org/download.html
[9] http://7-zip.org
Folders
-------
In the following I assume the following folder structure:-
D:\svn
D:\svn\httpd
D:\svn\httpd\srclib\apr
D:\svn\httpd\srclib\apr-iconv
D:\svn\httpd\srclib\apr-util
D:\svn\httpd\srclib\openssl
D:\svn\httpd\srclib\pcre
D:\svn\serf
D:\svn\sqlite
D:\svn\src
D:\svn\zlib
===============================================================================
1) Apache httpd
Download the latest httpd source tarball: https://httpd.apache.org/download.cgi
Unpack the source to:
D:\svn\httpd
-------------------------------------------------------------------------------
2) APR source tarballs
Download from: https://apr.apache.org/download.cgi
Unpack the archives to the * httpd\srclib * folder:
D:\svn\httpd\srclib\apr
D:\svn\httpd\srclib\apr-iconv
D:\svn\httpd\srclib\apr-util
Remember to remove the version numbers from the folder names!
Note: last I tried, apr 1.5.2 causes a number of build errors:
http://mail-archives.apache.org/mod_mbox/apr-dev/201504.mbox/%3c553DB43A.3020408@gknw.net%3e
Reverting to 1.5.1... works
-------------------------------------------------------------------------------
3) OpenSSL
Download the latest OpenSSL source: http://www.openssl.org/source/
Unpack the source to the srclib folder:
D:\svn\httpd\srclib\openssl
See also: http://wiki.openssl.org/index.php/Compilation_and_Installation
-------------------------------------------------------------------------------
4) Download the latest PCRE source: http://www.pcre.org/
Unpack the PCRE (Perl Compatible Regular Expressions) to the srclib folder:
D:\svn\httpd\srclib\pcre
-------------------------------------------------------------------------------
5) Fix line endings
Run `lineends.pl` from the root folder to convert line endings from *nix to
windoze standard (NB: recurses down from current folder):
{{{
d:\> cd svn\httpd
d:\svn\httpd\> perl srclib\apr\build\lineends.pl
}}}
-------------------------------------------------------------------------------
6) Build OpenSSL
-----------------
Run the following in a * Visual Studio * command prompt window...
{{{
cd \svn\httpd\srclib\openssl
perl Configure no-asm no-comp no-rc5 no-idea no-ssl2 enable-mdc2 VC-WIN32
ms\do_ms.bat
nmake -f ms\ntdll.mak
}}}
Note: specifying "no-ssl2 no-ssl3" will result in linker errors!
-------------------------------------------------------------------------------
7) Build PCRE
--------------
[from http://wiki.greenstone.org/doku.php?id=en:developer:compiling_apache]
- Read `srclib/pcre/NON-AUTOTOOLS-BUILD`
section "BUILDING PCRE ON WINDOWS WITH CMAKE"
- Run the CMake GUI from a VS2008 command prompt ("start cmake-gui")
- Specify the source and output folders as ...\srclib\pcre
- Hit `Configure`
- Select `NMake Makefiles` for the generator, using default native compilers
- Edit the red block of values:
- BUILD_SHARED LIBS
- Set CMAKE_BUILD_TYPES to RelWithDebInfo
- Set CMAKE_INSTALL_PREFIX to the \srclib\pcre folder
? Set both unicode support flags (seems likely to be useful?)
? not sure about PCRE_NEWLINE for windoze? default: LF (*nix)
- Hit `Configure` again, then `Generate` (then close cmake-gui).
- From the output folder (e.g. srclib\pcre):
{{{
nmake -f makefile
}}}
...should make pcre.dll in the pcre folder, where the apache makefile expects
it to be.
-------------------------------------------------------------------------------
8) Prepare apr, apr-util, apr-iconv, httpd & modules...
------------------------------------------------------
** First ** convert the project files (this avoids lots of RC errors about
embedded quotes around strings in descriptions) using:
{{{
cd \svn\httpd
perl srclib\apr\build\cvtdsp.pl -2005
}}}
** Second ** clear out (rename?) any old \apache24 folder...
The command-line build did not work, so open the `apache.dsw` file:
{{{
start apache.dsw
}}}
...and allow the projects to convert to VS2008 format.
*******
*** Save and close, then add-in the mod_wsgi module...
*******
Extract & rename the mod_wsgi "server" folder into the httpd tree:
- from: mod_wsgi-x.y.z.zip\mod_wsgi-x.y.z\src\server
- to : httpd\modules\wsgi
Copy and edit the "autoindex" project file:
{{{
copy modules\generators\mod_autoindex.vcxproj modules\wsgi\mod_wsgi.vcxproj
}}}
- Edit the vcxproj file:
- replace all `autoindex` => `wsgi`
- deleted the <ProjectGUID /> value (will be recreated)
(Re)Start Visual Studio ("start apache.sln")...
- add the project into the httpd solution (like mod_autoindex).
- add all modules\wsgi\*.c files into the project (NB: mod_wsgi.c already included)
- use the properties dialog on the `mod_wsgi` node to:
- add the Python27\include folder to the compiler options (all builds)
- C/C++ > General > Additional Include Directories
- add the Python27\libs folder to the linker options (all builds)
- Linker > General > Additional Library Directories
- add the extra linker dependencies (Release builds):
- Linker > General > Additional Dependencies
- $(SolutionDir)release\libhttpd.lib
- $(SolutionDir)srclib\apr\release\libapr-1.lib
- $(SolutionDir)srclib\apr-util\release\libaprutil-1.lib
-------------------------------------------------------------------------------
9) Build!
- Set for Release Win32 build.
- Select and Build the InstallBin project
The InstallBin target should create an Apache24 folder in the root, containing
all of the files required to run httpd (ish).
Copy the mod_wsgi module to the output folder:
{{{
copy modules\wsgi\Release\mod_wsgi.so \apache24\modules
}}}
===============================================================================
================================================================ SUBVERSION ===
===============================================================================
Primary resources:
[1] The INSTALL file in the root of the source tree
Blogs:-
[2] http://blogs.collab.net/subversion/building-subversion-on-windows-a-walk-through
[3] http://svn.haxx.se/dev/archive-2013-04/0332.shtml (Ben Reser)
Download the latest sources:
D:\svn\serf
D:\svn\sqlite
D:\svn\src
D:\svn\zlib
- serf: http://serf.apache.org/download
- sqlite "amalgamation": https://www.sqlite.org/download.html
- subversion: http://subversion.apache.org/download/
- zlib: http://www.zlib.net/ (ZLib source in ZIP file, half-way down)
-------------------------------------------------------------------------------
10) Build zlib:
Extract zlib source to \svn\zlib
{{{
cd \svn\zlib
nmake -f win32\makefile.msc
}}}
Note 1: last I looked there were bugs with the MS ASM build => not using asm.
Note 2: 1.9.x requires zlibstat:
edit D:\svn\zlib\contrib\vstudio\vc9\zlibstat.vcproj:
- search "ZLIB_WINAPI" replace ""
{{{
vcbuild /rebuild contrib\vstudio\vc9\zlibvc.sln "ReleaseWithoutAsm|Win32"
#copy `zlib\contrib\vstudio\vc9\x86\ZlibDllReleaseWithoutAsm\zlibwapi.dll`
# - to `zlib\zlib.dll`
copy zlib\contrib\vstudio\vc9\x86\ZlibStatReleaseWithoutAsm\zlibstat.lib zlib\zlibstat.lib
}}}
(see: D:\svn\zlib\contrib\vstudio\readme.txt)
(and: http://www.tannerhelland.com/5076/compile-zlib-winapi-wapi-stdcall/)
(but: adding ZLIB_WINAPI is _wrong_ ~ it needs to be _removed_ from the projects!)
-------------------------------------------------------------------------------
11) Build serf:
Extract serf source to \svn\serf
Build from VS cmd window (requires SCons to be installed):
{{{
cd \svn\serf
c:\Python27\Scripts\scons.py APR=\svn\httpd\srclib\apr
APU=\svn\httpd\srclib\apr-util
OPENSSL=\svn\httpd\srclib\openssl
ZLIB=\svn\zlib
}}}
-------------------------------------------------------------------------------
12) Unpack sqlite:
Extract the sqlite "amalgamation" source to \svn\sqlite
-------------------------------------------------------------------------------
13) Subversion:
Unpack the ZIP archive into D:\svn\src
Configure:
{{{
D:\svn\src> gen-make.py -t vcproj --vsnet-version=2008
--with-httpd=D:\svn\httpd
--with-openssl=d:\svn\httpd\srclib\openssl
--with-serf=d:\svn\serf
--with-sqlite=d:\svn\sqlite
--with-swig=c:\programs\swig
--with-zlib=d:\svn\zlib
}}}
Open the subversion_vcnet solution file ("start subversion_vcnet.sln"):
- Select for a Release build
- Check that the `__ALL_TESTS__` project is set as the default
- Build...
This resulted in instances of [3] errors:-
[1] fatal error C1083: Cannot open include file: 'boost/shared_ptr.hpp': No such file or directory
[2] 9x fatal error C1083: Cannot open include file: 'stdbool.h': No such file or directory
[3] fatal error C1083: Cannot open include file: 'swigutil_pl__pre_perl.h': No such file or directory
...and all come from bindings I do not need (and am not configured to build).
---
Run the tests (copy most of the following to e.g. "run-tests.bat")...
{{{
cd \svn\src
setlocal
set path=%path%;d:\Apache24\bin
set path=%path%;d:\svn\httpd\srclib\apr\Release
set path=%path%;d:\svn\httpd\srclib\apr-util\Release
set path=%path%;d:\svn\httpd\srclib\apr-iconv\Release
win-tests.py --release --cleanup --parallel
win-tests.py -r -c --httpd-dir=d:\Apache24 --httpd-daemon
}}}
-------------------------------------------------------------------------------
14) Python bindings...
http://www.lejordet.com/2009/03/compiling-subversion-python-bindings-on-windows/
...or: D:\svn\src\subversion\bindings\swig\INSTALL
These should have built Ok but need to be collected together for installation.
A) Create a <py-bind>\svn folder somewhere and copy:
- svn\src\subversion\bindings\swig\python\svn\*.py
B) Create a <py-bind>\libsvn folder and copy:
- svn\src\subversion\bindings\swig\python\*.py
- svn\src\Release\subversion\bindings\swig\python\*.pyd
- svn\src\Release\subversion\bindings\swig\python\libsvn_swig_py\*.dll
These files should be installed to the server:-
- C:\Python27\Lib\site-packages\(lib)svn
{{{
cd \svn\src
mkdir \Apache24\python\svn
copy subversion\bindings\swig\python\svn\*.py \Apache24\svn\svn
mkdir \Apache24\python\libsvn
copy subversion\bindings\swig\python\*.py \Apache24\svn\libsvn
copy Release\subversion\bindings\swig\python\*.pyd \Apache24\svn\libsvn
copy Release\subversion\bindings\swig\python\libsvn_swig_py\*.dll \Apache24\svn\libsvn
}}}
===============================================================================
Installation
...is mainly left for the user to match their local systems.
Copy the required files:
- Apache httpd
- remember to exclude the *.pdb files (if not required)
- should already include mod_wsgi
- Subversion (main)
- D:\svn\src\Release\*.dll
- D:\svn\src\Release\subversion\*\svn*.exe (except `svndiff-test.exe`)
- D:\svn\src\Release\tools\*\svnauthz*.exe (from server-side)
- Subversion (Python bindings)
- \Apache24\python (see 13 above) copied to server's python Lib\site-packages folder
- Subversion (httpd modules)
- D:\svn\src\Release\*.so (to apache \modules folder)
Now you "just" need to install the httpd service and fix the httpd "conf" files
as well as set appropriate folder permissions etc. as required by local policy.
(end)
Re: Compiling svn + httpd for windows python 2.x
Posted by Pavel Lyalyakin <pa...@visualsvn.com>.
Hello Mark,
On Thu, Jun 23, 2016 at 2:39 PM, Cooke, Mark <ma...@siemens.com> wrote:
>
> Hi Pavel,
>
> > -----Original Message-----
> > From: Pavel Lyalyakin [mailto:pavel.lyalyakin@visualsvn.com]
> > Sent: 22 June 2016 17:11
> >
> > Hello Mark,
> >
> > On Wed, Jun 22, 2016 at 1:46 PM, Cooke, Mark <ma...@siemens.com> wrote:
> > >
> > > Folks,
> > >
> > > We use subversion with Trac behind httpd on Windows Server. As Trac is
> > written in "old" Python (2.x), I have had to resort to building everything
> > from source. This is not simple and so I thought I would publish my notes
> > here in case it helps anyone else and in the hope that if I have made any
> > mistakes, someone will be kind enough to point them out to me!
> >
> > You say that you had to resort to building SVN and HTTPD on Windows
> > from source. Does it mean that all the existing SVN server packages
> > such as VisualSVN Server don't work for you by some reason? Why?
> >
> > I'm asking because VisualSVN Server comes with Python 2.7.x bindings
> > and `mod_wsgi` which should perfectly work with Trac.
>
> At the risk of going off-topic for the svn-users list... how do I integrate Trac with VisualSVN? I have got VisualSVN installed in a test server but the `conf` file warns against modifications and web searches have provided no hints except links to an apparently defunct Trac plugin [1] including the comment "The new web browser is powered by HTML5 and ensures easy integration with third-party issue trackers" and a version of the svn book[2]
>
> [1] https://www.visualsvn.com/server/trac/
> [2] https://www.visualsvn.com/support/svnbook/serverconfig/httpd/
>
> Cheers,
>
> ~ Mark C
>
> > --
> > With best regards,
> > Pavel Lyalyakin
> > VisualSVN Team
Here are the basic steps. The main idea is to modify httpd-custom.conf
file instead of httpd.conf.
1. Modify "%VISUALSVN_SEREVR%conf\httpd-custom.conf" file instead of
httpd.conf.
2. Load mod_wsgi module by adding the line to the httpd-custom.conf.
[[[
LoadModule wsgi_module bin/mod_wsgi.so
]]]
3. Generate the trac.wsgi script by running `tracadmin deploy` and
specify WSGIScriptAlias directory in httpd-custom.conf pointing to
trac.wsgi file.
4. Add the desired <directory> configuration to httpd-custom.conf
file.
Generally speaking, the steps and some examples are specified on
the Trac wiki page: https://trac.edgewall.org/wiki/TracModWSGI.
NOTE: The latest version of Python 2.7 is 2.7.11 and it has a bug in
its installer on Windows. See https://bugs.python.org/issue25824. The
problem may cause service load failures when loading `mod_wsgi`. The
workaround specified at https://bugs.python.org/msg256500 should solve
the problem. In short, after installing Python 2.7.11 you should
adjust the Windows Registry key mentioned in the workaround. The bug
has to be solved in the upcoming Python 2.7.x patch update AFAIK.
--
With best regards,
Pavel Lyalyakin
VisualSVN Team
RE: Compiling svn + httpd for windows python 2.x
Posted by "Cooke, Mark" <ma...@siemens.com>.
Hi Pavel,
> -----Original Message-----
> From: Pavel Lyalyakin [mailto:pavel.lyalyakin@visualsvn.com]
> Sent: 22 June 2016 17:11
>
> Hello Mark,
>
> On Wed, Jun 22, 2016 at 1:46 PM, Cooke, Mark <ma...@siemens.com> wrote:
> >
> > Folks,
> >
> > We use subversion with Trac behind httpd on Windows Server. As Trac is
> written in "old" Python (2.x), I have had to resort to building everything
> from source. This is not simple and so I thought I would publish my notes
> here in case it helps anyone else and in the hope that if I have made any
> mistakes, someone will be kind enough to point them out to me!
>
> You say that you had to resort to building SVN and HTTPD on Windows
> from source. Does it mean that all the existing SVN server packages
> such as VisualSVN Server don't work for you by some reason? Why?
>
> I'm asking because VisualSVN Server comes with Python 2.7.x bindings
> and `mod_wsgi` which should perfectly work with Trac.
At the risk of going off-topic for the svn-users list... how do I integrate Trac with VisualSVN? I have got VisualSVN installed in a test server but the `conf` file warns against modifications and web searches have provided no hints except links to an apparently defunct Trac plugin [1] including the comment "The new web browser is powered by HTML5 and ensures easy integration with third-party issue trackers" and a version of the svn book[2]
[1] https://www.visualsvn.com/server/trac/
[2] https://www.visualsvn.com/support/svnbook/serverconfig/httpd/
Cheers,
~ Mark C
> --
> With best regards,
> Pavel Lyalyakin
> VisualSVN Team
Re: Compiling svn + httpd for windows python 2.x
Posted by Pavel Lyalyakin <pa...@visualsvn.com>.
Hello Mark,
On Wed, Jun 22, 2016 at 1:46 PM, Cooke, Mark <ma...@siemens.com> wrote:
>
> Folks,
>
> We use subversion with Trac behind httpd on Windows Server. As Trac is written in "old" Python (2.x), I have had to resort to building everything from source. This is not simple and so I thought I would publish my notes here in case it helps anyone else and in the hope that if I have made any mistakes, someone will be kind enough to point them out to me!
You say that you had to resort to building SVN and HTTPD on Windows
from source. Does it mean that all the existing SVN server packages
such as VisualSVN Server don't work for you by some reason? Why?
I'm asking because VisualSVN Server comes with Python 2.7.x bindings
and `mod_wsgi` which should perfectly work with Trac.
--
With best regards,
Pavel Lyalyakin
VisualSVN Team
RE: Compiling svn + httpd for windows python 2.x
Posted by "Cooke, Mark" <ma...@siemens.com>.
> -----Original Message-----
> From: Stefan Hett [mailto:stefan@egosoft.com]
> Sent: 22 June 2016 17:03
>
> On 6/22/2016 5:30 PM, Cooke, Mark wrote:
> >
> >> -----Original Message-----
> >> From: Stefan Hett [mailto:stefan@egosoft.com]
> >> Sent: 22 June 2016 13:25
> >>
> >> On 6/22/2016 2:01 PM, Cooke, Mark wrote:
> >>>> -----Original Message-----
> >>>> From: Stefan Hett [mailto:stefan@egosoft.com]
> >>>> Sent: 22 June 2016 12:38
> >>>>
> >>>> Hi Mark,
> >>>>
> >>>>> Folks,
> >>>>>
> >>>>> We use subversion with Trac behind httpd on Windows Server. As Trac
> >>>>> is written in "old" Python (2.x), I have had to resort to building
> >>>>> everything from source. This is not simple and so I thought I would
> >>>>> publish my notes here in case it helps anyone else and in the hope
> >>>>> that if I have made any mistakes, someone will be kind enough to
> >>>>> point them out to me!
> >>>>>
> >>>>> ~ Mark C
> >>>>>
> >>>>> These are my notes for building Apache httpd and subversion for use
> >>>>> with Trac [1]. All of the components need to be built using the same
> >>>>> compiler to avoid run-time issues and, since Trac currently relies on
> >>>>> Python 2.x, that means Visual Studio 2008.
> >>>> Why does Python 2.x (aka: 2.7.11) imply having to use VS 2008? MaxSVN
> >>>> [1] is built using VS 2015 Update 1 (the upcoming builds will use VS
> >>>> 2015 Update 2) in combination with Python 2.7.11 and I'm unaware of any
> >>>> issue related to that.
> >>>>
> >>>> [1] http://www.luke1410.de/typo3/index.php?id=97
> >>> I cannot remember specific references now but when I looked into this
> >>> before it is because the official 2.x line is compiled using VC9. When
> >>> memory is passed between applications relying on different version of
> >>> the CRT then you can end up with hard to diagnose memory corruptions
> >>> that eventually cause problems.
> >> This is correct. If you are building a project linking in the CRT, you
> >> should ensure that all DLLs and libraries you build with use the same
> >> CRT. Mixing different CRT versions is unsupported and you will have
> >> undefined behavior. The possibilities for memory corruptions is the most
> >> prominent effect one might observe, but there are other problems/issues
> >> this will cause too.
> >>> Digging a bit I found this thread [1] that highlights a similar CRT
> >>> issue in python modules such as psycopg2 (which I use for PostGreSQL)
> >>> that use the CRT internally...
> >>>
> >>> [1] https://groups.google.com/forum/#!topic/modwsgi/ATtKX6qWLXc
> >> As far as I skimmed through the thread the problem here is not with
> >> python being compiled using VC9, but rather the module the python script
> >> tries to load (psycopg2, which according to the web seems to be some
> >> postgre-sql-related library) was compiled with a different CRT than the
> >> Apache binaries. That's a general issue, yes, and that's why I don't
> >> distribute the svn-apache modules with MaxSVN, even though they are
> >> built as part of the buildprocess (and utilized for testing). It's
> >> imperative that the CRT for the Apache modules matches the ones used to
> >> build Apache and that only the distributor of the specific Windows
> >> Apache module can ensure.
> > ...and that is my problem: Trac is running under py2.7 (via mod_wsgi from
> > httpd) and using psycopg2 to connect to a PostGreSQL backend. AFAIK, python
> > 2.7.x windows extensions should be built using VC++ 2008 to match the python
> > build, so it is best if I build httpd, mod_wsgi and svn using that compiler
> > too (especially as local policy means I need to be able to apply the latest
> > e.g. OpenSSL updates as they come out).
>
> Certainly that's working. But since you are recompiling psycopg2
> yourself, why don't you just recompile that then using VS 2015? Or are
> there other python libs involved?
I am not compiling python or any of the support libs that we use, I decided to compile httpd and svn instead.
I will have another go with VisualSVN again to see if it works for us now.
> --
> Regards,
> Stefan Hett
Re: Compiling svn + httpd for windows python 2.x
Posted by Stefan Hett <st...@egosoft.com>.
On 6/22/2016 5:30 PM, Cooke, Mark wrote:
>
>> -----Original Message-----
>> From: Stefan Hett [mailto:stefan@egosoft.com]
>> Sent: 22 June 2016 13:25
>> To: users@subversion.apache.org
>> Subject: Re: Compiling svn + httpd for windows python 2.x
>>
>> On 6/22/2016 2:01 PM, Cooke, Mark wrote:
>>>> -----Original Message-----
>>>> From: Stefan Hett [mailto:stefan@egosoft.com]
>>>> Sent: 22 June 2016 12:38
>>>>
>>>> Hi Mark,
>>>>
>>>>> Folks,
>>>>>
>>>>> We use subversion with Trac behind httpd on Windows Server. As Trac
>>>>> is written in "old" Python (2.x), I have had to resort to building
>>>>> everything from source. This is not simple and so I thought I would
>>>>> publish my notes here in case it helps anyone else and in the hope
>>>>> that if I have made any mistakes, someone will be kind enough to
>>>>> point them out to me!
>>>>>
>>>>> ~ Mark C
>>>>>
>>>>> These are my notes for building Apache httpd and subversion for use
>>>>> with Trac [1]. All of the components need to be built using the same
>>>>> compiler to avoid run-time issues and, since Trac currently relies on
>>>>> Python 2.x, that means Visual Studio 2008.
>>>> Why does Python 2.x (aka: 2.7.11) imply having to use VS 2008? MaxSVN
>>>> [1] is built using VS 2015 Update 1 (the upcoming builds will use VS
>>>> 2015 Update 2) in combination with Python 2.7.11 and I'm unaware of any
>>>> issue related to that.
>>>>
>>>> [1] http://www.luke1410.de/typo3/index.php?id=97
>>> I cannot remember specific references now but when I looked into this
>>> before it is because the official 2.x line is compiled using VC9. When
>>> memory is passed between applications relying on different version of
>>> the CRT then you can end up with hard to diagnose memory corruptions
>>> that eventually cause problems.
>> This is correct. If you are building a project linking in the CRT, you
>> should ensure that all DLLs and libraries you build with use the same
>> CRT. Mixing different CRT versions is unsupported and you will have
>> undefined behavior. The possibilities for memory corruptions is the most
>> prominent effect one might observe, but there are other problems/issues
>> this will cause too.
>>> Digging a bit I found this thread [1] that highlights a similar CRT
>>> issue in python modules such as psycopg2 (which I use for PostGreSQL)
>>> that use the CRT internally...
>>>
>>> [1] https://groups.google.com/forum/#!topic/modwsgi/ATtKX6qWLXc
>> As far as I skimmed through the thread the problem here is not with
>> python being compiled using VC9, but rather the module the python script
>> tries to load (psycopg2, which according to the web seems to be some
>> postgre-sql-related library) was compiled with a different CRT than the
>> Apache binaries. That's a general issue, yes, and that's why I don't
>> distribute the svn-apache modules with MaxSVN, even though they are
>> built as part of the buildprocess (and utilized for testing). It's
>> imperative that the CRT for the Apache modules matches the ones used to
>> build Apache and that only the distributor of the specific Windows
>> Apache module can ensure.
> ...and that is my problem: Trac is running under py2.7 (via mod_wsgi from httpd) and using psycopg2 to connect to a PostGreSQL backend. AFAIK, python 2.7.x windows extensions should be built using VC++ 2008 to match the python build, so it is best if I build httpd, mod_wsgi and svn using that compiler too (especially as local policy means I need to be able to apply the latest e.g. OpenSSL updates as they come out).
Certainly that's working. But since you are recompiling psycopg2
yourself, why don't you just recompile that then using VS 2015? Or are
there other python libs involved?
--
Regards,
Stefan Hett
RE: Compiling svn + httpd for windows python 2.x
Posted by "Cooke, Mark" <ma...@siemens.com>.
> -----Original Message-----
> From: Stefan Hett [mailto:stefan@egosoft.com]
> Sent: 22 June 2016 13:25
> To: users@subversion.apache.org
> Subject: Re: Compiling svn + httpd for windows python 2.x
>
> On 6/22/2016 2:01 PM, Cooke, Mark wrote:
> >> -----Original Message-----
> >> From: Stefan Hett [mailto:stefan@egosoft.com]
> >> Sent: 22 June 2016 12:38
> >>
> >> Hi Mark,
> >>
> >>> Folks,
> >>>
> >>> We use subversion with Trac behind httpd on Windows Server. As Trac
> >>> is written in "old" Python (2.x), I have had to resort to building
> >>> everything from source. This is not simple and so I thought I would
> >>> publish my notes here in case it helps anyone else and in the hope
> >>> that if I have made any mistakes, someone will be kind enough to
> >>> point them out to me!
> >>>
> >>> ~ Mark C
> >>>
> >>> These are my notes for building Apache httpd and subversion for use
> >>> with Trac [1]. All of the components need to be built using the same
> >>> compiler to avoid run-time issues and, since Trac currently relies on
> >>> Python 2.x, that means Visual Studio 2008.
> >>
> >> Why does Python 2.x (aka: 2.7.11) imply having to use VS 2008? MaxSVN
> >> [1] is built using VS 2015 Update 1 (the upcoming builds will use VS
> >> 2015 Update 2) in combination with Python 2.7.11 and I'm unaware of any
> >> issue related to that.
> >>
> >> [1] http://www.luke1410.de/typo3/index.php?id=97
> >
> > I cannot remember specific references now but when I looked into this
> > before it is because the official 2.x line is compiled using VC9. When
> > memory is passed between applications relying on different version of
> > the CRT then you can end up with hard to diagnose memory corruptions
> > that eventually cause problems.
>
> This is correct. If you are building a project linking in the CRT, you
> should ensure that all DLLs and libraries you build with use the same
> CRT. Mixing different CRT versions is unsupported and you will have
> undefined behavior. The possibilities for memory corruptions is the most
> prominent effect one might observe, but there are other problems/issues
> this will cause too.
> >
> > Digging a bit I found this thread [1] that highlights a similar CRT
> > issue in python modules such as psycopg2 (which I use for PostGreSQL)
> > that use the CRT internally...
> >
> > [1] https://groups.google.com/forum/#!topic/modwsgi/ATtKX6qWLXc
>
> As far as I skimmed through the thread the problem here is not with
> python being compiled using VC9, but rather the module the python script
> tries to load (psycopg2, which according to the web seems to be some
> postgre-sql-related library) was compiled with a different CRT than the
> Apache binaries. That's a general issue, yes, and that's why I don't
> distribute the svn-apache modules with MaxSVN, even though they are
> built as part of the buildprocess (and utilized for testing). It's
> imperative that the CRT for the Apache modules matches the ones used to
> build Apache and that only the distributor of the specific Windows
> Apache module can ensure.
...and that is my problem: Trac is running under py2.7 (via mod_wsgi from httpd) and using psycopg2 to connect to a PostGreSQL backend. AFAIK, python 2.7.x windows extensions should be built using VC++ 2008 to match the python build, so it is best if I build httpd, mod_wsgi and svn using that compiler too (especially as local policy means I need to be able to apply the latest e.g. OpenSSL updates as they come out).
> > If you can guarantee to me that isn't going to happen then it would make my
> > life easier!
>
> I might miss something here which exceeds my knowledge (tbh I've only
> average experience with Apache httpd and python), but as long as there's
> nothing pulled in from python when you build Apache (and I'm quite sure
> that's not the case), python is just used as a build tool (for building
> SVN, Apache) and lateron as a script tool (from Apache). It doesn't
> matter how the python executable is build in this regard and which CRT
> it uses. It however matters a lot for the modules you'd pull in with the
> python import statement.
Hmm, I started down this path because I had problems and I think they were as mentioned above. Life was good when Apache provided VC9 windows binaries and I could get svn with the python bindings from alagazam...
~ Mark C
> Regards,
> Stefan Hett
Re: Compiling svn + httpd for windows python 2.x
Posted by Stefan Hett <st...@egosoft.com>.
On 6/22/2016 2:01 PM, Cooke, Mark wrote:
>> -----Original Message-----
>> From: Stefan Hett [mailto:stefan@egosoft.com]
>> Sent: 22 June 2016 12:38
>>
>> Hi Mark,
>>
>>> Folks,
>>>
>>> We use subversion with Trac behind httpd on Windows Server. As Trac is
>>> written in "old" Python (2.x), I have had to resort to building everything
>>> from source. This is not simple and so I thought I would publish my notes
>>> here in case it helps anyone else and in the hope that if I have made any
>>> mistakes, someone will be kind enough to point them out to me!
>>>
>>> ~ Mark C
>>>
>>> These are my notes for building Apache httpd and subversion for use with
>>> Trac [1]. All of the components need to be built using the same compiler to
>>> avoid run-time issues and, since Trac currently relies on Python 2.x, that
>>> means Visual Studio 2008.
>> Why does Python 2.x (aka: 2.7.11) imply having to use VS 2008? MaxSVN
>> [1] is built using VS 2015 Update 1 (the upcoming builds will use VS
>> 2015 Update 2) in combination with Python 2.7.11 and I'm unaware of any
>> issue related to that.
>>
>> [1] http://www.luke1410.de/typo3/index.php?id=97
> I cannot remember specific references now but when I looked into this before it is because the official 2.x line is compiled using VC9. When memory is passed between applications relying on different version of the CRT then you can end up with hard to diagnose memory corruptions that eventually cause problems.
This is correct. If you are building a project linking in the CRT, you
should ensure that all DLLs and libraries you build with use the same
CRT. Mixing different CRT versions is unsupported and you will have
undefined behavior. The possibilities for memory corruptions is the most
prominent effect one might observe, but there are other problems/issues
this will cause too.
> Digging a bit I found this thread [1] that highlights a similar CRT issue in python modules such as psycopg2 (which I use for PostGreSQL) that use the CRT internally...
>
> [1] https://groups.google.com/forum/#!topic/modwsgi/ATtKX6qWLXc
As far as I skimmed through the thread the problem here is not with
python being compiled using VC9, but rather the module the python script
tries to load (psycopg2, which according to the web seems to be some
postgre-sql-related library) was compiled with a different CRT than the
Apache binaries. That's a general issue, yes, and that's why I don't
distribute the svn-apache modules with MaxSVN, even though they are
built as part of the buildprocess (and utilized for testing). It's
imperative that the CRT for the Apache modules matches the ones used to
build Apache and that only the distributor of the specific Windows
Apache module can ensure.
> If you can guarantee to me that isn't going to happen then it would make my life easier!
I might miss something here which exceeds my knowledge (tbh I've only
average experience with Apache httpd and python), but as long as there's
nothing pulled in from python when you build Apache (and I'm quite sure
that's not the case), python is just used as a build tool (for building
SVN, Apache) and lateron as a script tool (from Apache). It doesn't
matter how the python executable is build in this regard and which CRT
it uses. It however matters a lot for the modules you'd pull in with the
python import statement.
Regards,
Stefan Hett
RE: Compiling svn + httpd for windows python 2.x
Posted by "Cooke, Mark" <ma...@siemens.com>.
> -----Original Message-----
> From: Stefan Hett [mailto:stefan@egosoft.com]
> Sent: 22 June 2016 12:38
>
> Hi Mark,
>
> > Folks,
> >
> > We use subversion with Trac behind httpd on Windows Server. As Trac is
> > written in "old" Python (2.x), I have had to resort to building everything
> > from source. This is not simple and so I thought I would publish my notes
> > here in case it helps anyone else and in the hope that if I have made any
> > mistakes, someone will be kind enough to point them out to me!
> >
> > ~ Mark C
> >
> > These are my notes for building Apache httpd and subversion for use with
> > Trac [1]. All of the components need to be built using the same compiler to
> > avoid run-time issues and, since Trac currently relies on Python 2.x, that
> > means Visual Studio 2008.
>
> Why does Python 2.x (aka: 2.7.11) imply having to use VS 2008? MaxSVN
> [1] is built using VS 2015 Update 1 (the upcoming builds will use VS
> 2015 Update 2) in combination with Python 2.7.11 and I'm unaware of any
> issue related to that.
>
> [1] http://www.luke1410.de/typo3/index.php?id=97
I cannot remember specific references now but when I looked into this before it is because the official 2.x line is compiled using VC9. When memory is passed between applications relying on different version of the CRT then you can end up with hard to diagnose memory corruptions that eventually cause problems.
Digging a bit I found this thread [1] that highlights a similar CRT issue in python modules such as psycopg2 (which I use for PostGreSQL) that use the CRT internally...
[1] https://groups.google.com/forum/#!topic/modwsgi/ATtKX6qWLXc
If you can guarantee to me that isn't going to happen then it would make my life easier!
~ Mark C
Re: Compiling svn + httpd for windows python 2.x
Posted by Stefan Hett <st...@egosoft.com>.
Hi Mark,
> Folks,
>
> We use subversion with Trac behind httpd on Windows Server. As Trac is written in "old" Python (2.x), I have had to resort to building everything from source. This is not simple and so I thought I would publish my notes here in case it helps anyone else and in the hope that if I have made any mistakes, someone will be kind enough to point them out to me!
>
> ~ Mark C
>
> These are my notes for building Apache httpd and subversion for use with Trac [1]. All of the components need to be built using the same compiler to avoid run-time issues and, since Trac currently relies on Python 2.x, that means Visual Studio 2008.
Why does Python 2.x (aka: 2.7.11) imply having to use VS 2008? MaxSVN
[1] is built using VS 2015 Update 1 (the upcoming builds will use VS
2015 Update 2) in combination with Python 2.7.11 and I'm unaware of any
issue related to that.
[1] http://www.luke1410.de/typo3/index.php?id=97
--
Regards,
Stefan Hett