You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pylucene-dev@lucene.apache.org by Jeremy Michels <je...@gmail.com> on 2012/04/28 20:14:13 UTC
Building PyLucene on Windows
Would you be willing to share your makefile? I am having quite a bit of
trouble with this as well.
Re: Building PyLucene on Windows
Posted by Bill Janssen <ja...@parc.com>.
Andi Vajda <va...@apache.org> wrote:
> The patch to setuptools is for Linux only unless I'm mistaken.
Yes, you'll see if you look at that script that I don't apply any
patches to setuptools before I install it. I do have to apply a patch
to jcc's setup.py to generate the appropriate .lib file, though.
http://uplib.parc.com/hg/uplib/file/2013b08512fa/patches/jcc-2.9-mingw-PATCH
Not sure if that's still necessary.
Bill
Re: Building PyLucene on Windows
Posted by Andi Vajda <va...@apache.org>.
On Apr 28, 2012, at 16:35, Bill Janssen <ja...@parc.com> wrote:
> Jeremy Michels <je...@gmail.com> wrote:
>
>> Would you be willing to share your makefile? I am having quite a bit
>> of trouble with this as well.
>
> I build PyLucene on Windows regularly. I use mingw to compile and link.
> My build script is at
> http://uplib.parc.com/hg/uplib/file/2013b08512fa/win32/build-windows-prereqs.sh;
> look for "-- jcc --". Note that you must have an appropriately patched
> setuptools installed, first.
The patch to setuptools is for Linux only unless I'm mistaken.
Andi..
>
> Rather than editing the Makefile for PyLucene, I just override the
> appropriate make variables in the build environment.
>
> Bill
Re: Building PyLucene on Windows
Posted by Bill Janssen <ja...@parc.com>.
Jeremy Michels <je...@gmail.com> wrote:
> Would you be willing to share your makefile? I am having quite a bit
> of trouble with this as well.
I build PyLucene on Windows regularly. I use mingw to compile and link.
My build script is at
http://uplib.parc.com/hg/uplib/file/2013b08512fa/win32/build-windows-prereqs.sh;
look for "-- jcc --". Note that you must have an appropriately patched
setuptools installed, first.
Rather than editing the Makefile for PyLucene, I just override the
appropriate make variables in the build environment.
Bill
AW: AW: Building PyLucene on Windows
Posted by Thomas Koch <ko...@orbiteam.de>.
Jeremy ,
not sure if I can be of great help here either ... the error tells me that
compiling the JCC-generated lucene-sources (i.e. C++ code) fails. This could
either mean
1) JCC code generator failed, i.e. your JCC build is broken
2) your compiler is unable to understand the generated code
Ad1) I attach a directory listing of my build/_lucene dir so you can check
at least if file size matches...
I doubt that JCC is broken if the build went fine.
Ad2) The error looks like some pre-processor issue - e.g. in functions.cpp
there is
#if defined(_MSC_VER) || defined(__SUNPRO_CC)
which is a pre-processor statement (or macro) that should be understood by
recent compilers.
I cant' see where the error actually occurs (\_lucene\[F.h(1) ??) but my
strong guess it's the compiler that's crap.
> Windows SDK C/C++ compiler (v7.0)
As mentioned before I'd advise to use the version of Visual Studio C++
compiler that's used to build your Python version!
Start your python interpreter:
Python 2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
This tells you it's compiled with Visual C++ 2008 (MSC_VER=1500)
Cf.
stackoverflow.com/questions/2676763/what-version-of-visual-studio-is-this-py
thon-compiled-with
Regards
Thomas
> -----Ursprüngliche Nachricht-----
> Von: Jeremy Michels [mailto:jeremy.d.michels@gmail.com]
> Gesendet: Montag, 30. April 2012 18:20
> An: pylucene-dev@lucene.apache.org
> Betreff: Re: AW: Building PyLucene on Windows
>
> Tom -
>
> Thank you for sharing your expertise on this matter. I was unable to use
the
> pre-built binaries as I am using a 64-bit system (attempting to do so
would
> results in the error: ImportError: DLL load failed: %1 is not a valid
Win32
> application).
>
> This is my setup:
> Windows 7 (64-bit)
> Python 2.7.2 (64 bit)
> Java 1.7 (jdk1.7.0)
> Ant 1.8.3
> Windows SDK C/C++ compiler (v7.0)
> Cygwin
>
> I was able to build JCC without any apparent problems. Below are my
> Makefile settings:
>
> ANT=/cygdrive/c/Program\ Files/Ant/apache-ant-1.8.3/bin/ant
> JAVA_HOME=C:\\Program Files\\Java\\jdk1.7.0
> PREFIX_PYTHON=/cygdrive/c/Program\ Files/Python
> PYTHON=$(PREFIX_PYTHON)/python.exe
> JCC=$(PYTHON) -m jcc --shared
> NUM_FILES=4
>
> Previously I had the "--find-jvm-dll" option included on the "JCC"
> line, but this gave me the error "ValueError: jvm.dll could not be found,"
so I
> removed the option and added the location of the jvm.dll file to my path
and
> I no longer received this error.
>
> However, running "make" with the specifications given above results in the
> error:
>
> c:\users\jeremy\downloads\pylucene-3.5.0-3\build\_lucene\[F.h(1) : fatal
> error C
> 1016: #if[n]def expected an identifier
> error: command 'cl.exe' failed with exit status 2
> Makefile:312: recipe for target `compile' failed
> make: *** [compile] Error 1
>
> To be honest, I don't really know what this means and am feeling a bit out
of
> my league at this point. However, any further guidance you might have
> would be appreciated.
>
> Best Regards,
>
> Jeremy
Re: AW: Building PyLucene on Windows
Posted by Andi Vajda <va...@apache.org>.
On May 3, 2012, at 14:35, Jeremy Michels <je...@gmail.com> wrote:
> Thanks again for your help. I got the latest JCC and this seemed to
> solve the problem, as Andi suggested. However, running 'make' then
> gave me the error:
>
> Traceback (most recent call last):
> File "C:\Program Files\Python\lib\runpy.py", line 162, in _run_module_as_main
> "__main__", fname, loader, pkg_name)
> File "C:\Program Files\Python\lib\runpy.py", line 72, in _run_code
> exec code in run_globals
> File "C:\Program Files\Python\lib\site-packages\jcc-2.12-py2.7-win-amd64.egg\j
> cc\__main__.py", line 104, in <module>
> cpp.jcc(sys.argv)
> File "C:\Program Files\Python\lib\site-packages\jcc-2.12-py2.7-win-amd64.egg\j
> cc\cpp.py", line 644, in jcc
> raise ValueError, (cls, 'python class name already in use, use --rename', na
> me, pythonNames[name])
> ValueError: (<Class: class org.apache.lucene.analysis.de.GermanStemmer>, 'python
> class name already in use, use --rename', u'GermanStemmer', <Class: class org.t
> artarus.snowball.ext.GermanStemmer>)
> Makefile:312: recipe for target `compile' failed
> make: *** [compile] Error 127
>
> I couldn't figure out how to correctly add the rename argument in the
> Makefile.
> Eventually I got frustrated, and just edited out the line
> (line 644) raising the exception in the cpp.py file, adding a 'pass'
> statement instead.
That's going to be causing random failures and crashes. You should instead use the latest PyLucene code in the branch_3x branch. This is turning into the 3.6 release shortly.
Andi..
> Then the installation seemed to go ahead fine.
> However, the wisdom of my workaround seemed questionable, at best.
> What are your thoughts? What is the appropriate syntax for 'rename' ?
>
> Thanks again,
>
> Jeremy
>
>
> On Thu, May 3, 2012 at 7:45 AM, Andi Vajda <va...@apache.org> wrote:
>> Hi Jeremy,
>>
>> On Apr 30, 2012, at 9:19, Jeremy Michels <je...@gmail.com> wrote:
>>
>>> Thank you for sharing your expertise on this matter. I was unable to
>>> use the pre-built binaries as I am using a 64-bit system (attempting
>>> to do so would results in the error: ImportError: DLL load failed: %1
>>> is not a valid Win32 application).
>>>
>>> This is my setup:
>>> Windows 7 (64-bit)
>>> Python 2.7.2 (64 bit)
>>> Java 1.7 (jdk1.7.0)
>>> Ant 1.8.3
>>> Windows SDK C/C++ compiler (v7.0)
>>> Cygwin
>>>
>>> I was able to build JCC without any apparent problems. Below are my
>>> Makefile settings:
>>>
>>> ANT=/cygdrive/c/Program\ Files/Ant/apache-ant-1.8.3/bin/ant
>>> JAVA_HOME=C:\\Program Files\\Java\\jdk1.7.0
>>> PREFIX_PYTHON=/cygdrive/c/Program\ Files/Python
>>> PYTHON=$(PREFIX_PYTHON)/python.exe
>>> JCC=$(PYTHON) -m jcc --shared
>>> NUM_FILES=4
>>>
>>> Previously I had the "--find-jvm-dll" option included on the "JCC"
>>> line, but this gave me the error "ValueError: jvm.dll could not be
>>> found," so I removed the option and added the location of the jvm.dll
>>> file to my path and I no longer received this error.
>>>
>>> However, running "make" with the specifications given above results in
>>> the error:
>>>
>>> c:\users\jeremy\downloads\pylucene-3.5.0-3\build\_lucene\[F.h(1) : fatal error C
>>
>> I've seen that error and it seems to be a bug in JCC related to using Java 7 and generics (which is fine).
>>
>> I fixed that bug a couple of weeks ago. You should get the latest sources from the branch_3_x jcc sources or wait for the PyLucene 3.6 release which should happen within a week.
>>
>> Andi..
>>
>>> 1016: #if[n]def expected an identifier
>>> error: command 'cl.exe' failed with exit status 2
>>> Makefile:312: recipe for target `compile' failed
>>> make: *** [compile] Error 1
>>>
>>> To be honest, I don't really know what this means and am feeling a bit
>>> out of my league at this point. However, any further guidance you
>>> might have would be appreciated.
>>>
>>> Best Regards,
>>>
>>> Jeremy
Re: AW: Building PyLucene on Windows
Posted by Jeremy Michels <je...@gmail.com>.
Thanks again for your help. I got the latest JCC and this seemed to
solve the problem, as Andi suggested. However, running 'make' then
gave me the error:
Traceback (most recent call last):
File "C:\Program Files\Python\lib\runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "C:\Program Files\Python\lib\runpy.py", line 72, in _run_code
exec code in run_globals
File "C:\Program Files\Python\lib\site-packages\jcc-2.12-py2.7-win-amd64.egg\j
cc\__main__.py", line 104, in <module>
cpp.jcc(sys.argv)
File "C:\Program Files\Python\lib\site-packages\jcc-2.12-py2.7-win-amd64.egg\j
cc\cpp.py", line 644, in jcc
raise ValueError, (cls, 'python class name already in use, use --rename', na
me, pythonNames[name])
ValueError: (<Class: class org.apache.lucene.analysis.de.GermanStemmer>, 'python
class name already in use, use --rename', u'GermanStemmer', <Class: class org.t
artarus.snowball.ext.GermanStemmer>)
Makefile:312: recipe for target `compile' failed
make: *** [compile] Error 127
I couldn't figure out how to correctly add the rename argument in the
Makefile. Eventually I got frustrated, and just edited out the line
(line 644) raising the exception in the cpp.py file, adding a 'pass'
statement instead. Then the installation seemed to go ahead fine.
However, the wisdom of my workaround seemed questionable, at best.
What are your thoughts? What is the appropriate syntax for 'rename' ?
Thanks again,
Jeremy
On Thu, May 3, 2012 at 7:45 AM, Andi Vajda <va...@apache.org> wrote:
> Hi Jeremy,
>
> On Apr 30, 2012, at 9:19, Jeremy Michels <je...@gmail.com> wrote:
>
>> Thank you for sharing your expertise on this matter. I was unable to
>> use the pre-built binaries as I am using a 64-bit system (attempting
>> to do so would results in the error: ImportError: DLL load failed: %1
>> is not a valid Win32 application).
>>
>> This is my setup:
>> Windows 7 (64-bit)
>> Python 2.7.2 (64 bit)
>> Java 1.7 (jdk1.7.0)
>> Ant 1.8.3
>> Windows SDK C/C++ compiler (v7.0)
>> Cygwin
>>
>> I was able to build JCC without any apparent problems. Below are my
>> Makefile settings:
>>
>> ANT=/cygdrive/c/Program\ Files/Ant/apache-ant-1.8.3/bin/ant
>> JAVA_HOME=C:\\Program Files\\Java\\jdk1.7.0
>> PREFIX_PYTHON=/cygdrive/c/Program\ Files/Python
>> PYTHON=$(PREFIX_PYTHON)/python.exe
>> JCC=$(PYTHON) -m jcc --shared
>> NUM_FILES=4
>>
>> Previously I had the "--find-jvm-dll" option included on the "JCC"
>> line, but this gave me the error "ValueError: jvm.dll could not be
>> found," so I removed the option and added the location of the jvm.dll
>> file to my path and I no longer received this error.
>>
>> However, running "make" with the specifications given above results in
>> the error:
>>
>> c:\users\jeremy\downloads\pylucene-3.5.0-3\build\_lucene\[F.h(1) : fatal error C
>
> I've seen that error and it seems to be a bug in JCC related to using Java 7 and generics (which is fine).
>
> I fixed that bug a couple of weeks ago. You should get the latest sources from the branch_3_x jcc sources or wait for the PyLucene 3.6 release which should happen within a week.
>
> Andi..
>
>> 1016: #if[n]def expected an identifier
>> error: command 'cl.exe' failed with exit status 2
>> Makefile:312: recipe for target `compile' failed
>> make: *** [compile] Error 1
>>
>> To be honest, I don't really know what this means and am feeling a bit
>> out of my league at this point. However, any further guidance you
>> might have would be appreciated.
>>
>> Best Regards,
>>
>> Jeremy
Re: AW: Building PyLucene on Windows
Posted by Andi Vajda <va...@apache.org>.
Hi Jeremy,
On Apr 30, 2012, at 9:19, Jeremy Michels <je...@gmail.com> wrote:
> Thank you for sharing your expertise on this matter. I was unable to
> use the pre-built binaries as I am using a 64-bit system (attempting
> to do so would results in the error: ImportError: DLL load failed: %1
> is not a valid Win32 application).
>
> This is my setup:
> Windows 7 (64-bit)
> Python 2.7.2 (64 bit)
> Java 1.7 (jdk1.7.0)
> Ant 1.8.3
> Windows SDK C/C++ compiler (v7.0)
> Cygwin
>
> I was able to build JCC without any apparent problems. Below are my
> Makefile settings:
>
> ANT=/cygdrive/c/Program\ Files/Ant/apache-ant-1.8.3/bin/ant
> JAVA_HOME=C:\\Program Files\\Java\\jdk1.7.0
> PREFIX_PYTHON=/cygdrive/c/Program\ Files/Python
> PYTHON=$(PREFIX_PYTHON)/python.exe
> JCC=$(PYTHON) -m jcc --shared
> NUM_FILES=4
>
> Previously I had the "--find-jvm-dll" option included on the "JCC"
> line, but this gave me the error "ValueError: jvm.dll could not be
> found," so I removed the option and added the location of the jvm.dll
> file to my path and I no longer received this error.
>
> However, running "make" with the specifications given above results in
> the error:
>
> c:\users\jeremy\downloads\pylucene-3.5.0-3\build\_lucene\[F.h(1) : fatal error C
I've seen that error and it seems to be a bug in JCC related to using Java 7 and generics (which is fine).
I fixed that bug a couple of weeks ago. You should get the latest sources from the branch_3_x jcc sources or wait for the PyLucene 3.6 release which should happen within a week.
Andi..
> 1016: #if[n]def expected an identifier
> error: command 'cl.exe' failed with exit status 2
> Makefile:312: recipe for target `compile' failed
> make: *** [compile] Error 1
>
> To be honest, I don't really know what this means and am feeling a bit
> out of my league at this point. However, any further guidance you
> might have would be appreciated.
>
> Best Regards,
>
> Jeremy
Re: AW: Building PyLucene on Windows
Posted by Jeremy Michels <je...@gmail.com>.
Tom -
Thank you for sharing your expertise on this matter. I was unable to
use the pre-built binaries as I am using a 64-bit system (attempting
to do so would results in the error: ImportError: DLL load failed: %1
is not a valid Win32 application).
This is my setup:
Windows 7 (64-bit)
Python 2.7.2 (64 bit)
Java 1.7 (jdk1.7.0)
Ant 1.8.3
Windows SDK C/C++ compiler (v7.0)
Cygwin
I was able to build JCC without any apparent problems. Below are my
Makefile settings:
ANT=/cygdrive/c/Program\ Files/Ant/apache-ant-1.8.3/bin/ant
JAVA_HOME=C:\\Program Files\\Java\\jdk1.7.0
PREFIX_PYTHON=/cygdrive/c/Program\ Files/Python
PYTHON=$(PREFIX_PYTHON)/python.exe
JCC=$(PYTHON) -m jcc --shared
NUM_FILES=4
Previously I had the "--find-jvm-dll" option included on the "JCC"
line, but this gave me the error "ValueError: jvm.dll could not be
found," so I removed the option and added the location of the jvm.dll
file to my path and I no longer received this error.
However, running "make" with the specifications given above results in
the error:
c:\users\jeremy\downloads\pylucene-3.5.0-3\build\_lucene\[F.h(1) : fatal error C
1016: #if[n]def expected an identifier
error: command 'cl.exe' failed with exit status 2
Makefile:312: recipe for target `compile' failed
make: *** [compile] Error 1
To be honest, I don't really know what this means and am feeling a bit
out of my league at this point. However, any further guidance you
might have would be appreciated.
Best Regards,
Jeremy
AW: Building PyLucene on Windows
Posted by Thomas Koch <ko...@orbiteam.de>.
Jeremy, (and Chris)
note that there are some pre-built binaries of PyLucene/JCC available for
Windows and other platforms in the pylucene extras project:
http://code.google.com/a/apache-extras.org/p/pylucene-extra/
Anyway, I can shortly describe my setup for the build process :
I am currently using
- pylucene-3.x with Python 2.6 and Pyrthon 2.7 on windows (win7-32) with
the following tools:
- Java 1.6 (jdk1.6.0_06)
- Ant 1.8.0
- MS-Visual-Studio9 compiler (Microsoft Visual C++ 2008 Express Edition)
- Cygwin (for make basically)
I also tried a different toolchain with MinGW and GNU Make on windows once,
but had some problems so I'm currently happy with the above setup.
Here are the instructions for the step-by-step build process:
I. download the sources
II. build JCC
0. make sure to set JAVAHOME (or edit jcc/setup.py ...)
e.g. JAVA_HOME=C:\Program Files\Java\jdk1.6.0_06
1. cd jcc
2. python setup.py build
3. python setup.py install
III. build PyLucene
0. make sure you have Python, Java and Ant installed (I also use cygwin for
make process)
1. edit the Makefile and adapt to your configuration/environment
- there are some pre-set sample configs you may choose (i.e. uncomment)
- you may need to change path details and version numbers ...
e.g. (my env)
# Windows (Win32, Python 2.7, Java 1.6, ant 1.8.0 - all 32-Bit!)
ANT=/cygdrive/f/devel/apache-ant-1.8.0/bin/ant
JAVA_HOME=C:\\Program Files\\Java\\jdk1.6.0_06
PREFIX_PYTHON=/cygdrive/c/Python27
PYTHON=$(PREFIX_PYTHON)/python.exe
JCC=$(PYTHON) -m jcc --shared --find-jvm-dll
NUM_FILES=4
2. (in cygwin) run make i.e.
make
make install
make test
Hope that helps!
kind regards
Thomas
--
OrbiTeam Software GmbH & Co. KG, Germany
http://www.orbiteam.de
> -----Ursprüngliche Nachricht-----
> Von: Jeremy Michels [mailto:jeremy.d.michels@gmail.com]
> Gesendet: Samstag, 28. April 2012 20:14
> An: pylucene-dev@lucene.apache.org
> Betreff: Building PyLucene on Windows
>
> Would you be willing to share your makefile? I am having quite a bit of
> trouble with this as well.