You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@lucene.apache.org by Andi Vajda <va...@apache.org> on 2009/03/11 18:06:47 UTC
[VOTE] Release PyLucene 2.4.1
With the recent release of Java Lucene 2.4.1, it's high time PyLucene get
its first release out as an Apache subproject. I hope I got all the steps
right and all the things into the right place.
Release candidate 1 of PyLucene 2.4.1 is available from:
http://people.apache.org/~vajda/staging_area/
The list of changes is available from:
http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_2_4/CHANGES
Please vote to release this artifacts as PyLucene 2.4.1.
Thanks !
Andi..
ps: the KEYS file for PyLucene release signing is at:
http://svn.apache.org/repos/asf/lucene/pylucene/dist/KEYS
Re: [VOTE] Release PyLucene 2.4.1
Posted by Michael McCandless <lu...@mikemccandless.com>.
Andi, one small thing: when you upload a new RC, you should remove the
old ones just to reduce chances for confusion...
Mike
Andi Vajda wrote:
>
> On Wed, 11 Mar 2009, Aric Coady wrote:
>
>> On Mar 11, 2009, at 10:06 AM, Andi Vajda wrote:
>>> With the recent release of Java Lucene 2.4.1, it's high time
>>> PyLucene get its first release out as an Apache subproject. I hope
>>> I got all the steps right and all the things into the right place.
>>
>> Thanks.
>>
>>> Release candidate 1 of PyLucene 2.4.1 is available from:
>>>
>>> http://people.apache.org/~vajda/staging_area/
>>
>> Make install is failing with python 2.6 because of a conflict
>> between the new collections module and the builtin one.
>
> Ugh. I moved the collections.py file to a python sub-directory.
> This is only a problem during the build. The collections.py file is
> installed into the lucene egg where it is accessible from under the
> lucene package then.
>
> I also backed out the earlier config.py build trick as it fails when
> install is used without build first.
>
> I uploaded a new release candidate, rc3, to the staging area:
> http://people.apache.org/~vajda/staging_area/
>
> Thank you for the bug report.
>
> Andi..
>
>>
>> ...
>> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
>> python2.6/site-packages/setuptools-0.6c9-py2.6.egg/setuptools/
>> command/easy_install.py", line 21, in <module>
>> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
>> python2.6/site-packages/setuptools-0.6c9-py2.6.egg/setuptools/
>> package_index.py", line 2, in <module>
>> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
>> python2.6/urlparse.py", line 84, in <module>
>> from collections import namedtuple
>> ImportError: cannot import name namedtuple
>> make: *** [install] Error 255
>>
>> Probably will be necessary to rename or move collections.py.
>>
>> -Coady
Re: [VOTE] Release PyLucene 2.4.1
Posted by Michael McCandless <lu...@mikemccandless.com>.
Bill Janssen wrote:
> Michael McCandless <lu...@mikemccandless.com> wrote:
>
>> Michael McCandless wrote:
>>
>>>>> Ahhhh OK; this is quite confusing.
>>>>
>>>> Everyone seems to find it so. I advise Mac users at PARC to just
>>>> use
>>>> the system Python (unless and until they really know what they are
>>>> doing).
>>>
>>> I will now do the same to all Mac+Python people I know & meet ;)
>>
>> Though...
>>
>> It'd be much better if installs from python.org could cleanly co-
>> exist
>> with
>> Apple's default installs.
>>
>> Is there some suggestion/patch we could make to python.org to do
>> that?
>
> Well, we yak about it every now and then on the Mac-SIG Python list.
> And it more or less does, as long as you keep /usr/local/bin off your
> PATH. But there are all these meta-issues about people's paths, and
Ahh.... so I *did* have /usr/local/bin on my path.
> Apple not releasing Python updates, etc. IMO, this is really caused
> by
> developer-consumer culture clash. Apple builds for consumers, who
> will
> always just use the Python they got with the machine, and expects
> developers to know what they're doing. (Linux, on the other hand,
> builds for developers, and expects consumers to know what they're
> doing :-).
:) Makes sense I guess. I just don't know what I'm doing ;)
> A few more rules I find useful on the Mac:
>
> * never trust anything in /usr/local/bin to work
> * never use fink -- build from source
> * never use macports -- build from source
> * never set LD_LIBRARY_PATH or DYLD_LIBRARY_PATH
> (good advice on Linux, too)
> * never trust a package which requires you to break one of the
> above rules
These are excellent.
> Isn't this off-topic?
Yeah, sure is... I'll stop asking questions!
Mike
Re: [VOTE] Release PyLucene 2.4.1
Posted by Bill Janssen <ja...@parc.com>.
Michael McCandless <lu...@mikemccandless.com> wrote:
> Michael McCandless wrote:
>
> >>> Ahhhh OK; this is quite confusing.
> >>
> >> Everyone seems to find it so. I advise Mac users at PARC to just use
> >> the system Python (unless and until they really know what they are
> >> doing).
> >
> > I will now do the same to all Mac+Python people I know & meet ;)
>
> Though...
>
> It'd be much better if installs from python.org could cleanly co-exist
> with
> Apple's default installs.
>
> Is there some suggestion/patch we could make to python.org to do that?
Well, we yak about it every now and then on the Mac-SIG Python list.
And it more or less does, as long as you keep /usr/local/bin off your
PATH. But there are all these meta-issues about people's paths, and
Apple not releasing Python updates, etc. IMO, this is really caused by
developer-consumer culture clash. Apple builds for consumers, who will
always just use the Python they got with the machine, and expects
developers to know what they're doing. (Linux, on the other hand,
builds for developers, and expects consumers to know what they're doing :-).
A few more rules I find useful on the Mac:
* never trust anything in /usr/local/bin to work
* never use fink -- build from source
* never use macports -- build from source
* never set LD_LIBRARY_PATH or DYLD_LIBRARY_PATH
(good advice on Linux, too)
* never trust a package which requires you to break one of the above rules
Isn't this off-topic?
Bill
Re: [VOTE] Release PyLucene 2.4.1
Posted by Andi Vajda <va...@apache.org>.
On Mar 12, 2009, at 13:07, Michael McCandless
<lu...@mikemccandless.com> wrote:
> Michael McCandless wrote:
>
>>>> Ahhhh OK; this is quite confusing.
>>>
>>> Everyone seems to find it so. I advise Mac users at PARC to just
>>> use
>>> the system Python (unless and until they really know what they are
>>> doing).
>>
>> I will now do the same to all Mac+Python people I know & meet ;)
>
> Though...
>
> It'd be much better if installs from python.org could cleanly co-
> exist with
> Apple's default installs.
Oh it does work fine provided you tell the linker the correct place to
pickup the framework from.
I have three or four python installs on my system and it works fine.
Andi..
>
>
> Is there some suggestion/patch we could make to python.org to do that?
>
> (I have no idea what that'd be).
>
> How does python handle such coexistence on other platforms (Linux,
> FreeBSD, Solaris, etc., I guess, since MSFT doesn't pre-install
> python)?
>
> Mike
Re: [VOTE] Release PyLucene 2.4.1
Posted by Michael McCandless <lu...@mikemccandless.com>.
Michael McCandless wrote:
>>> Ahhhh OK; this is quite confusing.
>>
>> Everyone seems to find it so. I advise Mac users at PARC to just use
>> the system Python (unless and until they really know what they are
>> doing).
>
> I will now do the same to all Mac+Python people I know & meet ;)
Though...
It'd be much better if installs from python.org could cleanly co-exist
with
Apple's default installs.
Is there some suggestion/patch we could make to python.org to do that?
(I have no idea what that'd be).
How does python handle such coexistence on other platforms (Linux,
FreeBSD, Solaris, etc., I guess, since MSFT doesn't pre-install python)?
Mike
Re: [VOTE] Release PyLucene 2.4.1
Posted by Michael McCandless <lu...@mikemccandless.com>.
Bill Janssen wrote:
> Michael McCandless <lu...@mikemccandless.com> wrote:
>
>> Bill Janssen wrote:
>>
>>> Michael McCandless <lu...@mikemccandless.com> wrote:
>>>
>>>> But why do I have two separate 2.5 Python installs, compiled w/
>>>> different options, in the first place? It's weird and I'd like to
>>>> fix
>>>> it, I think (outside of building PyLucene, for my own sanity).
>>>
>>> One (the one in /Library) is the Python.org install, which I avoid
>>> religously on the Mac, for exactly these reasons. You should be
>>> able
>>> to delete it without problems, and also delete the Python
>>> executables
>>> and similar stuff in /usr/local/bin/.
>>>
>>> I say you *should* be able to, but I'm always leery of this. Try
>>> just
>>> removing all the read and execute privileges first, and see how
>>> things
>>> go. Then you can actually delete it when/if you need space.
>>
>> Ahhhh OK; this is quite confusing.
>
> Everyone seems to find it so. I advise Mac users at PARC to just use
> the system Python (unless and until they really know what they are
> doing).
I will now do the same to all Mac+Python people I know & meet ;)
> This is further complicated by the fact that OS X also uses
> /Library/Python/2.5/site-packages/ as the place for users to install
> additional Python modules; this, for instance, is where your "lucene"
> and "JCC" modules will wind up. So it's not *all* of "/Library", just
> "/Library/Frameworks/Python.framework/", that you want to avoid.
Yeah, I had also noticed this confusing yet-another-difference...
I just renamed my /Library/Framework/Python.framework{,.old} and that
did
the trick.
Mike
Re: [VOTE] Release PyLucene 2.4.1
Posted by Bill Janssen <ja...@parc.com>.
Michael McCandless <lu...@mikemccandless.com> wrote:
> Bill Janssen wrote:
>
> > Michael McCandless <lu...@mikemccandless.com> wrote:
> >
> >> But why do I have two separate 2.5 Python installs, compiled w/
> >> different options, in the first place? It's weird and I'd like to
> >> fix
> >> it, I think (outside of building PyLucene, for my own sanity).
> >
> > One (the one in /Library) is the Python.org install, which I avoid
> > religously on the Mac, for exactly these reasons. You should be able
> > to delete it without problems, and also delete the Python executables
> > and similar stuff in /usr/local/bin/.
> >
> > I say you *should* be able to, but I'm always leery of this. Try just
> > removing all the read and execute privileges first, and see how things
> > go. Then you can actually delete it when/if you need space.
>
> Ahhhh OK; this is quite confusing.
Everyone seems to find it so. I advise Mac users at PARC to just use
the system Python (unless and until they really know what they are
doing).
This is further complicated by the fact that OS X also uses
/Library/Python/2.5/site-packages/ as the place for users to install
additional Python modules; this, for instance, is where your "lucene"
and "JCC" modules will wind up. So it's not *all* of "/Library", just
"/Library/Frameworks/Python.framework/", that you want to avoid.
Bill
Re: [VOTE] Release PyLucene 2.4.1
Posted by Michael McCandless <lu...@mikemccandless.com>.
Bill Janssen wrote:
> Michael McCandless <lu...@mikemccandless.com> wrote:
>
>> But why do I have two separate 2.5 Python installs, compiled w/
>> different options, in the first place? It's weird and I'd like to
>> fix
>> it, I think (outside of building PyLucene, for my own sanity).
>
> One (the one in /Library) is the Python.org install, which I avoid
> religously on the Mac, for exactly these reasons. You should be able
> to delete it without problems, and also delete the Python executables
> and similar stuff in /usr/local/bin/.
>
> I say you *should* be able to, but I'm always leery of this. Try just
> removing all the read and execute privileges first, and see how things
> go. Then you can actually delete it when/if you need space.
Ahhhh OK; this is quite confusing.
I will attempt to migrate away from the Python in /Library/...
Thanks Bill!
Mike
Re: [VOTE] Release PyLucene 2.4.1
Posted by Bill Janssen <ja...@parc.com>.
Michael McCandless <lu...@mikemccandless.com> wrote:
> But why do I have two separate 2.5 Python installs, compiled w/
> different options, in the first place? It's weird and I'd like to fix
> it, I think (outside of building PyLucene, for my own sanity).
One (the one in /Library) is the Python.org install, which I avoid
religously on the Mac, for exactly these reasons. You should be able
to delete it without problems, and also delete the Python executables
and similar stuff in /usr/local/bin/.
I say you *should* be able to, but I'm always leery of this. Try just
removing all the read and execute privileges first, and see how things
go. Then you can actually delete it when/if you need space.
Bill
Re: [VOTE] Release PyLucene 2.4.1
Posted by Michael McCandless <lu...@mikemccandless.com>.
Andi Vajda wrote:
>
> On Thu, 12 Mar 2009, Michael McCandless wrote:
>
>> Anyway, finally, a little bruised from the whole process, and
>> wondering what other Python things on my Mac may now be broken... I
>> was able to run the tests successfully!
>
> Do not pollute your env with soft links. Use -F instead. Really.
OK, I'll test with -F/... instead of symlink.
But why do I have two separate 2.5 Python installs, compiled w/
different options, in the first place? It's weird and I'd like to fix
it, I think (outside of building PyLucene, for my own sanity).
>> Maybe it's ambitious... but I'd really love a "for the impatient...",
>> in INSTALL, like this:
>>
>> pushd jcc
>> <edit setup.py to configure for your env>
>> python setup.py build
>> sudo python setup.py install
>> popd
>> <edit Makefile to configure for your env>
>> make
>> sudo make install
>> make test
>
> Normally, this is all that's needed. But you're making the two
> manual edit steps smaller in importance than they really are. Still,
> adding such a paragraph to the docs can't hurt.
Yeah true. It'd be good to have a place (wiki maybe) where we call
can put our "interesting flags we had to add" in each of those steps,
for different platforms, and then reference those places in the above
for-the-impatient instructions.
>> I assume warnings like this are OK on OS X?:
>>
>> Warning: renaming static method 'toString' on class java.lang.Long
>> to 'toString_' since it is shadowed by non-static method of same
>> name.
>
> This is not specific to OS X but specific to that Java class and
> Python's inability to have static and non static methods named the
> same.
OK makes sense.
>> and also these ones?:
>>
>> cc1plus: warning: command line option "-Wstrict-prototypes" is
>> valid for C/ObjC but not for C++
>
> That's noise from gcc caused by distutils/setuptools not
> understanding C++ too well and passing it flags it shouldn't.
OK.
Mike
Re: [VOTE] Release PyLucene 2.4.1
Posted by Michael McCandless <lu...@mikemccandless.com>.
jcc/jcc/config.py? Indeed it reflects my addition of -F/System/...,
with this:
LFLAGS=['-framework', 'JavaVM', '-F/System/Library/Frameworks', '-
framework', 'Python']
But, weirdly, I don't then see that in the final linking step when
running "make" in the pylucene dir. Here's the final line:
g++ -Wl,-F. -bundle -undefined dynamic_lookup -arch i386 -arch ppc
build/temp.macosx-10.5-i386-2.5/build/_lucene/__init__.o build/
temp.macosx-10.5-i386-2.5/build/_lucene/__wrap01__.o build/
temp.macosx-10.5-i386-2.5/build/_lucene/__wrap02__.o build/
temp.macosx-10.5-i386-2.5/build/_lucene/__wrap03__.o build/
temp.macosx-10.5-i386-2.5/build/_lucene/functions.o build/
temp.macosx-10.5-i386-2.5/build/_lucene/JArray.o build/
temp.macosx-10.5-i386-2.5/build/_lucene/JObject.o build/
temp.macosx-10.5-i386-2.5/build/_lucene/lucene.o build/
temp.macosx-10.5-i386-2.5/build/_lucene/types.o -L/Library/Python/2.5/
site-packages/JCC-2.2-py2.5-macosx-10.5-i386.egg -ljcc -o build/
lib.macosx-10.5-i386-2.5/lucene/_lucene.so -framework JavaVM -
framework Python -Wl,-S -Wl,-rpath /Library/Python/2.5/site-packages/
JCC-2.2-py2.5-macosx-10.5-i386.egg
Here's my config in the Makefile:
PREFIX_PYTHON=/usr
ANT=ant
PYTHON=$(PREFIX_PYTHON)/bin/python
JCC=$(PYTHON) -m jcc --shared
NUM_FILES=2
This must be something silly in my env...
Mike
On Mar 12, 2009, at 4:18 PM, Christian Heimes wrote:
> Michael McCandless wrote:
>> Weird, that was not what I was seeing.
>>
>> I think I cd'd to jcc, added -F into setup.py, "rm -r build", python
>> (coming from the right framework) setup.py build (confirmed -F
>> worked),
>> sudo python setup.py install, cd .., make clean, make, and I didn't
>> see
>> the -F flag on that g++ line.
>>
>> Hmmm.... maybe somehow I was getting an older JCC install (in the
>> wrong
>> framework) when running that make.
>>
>> Let me go try it again (now that I've nuked the wrong framework).
>
> Check out jcc/config.py. It's auto-created during the build process
> of JCC.
>
> Christian
Re: [VOTE] Release PyLucene 2.4.1
Posted by Christian Heimes <li...@cheimes.de>.
Michael McCandless wrote:
> Weird, that was not what I was seeing.
>
> I think I cd'd to jcc, added -F into setup.py, "rm -r build", python
> (coming from the right framework) setup.py build (confirmed -F worked),
> sudo python setup.py install, cd .., make clean, make, and I didn't see
> the -F flag on that g++ line.
>
> Hmmm.... maybe somehow I was getting an older JCC install (in the wrong
> framework) when running that make.
>
> Let me go try it again (now that I've nuked the wrong framework).
Check out jcc/config.py. It's auto-created during the build process of JCC.
Christian
Re: [VOTE] Release PyLucene 2.4.1
Posted by Michael McCandless <lu...@mikemccandless.com>.
Andi Vajda wrote:
>
> On Mar 12, 2009, at 12:40, Michael McCandless <lucene@mikemccandless.com
> > wrote:
>
>>
>> Andi Vajda wrote:
>>
>>>
>>> On Thu, 12 Mar 2009, Michael McCandless wrote:
>>>
>>>> Anyway, finally, a little bruised from the whole process, and
>>>> wondering what other Python things on my Mac may now be broken... I
>>>> was able to run the tests successfully!
>>>
>>> Do not pollute your env with soft links. Use -F instead. Really.
>>
>> OK so I got -F added in JCC's setup.py, but then when I built
>> pylucene,
>> it also needed the -F but I couldn't figure out where to add it?
>
> No you don't. The flags you built JCC with are remembered and reused
> by JCC to build your extensions. This is what this thread about
> config.py is about.
Weird, that was not what I was seeing.
I think I cd'd to jcc, added -F into setup.py, "rm -r build", python
(coming from the right framework) setup.py build (confirmed -F
worked), sudo python setup.py install, cd .., make clean, make, and I
didn't see the -F flag on that g++ line.
Hmmm.... maybe somehow I was getting an older JCC install (in the
wrong framework) when running that make.
Let me go try it again (now that I've nuked the wrong framework).
Mike
Re: [VOTE] Release PyLucene 2.4.1
Posted by Andi Vajda <va...@apache.org>.
On Mar 12, 2009, at 12:40, Michael McCandless
<lu...@mikemccandless.com> wrote:
>
> Andi Vajda wrote:
>
>>
>> On Thu, 12 Mar 2009, Michael McCandless wrote:
>>
>>> Anyway, finally, a little bruised from the whole process, and
>>> wondering what other Python things on my Mac may now be broken... I
>>> was able to run the tests successfully!
>>
>> Do not pollute your env with soft links. Use -F instead. Really.
>
> OK so I got -F added in JCC's setup.py, but then when I built
> pylucene,
> it also needed the -F but I couldn't figure out where to add it?
No you don't. The flags you built JCC with are remembered and reused
by JCC to build your extensions. This is what this thread about
config.py is about.
Andi..
>
>
> But then I tried Bill's suggestion (nuking the Python framework under
> /Library/...) and then apparently g++ figured out -framework Python
> should
> simply fallback to the one in /System/Library/... and then
> everything just
> worked.
>
> Sheesh.
>
> OK now I'll start testing the release!
>
> Mike
Re: [VOTE] Release PyLucene 2.4.1
Posted by Michael McCandless <lu...@mikemccandless.com>.
Andi Vajda wrote:
>
> On Thu, 12 Mar 2009, Michael McCandless wrote:
>
>> Anyway, finally, a little bruised from the whole process, and
>> wondering what other Python things on my Mac may now be broken... I
>> was able to run the tests successfully!
>
> Do not pollute your env with soft links. Use -F instead. Really.
OK so I got -F added in JCC's setup.py, but then when I built pylucene,
it also needed the -F but I couldn't figure out where to add it?
But then I tried Bill's suggestion (nuking the Python framework under
/Library/...) and then apparently g++ figured out -framework Python
should
simply fallback to the one in /System/Library/... and then everything
just
worked.
Sheesh.
OK now I'll start testing the release!
Mike
Re: [VOTE] Release PyLucene 2.4.1
Posted by Andi Vajda <va...@apache.org>.
On Thu, 12 Mar 2009, Michael McCandless wrote:
> Anyway, finally, a little bruised from the whole process, and
> wondering what other Python things on my Mac may now be broken... I
> was able to run the tests successfully!
Do not pollute your env with soft links. Use -F instead. Really.
> Maybe it's ambitious... but I'd really love a "for the impatient...",
> in INSTALL, like this:
>
> pushd jcc
> <edit setup.py to configure for your env>
> python setup.py build
> sudo python setup.py install
> popd
> <edit Makefile to configure for your env>
> make
> sudo make install
> make test
Normally, this is all that's needed. But you're making the two manual edit
steps smaller in importance than they really are. Still, adding such a
paragraph to the docs can't hurt.
> I assume warnings like this are OK on OS X?:
>
> Warning: renaming static method 'toString' on class java.lang.Long to
> 'toString_' since it is shadowed by non-static method of same name.
This is not specific to OS X but specific to that Java class and Python's
inability to have static and non static methods named the same.
> and also these ones?:
>
> cc1plus: warning: command line option "-Wstrict-prototypes" is valid for
> C/ObjC but not for C++
That's noise from gcc caused by distutils/setuptools not understanding C++
too well and passing it flags it shouldn't.
Andi..
>
> Mike
>
> Michael McCandless wrote:
>
>>
>> OK I was able to finally build (OS X 10.5.6) and run all tests
>> successfully.
>>
>> I hit a few OS X issues that I think are my silliness/confused
>> environment. I'll try to document them here, for future Google
>> diggers, but note that I sort of desperately rushed though things
>> trying to get it to work so I'm not 100% certain about all the details
>> below...
>>
>> It turns out, I have python frameworks installed under
>> /System/Library/Frameworks/... as well as /Library/Frameworks/... I'm
>> not sure if this is normal or what I did to cause it, but that caused
>> problems because the jcc build was going against one of them and the
>> pylucene build, against the other one. Python 2.5 was installed at
>> both places...
>>
>> At first I hit this error:
>>
>> ld: -rpath can only be used when targeting Mac OS X 10.5 or later
>>
>> when building pylucene, apparently, because one of my frameworks
>> targeted OS X 10.3. Somehow I got past that and then...
>>
>> When building pylucene, I hit this:
>>
>> Fatal Python error: Interpreter not initialized (version
>> mismatch?)
>>
>> when "python -m jcc" tried to run, which was also caused by the two
>> frameworks. So I edited pylucene's Makefile to force it to use the
>> same python framework (hardwired the full path to the python exe), and
>> then all worked.
>>
>> (I've also symlink'd my two different 2.5 Python frameworks
>> together... let's see if anything breaks).
>>
>> At one point I also hit this:
>>
>> _NSJVMLoadLibrary: NSAddLibrary failed for
>> c/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Libraries/libjvm.dylib
>>
>> from JCC when it tries to init the JVM, previously discussed here:
>>
>> http://mail-archives.apache.org/mod_mbox/lucene-pylucene-dev/200902.mbox/%3c35483342-7332-4AA4-8C5A-60693D656971@mikemccandless.com%3e
>>
>> Downgrading to JDK 1.5 fixed, this for some reason.
>>
>> Mike
>>
>> Andi Vajda wrote:
>>
>>>
>>> On Wed, 11 Mar 2009, Aric Coady wrote:
>>>
>>>> On Mar 11, 2009, at 10:06 AM, Andi Vajda wrote:
>>>>> With the recent release of Java Lucene 2.4.1, it's high time PyLucene
>>>>> get its first release out as an Apache subproject. I hope I got all the
>>>>> steps right and all the things into the right place.
>>>>
>>>> Thanks.
>>>>
>>>>> Release candidate 1 of PyLucene 2.4.1 is available from:
>>>>>
>>>>> http://people.apache.org/~vajda/staging_area/
>>>>
>>>> Make install is failing with python 2.6 because of a conflict between the
>>>> new collections module and the builtin one.
>>>
>>> Ugh. I moved the collections.py file to a python sub-directory.
>>> This is only a problem during the build. The collections.py file is
>>> installed into the lucene egg where it is accessible from under the lucene
>>> package then.
>>>
>>> I also backed out the earlier config.py build trick as it fails when
>>> install is used without build first.
>>>
>>> I uploaded a new release candidate, rc3, to the staging area:
>>> http://people.apache.org/~vajda/staging_area/
>>>
>>> Thank you for the bug report.
>>>
>>> Andi..
>>>
>>>>
>>>> ...
>>>> File
>>>> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg/setuptools/command/easy_install.py",
>>>> line 21, in <module>
>>>> File
>>>> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg/setuptools/package_index.py",
>>>> line 2, in <module>
>>>> File
>>>> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urlparse.py",
>>>> line 84, in <module>
>>>> from collections import namedtuple
>>>> ImportError: cannot import name namedtuple
>>>> make: *** [install] Error 255
>>>>
>>>> Probably will be necessary to rename or move collections.py.
>>>>
>>>> -Coady
>
Re: [VOTE] Release PyLucene 2.4.1
Posted by Michael McCandless <lu...@mikemccandless.com>.
Anyway, finally, a little bruised from the whole process, and
wondering what other Python things on my Mac may now be broken... I
was able to run the tests successfully!
Maybe it's ambitious... but I'd really love a "for the impatient...",
in INSTALL, like this:
pushd jcc
<edit setup.py to configure for your env>
python setup.py build
sudo python setup.py install
popd
<edit Makefile to configure for your env>
make
sudo make install
make test
I assume warnings like this are OK on OS X?:
Warning: renaming static method 'toString' on class java.lang.Long
to 'toString_' since it is shadowed by non-static method of same name.
and also these ones?:
cc1plus: warning: command line option "-Wstrict-prototypes" is
valid for C/ObjC but not for C++
Mike
Michael McCandless wrote:
>
> OK I was able to finally build (OS X 10.5.6) and run all tests
> successfully.
>
> I hit a few OS X issues that I think are my silliness/confused
> environment. I'll try to document them here, for future Google
> diggers, but note that I sort of desperately rushed though things
> trying to get it to work so I'm not 100% certain about all the details
> below...
>
> It turns out, I have python frameworks installed under
> /System/Library/Frameworks/... as well as /Library/Frameworks/... I'm
> not sure if this is normal or what I did to cause it, but that caused
> problems because the jcc build was going against one of them and the
> pylucene build, against the other one. Python 2.5 was installed at
> both places...
>
> At first I hit this error:
>
> ld: -rpath can only be used when targeting Mac OS X 10.5 or later
>
> when building pylucene, apparently, because one of my frameworks
> targeted OS X 10.3. Somehow I got past that and then...
>
> When building pylucene, I hit this:
>
> Fatal Python error: Interpreter not initialized (version
> mismatch?)
>
> when "python -m jcc" tried to run, which was also caused by the two
> frameworks. So I edited pylucene's Makefile to force it to use the
> same python framework (hardwired the full path to the python exe), and
> then all worked.
>
> (I've also symlink'd my two different 2.5 Python frameworks
> together... let's see if anything breaks).
>
> At one point I also hit this:
>
> _NSJVMLoadLibrary: NSAddLibrary failed for c/System/Library/
> Frameworks/JavaVM.framework/Versions/CurrentJDK/Libraries/libjvm.dylib
>
> from JCC when it tries to init the JVM, previously discussed here:
>
> http://mail-archives.apache.org/mod_mbox/lucene-pylucene-dev/200902.mbox/%3c35483342-7332-4AA4-8C5A-60693D656971@mikemccandless.com%3e
>
> Downgrading to JDK 1.5 fixed, this for some reason.
>
> Mike
>
> Andi Vajda wrote:
>
>>
>> On Wed, 11 Mar 2009, Aric Coady wrote:
>>
>>> On Mar 11, 2009, at 10:06 AM, Andi Vajda wrote:
>>>> With the recent release of Java Lucene 2.4.1, it's high time
>>>> PyLucene get its first release out as an Apache subproject. I
>>>> hope I got all the steps right and all the things into the right
>>>> place.
>>>
>>> Thanks.
>>>
>>>> Release candidate 1 of PyLucene 2.4.1 is available from:
>>>>
>>>> http://people.apache.org/~vajda/staging_area/
>>>
>>> Make install is failing with python 2.6 because of a conflict
>>> between the new collections module and the builtin one.
>>
>> Ugh. I moved the collections.py file to a python sub-directory.
>> This is only a problem during the build. The collections.py file is
>> installed into the lucene egg where it is accessible from under the
>> lucene package then.
>>
>> I also backed out the earlier config.py build trick as it fails
>> when install is used without build first.
>>
>> I uploaded a new release candidate, rc3, to the staging area:
>> http://people.apache.org/~vajda/staging_area/
>>
>> Thank you for the bug report.
>>
>> Andi..
>>
>>>
>>> ...
>>> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
>>> python2.6/site-packages/setuptools-0.6c9-py2.6.egg/setuptools/
>>> command/easy_install.py", line 21, in <module>
>>> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
>>> python2.6/site-packages/setuptools-0.6c9-py2.6.egg/setuptools/
>>> package_index.py", line 2, in <module>
>>> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
>>> python2.6/urlparse.py", line 84, in <module>
>>> from collections import namedtuple
>>> ImportError: cannot import name namedtuple
>>> make: *** [install] Error 255
>>>
>>> Probably will be necessary to rename or move collections.py.
>>>
>>> -Coady
>
Re: [VOTE] Release PyLucene 2.4.1
Posted by Andi Vajda <va...@apache.org>.
On Thu, 12 Mar 2009, Michael McCandless wrote:
> OK I was able to finally build (OS X 10.5.6) and run all tests
> successfully.
>
> I hit a few OS X issues that I think are my silliness/confused
> environment. I'll try to document them here, for future Google
> diggers, but note that I sort of desperately rushed though things
> trying to get it to work so I'm not 100% certain about all the details
> below...
>
> It turns out, I have python frameworks installed under
> /System/Library/Frameworks/... as well as /Library/Frameworks/... I'm
> not sure if this is normal or what I did to cause it, but that caused
> problems because the jcc build was going against one of them and the
> pylucene build, against the other one. Python 2.5 was installed at
> both places...
>
> At first I hit this error:
>
> ld: -rpath can only be used when targeting Mac OS X 10.5 or later
>
> when building pylucene, apparently, because one of my frameworks
> targeted OS X 10.3. Somehow I got past that and then...
>
> When building pylucene, I hit this:
>
> Fatal Python error: Interpreter not initialized (version
> mismatch?)
Yes, this is the symptom when different python versions are loaded into the
same process. The easiest way to avoid that is to be sure that the linker is
going to be using the same python as the 'python' command.
The way to do this is add a -F/path/to/frameworks-for-python in front of the
-f Python args in LFLAGS for darwin in JCC's setup.py, telling the linker
explicitely where to find the python framework to link against.
I hit this problem when I test against Python 2.6 for which I have a
separate install.
Andi..
>
> when "python -m jcc" tried to run, which was also caused by the two
> frameworks. So I edited pylucene's Makefile to force it to use the
> same python framework (hardwired the full path to the python exe), and
> then all worked.
>
> (I've also symlink'd my two different 2.5 Python frameworks
> together... let's see if anything breaks).
>
> At one point I also hit this:
>
> _NSJVMLoadLibrary: NSAddLibrary failed for
> c/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Libraries/libjvm.dylib
>
> from JCC when it tries to init the JVM, previously discussed here:
>
> http://mail-archives.apache.org/mod_mbox/lucene-pylucene-dev/200902.mbox/%3c35483342-7332-4AA4-8C5A-60693D656971@mikemccandless.com%3e
>
> Downgrading to JDK 1.5 fixed, this for some reason.
>
> Mike
>
> Andi Vajda wrote:
>
>>
>> On Wed, 11 Mar 2009, Aric Coady wrote:
>>
>>> On Mar 11, 2009, at 10:06 AM, Andi Vajda wrote:
>>>> With the recent release of Java Lucene 2.4.1, it's high time PyLucene get
>>>> its first release out as an Apache subproject. I hope I got all the steps
>>>> right and all the things into the right place.
>>>
>>> Thanks.
>>>
>>>> Release candidate 1 of PyLucene 2.4.1 is available from:
>>>>
>>>> http://people.apache.org/~vajda/staging_area/
>>>
>>> Make install is failing with python 2.6 because of a conflict between the
>>> new collections module and the builtin one.
>>
>> Ugh. I moved the collections.py file to a python sub-directory.
>> This is only a problem during the build. The collections.py file is
>> installed into the lucene egg where it is accessible from under the lucene
>> package then.
>>
>> I also backed out the earlier config.py build trick as it fails when
>> install is used without build first.
>>
>> I uploaded a new release candidate, rc3, to the staging area:
>> http://people.apache.org/~vajda/staging_area/
>>
>> Thank you for the bug report.
>>
>> Andi..
>>
>>>
>>> ...
>>> File
>>> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg/setuptools/command/easy_install.py",
>>> line 21, in <module>
>>> File
>>> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg/setuptools/package_index.py",
>>> line 2, in <module>
>>> File
>>> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urlparse.py",
>>> line 84, in <module>
>>> from collections import namedtuple
>>> ImportError: cannot import name namedtuple
>>> make: *** [install] Error 255
>>>
>>> Probably will be necessary to rename or move collections.py.
>>>
>>> -Coady
Re: [VOTE] Release PyLucene 2.4.1
Posted by Michael McCandless <lu...@mikemccandless.com>.
OK I was able to finally build (OS X 10.5.6) and run all tests
successfully.
I hit a few OS X issues that I think are my silliness/confused
environment. I'll try to document them here, for future Google
diggers, but note that I sort of desperately rushed though things
trying to get it to work so I'm not 100% certain about all the details
below...
It turns out, I have python frameworks installed under
/System/Library/Frameworks/... as well as /Library/Frameworks/... I'm
not sure if this is normal or what I did to cause it, but that caused
problems because the jcc build was going against one of them and the
pylucene build, against the other one. Python 2.5 was installed at
both places...
At first I hit this error:
ld: -rpath can only be used when targeting Mac OS X 10.5 or later
when building pylucene, apparently, because one of my frameworks
targeted OS X 10.3. Somehow I got past that and then...
When building pylucene, I hit this:
Fatal Python error: Interpreter not initialized (version
mismatch?)
when "python -m jcc" tried to run, which was also caused by the two
frameworks. So I edited pylucene's Makefile to force it to use the
same python framework (hardwired the full path to the python exe), and
then all worked.
(I've also symlink'd my two different 2.5 Python frameworks
together... let's see if anything breaks).
At one point I also hit this:
_NSJVMLoadLibrary: NSAddLibrary failed for c/System/Library/
Frameworks/JavaVM.framework/Versions/CurrentJDK/Libraries/libjvm.dylib
from JCC when it tries to init the JVM, previously discussed here:
http://mail-archives.apache.org/mod_mbox/lucene-pylucene-dev/200902.mbox/%3c35483342-7332-4AA4-8C5A-60693D656971@mikemccandless.com%3e
Downgrading to JDK 1.5 fixed, this for some reason.
Mike
Andi Vajda wrote:
>
> On Wed, 11 Mar 2009, Aric Coady wrote:
>
>> On Mar 11, 2009, at 10:06 AM, Andi Vajda wrote:
>>> With the recent release of Java Lucene 2.4.1, it's high time
>>> PyLucene get its first release out as an Apache subproject. I hope
>>> I got all the steps right and all the things into the right place.
>>
>> Thanks.
>>
>>> Release candidate 1 of PyLucene 2.4.1 is available from:
>>>
>>> http://people.apache.org/~vajda/staging_area/
>>
>> Make install is failing with python 2.6 because of a conflict
>> between the new collections module and the builtin one.
>
> Ugh. I moved the collections.py file to a python sub-directory.
> This is only a problem during the build. The collections.py file is
> installed into the lucene egg where it is accessible from under the
> lucene package then.
>
> I also backed out the earlier config.py build trick as it fails when
> install is used without build first.
>
> I uploaded a new release candidate, rc3, to the staging area:
> http://people.apache.org/~vajda/staging_area/
>
> Thank you for the bug report.
>
> Andi..
>
>>
>> ...
>> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
>> python2.6/site-packages/setuptools-0.6c9-py2.6.egg/setuptools/
>> command/easy_install.py", line 21, in <module>
>> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
>> python2.6/site-packages/setuptools-0.6c9-py2.6.egg/setuptools/
>> package_index.py", line 2, in <module>
>> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
>> python2.6/urlparse.py", line 84, in <module>
>> from collections import namedtuple
>> ImportError: cannot import name namedtuple
>> make: *** [install] Error 255
>>
>> Probably will be necessary to rename or move collections.py.
>>
>> -Coady
Re: [VOTE] Release PyLucene 2.4.1
Posted by Andi Vajda <va...@apache.org>.
On Wed, 11 Mar 2009, Aric Coady wrote:
> On Mar 11, 2009, at 10:06 AM, Andi Vajda wrote:
>> With the recent release of Java Lucene 2.4.1, it's high time PyLucene get
>> its first release out as an Apache subproject. I hope I got all the steps
>> right and all the things into the right place.
>
> Thanks.
>
>> Release candidate 1 of PyLucene 2.4.1 is available from:
>>
>> http://people.apache.org/~vajda/staging_area/
>
> Make install is failing with python 2.6 because of a conflict between the new
> collections module and the builtin one.
Ugh. I moved the collections.py file to a python sub-directory.
This is only a problem during the build. The collections.py file is
installed into the lucene egg where it is accessible from under the lucene
package then.
I also backed out the earlier config.py build trick as it fails when install
is used without build first.
I uploaded a new release candidate, rc3, to the staging area:
http://people.apache.org/~vajda/staging_area/
Thank you for the bug report.
Andi..
>
> ...
> File
> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg/setuptools/command/easy_install.py",
> line 21, in <module>
> File
> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg/setuptools/package_index.py",
> line 2, in <module>
> File
> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urlparse.py",
> line 84, in <module>
> from collections import namedtuple
> ImportError: cannot import name namedtuple
> make: *** [install] Error 255
>
> Probably will be necessary to rename or move collections.py.
>
> -Coady
Re: [VOTE] Release PyLucene 2.4.1
Posted by Aric Coady <ar...@gmail.com>.
On Mar 11, 2009, at 10:06 AM, Andi Vajda wrote:
> With the recent release of Java Lucene 2.4.1, it's high time
> PyLucene get its first release out as an Apache subproject. I hope I
> got all the steps right and all the things into the right place.
Thanks.
> Release candidate 1 of PyLucene 2.4.1 is available from:
>
> http://people.apache.org/~vajda/staging_area/
Make install is failing with python 2.6 because of a conflict between
the new collections module and the builtin one.
...
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/setuptools-0.6c9-py2.6.egg/setuptools/command/
easy_install.py", line 21, in <module>
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/site-packages/setuptools-0.6c9-py2.6.egg/setuptools/
package_index.py", line 2, in <module>
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/urlparse.py", line 84, in <module>
from collections import namedtuple
ImportError: cannot import name namedtuple
make: *** [install] Error 255
Probably will be necessary to rename or move collections.py.
-Coady
Re: [VOTE] Release PyLucene 2.4.1
Posted by Andi Vajda <va...@apache.org>.
On Wed, 11 Mar 2009, Grant Ingersoll wrote:
>>> 1. I think you need to upload your Key to a public server (i.e.
>>> pgp.mit.edu)
>>
>> How do I do that ?
>
> http://pgp.mit.edu:11371/ There's a place to submit a key there.
Done.
Andi..
Re: [VOTE] Release PyLucene 2.4.1
Posted by Grant Ingersoll <gs...@apache.org>.
On Mar 11, 2009, at 2:48 PM, Andi Vajda wrote:
>
> On Wed, 11 Mar 2009, Grant Ingersoll wrote:
>
>> First off, I'm a total Python newbie, so bear with me...
>>
>> Couple of comments:
>>
>> 1. I think you need to upload your Key to a public server (i.e.
>> pgp.mit.edu)
>
> How do I do that ?
http://pgp.mit.edu:11371/ There's a place to submit a key there.
>
> Alternatively, it seems that you can import my key from the KEYS
> file in the staging area of the release artifacts.
> But I'd be glad to upload my key someplace too, how ?
>
>> See:
>> @molly[1003]$ gpg --verify pylucene-2.4.1-rc1-src.tar.gz.asc
>> gpg: Signature made Wed Mar 11 12:42:58 2009 EDT using DSA key ID
>> 7DADCFA5
>> gpg: requesting key 7DADCFA5 from hkp server subkeys.pgp.net
>> gpgkeys: key 39815DB27DADCFA5 not found on keyserver
>> gpg: no valid OpenPGP data found.
>> gpg: Total number processed: 0
>> gpg: Can't check signature: public key not found
>>
>> Also, http://pgp.mit.edu:11371/pks/lookup?search=Andi+Vajda&op=index
>>
>> At some point, too, it would be good to get you into the ASF web of
>> trust by signing keys with others in the ASF.
>
> That'd be good too.
>
>> 2. I see the samples, but how about some directions on how to
>> actually run them? I see http://lucene.apache.org/pylucene/documentation/readme.html
>> . I tried doing:
>> grantingersoll@molly[1018]$ python index.py
>> Traceback (most recent call last):
>> File "index.py", line 2, in <module>
>> import os, sys, tarfile, lucene
>> ImportError: No module named lucene
>>
>
> Did you actually build and install the software ?
>
>> In the LIA directory, but got the error above. Again, I'm a total
>> newbie. Do I need to make something first? I tried make up at the
>> top level, but it seems to be checking out from SVN and then I got:
>> Exported revision 752549.
>> cd lucene-java-2.4.1; -Dversion=2.4.1
>> /bin/sh: -Dversion=2.4.1: command not found
>> make: *** [lucene-java-2.4.1/build/lucene-core-2.4.1.jar] Error 127
>
> You need to build JCC and PyLucene, in that order and read the
> respective installation instructions first. Building them is not
> completely trivial and requires some hand editing of PyLucene's
> Makefile and JCC's setup.py. If you are on Mac OS X, the latter
> should be most likely correct out of the box as Apple's Java is
> installed in a predictable place. Still, reviewing the settings in
> setup.py is recommended.
>
> Under "Requirements", at http://lucene.apache.org/pylucene/ you can
> find the links to the installation instructions.
>
>> Sorry for my ignorance. Pointers to more docs that explain what to
>> do are fine.
>
> http://lucene.apache.org/pylucene/jcc/documentation/install.html
> http://lucene.apache.org/pylucene/documentation/install.html
>
> After you've built and installed both, you can run 'make test',
> which runs all tests and samples. Running the LIA samples is as
> simple as invoking any of the files in the samples/LuceneInAction
> directory with python.
>
> PyLucene and JCC involve Python, Java and C++. Some understanding of
> all three languages/environments helps with build issues.
OK, I will read some more.
Re: [VOTE] Release PyLucene 2.4.1
Posted by Andi Vajda <va...@apache.org>.
On Wed, 11 Mar 2009, Grant Ingersoll wrote:
> First off, I'm a total Python newbie, so bear with me...
>
> Couple of comments:
>
> 1. I think you need to upload your Key to a public server (i.e. pgp.mit.edu)
How do I do that ?
Alternatively, it seems that you can import my key from the KEYS file in the
staging area of the release artifacts.
But I'd be glad to upload my key someplace too, how ?
> See:
> @molly[1003]$ gpg --verify pylucene-2.4.1-rc1-src.tar.gz.asc
> gpg: Signature made Wed Mar 11 12:42:58 2009 EDT using DSA key ID 7DADCFA5
> gpg: requesting key 7DADCFA5 from hkp server subkeys.pgp.net
> gpgkeys: key 39815DB27DADCFA5 not found on keyserver
> gpg: no valid OpenPGP data found.
> gpg: Total number processed: 0
> gpg: Can't check signature: public key not found
>
> Also, http://pgp.mit.edu:11371/pks/lookup?search=Andi+Vajda&op=index
>
> At some point, too, it would be good to get you into the ASF web of trust by
> signing keys with others in the ASF.
That'd be good too.
> 2. I see the samples, but how about some directions on how to actually run
> them? I see http://lucene.apache.org/pylucene/documentation/readme.html. I
> tried doing:
> grantingersoll@molly[1018]$ python index.py
> Traceback (most recent call last):
> File "index.py", line 2, in <module>
> import os, sys, tarfile, lucene
> ImportError: No module named lucene
>
Did you actually build and install the software ?
> In the LIA directory, but got the error above. Again, I'm a total newbie.
> Do I need to make something first? I tried make up at the top level, but it
> seems to be checking out from SVN and then I got:
> Exported revision 752549.
> cd lucene-java-2.4.1; -Dversion=2.4.1
> /bin/sh: -Dversion=2.4.1: command not found
> make: *** [lucene-java-2.4.1/build/lucene-core-2.4.1.jar] Error 127
You need to build JCC and PyLucene, in that order and read the respective
installation instructions first. Building them is not completely trivial and
requires some hand editing of PyLucene's Makefile and JCC's setup.py. If you
are on Mac OS X, the latter should be most likely correct out of the box as
Apple's Java is installed in a predictable place. Still, reviewing the
settings in setup.py is recommended.
Under "Requirements", at http://lucene.apache.org/pylucene/ you can find the
links to the installation instructions.
> Sorry for my ignorance. Pointers to more docs that explain what to do are
> fine.
http://lucene.apache.org/pylucene/jcc/documentation/install.html
http://lucene.apache.org/pylucene/documentation/install.html
After you've built and installed both, you can run 'make test', which runs
all tests and samples. Running the LIA samples is as simple as invoking any
of the files in the samples/LuceneInAction directory with python.
PyLucene and JCC involve Python, Java and C++. Some understanding of all
three languages/environments helps with build issues.
Andi..
Re: [VOTE] Release PyLucene 2.4.1
Posted by Grant Ingersoll <gs...@apache.org>.
Hi Andi,
First off, I'm a total Python newbie, so bear with me...
Couple of comments:
1. I think you need to upload your Key to a public server (i.e.
pgp.mit.edu)
See:
@molly[1003]$ gpg --verify pylucene-2.4.1-rc1-src.tar.gz.asc
gpg: Signature made Wed Mar 11 12:42:58 2009 EDT using DSA key ID
7DADCFA5
gpg: requesting key 7DADCFA5 from hkp server subkeys.pgp.net
gpgkeys: key 39815DB27DADCFA5 not found on keyserver
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
gpg: Can't check signature: public key not found
Also, http://pgp.mit.edu:11371/pks/lookup?search=Andi+Vajda&op=index
At some point, too, it would be good to get you into the ASF web of
trust by signing keys with others in the ASF.
2. I see the samples, but how about some directions on how to actually
run them? I see http://lucene.apache.org/pylucene/documentation/readme.html
. I tried doing:
grantingersoll@molly[1018]$ python index.py
Traceback (most recent call last):
File "index.py", line 2, in <module>
import os, sys, tarfile, lucene
ImportError: No module named lucene
In the LIA directory, but got the error above. Again, I'm a total
newbie. Do I need to make something first? I tried make up at the
top level, but it seems to be checking out from SVN and then I got:
Exported revision 752549.
cd lucene-java-2.4.1; -Dversion=2.4.1
/bin/sh: -Dversion=2.4.1: command not found
make: *** [lucene-java-2.4.1/build/lucene-core-2.4.1.jar] Error 127
Sorry for my ignorance. Pointers to more docs that explain what to do
are fine.
-Grant
On Mar 11, 2009, at 1:06 PM, Andi Vajda wrote:
>
> With the recent release of Java Lucene 2.4.1, it's high time
> PyLucene get its first release out as an Apache subproject. I hope I
> got all the steps right and all the things into the right place.
>
> Release candidate 1 of PyLucene 2.4.1 is available from:
>
> http://people.apache.org/~vajda/staging_area/
>
> The list of changes is available from:
>
> http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_2_4/CHANGES
>
> Please vote to release this artifacts as PyLucene 2.4.1.
>
> Thanks !
>
> Andi..
>
> ps: the KEYS file for PyLucene release signing is at:
> http://svn.apache.org/repos/asf/lucene/pylucene/dist/KEYS
Re: [VOTE] Release PyLucene 2.4.1
Posted by Grant Ingersoll <gs...@apache.org>.
Hi Andi,
First off, I'm a total Python newbie, so bear with me...
Couple of comments:
1. I think you need to upload your Key to a public server (i.e.
pgp.mit.edu)
See:
@molly[1003]$ gpg --verify pylucene-2.4.1-rc1-src.tar.gz.asc
gpg: Signature made Wed Mar 11 12:42:58 2009 EDT using DSA key ID
7DADCFA5
gpg: requesting key 7DADCFA5 from hkp server subkeys.pgp.net
gpgkeys: key 39815DB27DADCFA5 not found on keyserver
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
gpg: Can't check signature: public key not found
Also, http://pgp.mit.edu:11371/pks/lookup?search=Andi+Vajda&op=index
At some point, too, it would be good to get you into the ASF web of
trust by signing keys with others in the ASF.
2. I see the samples, but how about some directions on how to actually
run them? I see http://lucene.apache.org/pylucene/documentation/readme.html
. I tried doing:
grantingersoll@molly[1018]$ python index.py
Traceback (most recent call last):
File "index.py", line 2, in <module>
import os, sys, tarfile, lucene
ImportError: No module named lucene
In the LIA directory, but got the error above. Again, I'm a total
newbie. Do I need to make something first? I tried make up at the
top level, but it seems to be checking out from SVN and then I got:
Exported revision 752549.
cd lucene-java-2.4.1; -Dversion=2.4.1
/bin/sh: -Dversion=2.4.1: command not found
make: *** [lucene-java-2.4.1/build/lucene-core-2.4.1.jar] Error 127
Sorry for my ignorance. Pointers to more docs that explain what to do
are fine.
-Grant
On Mar 11, 2009, at 1:06 PM, Andi Vajda wrote:
>
> With the recent release of Java Lucene 2.4.1, it's high time
> PyLucene get its first release out as an Apache subproject. I hope I
> got all the steps right and all the things into the right place.
>
> Release candidate 1 of PyLucene 2.4.1 is available from:
>
> http://people.apache.org/~vajda/staging_area/
>
> The list of changes is available from:
>
> http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_2_4/CHANGES
>
> Please vote to release this artifacts as PyLucene 2.4.1.
>
> Thanks !
>
> Andi..
>
> ps: the KEYS file for PyLucene release signing is at:
> http://svn.apache.org/repos/asf/lucene/pylucene/dist/KEYS
Re: [VOTE] Release PyLucene 2.4.1.y
Posted by Andi Vajda <va...@apache.org>.
On Mar 12, 2009, at 16:32, Christian Heimes <li...@cheimes.de> wrote:
> Andi Vajda wrote:
>> That's the problem. If I change my env or edit setup.py and config.py
>> exists already because I already did a build, it is not going to be
>> re-generated if I do 'python setup.py install' even though it should
>> since 'install' calls 'build' (and people expect it to) yet 'build'
>> doesn't appear in self.distutils.commands.
>
> Oh heck, I totally forgot that use case. How about:
>
> jcc/config.py is (re)created when:
>
> * the file doesn't exist
> * the modification time of setup.py is newer than the mtime
> of config.py
Simple enough.
> * at least one environ variable starts with "JCC"
> * at least one distutils command starts with "bdist" or "build"
Both scary.
It seems that the number of heuristics to introduce to fix this minor
inconvenience is oversized.
How about we address this differently by emitting a warning when
config.py is overwritten and 'install' is used (only) and it was
generated from JCC_ env vars earlier ?
Andi..
>
>
> Christian
Re: [VOTE] Release PyLucene 2.4.1.y
Posted by Christian Heimes <li...@cheimes.de>.
Andi Vajda wrote:
> That's the problem. If I change my env or edit setup.py and config.py
> exists already because I already did a build, it is not going to be
> re-generated if I do 'python setup.py install' even though it should
> since 'install' calls 'build' (and people expect it to) yet 'build'
> doesn't appear in self.distutils.commands.
Oh heck, I totally forgot that use case. How about:
jcc/config.py is (re)created when:
* the file doesn't exist
* the modification time of setup.py is newer than the mtime
of config.py
* at least one environ variable starts with "JCC"
* at least one distutils command starts with "bdist" or "build"
Christian
Re: [VOTE] Release PyLucene 2.4.1
Posted by Andi Vajda <va...@apache.org>.
On Mar 12, 2009, at 13:10, Christian Heimes <li...@cheimes.de> wrote:
> Andi Vajda wrote:
>>> if not os.path.exists(self.jcc_config_file):
>>> create_config = 1
>>> else:
>>> create_config = 0
>>>
>>> for command in self.distribution.commands:
>>> if command.startswith(("build", "bdist")):
>>> create_config = 1
>>> break
>>>
>>> if create_config:
>>> self.create_jcc_config()
>>>
>>>
>>> I might have missed a scenario or two where the code still to
>>> create a
>>> correct config.py. It should cover about 99.99% of all possible
>>> cases.
>>
>> It doesn't cover the case where one just calls 'install' without
>> 'build'
>> and a build is actually needed. build is invoked, as expected, but
>> doesn't appear in the commands list. In other words, not all build
>> commands start with 'build'. 'install' behaves like a build command
>> too
>> (in addition to installing stuff).
>
> This case is covered by the first check "if not
> os.path.exists(self.jcc_config_file)". When a user does the "tar -
> xzf &&
> python2.5 setup.py install" dance, the config file does not yet exist.
>
> The first if/else block makes sure it's always created. The second
> block
> ensures the file is not re-created unless the user invokes setup.py
> with a build or bdist command.
That's the problem. If I change my env or edit setup.py and config.py
exists already because I already did a build, it is not going to be re-
generated if I do 'python setup.py install' even though it should
since 'install' calls 'build' (and people expect it to) yet 'build'
doesn't appear in self.distutils.commands.
Andi..
>
>
> Christian
Re: [VOTE] Release PyLucene 2.4.1
Posted by Christian Heimes <li...@cheimes.de>.
Andi Vajda wrote:
>> if not os.path.exists(self.jcc_config_file):
>> create_config = 1
>> else:
>> create_config = 0
>>
>> for command in self.distribution.commands:
>> if command.startswith(("build", "bdist")):
>> create_config = 1
>> break
>>
>> if create_config:
>> self.create_jcc_config()
>>
>>
>> I might have missed a scenario or two where the code still to create a
>> correct config.py. It should cover about 99.99% of all possible cases.
>
> It doesn't cover the case where one just calls 'install' without 'build'
> and a build is actually needed. build is invoked, as expected, but
> doesn't appear in the commands list. In other words, not all build
> commands start with 'build'. 'install' behaves like a build command too
> (in addition to installing stuff).
This case is covered by the first check "if not
os.path.exists(self.jcc_config_file)". When a user does the "tar -xzf &&
python2.5 setup.py install" dance, the config file does not yet exist.
The first if/else block makes sure it's always created. The second block
ensures the file is not re-created unless the user invokes setup.py
with a build or bdist command.
Christian
Re: [VOTE] Release PyLucene 2.4.1
Posted by Andi Vajda <va...@apache.org>.
On Thu, 12 Mar 2009, Christian Heimes wrote:
> Andi Vajda wrote:
>> In other words, you're fixing this with this patch by introducing
>> another such inconvenience, both with a silent failure. It doesn't look
>> like the trade off is worthwhile.
>>
>> This is not saying anything about the quality or cleverness of the patch
>> itself. It taught me something about how do customize build.
>>
>> Since you've already put in place the build_py customization, it would
>> be a shame to lose this. Is there a better way to implement run() so
>> that the
>> logic for deciding on generating config.py doesn't have the flaws above
>> ? Is there a way to have config.py created only during a real 'build' ?
>
> Forsooth, it's tricky to get the creation of config.py right. I can't
> find a way to detect a real 'build' (as you said).
> You could follow a different path though. The config.py file should be
> (re)created when it doesn't not yet exist or whenever the user builds
> JCC. All build and binary distribution commands (binary egg, Windows
> installer etc) start with either 'build' or 'bdist'.
>
>
> if not os.path.exists(self.jcc_config_file):
> create_config = 1
> else:
> create_config = 0
>
> for command in self.distribution.commands:
> if command.startswith(("build", "bdist")):
> create_config = 1
> break
>
> if create_config:
> self.create_jcc_config()
>
>
> I might have missed a scenario or two where the code still to create a
> correct config.py. It should cover about 99.99% of all possible cases.
It doesn't cover the case where one just calls 'install' without 'build' and
a build is actually needed. build is invoked, as expected, but doesn't
appear in the commands list. In other words, not all build commands start
with 'build'. 'install' behaves like a build command too (in addition to
installing stuff).
Andi..
Re: [VOTE] Release PyLucene 2.4.1
Posted by Christian Heimes <li...@cheimes.de>.
Andi Vajda wrote:
> In other words, you're fixing this with this patch by introducing
> another such inconvenience, both with a silent failure. It doesn't look
> like the trade off is worthwhile.
>
> This is not saying anything about the quality or cleverness of the patch
> itself. It taught me something about how do customize build.
>
> Since you've already put in place the build_py customization, it would
> be a shame to lose this. Is there a better way to implement run() so
> that the
> logic for deciding on generating config.py doesn't have the flaws above
> ? Is there a way to have config.py created only during a real 'build' ?
Forsooth, it's tricky to get the creation of config.py right. I can't
find a way to detect a real 'build' (as you said).
You could follow a different path though. The config.py file should be
(re)created when it doesn't not yet exist or whenever the user builds
JCC. All build and binary distribution commands (binary egg, Windows
installer etc) start with either 'build' or 'bdist'.
if not os.path.exists(self.jcc_config_file):
create_config = 1
else:
create_config = 0
for command in self.distribution.commands:
if command.startswith(("build", "bdist")):
create_config = 1
break
if create_config:
self.create_jcc_config()
I might have missed a scenario or two where the code still to create a
correct config.py. It should cover about 99.99% of all possible cases.
Christian
Re: [VOTE] Release PyLucene 2.4.1
Posted by Andi Vajda <va...@apache.org>.
On Thu, 12 Mar 2009, Christian Heimes wrote:
> I've attached a patch that hooks into the build process. With the patch
> the config file is created right before the .py files are copied into
> the build directory.
Thanks for the patch. A few comments:
def run(self):
if (not os.path.exists(self.jcc_config_file) or
not "install" in self.distribution.commands):
self.create_jcc_config()
It looks to me as If one edits setup.py or changes one of the JCC_ env vars
and then runs 'python setup.py build install' again, with config.py
existing, the changes made are not taken into account, config.py is not
getting regenerated.
The logic here is tricky to get right and the failure silent (as before).
Basically, what we want is config.py to _not_ be regenerated during install,
only during 'build'.
The problem is that, regardless of whether it needs to be, your build_py's
run() is also always called when just 'install' is requested. I've verified
this with pdb and by inspecting self.distribution.commands inside run().
The check for existence then has the problem mentioned above and the check
for 'install' not being in self.distribution.commands, made to force
re-creation when just 'build' is requested breaks down when both 'build'
and 'install' are requested together.
I'm afraid that, at this point, this patch is not improving much on the
situation that got this started, the minor inconvenience (I quote) of having
to tell sudo to inherit the environment so that config.py doesn't get
smashed with defaults during install.
In other words, you're fixing this with this patch by introducing another
such inconvenience, both with a silent failure. It doesn't look like the
trade off is worthwhile.
This is not saying anything about the quality or cleverness of the patch
itself. It taught me something about how do customize build.
Since you've already put in place the build_py customization, it would be
a shame to lose this. Is there a better way to implement run() so that the
logic for deciding on generating config.py doesn't have the flaws above ? Is
there a way to have config.py created only during a real 'build' ?
> The MANIFEST.in file must be placed next to setup.py. It prevents
> jcc/config.py from getting placed in a source release.
Thanks, I added that one.
Andi..
Re: [VOTE] Release PyLucene 2.4.1
Posted by Christian Heimes <li...@cheimes.de>.
Christian Heimes wrote:
[...]
I forgot to include the changes in jcc/python.py. You have to rename
SHARED to ENABLE_SHARED.
Christian
Re: [VOTE] Release PyLucene 2.4.1
Posted by Christian Heimes <li...@cheimes.de>.
Andi Vajda wrote:
> Does 'install' without 'build' invoke 'build' ?
> If that's the case, then yeah, there is a problem and this should be
> backed out. If not, then where is the problem ? 'install' without a
> prior 'build' would make no sense, no ?
Yes, it does. Every step in the installation process knows about its
dependencies. For example install depends amongst others on build, build
depends on build_py and build_ext and so forth. Read the output of
setup.py install carefully and you'll notice the "running spam" lines.
$ sudo python2.5 setup.py install
[sudo] password for heimes:
running install
running bdist_egg
running egg_info
writing JCC.egg-info/PKG-INFO
writing top-level names to JCC.egg-info/top_level.txt
writing dependency_links to JCC.egg-info/dependency_links.txt
writing manifest file 'JCC.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
copying jcc/classes/org/apache/jcc/PythonVM.class ->
build/lib.linux-x86_64-2.5/jcc/classes/org/apache/jcc
copying jcc/classes/org/apache/jcc/PythonException.class ->
build/lib.linux-x86_64-2.5/jcc/classes/org/apache/jcc
running build_ext
...
I've attached a patch that hooks into the build process. With the patch
the config file is created right before the .py files are copied into
the build directory.
The MANIFEST.in file must be placed next to setup.py. It prevents
jcc/config.py from getting placed in a source release.
HTH
Christian
Re: [VOTE] Release PyLucene 2.4.1
Posted by Andi Vajda <va...@apache.org>.
On Wed, 11 Mar 2009, Christian Heimes wrote:
> Andi Vajda wrote:
>> Fixed. config.py is not written when 'build' is not in the command line
>> args anymore. I uploaded a new release candidate, rc2, to
>> http://people.apache.org/~vajda/staging_area.
>>
>> Thank you for the bug report.
>
> Your solution breaks an installation when a user doesn't call "python
> setup.py build" before she uses "python setup.py install" to install JCC.
Does 'install' without 'build' invoke 'build' ?
If that's the case, then yeah, there is a problem and this should be backed
out. If not, then where is the problem ? 'install' without a prior 'build'
would make no sense, no ?
> I think the best solution to the issue is a custom build_ext command
> that writes the config file after the extensions have been build. If you
> like I can try to come up with a suitable solution.
That may be a little heavy handed for such a uncommon problem but if you'd
like to give it try, by all means !!
Andi..
Re: [VOTE] Release PyLucene 2.4.1
Posted by Christian Heimes <li...@cheimes.de>.
Andi Vajda wrote:
> Fixed. config.py is not written when 'build' is not in the command line
> args anymore. I uploaded a new release candidate, rc2, to
> http://people.apache.org/~vajda/staging_area.
>
> Thank you for the bug report.
Your solution breaks an installation when a user doesn't call "python
setup.py build" before she uses "python setup.py install" to install JCC.
I think the best solution to the issue is a custom build_ext command
that writes the config file after the extensions have been build. If you
like I can try to come up with a suitable solution.
Christian
Re: [VOTE] Release PyLucene 2.4.1
Posted by Andi Vajda <va...@apache.org>.
On Wed, 11 Mar 2009, Andi Vajda wrote:
>> Could you stop setup.py from overwriting config.py when the file already
>> exists and "install" is used as distutils command?
>
> Not when the file already exists since that would prevent changes to setup.py
> from being reflected in config.py but it shouldn't be overwritten during
> 'install'. That's a bug.
Fixed. config.py is not written when 'build' is not in the command line
args anymore. I uploaded a new release candidate, rc2, to
http://people.apache.org/~vajda/staging_area.
Thank you for the bug report.
Andi..
Re: [VOTE] Release PyLucene 2.4.1
Posted by Andi Vajda <va...@apache.org>.
On Wed, 11 Mar 2009, Christian Heimes wrote:
> Andi Vajda schrieb:
>>
>> With the recent release of Java Lucene 2.4.1, it's high time PyLucene
>> get its first release out as an Apache subproject. I hope I got all the
>> steps right and all the things into the right place.
>>
>> Release candidate 1 of PyLucene 2.4.1 is available from:
>>
>> http://people.apache.org/~vajda/staging_area/
>>
>> The list of changes is available from:
>>
>>
>> http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_2_4/CHANGES
>>
>>
>> Please vote to release this artifacts as PyLucene 2.4.1.
>
> I ran into a minor inconvenience. I've build and installed JCC with
>
> JCC_JDK=/usr/lib/jvm/java-6-sun python2.5 setup.py build
> sudo python2.5 setup.py install
>
> The second call to setup.py overwrites JCC's config.py file with default
> settings thus breaking the compilation of PyLucene.
>
> JCC_JDK=/usr/lib/jvm/java-6-sun sudo python2.5 setup.py install
>
> doesn't work either because sudo filters out most environment vars for
> security reasons. In order to build JCC as ordinary user I had to use
>
> JCC_JDK=/usr/lib/jvm/java-6-sun sudo -E python2.5 setup.py install
>
> Could you stop setup.py from overwriting config.py when the file already
> exists and "install" is used as distutils command?
Not when the file already exists since that would prevent changes to
setup.py from being reflected in config.py but it shouldn't be overwritten
during 'install'. That's a bug.
Andi..
Re: [VOTE] Release PyLucene 2.4.1
Posted by Christian Heimes <li...@cheimes.de>.
Andi Vajda schrieb:
>
> With the recent release of Java Lucene 2.4.1, it's high time PyLucene
> get its first release out as an Apache subproject. I hope I got all the
> steps right and all the things into the right place.
>
> Release candidate 1 of PyLucene 2.4.1 is available from:
>
> http://people.apache.org/~vajda/staging_area/
>
> The list of changes is available from:
>
>
> http://svn.apache.org/repos/asf/lucene/pylucene/branches/pylucene_2_4/CHANGES
>
>
> Please vote to release this artifacts as PyLucene 2.4.1.
I ran into a minor inconvenience. I've build and installed JCC with
JCC_JDK=/usr/lib/jvm/java-6-sun python2.5 setup.py build
sudo python2.5 setup.py install
The second call to setup.py overwrites JCC's config.py file with default
settings thus breaking the compilation of PyLucene.
JCC_JDK=/usr/lib/jvm/java-6-sun sudo python2.5 setup.py install
doesn't work either because sudo filters out most environment vars for
security reasons. In order to build JCC as ordinary user I had to use
JCC_JDK=/usr/lib/jvm/java-6-sun sudo -E python2.5 setup.py install
Could you stop setup.py from overwriting config.py when the file already
exists and "install" is used as distutils command?
Christian