Multiple Indexes - Merge?

[João Rodrigues <an...@gmail.com>]

Hello all! I am new to Lucene and I'm using PyLucene's extension in my
project. I'm using it to index a large volume of data (index size estimated
in 75GB) over the web and as my internet connection isn't that good enough,
I am asking you guys, probably more experienced than I am, if there is a way
of, if the index is cut short (my script, if connection fails, will optimize
and close the index, just as if it was done), to start from where it ended.
Another option was to merge several "little" indexes.

Hope you can help me out on this one.

Thanks in advance

João Rodrigues

Re: Multiple Indexes - Merge?

[Chris Hostetter <ho...@fucit.org>]

: Alright, thanks for the answers. I'd tried the pylucene lists and they've
: directed me here. I'm sorry for the trouble.

No trouble at all ... I just wanted to make sure you were aware of all the 
resources available to you.


-Hoss

Re: Multiple Indexes - Merge?

[João Rodrigues <an...@gmail.com>]

Alright, thanks for the answers. I'd tried the pylucene lists and they've
directed me here. I'm sorry for the trouble.

Again, thanks for the quick reply, it'll be useful indeed :)

João

Re: Multiple Indexes - Merge?

[Chris Hostetter <ho...@fucit.org>]

: in 75GB) over the web and as my internet connection isn't that good enough,
: I am asking you guys, probably more experienced than I am, if there is a way
: of, if the index is cut short (my script, if connection fails, will optimize
: and close the index, just as if it was done), to start from where it ended.
: Another option was to merge several "little" indexes.

Well.. the short answer is "yes" you can merge indexes ... the 
IndexReader.addIndexes method will do this for you.  In general though, 
your problem statement doesn't seem to really relate to Lucene, or require 
merging of indexes ...if your app encounters a problem unrelated to 
luncene (ie: a network problem) and closes the index cleanly, then when 
the problem goes away your app can open an IndexWriter on the index and 
start right back up where it left off.




-Hoss