opening index readers and writers to keep indexes updated

[Paulo Silveira <pa...@paulo.com.br>]

Hello everybody.

I have a big index that will be stored in the FS. I have lots of
updates, insertions and deletions in the index, and I would like to
minimize the number of "phatom reads".

I ve seen in the wiki this link:

http://wiki.apache.org/jakarta-lucene/UpdatingAnIndex

So, what about my idea:

- A singleton SearchEngine
- SearchEngine has a IndexReader attribute, and _no_ index writer.
- When inserting a new Document, open a writer, writes, close and the
replace the old reader for a new one.
- When deleting, open a reader, delete, replace the older reader for
this opened reader.
- The update method deletes than inserts.
- When searching, use the attribute IndexReader

Now some questions:

Is it too bad to open/close a writer every time I write on the index?

What about opening a new IndexReader for each new search, instead of
keeping the IndexReader attribute?

Would it be a lot better to batch updates? I could have a stack of
documents to be updated, and I would only write to the indexes when
the documentsToBeUpdated.size() reaches a certain number.

thanks a lot.
--
Paulo Silveira
http://www.paulo.com.br/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: opening index readers and writers to keep indexes updated

[Otis Gospodnetic <ot...@yahoo.com>]

Definitely batch your adds/updates/deletes, and reuse the IndexReader as you described instead of opening a new one for every search.  I _believe_ you can keep the same IndexWriter for adds, as long as you don't overlap it with an IndexReader that does deletes.  If you have Lucene in Action, check out Concurrency Rules, section 2.9.1: http://www.lucenebook.com/search?query=concurrency+rules

Otis


----- Original Message ----
From: Paulo Silveira <pa...@paulo.com.br>
To: java-user@lucene.apache.org
Sent: Thu 09 Feb 2006 10:59:17 AM EST
Subject: opening index readers and writers to keep indexes updated

Hello everybody.

I have a big index that will be stored in the FS. I have lots of
updates, insertions and deletions in the index, and I would like to
minimize the number of "phatom reads".

I ve seen in the wiki this link:

http://wiki.apache.org/jakarta-lucene/UpdatingAnIndex

So, what about my idea:

- A singleton SearchEngine
- SearchEngine has a IndexReader attribute, and _no_ index writer.
- When inserting a new Document, open a writer, writes, close and the
replace the old reader for a new one.
- When deleting, open a reader, delete, replace the older reader for
this opened reader.
- The update method deletes than inserts.
- When searching, use the attribute IndexReader

Now some questions:

Is it too bad to open/close a writer every time I write on the index?

What about opening a new IndexReader for each new search, instead of
keeping the IndexReader attribute?

Would it be a lot better to batch updates? I could have a stack of
documents to be updated, and I would only write to the indexes when
the documentsToBeUpdated.size() reaches a certain number.

thanks a lot.
--
Paulo Silveira
http://www.paulo.com.br/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org





---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org