Search and Update one index with two processes simultaneously

[br...@gsa.gov]

Hello,

Is it possible to have two separate process, one performing searches, and
the other performing updates on the same index?  I have a system in
production that uses this design and occasionally the search program grinds
to a halt.  I first suspected that this was just a load issue, but there
isn't that much load (peak times average 2-3 requests per second, with
occasional bursts of 10-20 requests) and I can't replicate the problem.
The logs show that when the slowdown occurs we are usually answering
requests to search at first, but ongoing searches have stopped finishing
(somewhere inside IndexSearcher.search()).  There doesn't seem to be a
single expensive query that might be bringing us to our kness either.  So,
I was wondering if maybe it is possible that this is a race condition
caused by our update program, which is a separate program that updates the
index while it is being searched.

Some basic info:

The search program uses a single IndexSearcher to perform all searches.
Results are collected with a HitCollector which uses the same IndexSearcher
to extract each document - there is a requirement that the documents be
returned in a specific order, so we have an external structure to determine
the order, once the ID (not the internal ID) has been extracted.
A separate HitCollector is used for each search.
This IndexSearcher in the search program is swapped for a new one when the
update program has finished an update cycle and notifies the search
program.
The index is about 90k documents, average query returns less than 100 hits.

Thanks for any information, or just for your opinion.

Brad



---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org