Concurrency Control & Searching Performance

[Li Bing <lb...@gmail.com>]

Hi, all,

I have used Lucene.NET for some days. I noticed one issue recently. Now the
size of my index becomes large. The total is more than 10G. To increase the
performance, I split the size into smaller ones. But since more data should
be crawled from the Web, the size of index is being increased even when
queries are processed on the same index. My question is whether Lucene.NET
has a proper concurrency control mechanism so that the updating of the index
does not lower the performance of searching?

As I know, Hadoop considers the issue. Since the index of Web data has the
property of write-once & read-many, usually it is not required to consider
the consistency problem. Therefore, the performance of searching must be
good enough.

What about Lucene.NET? Does it has the similar solution? If not, how do you
handle the problem?

Thanks so much!
LB