You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@lucenenet.apache.org by Li Bing <lb...@gmail.com> on 2010/07/22 20:02:40 UTC
Consistency and Performance
Dear all,
I asked the similar question one week ago. But I have not got any answers
until now. I just clarifies my questions as follows.
When the size of an index is large and it is being updated, should the
operation of searching be suspended until the updating is done? In my
system, since I use .NET concurrency control to manage the consistency, the
performance becomes lower for the above issue.
How does Lucene.NET handle this inside? If I remove my concurrency control,
the performance will not be affected by updating?
Thanks,
Bing
RE: Consistency and Performance
Posted by Jean-Francois Beaulac <je...@hotmail.com>.
Hi,
If you need to suspend searching or not ultimately depends on your application's requirement, but you usually dont need to.
I recommend you read :http://wiki.apache.org/lucene-java/LuceneFAQ#Does_Lucene_allow_searching_and_indexing_simultaneously.3F
Basically your searcher will only see what was available when it was opened on the index, to see new changes after updating the index you need to re-open the searcher (see http://lucene.apache.org/java/2_9_2/api/core/org/apache/lucene/index/IndexReader.html#reopen())
You may have useless files hanging in your index directory after optimizing if you had searchers opened on the index while you closed the IndexWriter. See the next link for more details.
http://wiki.apache.org/lucene-java/LuceneFAQ#Why_do_I_have_a_deletable_file_.28and_old_segment_files_remain.29_after_running_optimize.3F
cheers,jf
> Date: Fri, 23 Jul 2010 02:02:40 +0800
> Subject: Consistency and Performance
> From: lblabs@gmail.com
> To: lucene-net-user@incubator.apache.org
>
> Dear all,
>
> I asked the similar question one week ago. But I have not got any answers
> until now. I just clarifies my questions as follows.
>
> When the size of an index is large and it is being updated, should the
> operation of searching be suspended until the updating is done? In my
> system, since I use .NET concurrency control to manage the consistency, the
> performance becomes lower for the above issue.
>
> How does Lucene.NET handle this inside? If I remove my concurrency control,
> the performance will not be affected by updating?
>
> Thanks,
> Bing
_________________________________________________________________
Turn down-time into play-time with Messenger games
http://go.microsoft.com/?linkid=9734385
RE: Consistency and Performance
Posted by "Nicholas Paldino [.NET/C# MVP]" <ca...@caspershouse.com>.
There is something I've witnessed though which one should be aware
of when sharing an IndexWriter and doing multiple writes on multiple threads
with the desire to commit or rollback a distinct units of work on those
threads.
The typical pattern of use is to have one IndexWriter open and call
GetReader on it to get your readers. This also means that you will add and
update documents through that one writer.
If you are trying to update two separate documents at the same time
and need to commit one and roll back the other, you have to synchronize
access to the IndexWriter, as the PrepareCommit/Commit/Rollback is across
the entire index and all the documents that have been added/updated since
the last call to Commit or Rollback.
- Nick
-----Original Message-----
From: Digy [mailto:digydigy@gmail.com]
Sent: Thursday, July 22, 2010 2:17 PM
To: lucene-net-user@lucene.apache.org; bing.li@asu.edu
Subject: RE: Consistency and Performance
Hi Li,
Lucene.Net is thread(+process) safe. So you don't have to implement your own
concurrency control.
You can create a single IndexWriter, make updates on it(even with many
threads) and acquire an up-to-date IndexReader (by IndexWriter's GetReader
method) whenever a search request arrives without thinking on "concurrency
control"
DIGY.
-----Original Message-----
From: Li Bing [mailto:lblabs@gmail.com]
Sent: Thursday, July 22, 2010 9:03 PM
To: lucene-net-user@incubator.apache.org
Subject: Consistency and Performance
Dear all,
I asked the similar question one week ago. But I have not got any answers
until now. I just clarifies my questions as follows.
When the size of an index is large and it is being updated, should the
operation of searching be suspended until the updating is done? In my
system, since I use .NET concurrency control to manage the consistency, the
performance becomes lower for the above issue.
How does Lucene.NET handle this inside? If I remove my concurrency control,
the performance will not be affected by updating?
Thanks,
Bing
RE: Consistency and Performance
Posted by Digy <di...@gmail.com>.
Hi Li,
Lucene.Net is thread(+process) safe. So you don't have to implement your own
concurrency control.
You can create a single IndexWriter, make updates on it(even with many
threads) and acquire an up-to-date IndexReader (by IndexWriter's GetReader
method) whenever a search request arrives without thinking on "concurrency
control"
DIGY.
-----Original Message-----
From: Li Bing [mailto:lblabs@gmail.com]
Sent: Thursday, July 22, 2010 9:03 PM
To: lucene-net-user@incubator.apache.org
Subject: Consistency and Performance
Dear all,
I asked the similar question one week ago. But I have not got any answers
until now. I just clarifies my questions as follows.
When the size of an index is large and it is being updated, should the
operation of searching be suspended until the updating is done? In my
system, since I use .NET concurrency control to manage the consistency, the
performance becomes lower for the above issue.
How does Lucene.NET handle this inside? If I remove my concurrency control,
the performance will not be affected by updating?
Thanks,
Bing