You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@lucenenet.apache.org by Khoa Vo <vo...@gmail.com> on 2011/03/16 03:07:50 UTC
[Lucene.Net] Adding document to a large index large time.
I am trying to add a new document to an existing optimized Index with about
600k documents.
It seems like the IndexWriter method AddDocument(doc) is taking a long time
(30 seconds or so)
Is this the usual behavior?
Note: Initially indexing all 600 documents only takes about 20 minutes.
It seems like something is blocking, but I am not sure.
Any insight would be helpful
-Khoa
Re: [Lucene.Net] Adding document to a large index large time.
Posted by Nic Wise <ni...@fastchicken.co.nz>.
Look up MaxMergeDocs and MergeFactor on IndexWriter. This may be set
to merge various segments when you add past a certain number of docs,
which can take a while.
How big is the index in general? I've worked with multi-GB indexes (up
to about 250GB) which function fine IF these are set correctly :)
(I dont have lucene.net docs or VS handy at the moment, so I'm going
off the java docs, but they are here:
http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/index/IndexWriter.html#getMaxMergeDocs()
http://lucene.apache.org/java/2_4_0/api/org/apache/lucene/index/IndexWriter.html#getMergeFactor()
(those are for 2.4, not sure which version Lucene.NET is coming off at
the moment)
On Wed, Mar 16, 2011 at 02:07, Khoa Vo <vo...@gmail.com> wrote:
> I am trying to add a new document to an existing optimized Index with about
> 600k documents.
>
> It seems like the IndexWriter method AddDocument(doc) is taking a long time
> (30 seconds or so)
>
> Is this the usual behavior?
>
> Note: Initially indexing all 600 documents only takes about 20 minutes.
>
> It seems like something is blocking, but I am not sure.
>
> Any insight would be helpful
>
> -Khoa
>
--
Nic Wise
t. +44 7788 592 806 | @fastchicken | http://www.linkedin.com/in/nicwise
b. http://www.fastchicken.co.nz/ | http://www.flickr.com/photos/nicwise
mobileAgent (for FreeAgent): get your accounts in your pocket.
http://goo.gl/IuBU
Trip Wallet: Keep track of your budget on the go: http://goo.gl/ePhKa