Performance Improvments?

["Palmer, Andrew MMI Woking" <an...@vnuinc.co.uk>]

Hi,

 

I am looking at changing the value BufferedIndexOutput.BUFFER_SIZE  from
1024 to maybe 8192.  Has anyone done anything similar and did they get
any performance improvements.  

 

Generally I am looking to reduce the time it takes to optimize the
index, currently a 20GB index takes about an hour.  I have at least four
indexes of this size and they are all in the critical path for the
production cycle.

 

Any other suggestion would be appreciated.

 

 

Andrew

 

 

Re: Performance Improvments?

[Doug Cutting <cu...@apache.org>]

Palmer, Andrew MMI Woking wrote:
> I am looking at changing the value BufferedIndexOutput.BUFFER_SIZE  from
> 1024 to maybe 8192.  Has anyone done anything similar and did they get
> any performance improvements.  

I doubt this will speed things much.

> Generally I am looking to reduce the time it takes to optimize the
> index, currently a 20GB index takes about an hour.  I have at least four
> indexes of this size and they are all in the critical path for the
> production cycle.

A RAID device with higher i/o throughput might help.

Or perhaps you can just optimize the index less often.  Instead use a 
very small merge factor (i.e., 2) to minimize the number of segments. 
This will make searches faster.  Increase minMergeDocs to 1000 or so to 
keep updates fast.

Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org