Re: Lucene CPU Utilization

[jw...@dicarta.com]

We're going to run into this issue when dealing with some of our larger customers.

What we plan on doing is to separate our indexers in to separate cpus, and then throttle them by using sleep(100) or some other number to be determined in testing.  We also plan on doing this over 2 weekends, since our user base on the weekend is probably 10% of that on a weekday.  We'll merge the generated indexes into a master index at the end of each weekend, so the users can search on half (hopefully) of the documents in the first week of deployment and all in the second week.

Jeff Wang
diCarta, Inc.

-----Original Message-----
From:	Amany Moussa [mailto:amany_moussa@yahoo.com]
Sent:	Mon 2/20/2006 12:22 PM
To:	java-user@lucene.apache.org
Cc:	
Subject:	Re: Lucene CPU Utilization
Thank you so much for your reply.

I know that you answered this question before. I just
wanted to post the question to receive more feedbacks
and share the information.

Thanks again.

Amany M.

--- Otis Gospodnetic <ot...@yahoo.com>
wrote:

> I think I answered that question just the other
> day.... privately...
> No, there is nothing in Lucene to help you with CPU
> utilization.
> However, if you are running this on a UNIX box of
> some kind, you can (re)nice the process and thus
> lower its priority, giving other processes more time
> with the CPU.  Windows may have something similar.
> 
> Otis
> 
> ----- Original Message ----
> From: Amany Moussa <am...@yahoo.com>
> To: java-user@lucene.apache.org
> Sent: Monday, February 20, 2006 9:50:57 AM
> Subject: Lucene CPU Utilization
> 
> 
> Hello,
> 
> I am building a Lucene index with over a million   
> documents retrieved from database. I am running the
> application on Unix, I am getting a 100% CPU
> utilization the moment the application start.
> The application creates a list of small indices in a
> temp directory then merge them all in the main index
> file.                                               
>  
> Is there any way I can tune the indexing process and
> reduce the CPU utilization.                         
> Thanks much.                              
> 
> Amany M.
> 
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam
> protection around 
> http://mail.yahoo.com 
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail:
> java-user-help@lucene.apache.org
> 
> 
> 
> 
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail:
> java-user-help@lucene.apache.org
> 
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org