You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Pavitar Singh <ps...@sprinklr.com> on 2012/07/12 02:11:58 UTC
Solr 4.0 Alpha taking lot of CPU
Hi,
We upgraded to Solr 4.0 Alpha and our CPU usage shot off to 400%.In
profiling we are getting following trace.
- *100.0%* -
*java.lang*.Thread.run<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
- *42 Collapsed methods
(show)<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
*
- *98.0%* - *org.apache.lucene.index*
.DocumentsWriter.updateDocument<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
- *77.0%* - *org.apache.lucene.index*
.DocumentsWriterPerThread.updateDocument<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
- *76.0%* - *org.apache.lucene.index*
.DocFieldProcessor.processDocument<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
- *76.0%* - *org.apache.lucene.index*
.DocInverterPerField.processFields<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
- *36.0%* - *org.apache.lucene.analysis.miscellaneous*
.TrimFilter.incrementToken<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
- *35.0%* - *org.apache.lucene.analysis.core*
.LowerCaseFilter.incrementToken<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
- *17.0%* - *org.apache.lucene.analysis.ngram*
.NGramTokenFilter.incrementToken<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
- *9.2%* - *org.apache.lucene.util*
.AttributeSource.clearAttributes<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
- *34.0%* - *org.apache.lucene.index*
.TermsHashPerField.add<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
- *12.0%* - *org.apache.lucene.index*
.FreqProxTermsWriterPerField.addTerm<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
- *11.0%* - *org.apache.lucene.index*
.FreqProxTermsWriterPerField.writeProx<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
- *6.4%* - *org.apache.lucene.index*
.TermsHashPerField.writeVInt<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
- *11.0%* - *
org.apache.lucene.analysis.tokenattributes*
.CharTermAttributeImpl.fillBytesRef<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
- *15.0%* - *org.apache.lucene.index*
.DocumentsWriterFlushControl.obtainAndLock<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
Re: Solr 4.0 Alpha taking lot of CPU
Posted by Pavitar Singh <ps...@sprinklr.com>.
In text format
100.0% - java.lang.Thread.run
42 Collapsed methods (show)
98.0% - org.apache.lucene.index.DocumentsWriter.updateDocument
77.0% - org.apache.lucene.index.DocumentsWriterPerThread.updateDocument
76.0% - org.apache.lucene.index.DocFieldProcessor.processDocument
76.0% - org.apache.lucene.index.DocInverterPerField.processFields
36.0% - org.apache.lucene.analysis.miscellaneous.TrimFilter.incrementToken
35.0% - org.apache.lucene.analysis.core.LowerCaseFilter.incrementToken
17.0% - org.apache.lucene.analysis.ngram.NGramTokenFilter.incrementToken
9.2% - org.apache.lucene.util.AttributeSource.clearAttributes
34.0% - org.apache.lucene.index.TermsHashPerField.add
12.0% - org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm
11.0% - org.apache.lucene.index.FreqProxTermsWriterPerField.writeProx
6.4% - org.apache.lucene.index.TermsHashPerField.writeVInt
11.0% -
org.apache.lucene.analysis.tokenattributes.CharTermAttributeImpl.fillBytesRef
15.0% - org.apache.lucene.index.DocumentsWriterFlushControl.obtainAndLock
On Thu, Jul 12, 2012 at 5:41 AM, Pavitar Singh <ps...@sprinklr.com> wrote:
> Hi,
>
> We upgraded to Solr 4.0 Alpha and our CPU usage shot off to 400%.In
> profiling we are getting following trace.
>
>
> - *100.0%* - *java.lang*.Thread.run<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *42 Collapsed methods (show)<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> *
> - *98.0%* - *org.apache.lucene.index*
> .DocumentsWriter.updateDocument<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *77.0%* - *org.apache.lucene.index*
> .DocumentsWriterPerThread.updateDocument<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *76.0%* - *org.apache.lucene.index*
> .DocFieldProcessor.processDocument<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *76.0%* - *org.apache.lucene.index*
> .DocInverterPerField.processFields<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *36.0%* - *
> org.apache.lucene.analysis.miscellaneous*
> .TrimFilter.incrementToken<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *35.0%* - *org.apache.lucene.analysis.core*
> .LowerCaseFilter.incrementToken<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *17.0%* - *org.apache.lucene.analysis.ngram*
> .NGramTokenFilter.incrementToken<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *9.2%* - *org.apache.lucene.util*
> .AttributeSource.clearAttributes<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *34.0%* - *org.apache.lucene.index*
> .TermsHashPerField.add<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *12.0%* - *org.apache.lucene.index*
> .FreqProxTermsWriterPerField.addTerm<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *11.0%* - *org.apache.lucene.index*
> .FreqProxTermsWriterPerField.writeProx<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *6.4%* - *org.apache.lucene.index*
> .TermsHashPerField.writeVInt<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *11.0%* - *
> org.apache.lucene.analysis.tokenattributes*
> .CharTermAttributeImpl.fillBytesRef<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *15.0%* - *org.apache.lucene.index*
> .DocumentsWriterFlushControl.obtainAndLock<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
>
>
Re: Solr 4.0 Alpha taking lot of CPU
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Wed, Jul 11, 2012 at 8:11 PM, Pavitar Singh <ps...@sprinklr.com> wrote:
> We upgraded to Solr 4.0 Alpha and our CPU usage shot off to 400%.In
> profiling we are getting following trace.
That could either be good or bad. Higher CPU can mean higher
concurrency. Have you benchmarked your indexing performance?
Example: going from 60 minutes for indexing and 200% average CPU usage
to 30 minutes at 400% CPU would generally be considered a good thing.
-Yonik
http://lucidimagination.com
>
> - *100.0%* -
> *java.lang*.Thread.run<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *42 Collapsed methods
> (show)<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> *
> - *98.0%* - *org.apache.lucene.index*
> .DocumentsWriter.updateDocument<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *77.0%* - *org.apache.lucene.index*
> .DocumentsWriterPerThread.updateDocument<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *76.0%* - *org.apache.lucene.index*
> .DocFieldProcessor.processDocument<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *76.0%* - *org.apache.lucene.index*
>
> .DocInverterPerField.processFields<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *36.0%* - *org.apache.lucene.analysis.miscellaneous*
>
> .TrimFilter.incrementToken<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *35.0%* - *org.apache.lucene.analysis.core*
>
> .LowerCaseFilter.incrementToken<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *17.0%* - *org.apache.lucene.analysis.ngram*
>
> .NGramTokenFilter.incrementToken<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *9.2%* - *org.apache.lucene.util*
>
> .AttributeSource.clearAttributes<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *34.0%* - *org.apache.lucene.index*
>
> .TermsHashPerField.add<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *12.0%* - *org.apache.lucene.index*
>
> .FreqProxTermsWriterPerField.addTerm<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *11.0%* - *org.apache.lucene.index*
>
> .FreqProxTermsWriterPerField.writeProx<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *6.4%* - *org.apache.lucene.index*
>
> .TermsHashPerField.writeVInt<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *11.0%* - *
> org.apache.lucene.analysis.tokenattributes*
>
> .CharTermAttributeImpl.fillBytesRef<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>
> - *15.0%* - *org.apache.lucene.index*
> .DocumentsWriterFlushControl.obtainAndLock<https://rpm.newrelic.com/accounts/132291/applications/834717/profiles/1266#>