You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by didier deshommes <df...@gmail.com> on 2009/09/25 00:32:08 UTC
OutOfMemoryError due to auto-warming
Hi there,
We are running solr and allocating 1GB to it and we keep having
OutOfMemoryErrors. We get messages like this:
Error during auto-warming of
key:org.apache.solr.search.QueryResultKey@c785194d:java.lang.OutOfMemoryError:
Java heap space
at java.util.Arrays.copyOfRange(Arrays.java:3209)
at java.lang.String.<init>(String.java:216)
at org.apache.lucene.index.TermBuffer.toTerm(TermBuffer.java:122)
at org.apache.lucene.index.SegmentTermEnum.term(SegmentTermEnum.java:169)
at org.apache.lucene.search.FieldCacheImpl$StringIndexCache.createValue(FieldCacheImpl.java:701)
at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:208)
at org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:676)
at org.apache.solr.search.MissingLastOrdComparator.setNextReader(MissingStringLastComparatorSource.java:181)
at org.apache.lucene.search.TopFieldCollector$OneComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:94)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:252)
at org.apache.lucene.search.Searcher.search(Searcher.java:173)
at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:988)
at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884)
at org.apache.solr.search.SolrIndexSearcher.access$000(SolrIndexSearcher.java:51)
at org.apache.solr.search.SolrIndexSearcher$3.regenerateItem(SolrIndexSearcher.java:332)
at org.apache.solr.search.LRUCache.warm(LRUCache.java:194)
at org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java:1481)
at org.apache.solr.core.SolrCore$3.call(SolrCore.java:1154)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
And like this:
Error during auto-warming of
key:org.apache.solr.search.QueryResultKey@33cf792:java.lang.OutOfMemoryError:
Java heap space
We've searched and one suggestion was to reduce the size of the
various caches that do sorting in solrconfig.xml
(http://osdir.com/ml/solr-user.lucene.apache.org/2009-05/msg01043.html).
Does this solution generally work? Can anyone think of any other
cause for this problem?
didier
Re: OutOfMemoryError due to auto-warming
Posted by didier deshommes <df...@gmail.com>.
On Thu, Sep 24, 2009 at 5:40 PM, Francis Yakin <fy...@liquid.com> wrote:
> You also can increase the JVM HeapSize if you have enough physical memory, like for example if you have 4GB physical, gives the JVM heapsize 2GB or 2.5GB.
Thanks,
we can definitely do that (we have 4GB available). I also forgot to
add that we're running a development version of solr (git clone from ~
3 weeks ago).
Thanks,
didier
>
> Francis
>
> -----Original Message-----
> From: didier deshommes [mailto:dfdeshom@gmail.com]
> Sent: Thursday, September 24, 2009 3:32 PM
> To: solr-user@lucene.apache.org
> Cc: Andrew Montalenti
> Subject: OutOfMemoryError due to auto-warming
>
> Hi there,
> We are running solr and allocating 1GB to it and we keep having
> OutOfMemoryErrors. We get messages like this:
>
> Error during auto-warming of
> key:org.apache.solr.search.QueryResultKey@c785194d:java.lang.OutOfMemoryError:
> Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3209)
> at java.lang.String.<init>(String.java:216)
> at org.apache.lucene.index.TermBuffer.toTerm(TermBuffer.java:122)
> at org.apache.lucene.index.SegmentTermEnum.term(SegmentTermEnum.java:169)
> at org.apache.lucene.search.FieldCacheImpl$StringIndexCache.createValue(FieldCacheImpl.java:701)
> at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:208)
> at org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:676)
> at org.apache.solr.search.MissingLastOrdComparator.setNextReader(MissingStringLastComparatorSource.java:181)
> at org.apache.lucene.search.TopFieldCollector$OneComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:94)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:252)
> at org.apache.lucene.search.Searcher.search(Searcher.java:173)
> at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:988)
> at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884)
> at org.apache.solr.search.SolrIndexSearcher.access$000(SolrIndexSearcher.java:51)
> at org.apache.solr.search.SolrIndexSearcher$3.regenerateItem(SolrIndexSearcher.java:332)
> at org.apache.solr.search.LRUCache.warm(LRUCache.java:194)
> at org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java:1481)
> at org.apache.solr.core.SolrCore$3.call(SolrCore.java:1154)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
>
> And like this:
> Error during auto-warming of
> key:org.apache.solr.search.QueryResultKey@33cf792:java.lang.OutOfMemoryError:
> Java heap space
>
> We've searched and one suggestion was to reduce the size of the
> various caches that do sorting in solrconfig.xml
> (http://osdir.com/ml/solr-user.lucene.apache.org/2009-05/msg01043.html).
> Does this solution generally work? Can anyone think of any other
> cause for this problem?
>
> didier
>
RE: OutOfMemoryError due to auto-warming
Posted by Francis Yakin <fy...@liquid.com>.
You also can increase the JVM HeapSize if you have enough physical memory, like for example if you have 4GB physical, gives the JVM heapsize 2GB or 2.5GB.
Francis
-----Original Message-----
From: didier deshommes [mailto:dfdeshom@gmail.com]
Sent: Thursday, September 24, 2009 3:32 PM
To: solr-user@lucene.apache.org
Cc: Andrew Montalenti
Subject: OutOfMemoryError due to auto-warming
Hi there,
We are running solr and allocating 1GB to it and we keep having
OutOfMemoryErrors. We get messages like this:
Error during auto-warming of
key:org.apache.solr.search.QueryResultKey@c785194d:java.lang.OutOfMemoryError:
Java heap space
at java.util.Arrays.copyOfRange(Arrays.java:3209)
at java.lang.String.<init>(String.java:216)
at org.apache.lucene.index.TermBuffer.toTerm(TermBuffer.java:122)
at org.apache.lucene.index.SegmentTermEnum.term(SegmentTermEnum.java:169)
at org.apache.lucene.search.FieldCacheImpl$StringIndexCache.createValue(FieldCacheImpl.java:701)
at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:208)
at org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:676)
at org.apache.solr.search.MissingLastOrdComparator.setNextReader(MissingStringLastComparatorSource.java:181)
at org.apache.lucene.search.TopFieldCollector$OneComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:94)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:252)
at org.apache.lucene.search.Searcher.search(Searcher.java:173)
at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:988)
at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884)
at org.apache.solr.search.SolrIndexSearcher.access$000(SolrIndexSearcher.java:51)
at org.apache.solr.search.SolrIndexSearcher$3.regenerateItem(SolrIndexSearcher.java:332)
at org.apache.solr.search.LRUCache.warm(LRUCache.java:194)
at org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java:1481)
at org.apache.solr.core.SolrCore$3.call(SolrCore.java:1154)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
And like this:
Error during auto-warming of
key:org.apache.solr.search.QueryResultKey@33cf792:java.lang.OutOfMemoryError:
Java heap space
We've searched and one suggestion was to reduce the size of the
various caches that do sorting in solrconfig.xml
(http://osdir.com/ml/solr-user.lucene.apache.org/2009-05/msg01043.html).
Does this solution generally work? Can anyone think of any other
cause for this problem?
didier
RE: OutOfMemoryError due to auto-warming
Posted by Francis Yakin <fy...@liquid.com>.
I reduced the size of queryResultCache in solrconfig seems to fix the issue as well.
<!-- Maximum number of documents to cache for any entry in the
queryResultCache. -->
<queryResultMaxDocsCached>200</queryResultMaxDocsCached>
>From 500
<!-- Maximum number of documents to cache for any entry in the
queryResultCache. -->
<queryResultMaxDocsCached>500</queryResultMaxDocsCached>
Francis
-----Original Message-----
From: didier deshommes [mailto:dfdeshom@gmail.com]
Sent: Thursday, September 24, 2009 3:32 PM
To: solr-user@lucene.apache.org
Cc: Andrew Montalenti
Subject: OutOfMemoryError due to auto-warming
Hi there,
We are running solr and allocating 1GB to it and we keep having
OutOfMemoryErrors. We get messages like this:
Error during auto-warming of
key:org.apache.solr.search.QueryResultKey@c785194d:java.lang.OutOfMemoryError:
Java heap space
at java.util.Arrays.copyOfRange(Arrays.java:3209)
at java.lang.String.<init>(String.java:216)
at org.apache.lucene.index.TermBuffer.toTerm(TermBuffer.java:122)
at org.apache.lucene.index.SegmentTermEnum.term(SegmentTermEnum.java:169)
at org.apache.lucene.search.FieldCacheImpl$StringIndexCache.createValue(FieldCacheImpl.java:701)
at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:208)
at org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:676)
at org.apache.solr.search.MissingLastOrdComparator.setNextReader(MissingStringLastComparatorSource.java:181)
at org.apache.lucene.search.TopFieldCollector$OneComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:94)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:252)
at org.apache.lucene.search.Searcher.search(Searcher.java:173)
at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:988)
at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884)
at org.apache.solr.search.SolrIndexSearcher.access$000(SolrIndexSearcher.java:51)
at org.apache.solr.search.SolrIndexSearcher$3.regenerateItem(SolrIndexSearcher.java:332)
at org.apache.solr.search.LRUCache.warm(LRUCache.java:194)
at org.apache.solr.search.SolrIndexSearcher.warm(SolrIndexSearcher.java:1481)
at org.apache.solr.core.SolrCore$3.call(SolrCore.java:1154)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
And like this:
Error during auto-warming of
key:org.apache.solr.search.QueryResultKey@33cf792:java.lang.OutOfMemoryError:
Java heap space
We've searched and one suggestion was to reduce the size of the
various caches that do sorting in solrconfig.xml
(http://osdir.com/ml/solr-user.lucene.apache.org/2009-05/msg01043.html).
Does this solution generally work? Can anyone think of any other
cause for this problem?
didier