You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Yuval Dotan <yu...@gmail.com> on 2012/04/30 10:31:46 UTC
Java out of memory - with fieldcache faceting
Hi Guys
I have a problem and i need your assistance
I get an exception when doing field cache faceting:
*/solr/select?q=*:*&facet=true&facet.field=src_ip_str&facet.limit=10*
<lst name="error">
<str name="msg">java.lang.OutOfMemoryError: Java heap space</str>
<str name="trace">
java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:449)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:277)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
at org.eclipse.jetty.server.Server.handle(Server.java:351) at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:890)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:944)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634) at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230) at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
at java.lang.Thread.run(Thread.java:679) Caused by:
java.lang.OutOfMemoryError: Java heap space at
org.apache.lucene.util.packed.Direct16.<init>(Direct16.java:38) at
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:267) at
org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:81) at
org.apache.lucene.search.FieldCacheImpl$DocTermsIndexCache.createValue(FieldCacheImpl.java:1178)
at
org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:248)
at
org.apache.lucene.search.FieldCacheImpl.getTermsIndex(FieldCacheImpl.java:1081)
at
org.apache.lucene.search.FieldCacheImpl.getTermsIndex(FieldCacheImpl.java:1077)
at
org.apache.solr.request.SimpleFacets.getFieldCacheCounts(SimpleFacets.java:459)
at
org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:310)
at
org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:396)
at
org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:205)
at
org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:81)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:204)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1541) at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:435)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:256)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
at org.eclipse.jetty.server.Server.handle(Server.java:351) at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
</str>
<int name="code">500</int>
</lst>
my index size : 2,083,704,454 docs
attached my solrconfig.xml and schema.xml
I'm working with apache-solr-4.0-2012-04-24_08-27-47 + Solr 4.0 (NRT
support) with RankingAlgorithm 1.4.2 (war file 33MB)
<http://solr-ra.tgels.com/solr-ra.jsp>
Thanks
Yuval
Re: Java out of memory - with fieldcache faceting
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Hi,
Tell us more about:
* what you facet on
* how many facet values are in each facet
* how much RAM you have
* 32 or 64 bit
* -Xmx you are using
* faceting method you are using
* ...
Otis
----
Performance Monitoring for Solr - http://sematext.com/spm/solr-performance-monitoring
>________________________________
> From: Yuval Dotan <yu...@gmail.com>
>To: solr-user@lucene.apache.org
>Sent: Monday, April 30, 2012 4:31 AM
>Subject: Java out of memory - with fieldcache faceting
>
>
>Hi Guys
>I have a problem and i need your assistance
>I get an exception when doing field cache faceting:
>
>
>/solr/select?q=*:*&facet=true&facet.field=src_ip_str&facet.limit=10
>
>
><lst name="error">
><str name="msg">java.lang.OutOfMemoryError: Java heap space</str>
><str name="trace">
>java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space at org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:449) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:277) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192) at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) at org.eclipse.jetty.server.Server.handle(Server.java:351) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:890) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:944) at
org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534) at java.lang.Thread.run(Thread.java:679) Caused by: java.lang.OutOfMemoryError: Java heap space at org.apache.lucene.util.packed.Direct16.<init>(Direct16.java:38) at org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:267) at org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:81) at org.apache.lucene.search.FieldCacheImpl$DocTermsIndexCache.createValue(FieldCacheImpl.java:1178) at
org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:248) at org.apache.lucene.search.FieldCacheImpl.getTermsIndex(FieldCacheImpl.java:1081) at org.apache.lucene.search.FieldCacheImpl.getTermsIndex(FieldCacheImpl.java:1077) at org.apache.solr.request.SimpleFacets.getFieldCacheCounts(SimpleFacets.java:459) at org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:310) at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:396) at org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:205) at org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:81) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:204) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1541) at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:435) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:256) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999) at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) at org.eclipse.jetty.server.Server.handle(Server.java:351) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
></str>
><int name="code">500</int>
></lst>
>
>
>my index size : 2,083,704,454 docs
>
>
>attached my solrconfig.xml and schema.xml
>
>
>I'm working with apache-solr-4.0-2012-04-24_08-27-47 + Solr 4.0 (NRT support) with RankingAlgorithm 1.4.2 (war file 33MB)
>
>
>Thanks
>Yuval
>
>
Re: Java out of memory - with fieldcache faceting
Posted by Dan Tuffery <da...@gmail.com>.
There's a Lucene/Solr memory size estimator spreadsheet in the SVN:
http://svn.apache.org/repos/asf/lucene/dev/trunk/dev-tools/size-estimator-lucene-solr.xls
Dan
On Mon, Apr 30, 2012 at 11:39 AM, Yuval Dotan <yu...@gmail.com> wrote:
> Thanks for the fast answer
> One more question:
> Is there a way to know (some formula) what is the size of memory i need for
> these actions?
>
> Thanks
> Yuval
>
> On Mon, Apr 30, 2012 at 11:50, Dan Tuffery <da...@gmail.com> wrote:
>
> > You need to add more memory to the JVM that is running Solr:
> >
> > http://wiki.apache.org/solr/SolrPerformanceFactors#OutOfMemoryErrors
> >
> > Dan
> >
> > On Mon, Apr 30, 2012 at 9:43 AM, Yuval Dotan <yu...@gmail.com>
> wrote:
> >
> > > Hi Guys
> > > I have a problem and i need your assistance
> > > I get an exception when doing field cache faceting (the enum method
> works
> > > perfectly):
> > >
> > > */solr/select?q=*:*&facet=true&facet.field=src_ip_str&facet.limit=10*
> > >
> > > <lst name="error">
> > > <str name="msg">java.lang.OutOfMemoryError: Java heap space</str>
> > > <str name="trace">
> > > java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
> > at
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:449)
> > > at
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:277)
> > > at
> > >
> >
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
> > > at
> > >
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
> > > at
> > >
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
> > > at
> > >
> >
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
> > > at
> > >
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
> > > at
> > >
> >
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
> > > at
> > >
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
> > > at
> > >
> >
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
> > > at
> > >
> >
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
> > > at
> > >
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> > > at
> > >
> >
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
> > > at
> > >
> >
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
> > > at
> > >
> >
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
> > > at org.eclipse.jetty.server.Server.handle(Server.java:351) at
> > >
> >
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
> > > at
> > >
> >
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
> > > at
> > >
> >
> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:890)
> > > at
> > >
> >
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:944)
> > > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634) at
> > > org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230)
> at
> > >
> >
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
> > > at
> > >
> >
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
> > > at
> > >
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
> > > at
> > >
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
> > > at java.lang.Thread.run(Thread.java:679) Caused by:
> > > java.lang.OutOfMemoryError: Java heap space at
> > > org.apache.lucene.util.packed.Direct16.<init>(Direct16.java:38) at
> > >
> org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:267)
> > at
> > >
> org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:81)
> > at
> > >
> >
> org.apache.lucene.search.FieldCacheImpl$DocTermsIndexCache.createValue(FieldCacheImpl.java:1178)
> > > at
> > >
> >
> org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:248)
> > > at
> > >
> >
> org.apache.lucene.search.FieldCacheImpl.getTermsIndex(FieldCacheImpl.java:1081)
> > > at
> > >
> >
> org.apache.lucene.search.FieldCacheImpl.getTermsIndex(FieldCacheImpl.java:1077)
> > > at
> > >
> >
> org.apache.solr.request.SimpleFacets.getFieldCacheCounts(SimpleFacets.java:459)
> > > at
> > >
> org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:310)
> > > at
> > >
> >
> org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:396)
> > > at
> > >
> >
> org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:205)
> > > at
> > >
> >
> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:81)
> > > at
> > >
> >
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:204)
> > > at
> > >
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
> > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1541) at
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:435)
> > > at
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:256)
> > > at
> > >
> >
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
> > > at
> > >
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
> > > at
> > >
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
> > > at
> > >
> >
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
> > > at
> > >
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
> > > at
> > >
> >
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
> > > at
> > >
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
> > > at
> > >
> >
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
> > > at
> > >
> >
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
> > > at
> > >
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> > > at
> > >
> >
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
> > > at
> > >
> >
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
> > > at
> > >
> >
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
> > > at org.eclipse.jetty.server.Server.handle(Server.java:351) at
> > >
> >
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
> > > </str>
> > > <int name="code">500</int>
> > > </lst>
> > >
> > > my index size : 2,083,704,454 docs
> > >
> > > attached my solrconfig.xml and schema.xml
> > >
> > > I'm working with apache-solr-4.0-2012-04-24_08-27-47 + Solr 4.0 (NRT
> > > support) with RankingAlgorithm 1.4.2 (war file 33MB) <
> > http://solr-ra.tgels.com/solr-ra.jsp>
> > >
> > > Thanks
> > > Yuval
> > >
> > >
> >
>
Re: Java out of memory - with fieldcache faceting
Posted by Yuval Dotan <yu...@gmail.com>.
Thanks for the fast answer
One more question:
Is there a way to know (some formula) what is the size of memory i need for
these actions?
Thanks
Yuval
On Mon, Apr 30, 2012 at 11:50, Dan Tuffery <da...@gmail.com> wrote:
> You need to add more memory to the JVM that is running Solr:
>
> http://wiki.apache.org/solr/SolrPerformanceFactors#OutOfMemoryErrors
>
> Dan
>
> On Mon, Apr 30, 2012 at 9:43 AM, Yuval Dotan <yu...@gmail.com> wrote:
>
> > Hi Guys
> > I have a problem and i need your assistance
> > I get an exception when doing field cache faceting (the enum method works
> > perfectly):
> >
> > */solr/select?q=*:*&facet=true&facet.field=src_ip_str&facet.limit=10*
> >
> > <lst name="error">
> > <str name="msg">java.lang.OutOfMemoryError: Java heap space</str>
> > <str name="trace">
> > java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
> at
> >
> org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:449)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:277)
> > at
> >
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
> > at
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
> > at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
> > at
> >
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
> > at
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
> > at
> > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
> > at
> >
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
> > at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
> > at
> >
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
> > at
> >
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
> > at org.eclipse.jetty.server.Server.handle(Server.java:351) at
> >
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
> > at
> >
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
> > at
> >
> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:890)
> > at
> >
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:944)
> > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634) at
> > org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230) at
> >
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
> > at
> >
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
> > at
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
> > at
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
> > at java.lang.Thread.run(Thread.java:679) Caused by:
> > java.lang.OutOfMemoryError: Java heap space at
> > org.apache.lucene.util.packed.Direct16.<init>(Direct16.java:38) at
> > org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:267)
> at
> > org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:81)
> at
> >
> org.apache.lucene.search.FieldCacheImpl$DocTermsIndexCache.createValue(FieldCacheImpl.java:1178)
> > at
> >
> org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:248)
> > at
> >
> org.apache.lucene.search.FieldCacheImpl.getTermsIndex(FieldCacheImpl.java:1081)
> > at
> >
> org.apache.lucene.search.FieldCacheImpl.getTermsIndex(FieldCacheImpl.java:1077)
> > at
> >
> org.apache.solr.request.SimpleFacets.getFieldCacheCounts(SimpleFacets.java:459)
> > at
> > org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:310)
> > at
> >
> org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:396)
> > at
> >
> org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:205)
> > at
> >
> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:81)
> > at
> >
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:204)
> > at
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1541) at
> >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:435)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:256)
> > at
> >
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
> > at
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
> > at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
> > at
> >
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
> > at
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
> > at
> > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
> > at
> >
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
> > at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
> > at
> >
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
> > at
> >
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
> > at org.eclipse.jetty.server.Server.handle(Server.java:351) at
> >
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
> > </str>
> > <int name="code">500</int>
> > </lst>
> >
> > my index size : 2,083,704,454 docs
> >
> > attached my solrconfig.xml and schema.xml
> >
> > I'm working with apache-solr-4.0-2012-04-24_08-27-47 + Solr 4.0 (NRT
> > support) with RankingAlgorithm 1.4.2 (war file 33MB) <
> http://solr-ra.tgels.com/solr-ra.jsp>
> >
> > Thanks
> > Yuval
> >
> >
>
Re: Java out of memory - with fieldcache faceting
Posted by Dan Tuffery <da...@gmail.com>.
You need to add more memory to the JVM that is running Solr:
http://wiki.apache.org/solr/SolrPerformanceFactors#OutOfMemoryErrors
Dan
On Mon, Apr 30, 2012 at 9:43 AM, Yuval Dotan <yu...@gmail.com> wrote:
> Hi Guys
> I have a problem and i need your assistance
> I get an exception when doing field cache faceting (the enum method works
> perfectly):
>
> */solr/select?q=*:*&facet=true&facet.field=src_ip_str&facet.limit=10*
>
> <lst name="error">
> <str name="msg">java.lang.OutOfMemoryError: Java heap space</str>
> <str name="trace">
> java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space at
> org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:449)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:277)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
> at org.eclipse.jetty.server.Server.handle(Server.java:351) at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
> at
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:890)
> at
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:944)
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634) at
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230) at
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
> at
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
> at java.lang.Thread.run(Thread.java:679) Caused by:
> java.lang.OutOfMemoryError: Java heap space at
> org.apache.lucene.util.packed.Direct16.<init>(Direct16.java:38) at
> org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:267) at
> org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:81) at
> org.apache.lucene.search.FieldCacheImpl$DocTermsIndexCache.createValue(FieldCacheImpl.java:1178)
> at
> org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:248)
> at
> org.apache.lucene.search.FieldCacheImpl.getTermsIndex(FieldCacheImpl.java:1081)
> at
> org.apache.lucene.search.FieldCacheImpl.getTermsIndex(FieldCacheImpl.java:1077)
> at
> org.apache.solr.request.SimpleFacets.getFieldCacheCounts(SimpleFacets.java:459)
> at
> org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:310)
> at
> org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:396)
> at
> org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:205)
> at
> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:81)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:204)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1541) at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:435)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:256)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
> at org.eclipse.jetty.server.Server.handle(Server.java:351) at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
> </str>
> <int name="code">500</int>
> </lst>
>
> my index size : 2,083,704,454 docs
>
> attached my solrconfig.xml and schema.xml
>
> I'm working with apache-solr-4.0-2012-04-24_08-27-47 + Solr 4.0 (NRT
> support) with RankingAlgorithm 1.4.2 (war file 33MB) <http://solr-ra.tgels.com/solr-ra.jsp>
>
> Thanks
> Yuval
>
>
Java out of memory - with fieldcache faceting
Posted by Yuval Dotan <yu...@gmail.com>.
Hi Guys
I have a problem and i need your assistance
I get an exception when doing field cache faceting (the enum method works
perfectly):
*/solr/select?q=*:*&facet=true&facet.field=src_ip_str&facet.limit=10*
<lst name="error">
<str name="msg">java.lang.OutOfMemoryError: Java heap space</str>
<str name="trace">
java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:449)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:277)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
at org.eclipse.jetty.server.Server.handle(Server.java:351) at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:890)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:944)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634) at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230) at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
at java.lang.Thread.run(Thread.java:679) Caused by:
java.lang.OutOfMemoryError: Java heap space at
org.apache.lucene.util.packed.Direct16.<init>(Direct16.java:38) at
org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:267) at
org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:81) at
org.apache.lucene.search.FieldCacheImpl$DocTermsIndexCache.createValue(FieldCacheImpl.java:1178)
at
org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:248)
at
org.apache.lucene.search.FieldCacheImpl.getTermsIndex(FieldCacheImpl.java:1081)
at
org.apache.lucene.search.FieldCacheImpl.getTermsIndex(FieldCacheImpl.java:1077)
at
org.apache.solr.request.SimpleFacets.getFieldCacheCounts(SimpleFacets.java:459)
at
org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:310)
at
org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:396)
at
org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:205)
at
org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:81)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:204)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1541) at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:435)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:256)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
at org.eclipse.jetty.server.Server.handle(Server.java:351) at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
</str>
<int name="code">500</int>
</lst>
my index size : 2,083,704,454 docs
attached my solrconfig.xml and schema.xml
I'm working with apache-solr-4.0-2012-04-24_08-27-47 + Solr 4.0 (NRT
support) with RankingAlgorithm 1.4.2 (war file 33MB)
<http://solr-ra.tgels.com/solr-ra.jsp>
Thanks
Yuval