You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by J Mohamed Zahoor <za...@indix.com> on 2013/05/17 09:17:30 UTC
Java heap space exception in 4.2.1
Hi
I moved to 4.2.1 from 4.1 recently.. everything was working fine until i added few more stats query..
Now i am getting this error frequently that solr does not run even for 2 minutes continuously.
All 5GB is getting used instantaneously in few queries...
SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
at org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:560)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:365)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485)
at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:856)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:722)
one thing i did was when we moved from 4.1 to 4.2.1 i changed only the solr.war and left other jars and config as it is...
WIll that create a problem...??
./zahoor
Re: Java heap space exception in 4.2.1
Posted by J Mohamed Zahoor <za...@indix.com>.
Hprof introspection shows that huge Double Array are using up 75% of heap space... which belongs to Lucen's FieldCache..
./zahoor
On 17-May-2013, at 12:47 PM, J Mohamed Zahoor <za...@indix.com> wrote:
> Hi
>
> I moved to 4.2.1 from 4.1 recently.. everything was working fine until i added few more stats query..
> Now i am getting this error frequently that solr does not run even for 2 minutes continuously.
> All 5GB is getting used instantaneously in few queries...
>
>
> SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
> at org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:560)
> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)
> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
> at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
> at org.eclipse.jetty.server.Server.handle(Server.java:365)
> at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485)
> at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
> at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937)
> at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998)
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:856)
> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
> at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
> at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
> at java.lang.Thread.run(Thread.java:722)
>
>
>
> one thing i did was when we moved from 4.1 to 4.2.1 i changed only the solr.war and left other jars and config as it is...
> WIll that create a problem...??
>
> ./zahoor
>
>
Re: Java heap space exception in 4.2.1
Posted by Erick Erickson <er...@gmail.com>.
400M docs is quite a large number of documents for a single piece of
hardware, and
if you're faceting over a large number of unique values, this will
chew up memory.
So it's not surprising that you're seeing OOMs, I suspect you just have too many
documents on a single machine..
Best
Erick
On Mon, May 27, 2013 at 3:11 AM, Jam Luo <co...@gmail.com> wrote:
> I am sorry about a type mistake 8,000,000,000 -> 800,000,000
>
>
> 2013/5/27 Jam Luo <co...@gmail.com>
>
>> I have the same problem. at 4.1 ,a solr instance could take 8,000,000,000
>> doc. but at 4.2.1, a instance only take 400,000,000 doc, it will oom at
>> facet query. the facet field was token by space.
>>
>> May 27, 2013 11:12:55 AM org.apache.solr.common.SolrException log
>> SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java
>> heap space
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
>> at
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1338)
>> at
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
>> at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
>> at
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
>> at
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>> at
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
>> at
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
>> at
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
>> at
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
>> at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>> at
>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
>> at
>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
>> at
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
>> at org.eclipse.jetty.server.Server.handle(Server.java:350)
>> at
>> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
>> at
>> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
>> at
>> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900)
>> at
>> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954)
>> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:851)
>> at
>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
>> at
>> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
>> at
>> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
>> at
>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:603)
>> at
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:538)
>> at java.lang.Thread.run(Thread.java:662)
>> Caused by: java.lang.OutOfMemoryError: Java heap space
>> at
>> org.apache.lucene.index.DocTermOrds.uninvert(DocTermOrds.java:448)
>> at
>> org.apache.solr.request.UnInvertedField.<init>(UnInvertedField.java:179)
>> at
>> org.apache.solr.request.UnInvertedField.getUnInvertedField(UnInvertedField.java:664)
>> at
>> org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:426)
>> at
>> org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:517)
>> at
>> org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:252)
>> at
>> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:78)
>> at
>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
>> at
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1825)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
>> at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
>> at
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1338)
>> at
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
>> at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
>> at
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
>> at
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>> at
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
>> at
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
>> at
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
>> at
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
>> at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>> at
>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
>> at
>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
>> at
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
>> at org.eclipse.jetty.server.Server.handle(Server.java:350)
>> at
>> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
>> at
>> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
>> at
>> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900)
>> at
>> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954)
>> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:851)
>>
>>
>>
>> 2013/5/19 J Mohamed Zahoor <za...@indix.com>
>>
>>>
>>> aah… was doing a facet on a double field which was having 6 decimal
>>> places…
>>> No surprise that the lucene cache got full…
>>>
>>> .z/ahoor
>>>
>>> On 17-May-2013, at 11:56 PM, J Mohamed Zahoor <za...@indix.com> wrote:
>>>
>>> > Memory increase a lot with queries which have facets…
>>> >
>>> >
>>> > ./Zahoor
>>> >
>>> >
>>> > On 17-May-2013, at 10:00 PM, Shawn Heisey <so...@elyograg.org> wrote:
>>> >
>>> >> On 5/17/2013 1:17 AM, J Mohamed Zahoor wrote:
>>> >>> I moved to 4.2.1 from 4.1 recently.. everything was working fine
>>> until i added few more stats query..
>>> >>> Now i am getting this error frequently that solr does not run even
>>> for 2 minutes continuously.
>>> >>> All 5GB is getting used instantaneously in few queries...
>>> >>
>>> >> Someone on IRC ran into memory problems upgrading from 4.0 to 4.2. It
>>> >> wasn't OOM errors, they were just using a lot more heap than they were
>>> >> before and running into constant full garbage collections.
>>> >>
>>> >> There is another message on this list about someone who upgraded from
>>> >> 3.5 to 4.2 and is having memory troubles.
>>> >>
>>> >> The person on IRC made most of their fields unstored and reindexed,
>>> >> which fixed the problem for them. They only needed a few fields
>>> stored.
>>> >>
>>> >> Because the IRC user was on 4.0, I originally thought it had something
>>> >> to do with compressed stored fields, but on this thread, they started
>>> >> with 4.1. If that was the released 4.1.0 and not a SNAPSHOT version,
>>> >> then they had compressed stored fields before the upgrade.
>>> >>
>>> >> The user on IRC was not using termvectors or docvalues, which would be
>>> >> potential pain points unique to 4.2.
>>> >>
>>> >> I'm using 4.2.1 with no trouble in my setup, but I do have a heap
>>> that's
>>> >> considerably larger than I need. There are no apparent memory leaks -
>>> >> it's been running for over a month with updates once a minute. I've
>>> >> finally switched over from the 3.5.0 index to the new one, so for the
>>> >> last few days, it has been also taking our full query load.
>>> >>
>>> >> What could have changed between 4.1 and 4.2 to cause dramatically
>>> >> increased memory usage?
>>> >>
>>> >> From my /admin/system:
>>> >>
>>> >> <date name="startTime">2013-04-05T15:52:55.751Z</date>
>>> >>
>>> >> Thanks,
>>> >> Shawn
>>> >>
>>> >
>>>
>>>
>>
Re: Java heap space exception in 4.2.1
Posted by Jam Luo <co...@gmail.com>.
I am sorry about a type mistake 8,000,000,000 -> 800,000,000
2013/5/27 Jam Luo <co...@gmail.com>
> I have the same problem. at 4.1 ,a solr instance could take 8,000,000,000
> doc. but at 4.2.1, a instance only take 400,000,000 doc, it will oom at
> facet query. the facet field was token by space.
>
> May 27, 2013 11:12:55 AM org.apache.solr.common.SolrException log
> SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java
> heap space
> at
> org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1338)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
> at org.eclipse.jetty.server.Server.handle(Server.java:350)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
> at
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900)
> at
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954)
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:851)
> at
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
> at
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
> at
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:603)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:538)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.lucene.index.DocTermOrds.uninvert(DocTermOrds.java:448)
> at
> org.apache.solr.request.UnInvertedField.<init>(UnInvertedField.java:179)
> at
> org.apache.solr.request.UnInvertedField.getUnInvertedField(UnInvertedField.java:664)
> at
> org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:426)
> at
> org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:517)
> at
> org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:252)
> at
> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:78)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1825)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1338)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
> at org.eclipse.jetty.server.Server.handle(Server.java:350)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
> at
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900)
> at
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954)
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:851)
>
>
>
> 2013/5/19 J Mohamed Zahoor <za...@indix.com>
>
>>
>> aah… was doing a facet on a double field which was having 6 decimal
>> places…
>> No surprise that the lucene cache got full…
>>
>> .z/ahoor
>>
>> On 17-May-2013, at 11:56 PM, J Mohamed Zahoor <za...@indix.com> wrote:
>>
>> > Memory increase a lot with queries which have facets…
>> >
>> >
>> > ./Zahoor
>> >
>> >
>> > On 17-May-2013, at 10:00 PM, Shawn Heisey <so...@elyograg.org> wrote:
>> >
>> >> On 5/17/2013 1:17 AM, J Mohamed Zahoor wrote:
>> >>> I moved to 4.2.1 from 4.1 recently.. everything was working fine
>> until i added few more stats query..
>> >>> Now i am getting this error frequently that solr does not run even
>> for 2 minutes continuously.
>> >>> All 5GB is getting used instantaneously in few queries...
>> >>
>> >> Someone on IRC ran into memory problems upgrading from 4.0 to 4.2. It
>> >> wasn't OOM errors, they were just using a lot more heap than they were
>> >> before and running into constant full garbage collections.
>> >>
>> >> There is another message on this list about someone who upgraded from
>> >> 3.5 to 4.2 and is having memory troubles.
>> >>
>> >> The person on IRC made most of their fields unstored and reindexed,
>> >> which fixed the problem for them. They only needed a few fields
>> stored.
>> >>
>> >> Because the IRC user was on 4.0, I originally thought it had something
>> >> to do with compressed stored fields, but on this thread, they started
>> >> with 4.1. If that was the released 4.1.0 and not a SNAPSHOT version,
>> >> then they had compressed stored fields before the upgrade.
>> >>
>> >> The user on IRC was not using termvectors or docvalues, which would be
>> >> potential pain points unique to 4.2.
>> >>
>> >> I'm using 4.2.1 with no trouble in my setup, but I do have a heap
>> that's
>> >> considerably larger than I need. There are no apparent memory leaks -
>> >> it's been running for over a month with updates once a minute. I've
>> >> finally switched over from the 3.5.0 index to the new one, so for the
>> >> last few days, it has been also taking our full query load.
>> >>
>> >> What could have changed between 4.1 and 4.2 to cause dramatically
>> >> increased memory usage?
>> >>
>> >> From my /admin/system:
>> >>
>> >> <date name="startTime">2013-04-05T15:52:55.751Z</date>
>> >>
>> >> Thanks,
>> >> Shawn
>> >>
>> >
>>
>>
>
Re: Java heap space exception in 4.2.1
Posted by Jam Luo <co...@gmail.com>.
I have the same problem. at 4.1 ,a solr instance could take 8,000,000,000
doc. but at 4.2.1, a instance only take 400,000,000 doc, it will oom at
facet query. the facet field was token by space.
May 27, 2013 11:12:55 AM org.apache.solr.common.SolrException log
SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java
heap space
at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1338)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
at org.eclipse.jetty.server.Server.handle(Server.java:350)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:851)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:603)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:538)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.OutOfMemoryError: Java heap space
at
org.apache.lucene.index.DocTermOrds.uninvert(DocTermOrds.java:448)
at
org.apache.solr.request.UnInvertedField.<init>(UnInvertedField.java:179)
at
org.apache.solr.request.UnInvertedField.getUnInvertedField(UnInvertedField.java:664)
at
org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:426)
at
org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:517)
at
org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:252)
at
org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:78)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1825)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1338)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
at org.eclipse.jetty.server.Server.handle(Server.java:350)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:851)
2013/5/19 J Mohamed Zahoor <za...@indix.com>
>
> aah… was doing a facet on a double field which was having 6 decimal places…
> No surprise that the lucene cache got full…
>
> .z/ahoor
>
> On 17-May-2013, at 11:56 PM, J Mohamed Zahoor <za...@indix.com> wrote:
>
> > Memory increase a lot with queries which have facets…
> >
> >
> > ./Zahoor
> >
> >
> > On 17-May-2013, at 10:00 PM, Shawn Heisey <so...@elyograg.org> wrote:
> >
> >> On 5/17/2013 1:17 AM, J Mohamed Zahoor wrote:
> >>> I moved to 4.2.1 from 4.1 recently.. everything was working fine until
> i added few more stats query..
> >>> Now i am getting this error frequently that solr does not run even for
> 2 minutes continuously.
> >>> All 5GB is getting used instantaneously in few queries...
> >>
> >> Someone on IRC ran into memory problems upgrading from 4.0 to 4.2. It
> >> wasn't OOM errors, they were just using a lot more heap than they were
> >> before and running into constant full garbage collections.
> >>
> >> There is another message on this list about someone who upgraded from
> >> 3.5 to 4.2 and is having memory troubles.
> >>
> >> The person on IRC made most of their fields unstored and reindexed,
> >> which fixed the problem for them. They only needed a few fields stored.
> >>
> >> Because the IRC user was on 4.0, I originally thought it had something
> >> to do with compressed stored fields, but on this thread, they started
> >> with 4.1. If that was the released 4.1.0 and not a SNAPSHOT version,
> >> then they had compressed stored fields before the upgrade.
> >>
> >> The user on IRC was not using termvectors or docvalues, which would be
> >> potential pain points unique to 4.2.
> >>
> >> I'm using 4.2.1 with no trouble in my setup, but I do have a heap that's
> >> considerably larger than I need. There are no apparent memory leaks -
> >> it's been running for over a month with updates once a minute. I've
> >> finally switched over from the 3.5.0 index to the new one, so for the
> >> last few days, it has been also taking our full query load.
> >>
> >> What could have changed between 4.1 and 4.2 to cause dramatically
> >> increased memory usage?
> >>
> >> From my /admin/system:
> >>
> >> <date name="startTime">2013-04-05T15:52:55.751Z</date>
> >>
> >> Thanks,
> >> Shawn
> >>
> >
>
>
Re: Java heap space exception in 4.2.1
Posted by J Mohamed Zahoor <za...@indix.com>.
aah… was doing a facet on a double field which was having 6 decimal places…
No surprise that the lucene cache got full…
.z/ahoor
On 17-May-2013, at 11:56 PM, J Mohamed Zahoor <za...@indix.com> wrote:
> Memory increase a lot with queries which have facets…
>
>
> ./Zahoor
>
>
> On 17-May-2013, at 10:00 PM, Shawn Heisey <so...@elyograg.org> wrote:
>
>> On 5/17/2013 1:17 AM, J Mohamed Zahoor wrote:
>>> I moved to 4.2.1 from 4.1 recently.. everything was working fine until i added few more stats query..
>>> Now i am getting this error frequently that solr does not run even for 2 minutes continuously.
>>> All 5GB is getting used instantaneously in few queries...
>>
>> Someone on IRC ran into memory problems upgrading from 4.0 to 4.2. It
>> wasn't OOM errors, they were just using a lot more heap than they were
>> before and running into constant full garbage collections.
>>
>> There is another message on this list about someone who upgraded from
>> 3.5 to 4.2 and is having memory troubles.
>>
>> The person on IRC made most of their fields unstored and reindexed,
>> which fixed the problem for them. They only needed a few fields stored.
>>
>> Because the IRC user was on 4.0, I originally thought it had something
>> to do with compressed stored fields, but on this thread, they started
>> with 4.1. If that was the released 4.1.0 and not a SNAPSHOT version,
>> then they had compressed stored fields before the upgrade.
>>
>> The user on IRC was not using termvectors or docvalues, which would be
>> potential pain points unique to 4.2.
>>
>> I'm using 4.2.1 with no trouble in my setup, but I do have a heap that's
>> considerably larger than I need. There are no apparent memory leaks -
>> it's been running for over a month with updates once a minute. I've
>> finally switched over from the 3.5.0 index to the new one, so for the
>> last few days, it has been also taking our full query load.
>>
>> What could have changed between 4.1 and 4.2 to cause dramatically
>> increased memory usage?
>>
>> From my /admin/system:
>>
>> <date name="startTime">2013-04-05T15:52:55.751Z</date>
>>
>> Thanks,
>> Shawn
>>
>
Re: Java heap space exception in 4.2.1
Posted by J Mohamed Zahoor <za...@indix.com>.
Memory increase a lot with queries which have facets…
./Zahoor
On 17-May-2013, at 10:00 PM, Shawn Heisey <so...@elyograg.org> wrote:
> On 5/17/2013 1:17 AM, J Mohamed Zahoor wrote:
>> I moved to 4.2.1 from 4.1 recently.. everything was working fine until i added few more stats query..
>> Now i am getting this error frequently that solr does not run even for 2 minutes continuously.
>> All 5GB is getting used instantaneously in few queries...
>
> Someone on IRC ran into memory problems upgrading from 4.0 to 4.2. It
> wasn't OOM errors, they were just using a lot more heap than they were
> before and running into constant full garbage collections.
>
> There is another message on this list about someone who upgraded from
> 3.5 to 4.2 and is having memory troubles.
>
> The person on IRC made most of their fields unstored and reindexed,
> which fixed the problem for them. They only needed a few fields stored.
>
> Because the IRC user was on 4.0, I originally thought it had something
> to do with compressed stored fields, but on this thread, they started
> with 4.1. If that was the released 4.1.0 and not a SNAPSHOT version,
> then they had compressed stored fields before the upgrade.
>
> The user on IRC was not using termvectors or docvalues, which would be
> potential pain points unique to 4.2.
>
> I'm using 4.2.1 with no trouble in my setup, but I do have a heap that's
> considerably larger than I need. There are no apparent memory leaks -
> it's been running for over a month with updates once a minute. I've
> finally switched over from the 3.5.0 index to the new one, so for the
> last few days, it has been also taking our full query load.
>
> What could have changed between 4.1 and 4.2 to cause dramatically
> increased memory usage?
>
> From my /admin/system:
>
> <date name="startTime">2013-04-05T15:52:55.751Z</date>
>
> Thanks,
> Shawn
>
Re: Java heap space exception in 4.2.1
Posted by Shawn Heisey <so...@elyograg.org>.
On 5/17/2013 1:17 AM, J Mohamed Zahoor wrote:
> I moved to 4.2.1 from 4.1 recently.. everything was working fine until i added few more stats query..
> Now i am getting this error frequently that solr does not run even for 2 minutes continuously.
> All 5GB is getting used instantaneously in few queries...
Someone on IRC ran into memory problems upgrading from 4.0 to 4.2. It
wasn't OOM errors, they were just using a lot more heap than they were
before and running into constant full garbage collections.
There is another message on this list about someone who upgraded from
3.5 to 4.2 and is having memory troubles.
The person on IRC made most of their fields unstored and reindexed,
which fixed the problem for them. They only needed a few fields stored.
Because the IRC user was on 4.0, I originally thought it had something
to do with compressed stored fields, but on this thread, they started
with 4.1. If that was the released 4.1.0 and not a SNAPSHOT version,
then they had compressed stored fields before the upgrade.
The user on IRC was not using termvectors or docvalues, which would be
potential pain points unique to 4.2.
I'm using 4.2.1 with no trouble in my setup, but I do have a heap that's
considerably larger than I need. There are no apparent memory leaks -
it's been running for over a month with updates once a minute. I've
finally switched over from the 3.5.0 index to the new one, so for the
last few days, it has been also taking our full query load.
What could have changed between 4.1 and 4.2 to cause dramatically
increased memory usage?
>From my /admin/system:
<date name="startTime">2013-04-05T15:52:55.751Z</date>
Thanks,
Shawn