You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by raulgrande83 <ra...@hotmail.com> on 2013/03/14 17:07:35 UTC

Out of Memory doing a query Solr 4.2

Hi 

After doing a query to Solr to get the uniqueIds (string of 20 characters)
of 700 documents in a collection, I'm getting an out of memory error using
Solr 4.2. I tried to increase the JVM-Memory 1G (from 3G to 4G) however this
didn't change anything.

This was working on 3.5. 

I've moved from 3.5 to 4.2.

Did anyone have the same problem?

Thanks


------------------------------------------

Details :

Solr 4.2
Solr Index 20G aprox.

JVM: IBM J9 VM(1.6.0.2.4)
JVM-Memory:4G
S.O. Linux
Processors 8
RAM: 101G

------------------------------------

org.apache.solr.common.SolrException log 
SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError 
        at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:651) 
        at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:364) 
        at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141) 
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) 
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) 
        at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:240) 
        at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:164) 
        at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:164) 
        at
org.apache.catalina.ha.session.JvmRouteBinderValve.invoke(JvmRouteBinderValve.java:218) 
        at
org.apache.catalina.ha.tcp.ReplicationValve.invoke(ReplicationValve.java:333) 
        at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100) 
        at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) 
        at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:394) 
        at
org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:284) 
        at
org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:322) 
        at
org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1714) 
        at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:898) 
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:920) 
        at java.lang.Thread.run(Thread.java:736) 
Caused by: java.lang.OutOfMemoryError 
        at java.util.Arrays.copyOfRange(Arrays.java:4114) 
        at java.util.Arrays.copyOf(Arrays.java:3833) 
        at java.lang.StringCoding.safeTrim(StringCoding.java:686) 
        at java.lang.StringCoding.access$300(StringCoding.java:41) 
        at
java.lang.StringCoding$StringDecoder.decode(StringCoding.java:739) 
        at java.lang.StringCoding.decode(StringCoding.java:746) 
        at java.lang.String.<init>(String.java:2036) 
        at java.lang.String.<init>(String.java:2011) 
        at
org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.readField(CompressingStoredFieldsReader.java:143) 
        at
org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.visitDocument(CompressingStoredFieldsReader.java:272) 
        at
org.apache.lucene.index.SegmentReader.document(SegmentReader.java:139) 
        at
org.apache.lucene.index.BaseCompositeReader.document(BaseCompositeReader.java:116) 
        at
org.apache.lucene.index.IndexReader.document(IndexReader.java:436) 
        at
org.apache.lucene.document.LazyDocument.getDocument(LazyDocument.java:65) 
        at
org.apache.lucene.document.LazyDocument.access$000(LazyDocument.java:36) 
        at
org.apache.lucene.document.LazyDocument$LazyField.stringValue(LazyDocument.java:105) 
        at org.apache.solr.schema.FieldType.toExternal(FieldType.java:346) 
        at org.apache.solr.schema.FieldType.toObject(FieldType.java:355) 
        at
org.apache.solr.response.BinaryResponseWriter$Resolver.getValue(BinaryResponseWriter.java:208) 
        at
org.apache.solr.response.BinaryResponseWriter$Resolver.getDoc(BinaryResponseWriter.java:186) 
        at
org.apache.solr.response.BinaryResponseWriter$Resolver.writeResultsBody(BinaryResponseWriter.java:147) 
        at
org.apache.solr.response.BinaryResponseWriter$Resolver.writeResults(BinaryResponseWriter.java:173) 
        at
org.apache.solr.response.BinaryResponseWriter$Resolver.resolve(BinaryResponseWriter.java:86) 
        at
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:154) 
        at
org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:144) 
        at
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:234) 
        at
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:149) 
        at
org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:92) 
        at
org.apache.solr.response.BinaryResponseWriter.write(BinaryResponseWriter.java:50) 
        at
org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:618) 
        at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:356) 
        at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141) 
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) 
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) 
        at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:240) 
        at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:164) 
        at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:164) 
        at
org.apache.catalina.ha.session.JvmRouteBinderValve.invoke(JvmRouteBinderValve.java:218) 
        at
org.apache.catalina.ha.tcp.ReplicationValve.invoke(ReplicationValve.java:333) 
        at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100) 
        at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) 
        at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:394) 




--
View this message in context: http://lucene.472066.n3.nabble.com/Out-of-Memory-doing-a-query-Solr-4-2-tp4047394.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Out of Memory doing a query Solr 4.2

Posted by Robert Muir <rc...@gmail.com>.
On Fri, Mar 15, 2013 at 6:46 AM, raulgrande83 <ra...@hotmail.com> wrote:
> Thank you for your help. I'm afraid it won't be so easy to change de jvm
> version, because it is required at the moment.
>
> It seems that Solr 4.2 supports Java 1.6 at least. Is that correct?
>
> Could you find any clue of what is happening in the attached traces? It
> would be great to know why it is happening now, because it was working for
> Solr 3.5.

Its probably not an OOM at all. instead its more likely IBM JVM is
probably miscompiling our code and producing large integers, like it
does quite often. For example, we had to disable testing it completely
recently for this reason. If someone were to report a JIRA issue that
mentioned IBM, I'd make the same comment there but in general not take
it seriously at all due to the kind of bugs i've seen from that JVM.

The fact that IBM JVM didnt miscompile 3.5's code is irrelevant.

Re: Out of Memory doing a query Solr 4.2

Posted by raulgrande83 <ra...@hotmail.com>.
Thank you for your help. I'm afraid it won't be so easy to change de jvm
version, because it is required at the moment. 

It seems that Solr 4.2 supports Java 1.6 at least. Is that correct?

Could you find any clue of what is happening in the attached traces? It
would be great to know why it is happening now, because it was working for
Solr 3.5.


Thanks again.



--
View this message in context: http://lucene.472066.n3.nabble.com/Out-of-Memory-doing-a-query-Solr-4-2-tp4047394p4047654.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Out of Memory doing a query Solr 4.2

Posted by Bernd Fehling <be...@uni-bielefeld.de>.
We are currently using
Oracle Corporation Java HotSpot(TM) 64-Bit Server VM (1.7.0_07 23.3-b01)

Runs excellent and also no memory parameter tweaking neccessary.
Give enough physical and JVM memory, use "-XX:+UseG1GC" and thats it.

Also no "saw tooth" and GC timeouts from JVM as with earlier versions.

Regards
Bernd


Am 15.03.2013 09:09, schrieb raulgrande83:
> Why? Could this be the cause of the problem? This was working ok for Solr
> 3.5.
> 
> Could you recommend me one ?
> 
> Thanks.
> 
> 
> 
> 
> 
> 
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Out-of-Memory-doing-a-query-Solr-4-2-tp4047394p4047621.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Out of Memory doing a query Solr 4.2

Posted by raulgrande83 <ra...@hotmail.com>.
Why? Could this be the cause of the problem? This was working ok for Solr
3.5.

Could you recommend me one ?

Thanks.






--
View this message in context: http://lucene.472066.n3.nabble.com/Out-of-Memory-doing-a-query-Solr-4-2-tp4047394p4047621.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Out of Memory doing a query Solr 4.2

Posted by Robert Muir <rc...@gmail.com>.
On Thu, Mar 14, 2013 at 12:07 PM, raulgrande83 <ra...@hotmail.com> wrote:
> JVM: IBM J9 VM(1.6.0.2.4)

I don't recommend using this JVM.