You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by sweety <sw...@yahoo.com> on 2013/12/10 15:37:37 UTC
Java heap space:out of memory
I just indexed 10 doc of total 15mb.For some queries it works fine but,
for some queries i get this error:
<response>
<lst name="error">
<str name="msg">java.lang.OutOfMemoryError: Java heap space</str>
<str name="trace">
java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:651)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:364)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:928)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:987)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:298)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at
java.lang.Thread.run(Unknown Source) Caused by: java.lang.OutOfMemoryError:
Java heap space
</str>
<int name="code">500</int>
</lst>
</response>
I have direclty indexed them into solr.
My schema.xml is:
<fields>
<field name="doc_id" type="uuid" indexed="true" stored="true" default="NEW"
multiValued="false"/>
<field name="id" type="integer" indexed="true" stored="true" required="true"
multiValued="false"/>
<field name="contents" type="text" indexed="true" stored="true"
multiValued="false"/>
<field name="author" type="title_text" indexed="true" stored="true"
multiValued="true"/>
<field name="title" type="title_text" indexed="true" stored="true"/>
<field name="_version_" type="long" indexed="true" stored="true"
multiValued="false"/>
<copyfield source="id" dest="text" />
<dynamicField name="ignored_*" type="text" indexed="false" stored="false"
multiValued="true"/>
<field name="spelltext" type="spell" indexed="true" stored="false"
multiValued="true" />
<copyField source="contents" dest="spelltext" />
</fields>
I dont understand for such small num of doc why do i get this error.
I havent studied much about solr performance details.
How to increase the heap size? because I need to index a lot more data
still.
Thanks in advance.
--
View this message in context: http://lucene.472066.n3.nabble.com/Java-heap-space-out-of-memory-tp4105903.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Java heap space:out of memory
Posted by Gora Mohanty <go...@mimirtech.com>.
On 10 December 2013 23:39, Shawn Heisey <so...@elyograg.org> wrote:
> On 12/10/2013 11:05 AM, sweety wrote:
>> sorry but i dont know how to check that?
>
> On the Solr admin UI dashboard, one of the numbers is JVM-Memory. There
> are three numbers below that.
This is great advice, and something that I had forgotten. This
also has the advantage of being independent of the container
that one is using: Jetty/Tomcat/whatever.
Specifically for Tomcat, as Michael had pointed out, please look
at the -Xmx and -Xms options when starting Tomcat.
Regards,
Gora
Re: Java heap space:out of memory
Posted by Michael Della Bitta <mi...@appinions.com>.
Just for future reference, I think Tomcat uses CATALINA_OPTS for the server
runtime options.
Michael Della Bitta
Applications Developer
o: +1 646 532 3062 | c: +1 917 477 7906
appinions inc.
“The Science of Influence Marketing”
18 East 41st Street
New York, NY 10017
t: @appinions <https://twitter.com/Appinions> | g+:
plus.google.com/appinions<https://plus.google.com/u/0/b/112002776285509593336/112002776285509593336/posts>
w: appinions.com <http://www.appinions.com/>
On Wed, Dec 11, 2013 at 2:14 AM, sweety <sw...@yahoo.com> wrote:
> You were right the changes made in JAVA_OPTs didn't show increase in the
> heap
> size, I made changes in the UI of tomcat
> Initial pool memory : 512 MB
> Maximum pool memory : 1024 MB
>
> Now the heap size has increased.
> Thanks you all for your suggestions,it really saved my time.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Java-heap-space-out-of-memory-tp4105903p4106082.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Java heap space:out of memory
Posted by sweety <sw...@yahoo.com>.
You were right the changes made in JAVA_OPTs didn't show increase in the heap
size, I made changes in the UI of tomcat
Initial pool memory : 512 MB
Maximum pool memory : 1024 MB
Now the heap size has increased.
Thanks you all for your suggestions,it really saved my time.
--
View this message in context: http://lucene.472066.n3.nabble.com/Java-heap-space-out-of-memory-tp4105903p4106082.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Java heap space:out of memory
Posted by Shawn Heisey <so...@elyograg.org>.
On 12/10/2013 11:47 PM, sweety wrote:
> yes,i did put the space,as in the image
Chances are good that wherever it was that you put the JAVA_OPTS is not
being used by the servlet container that's running, or perhaps you never
stopped and restarted the correct process.
Thanks,
Shawn
Re: Java heap space:out of memory
Posted by sweety <sw...@yahoo.com>.
yes,i did put the space,as in the image
--
View this message in context: http://lucene.472066.n3.nabble.com/Java-heap-space-out-of-memory-tp4105903p4106077.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Java heap space:out of memory
Posted by Shawn Heisey <so...@elyograg.org>.
On 12/10/2013 10:09 PM, sweety wrote:
> I have set : JAVA_OPTS as value: -Xms1024M-Xmx1024M
> But the dashboard still shows 64M,but now the usage is only 18%
> How could that be? yesterday it was 87%.
Due to Java's garbage collection memory model, memory usage will vary.
If "-Xms1024M-Xmx1024M" is the literal string you have used, it won't
work - you'll need "-Xms1024M -Xmx1024M" instead -- note the space that
I've added.
Thanks,
Shawn
Re: Java heap space:out of memory
Posted by Ahmet Arslan <io...@yahoo.com>.
Hi,
Did you put a space between flags? -Xms1g -Xmx1g
On Wednesday, December 11, 2013 7:09 AM, sweety <sw...@yahoo.com> wrote:
I have set : JAVA_OPTS as value: -Xms1024M-Xmx1024M
But the dashboard still shows 64M,but now the usage is only 18%
How could that be? yesterday it was 87%.
--
View this message in context: http://lucene.472066.n3.nabble.com/Java-heap-space-out-of-memory-tp4105903p4106069.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Java heap space:out of memory
Posted by sweety <sw...@yahoo.com>.
I have set : JAVA_OPTS as value: -Xms1024M-Xmx1024M
But the dashboard still shows 64M,but now the usage is only 18%
How could that be? yesterday it was 87%.
--
View this message in context: http://lucene.472066.n3.nabble.com/Java-heap-space-out-of-memory-tp4105903p4106069.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Java heap space:out of memory
Posted by Shawn Heisey <so...@elyograg.org>.
On 12/10/2013 11:26 AM, sweety wrote:
> okay thanks,
> here it is:
> max heap size : 63.56MB(it is howing 37.2% usage though)
> How to increase that size??
64MB is an extremely small heap size. Chances are good that because you
are running on Windows 7, Java is in client mode, especially if the Java
is 32bit rather than 64bit. General advice: always use a 64-bit CPU,
operating system, and Java. A 32-bit Java is unable to use a heap size
larger than 2GB.
You need to add options like the following to your servlet container
startup:
-server -d64 -Xms1024M-Xmx1024M
The "1024M" part of that says that you're allocating 1GB to Java. You
may need to make that number bigger or smaller to fit your specific needs.
http://www.jkstack.com/2012/12/how-to-set-tomcat-heap-size-jvm-heap.html
Thanks,
Shawn
Re: Java heap space:out of memory
Posted by sweety <sw...@yahoo.com>.
okay thanks,
here it is:
max heap size : 63.56MB(it is howing 37.2% usage though)
How to increase that size??
--
View this message in context: http://lucene.472066.n3.nabble.com/Java-heap-space-out-of-memory-tp4105903p4105952.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Java heap space:out of memory
Posted by Shawn Heisey <so...@elyograg.org>.
On 12/10/2013 11:05 AM, sweety wrote:
> sorry but i dont know how to check that?
On the Solr admin UI dashboard, one of the numbers is JVM-Memory. There
are three numbers below that. Those numbers are (from left to right and
top to bottom) currently used heap, current heap size, and max heap
size. It's the max heap size that you need for Gora's question.
You can also obtain the number using various java commandline/GUI tools
to talk to the java process, but the admin UI in your browser is very
easy, assuming it is working.
Thanks,
Shawn
Re: Java heap space:out of memory
Posted by sweety <sw...@yahoo.com>.
sorry but i dont know how to check that?
--
View this message in context: http://lucene.472066.n3.nabble.com/Java-heap-space-out-of-memory-tp4105903p4105947.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Java heap space:out of memory
Posted by Gora Mohanty <go...@mimirtech.com>.
On 10 December 2013 22:25, sweety <sw...@yahoo.com> wrote:
>
> 4gb ram.
> I m running on Windows 7,with Tomcat as webserver.
As Michael asked, how much of this memory is made available to Solr?
Regards,
Gora
Re: Java heap space:out of memory
Posted by sweety <sw...@yahoo.com>.
4gb ram.
I m running on Windows 7,with Tomcat as webserver.
--
View this message in context: http://lucene.472066.n3.nabble.com/Java-heap-space-out-of-memory-tp4105903p4105929.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Java heap space:out of memory
Posted by Michael Della Bitta <mi...@appinions.com>.
I'm not sure how you're running Solr, but generally if you're using the
Java command line to launch Jetty, you do something like this:
java -Xmx512m -jar start.jar
That would give you a half-gigabyte heap.
Are you running this on a machine with low overall RAM, or low available
RAM?
Michael Della Bitta
Applications Developer
o: +1 646 532 3062 | c: +1 917 477 7906
appinions inc.
“The Science of Influence Marketing”
18 East 41st Street
New York, NY 10017
t: @appinions <https://twitter.com/Appinions> | g+:
plus.google.com/appinions<https://plus.google.com/u/0/b/112002776285509593336/112002776285509593336/posts>
w: appinions.com <http://www.appinions.com/>
On Tue, Dec 10, 2013 at 9:37 AM, sweety <sw...@yahoo.com> wrote:
> I just indexed 10 doc of total 15mb.For some queries it works fine but,
> for some queries i get this error:
> <response>
> <lst name="error">
> <str name="msg">java.lang.OutOfMemoryError: Java heap space</str>
> <str name="trace">
> java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space at
>
> org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:651)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:364)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> at
>
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
> at
>
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
> at
>
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
> at
>
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:928)
> at
>
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
> at
>
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:987)
> at
>
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539)
> at
>
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:298)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at
> java.lang.Thread.run(Unknown Source) Caused by: java.lang.OutOfMemoryError:
> Java heap space
> </str>
> <int name="code">500</int>
> </lst>
> </response>
>
> I have direclty indexed them into solr.
> My schema.xml is:
> <fields>
>
> <field name="doc_id" type="uuid" indexed="true" stored="true" default="NEW"
> multiValued="false"/>
> <field name="id" type="integer" indexed="true" stored="true"
> required="true"
> multiValued="false"/>
> <field name="contents" type="text" indexed="true" stored="true"
> multiValued="false"/>
> <field name="author" type="title_text" indexed="true" stored="true"
> multiValued="true"/>
> <field name="title" type="title_text" indexed="true" stored="true"/>
>
> <field name="_version_" type="long" indexed="true" stored="true"
> multiValued="false"/>
> <copyfield source="id" dest="text" />
> <dynamicField name="ignored_*" type="text" indexed="false" stored="false"
> multiValued="true"/>
>
> <field name="spelltext" type="spell" indexed="true" stored="false"
> multiValued="true" />
> <copyField source="contents" dest="spelltext" />
> </fields>
>
> I dont understand for such small num of doc why do i get this error.
> I havent studied much about solr performance details.
> How to increase the heap size? because I need to index a lot more data
> still.
> Thanks in advance.
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Java-heap-space-out-of-memory-tp4105903.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>