You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "zhuojunjian (JIRA)" <ji...@apache.org> on 2013/04/11 15:13:15 UTC
[jira] [Commented] (SOLR-4698) "java.lang.OutOfMemoryError: Map
failed" when run in JRE64
[ https://issues.apache.org/jira/browse/SOLR-4698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628907#comment-13628907 ]
zhuojunjian commented on SOLR-4698:
-----------------------------------
no one can provide any idea?
> "java.lang.OutOfMemoryError: Map failed" when run in JRE64
> ------------------------------------------------------------
>
> Key: SOLR-4698
> URL: https://issues.apache.org/jira/browse/SOLR-4698
> Project: Solr
> Issue Type: Bug
> Components: scripts and tools
> Affects Versions: 4.0
> Environment: CPU:4 cores
> Memory:64G
> Hard Disk:2.5T
> OS:linux
> Reporter: zhuojunjian
> Fix For: 4.3
>
>
> we make solr run in tomcat. and now there are 10 million records desployed averagely in solrcloud which has 3 shards. each record's size is about 1.5k.
> first we used JRE32. then we found it was not fast enough when the current search request count reached 300.
> so we think maybe the memory assigned for solr is not enough. and we replace JRE32 with JRE64, and set "JAVA_OPTS="-Xms2048m -Xmx5120m"" in $TOMCAT_HOME/bin/catalina.sh. but when we restart the solrcloud, we find it failed. the error log is below:
> 2013-04-10 18:58:36,174 INFO org.apache.solr.core.SolrCore:847 - [metadata] CLOSING SolrCore org.apache.solr.core.SolrCore@74a638fc
> 2013-04-10 18:58:36,177 INFO org.apache.solr.core.SolrCore:1658 - [metadata] Closing main searcher on request.
> 2013-04-10 18:58:36,178 ERROR org.apache.solr.core.CoreContainer:875 - Unable to create core: metadata
> org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:721)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:566)
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:850)
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:534)
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:356)
> at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:308)
> at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:107)
> at org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:277)
> at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:258)
> at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:382)
> at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:103)
> at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4650)
> at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5306)
> at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
> at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
> at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
> at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:633)
> at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1114)
> at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1673)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1310)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1422)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:696)
> ... 24 more
> Caused by: java.io.IOException: Map failed
> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748)
> at org.apache.lucene.store.MMapDirectory.map(MMapDirectory.java:284)
> at org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:256)
> at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:224)
> at org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:232)
> at org.apache.lucene.codecs.lucene40.Lucene40PostingsReader.<init>(Lucene40PostingsReader.java:68)
> at org.apache.lucene.codecs.lucene40.Lucene40PostingsFormat.fieldsProducer(Lucene40PostingsFormat.java:316)
> at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:194)
> at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:233)
> at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:107)
> at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:57)
> at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62)
> at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:783)
> at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52)
> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:87)
> at org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:34)
> at org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:119)
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1286)
> at org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:119)
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1286)
> ... 26 more
> Caused by: java.lang.OutOfMemoryError: Map failed
> at sun.nio.ch.FileChannelImpl.map0(Native Method)
> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:745)
> ... 43 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [jira] [Commented] (SOLR-4698) "java.lang.OutOfMemoryError: Map
failed" when run in JRE64
Posted by Erick Erickson <er...@gmail.com>.
See: http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
Especially the section titled "How to configure my operating system
and Java VM to make optimal use of MMapDirectory?"
Best
Erick
On Thu, Apr 11, 2013 at 8:13 AM, zhuojunjian (JIRA) <ji...@apache.org> wrote:
>
> [ https://issues.apache.org/jira/browse/SOLR-4698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628907#comment-13628907 ]
>
> zhuojunjian commented on SOLR-4698:
> -----------------------------------
>
> no one can provide any idea?
>
>> "java.lang.OutOfMemoryError: Map failed" when run in JRE64
>> ------------------------------------------------------------
>>
>> Key: SOLR-4698
>> URL: https://issues.apache.org/jira/browse/SOLR-4698
>> Project: Solr
>> Issue Type: Bug
>> Components: scripts and tools
>> Affects Versions: 4.0
>> Environment: CPU:4 cores
>> Memory:64G
>> Hard Disk:2.5T
>> OS:linux
>> Reporter: zhuojunjian
>> Fix For: 4.3
>>
>>
>> we make solr run in tomcat. and now there are 10 million records desployed averagely in solrcloud which has 3 shards. each record's size is about 1.5k.
>> first we used JRE32. then we found it was not fast enough when the current search request count reached 300.
>> so we think maybe the memory assigned for solr is not enough. and we replace JRE32 with JRE64, and set "JAVA_OPTS="-Xms2048m -Xmx5120m"" in $TOMCAT_HOME/bin/catalina.sh. but when we restart the solrcloud, we find it failed. the error log is below:
>> 2013-04-10 18:58:36,174 INFO org.apache.solr.core.SolrCore:847 - [metadata] CLOSING SolrCore org.apache.solr.core.SolrCore@74a638fc
>> 2013-04-10 18:58:36,177 INFO org.apache.solr.core.SolrCore:1658 - [metadata] Closing main searcher on request.
>> 2013-04-10 18:58:36,178 ERROR org.apache.solr.core.CoreContainer:875 - Unable to create core: metadata
>> org.apache.solr.common.SolrException: Error opening new searcher
>> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:721)
>> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:566)
>> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:850)
>> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:534)
>> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:356)
>> at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:308)
>> at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:107)
>> at org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:277)
>> at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:258)
>> at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:382)
>> at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:103)
>> at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4650)
>> at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5306)
>> at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
>> at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
>> at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
>> at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:633)
>> at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1114)
>> at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1673)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:662)
>> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
>> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1310)
>> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1422)
>> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:696)
>> ... 24 more
>> Caused by: java.io.IOException: Map failed
>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748)
>> at org.apache.lucene.store.MMapDirectory.map(MMapDirectory.java:284)
>> at org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:256)
>> at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:224)
>> at org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:232)
>> at org.apache.lucene.codecs.lucene40.Lucene40PostingsReader.<init>(Lucene40PostingsReader.java:68)
>> at org.apache.lucene.codecs.lucene40.Lucene40PostingsFormat.fieldsProducer(Lucene40PostingsFormat.java:316)
>> at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:194)
>> at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:233)
>> at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:107)
>> at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:57)
>> at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62)
>> at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:783)
>> at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52)
>> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:87)
>> at org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:34)
>> at org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:119)
>> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1286)
>> at org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:119)
>> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1286)
>> ... 26 more
>> Caused by: java.lang.OutOfMemoryError: Map failed
>> at sun.nio.ch.FileChannelImpl.map0(Native Method)
>> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:745)
>> ... 43 more
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA administrators
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org