You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Frank Moss <fr...@gmail.com> on 2012/01/17 12:05:32 UTC
Error when opening a lucene index: Map failed
Hello,
I am having problems opening a lucene index. The index has been created on
the same machine. The size of index is 44G. Its a 64bit machine running
OpenSuse.
I have tried starting the java process with 1,2,3,4 GB of memory but still
get the same error. Any tips? The gc log shows that its running the GC even
though it has enough memory. But probably this error has something to do
with the memory mapped files using NIO. Anybody has seen anything like this
before? And if so, how did you solve it?
The output of "free -m" before starting the process:
free -m
total used free shared buffers cached
Mem: 24158 20803 3354 0 635 19387
-/+ buffers/cache: 780 23378
Swap: 2204 164 2039
Below is the error.
Thanks in advance.
FM
$ java -d64 -server -Xmx15g -Xms15g -XX:+UseConcMarkSweepGC
-verbose:gc -XX:MaxPermSize=512m -XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/home/frank/heap.dmp -jar lukeall-3.5.0.jar
[Full GC 207648K->28758K(15660544K), 0.1105290 secs]
[Full GC 61479K->15416K(15660544K), 0.0654310 secs]
[Full GC 69950K->15418K(15660544K), 0.0717170 secs]
[Full GC 69952K->15418K(15660544K), 0.0661720 secs]
java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:849)
at org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:265)
at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216)
at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:89)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:115)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:93)
at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:113)
at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:83)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:754)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:75)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:462)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:377)
at org.getopt.luke.Luke.openIndex(Unknown Source)
at org.getopt.luke.Luke.openOk(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at thinlet.Thinlet.invokeImpl(Unknown Source)
at thinlet.Thinlet.invoke(Unknown Source)
at java.awt.Component.dispatchEventImpl(Component.java:4861)
Caused by: java.lang.OutOfMemoryError: Map failed
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:846)
... 48 more
Re: Error when opening a lucene index: Map failed
Posted by Frank Moss <fr...@gmail.com>.
Thanks it solved the problem.
On Tue, Jan 17, 2012 at 12:12 PM, Ian Lea <ia...@gmail.com> wrote:
> Try $ ulimit -v unlimited
>
> --
> Ian.
>
>
> On Tue, Jan 17, 2012 at 11:05 AM, Frank Moss <fr...@gmail.com>
> wrote:
> > Hello,
> >
> > I am having problems opening a lucene index. The index has been created
> on
> > the same machine. The size of index is 44G. Its a 64bit machine running
> > OpenSuse.
> >
> > I have tried starting the java process with 1,2,3,4 GB of memory but
> still
> > get the same error. Any tips? The gc log shows that its running the GC
> even
> > though it has enough memory. But probably this error has something to do
> > with the memory mapped files using NIO. Anybody has seen anything like
> this
> > before? And if so, how did you solve it?
> > The output of "free -m" before starting the process:
> >
> > free -m
> > total used free shared buffers cached
> > Mem: 24158 20803 3354 0 635 19387
> > -/+ buffers/cache: 780 23378
> > Swap: 2204 164 2039
> >
> >
> > Below is the error.
> >
> > Thanks in advance.
> >
> > FM
> >
> >
> > $ java -d64 -server -Xmx15g -Xms15g -XX:+UseConcMarkSweepGC
> > -verbose:gc -XX:MaxPermSize=512m -XX:+HeapDumpOnOutOfMemoryError
> > -XX:HeapDumpPath=/home/frank/heap.dmp -jar lukeall-3.5.0.jar
> > [Full GC 207648K->28758K(15660544K), 0.1105290 secs]
> > [Full GC 61479K->15416K(15660544K), 0.0654310 secs]
> > [Full GC 69950K->15418K(15660544K), 0.0717170 secs]
> > [Full GC 69952K->15418K(15660544K), 0.0661720 secs]
> > java.io.IOException: Map failed
> > at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:849)
> > at
> org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:265)
> > at
> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216)
> > at
> org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:89)
> > at
> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:115)
> > at
> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:93)
> > at
> org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:113)
> > at
> org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:83)
> > at
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:754)
> > at
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:75)
> > at org.apache.lucene.index.IndexReader.open(IndexReader.java:462)
> > at org.apache.lucene.index.IndexReader.open(IndexReader.java:377)
> > at org.getopt.luke.Luke.openIndex(Unknown Source)
> > at org.getopt.luke.Luke.openOk(Unknown Source)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at thinlet.Thinlet.invokeImpl(Unknown Source)
> > at thinlet.Thinlet.invoke(Unknown Source)
> > at java.awt.Component.dispatchEventImpl(Component.java:4861)
> > Caused by: java.lang.OutOfMemoryError: Map failed
> > at sun.nio.ch.FileChannelImpl.map0(Native Method)
> > at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:846)
> > ... 48 more
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: Error when opening a lucene index: Map failed
Posted by Ian Lea <ia...@gmail.com>.
Try $ ulimit -v unlimited
--
Ian.
On Tue, Jan 17, 2012 at 11:05 AM, Frank Moss <fr...@gmail.com> wrote:
> Hello,
>
> I am having problems opening a lucene index. The index has been created on
> the same machine. The size of index is 44G. Its a 64bit machine running
> OpenSuse.
>
> I have tried starting the java process with 1,2,3,4 GB of memory but still
> get the same error. Any tips? The gc log shows that its running the GC even
> though it has enough memory. But probably this error has something to do
> with the memory mapped files using NIO. Anybody has seen anything like this
> before? And if so, how did you solve it?
> The output of "free -m" before starting the process:
>
> free -m
> total used free shared buffers cached
> Mem: 24158 20803 3354 0 635 19387
> -/+ buffers/cache: 780 23378
> Swap: 2204 164 2039
>
>
> Below is the error.
>
> Thanks in advance.
>
> FM
>
>
> $ java -d64 -server -Xmx15g -Xms15g -XX:+UseConcMarkSweepGC
> -verbose:gc -XX:MaxPermSize=512m -XX:+HeapDumpOnOutOfMemoryError
> -XX:HeapDumpPath=/home/frank/heap.dmp -jar lukeall-3.5.0.jar
> [Full GC 207648K->28758K(15660544K), 0.1105290 secs]
> [Full GC 61479K->15416K(15660544K), 0.0654310 secs]
> [Full GC 69950K->15418K(15660544K), 0.0717170 secs]
> [Full GC 69952K->15418K(15660544K), 0.0661720 secs]
> java.io.IOException: Map failed
> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:849)
> at org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:265)
> at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216)
> at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:89)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:115)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:93)
> at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:113)
> at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:83)
> at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:754)
> at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:75)
> at org.apache.lucene.index.IndexReader.open(IndexReader.java:462)
> at org.apache.lucene.index.IndexReader.open(IndexReader.java:377)
> at org.getopt.luke.Luke.openIndex(Unknown Source)
> at org.getopt.luke.Luke.openOk(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at thinlet.Thinlet.invokeImpl(Unknown Source)
> at thinlet.Thinlet.invoke(Unknown Source)
> at java.awt.Component.dispatchEventImpl(Component.java:4861)
> Caused by: java.lang.OutOfMemoryError: Map failed
> at sun.nio.ch.FileChannelImpl.map0(Native Method)
> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:846)
> ... 48 more
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org