You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by 王建新 <ji...@nufrontsoft.com> on 2008/04/19 11:05:39 UTC
A question about ParalellMultiSearcher and RMI
I want to use RAMDirectory to raise the peformance of lucene.
So I cut the index dir to 3 smaller index dirs(1G one index dir).
Then I use RAMDirectory to load the 3 dirs, and rebind the 3 searcher by RMI.(My server has 8G mem)
At last, I lookup the 3 remote searcher to new a ParalellMultiSearcher.
It works correctorly, but sometimes occur errors like :
java.rmi.ServerError: Error occurred in server thread; nested exception is:
java.lang.OutOfMemoryError: Java heap space
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:333)
at sun.rmi.transport.Transport$1.run(Transport.java:159)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(Unknown Source)
at sun.rmi.transport.StreamRemoteCall.executeCall(Unknown Source)
at sun.rmi.server.UnicastRef.invoke(Unknown Source)
at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(Unknown Source)
at java.rmi.server.RemoteObjectInvocationHandler.invoke(Unknown Source)
at $Proxy0.search(Unknown Source)
at org.apache.lucene.search.MultiSearcherThread.run(ParallelMultiSearcher.java:251)
Caused by: java.lang.OutOfMemoryError: Java heap space
Thanks.
Re: A question about ParalellMultiSearcher and RMI
Posted by 游泳池的鱼 <su...@gmail.com>.
In 32bit operate system JVM not support memory biger than 2g
2008/4/20, Chris <ch...@gmail.com>:
>
> You may try to use the java parameter with -xmx to set up your memory.
> If you have 8G Ram in system , but java VM only caluse the smaller usage.
>
> And I try to merage the index from disk to RAM . When Index is only about
> 1G
> size , but into the RAM .
> I try to trace the memory usage , over 1G (about 1.5~2G )...
> But if you only have 3G , I think the 8G memory that is good enough.
>
> above
> Chris.
>
> PCHOME ,Search Team @ Taiwan
>
>
> 2008/4/19, 王建新 <ji...@nufrontsoft.com>:
> >
> > I want to use RAMDirectory to raise the peformance of lucene.
> > So I cut the index dir to 3 smaller index dirs(1G one index dir).
> > Then I use RAMDirectory to load the 3 dirs, and rebind the 3 searcher by
> > RMI.(My server has 8G mem)
> > At last, I lookup the 3 remote searcher to new a ParalellMultiSearcher.
> > It works correctorly, but sometimes occur errors like :
> >
> > java.rmi.ServerError: Error occurred in server thread; nested exception
> > is:
> > java.lang.OutOfMemoryError: Java heap space
> > at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:333)
> > at sun.rmi.transport.Transport$1.run(Transport.java:159)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
> > at
> > sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
> > at
> >
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
> > at
> >
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
> > at java.lang.Thread.run(Thread.java:619)
> > at
> sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(Unknown
> > Source)
> > at sun.rmi.transport.StreamRemoteCall.executeCall(Unknown Source)
> > at sun.rmi.server.UnicastRef.invoke(Unknown Source)
> > at
> > java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(Unknown
> > Source)
> > at java.rmi.server.RemoteObjectInvocationHandler.invoke(Unknown Source)
> > at $Proxy0.search(Unknown Source)
> > at
> >
> org.apache.lucene.search.MultiSearcherThread.run(ParallelMultiSearcher.java:251)
> > Caused by: java.lang.OutOfMemoryError: Java heap space
> >
> > Thanks.
>
>
>
>
> --
> Chris Lin
> chrislin0426@gmail.com
> Taipei , Taiwan.
> -----------------------------------------------------------
>
Re: A question about ParalellMultiSearcher and RMI
Posted by Chris <ch...@gmail.com>.
You may try to use the java parameter with -xmx to set up your memory.
If you have 8G Ram in system , but java VM only caluse the smaller usage.
And I try to merage the index from disk to RAM . When Index is only about 1G
size , but into the RAM .
I try to trace the memory usage , over 1G (about 1.5~2G )...
But if you only have 3G , I think the 8G memory that is good enough.
above
Chris.
PCHOME ,Search Team @ Taiwan
2008/4/19, 王建新 <ji...@nufrontsoft.com>:
>
> I want to use RAMDirectory to raise the peformance of lucene.
> So I cut the index dir to 3 smaller index dirs(1G one index dir).
> Then I use RAMDirectory to load the 3 dirs, and rebind the 3 searcher by
> RMI.(My server has 8G mem)
> At last, I lookup the 3 remote searcher to new a ParalellMultiSearcher.
> It works correctorly, but sometimes occur errors like :
>
> java.rmi.ServerError: Error occurred in server thread; nested exception
> is:
> java.lang.OutOfMemoryError: Java heap space
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:333)
> at sun.rmi.transport.Transport$1.run(Transport.java:159)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:619)
> at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(Unknown
> Source)
> at sun.rmi.transport.StreamRemoteCall.executeCall(Unknown Source)
> at sun.rmi.server.UnicastRef.invoke(Unknown Source)
> at
> java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(Unknown
> Source)
> at java.rmi.server.RemoteObjectInvocationHandler.invoke(Unknown Source)
> at $Proxy0.search(Unknown Source)
> at
> org.apache.lucene.search.MultiSearcherThread.run(ParallelMultiSearcher.java:251)
> Caused by: java.lang.OutOfMemoryError: Java heap space
>
> Thanks.
--
Chris Lin
chrislin0426@gmail.com
Taipei , Taiwan.
-----------------------------------------------------------