Not enough space

[Xuesong Luo <xl...@successfactors.com>]

I set up solr1.2 to run snapshooter each time after a commit/optimize.
It worked fine for a while, but later I got the error message below
after sending the commit request. It seems jboss(4.0.GA) had problem
running snapshooter. The index size is 290m, the file system that solr
data directory is on has 2g free space. The swap space(/tmp) has 420m
free space. Both seem have enough space. I also tried running
snapshooter from command line and is able to create the snapshot without
causing any problem.

 

 

2007-07-03 14:49:20,923 ERROR [STDERR] Jul 3, 2007 2:49:20 PM
org.apache.solr.core.SolrException log

SEVERE: java.io.IOException: Not enough space

        at java.lang.UNIXProcess.forkAndExec(Native Method)

        at java.lang.UNIXProcess.<init>(UNIXProcess.java:53)

        at java.lang.ProcessImpl.start(ProcessImpl.java:65)

        at java.lang.ProcessBuilder.start(ProcessBuilder.java:451)

        at java.lang.Runtime.exec(Runtime.java:591)

        at
org.apache.solr.core.RunExecutableListener.exec(RunExecutableListener

.java:70)

        at
org.apache.solr.core.RunExecutableListener.postCommit(RunExecutableLi

stener.java:97)

        at
org.apache.solr.update.UpdateHandler.callPostCommitCallbacks(UpdateHa

ndler.java:99)

        at
org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandle

r2.java:514)

        at
org.apache.solr.handler.XmlUpdateRequestHandler.update(XmlUpdateReque

stHandler.java:214) 

        at
org.apache.solr.handler.XmlUpdateRequestHandler.doLegacyUpdate(XmlUpd

ateRequestHandler.java:355)

        at
org.apache.solr.servlet.SolrUpdateServlet.doPost(SolrUpdateServlet.ja

va:58)

        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)

        at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)

Re: Not enough space

[Yonik Seeley <yo...@apache.org>]

On 7/4/07, Chris Hostetter <ho...@fucit.org> wrote:
>
> : I set up solr1.2 to run snapshooter each time after a commit/optimize.
> : It worked fine for a while, but later I got the error message below
> : after sending the commit request. It seems jboss(4.0.GA) had problem
> : running snapshooter. The index size is 290m, the file system that solr
> : data directory is on has 2g free space. The swap space(/tmp) has 420m
>
> The message from the native code used by your JVM to do the forkAndExec
> seems to be a little missleading, googling for "IOException: Not enough
> space forkAndExec" turned up a bunch of examples where people have seem to
> have run into similar problems and the root cause is not enough swap.
>
> You may have 420MB of swap available, and your index size may only be
> 290MB but the size of your index isn't the issue -- what matters is that
> in order to to exec shapshooter, the JVM has to fork first, which causes
> your OS to make a copy of all the memory allocated to your JVM -- so
> whatever your heap size is, unless you have that much free mem at the time
> the forkAndExec call is made, it will try to use swap, if that much swap
> isn't available, it will fail with this error.

Right... except it's not as expensive as it sounds.  OS's like Linux
don't actually copy all the memory, they implement copy-on-write so
all that physical ram is not used (except for the extra page tables)
and swap is not actually used.  But because at the time of fork() the
kernel doesn't know if it will actually be followed by exec(), it
checks if enough memory+swap *would* be available if needed (to avoid
memory overcommit).

-Yonik

Re: Not enough space

[Chris Hostetter <ho...@fucit.org>]

: I set up solr1.2 to run snapshooter each time after a commit/optimize.
: It worked fine for a while, but later I got the error message below
: after sending the commit request. It seems jboss(4.0.GA) had problem
: running snapshooter. The index size is 290m, the file system that solr
: data directory is on has 2g free space. The swap space(/tmp) has 420m

The message from the native code used by your JVM to do the forkAndExec
seems to be a little missleading, googling for "IOException: Not enough
space forkAndExec" turned up a bunch of examples where people have seem to
have run into similar problems and the root cause is not enough swap.

You may have 420MB of swap available, and your index size may only be
290MB but the size of your index isn't the issue -- what matters is that
in order to to exec shapshooter, the JVM has to fork first, which causes
your OS to make a copy of all the memory allocated to your JVM -- so
whatever your heap size is, unless you have that much free mem at the time
the forkAndExec call is made, it will try to use swap, if that much swap
isn't available, it will fail with this error.




-Hoss

Re: Not enough space

[Brian Carmalt <bc...@contact.de>]

Search with Google for swap file linux linux or "distro name"

There is tons of info out there.

 
Am Donnerstag, den 25.09.2008, 02:07 -0700 schrieb sunnyfr:
> Hi,
> I've obviously the same error, I just don't know how do you add swap space ? 
> Thanks a lot,
> 
> 
> Yonik Seeley wrote:
> > 
> > On 7/5/07, Xuesong Luo <xl...@successfactors.com> wrote:
> >> Thanks, Chris and Yonik. You are right. I remember the heap size was
> >> over 500m when I got the Not enough space error message.
> >> Is there a best practice to avoid this kind of problem?
> > 
> > add more swap space.
> > 
> > -Yonik
> > 
> > 
> 

Re: Not enough space

[sunnyfr <jo...@gmail.com>]

Hi,
I've obviously the same error, I just don't know how do you add swap space ? 
Thanks a lot,


Yonik Seeley wrote:
> 
> On 7/5/07, Xuesong Luo <xl...@successfactors.com> wrote:
>> Thanks, Chris and Yonik. You are right. I remember the heap size was
>> over 500m when I got the Not enough space error message.
>> Is there a best practice to avoid this kind of problem?
> 
> add more swap space.
> 
> -Yonik
> 
> 

-- 
View this message in context: http://www.nabble.com/Not-enough-space-tp11423199p19665707.html
Sent from the Solr - User mailing list archive at Nabble.com.