You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Siraj Haider <si...@jobdiva.com> on 2009/11/30 23:50:42 UTC

Disk full while optimizing an index

Index optimization fails if we don't have enough space on the drive and 
leaves the hard drive almost full.  Is there a way not to even start 
optimization if we don't have enough space on drive?

regards
-siraj

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


RE: Disk full while optimizing an index

Posted by Uwe Schindler <uw...@thetaphi.de>.
And if you have open IndexReaders/Searchers at the same time use 3.5 as
factor (because some files were already deleted from directory, but still
occupy space - *nix delete on last close) :-)

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Jason Rutherglen [mailto:jason.rutherglen@gmail.com]
> Sent: Monday, November 30, 2009 11:55 PM
> To: java-user@lucene.apache.org
> Subject: Re: Disk full while optimizing an index
> 
> Siraj,
> 
> You could estimate the maximum size used during optimization at 2.5 (a
> sort of rough maximum) times your current index size, and not optimize
> if your index (at 2.5 times) would exceed your allowable disk space.
> 
> Jason
> 
> On Mon, Nov 30, 2009 at 2:50 PM, Siraj Haider <si...@jobdiva.com> wrote:
> > Index optimization fails if we don't have enough space on the drive and
> > leaves the hard drive almost full.  Is there a way not to even start
> > optimization if we don't have enough space on drive?
> >
> > regards
> > -siraj
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


RE: Disk full while optimizing an index

Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi siraj,

There is now way to find out the free space on a partition using Java 5
(Lucene 3.0) or Java 1.4 (Lucene 2.9) without any native JNI calls. So
Lucene cannot calculate it before optimizing.

With Java 6 it would be possible, but Lucene 3.0 is only allowed to use Java
5: File#getUsableSpace()
A workaround would be to only check it conditionally using reflection on
File.class.

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Siraj Haider [mailto:siraj@jobdiva.com]
> Sent: Monday, November 30, 2009 11:58 PM
> To: java-user@lucene.apache.org
> Subject: Re: Disk full while optimizing an index
> 
> Jason,
> Thank you for your suggestion.  That is what I am planning to do, but I
> overheard or read somewhere that the new lucene version can take care of
> that internally, so I was just trying to see if somebody know something
> about it.
> 
> regards
> -siraj
> 
> Jason Rutherglen wrote:
> > Siraj,
> >
> > You could estimate the maximum size used during optimization at 2.5 (a
> > sort of rough maximum) times your current index size, and not optimize
> > if your index (at 2.5 times) would exceed your allowable disk space.
> >
> > Jason
> >
> > On Mon, Nov 30, 2009 at 2:50 PM, Siraj Haider <si...@jobdiva.com> wrote:
> >
> >> Index optimization fails if we don't have enough space on the drive and
> >> leaves the hard drive almost full.  Is there a way not to even start
> >> optimization if we don't have enough space on drive?
> >>
> >> regards
> >> -siraj
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Re: Disk full while optimizing an index

Posted by Siraj Haider <si...@jobdiva.com>.
Jason,
Thank you for your suggestion.  That is what I am planning to do, but I 
overheard or read somewhere that the new lucene version can take care of 
that internally, so I was just trying to see if somebody know something 
about it.

regards
-siraj

Jason Rutherglen wrote:
> Siraj,
>
> You could estimate the maximum size used during optimization at 2.5 (a
> sort of rough maximum) times your current index size, and not optimize
> if your index (at 2.5 times) would exceed your allowable disk space.
>
> Jason
>
> On Mon, Nov 30, 2009 at 2:50 PM, Siraj Haider <si...@jobdiva.com> wrote:
>   
>> Index optimization fails if we don't have enough space on the drive and
>> leaves the hard drive almost full.  Is there a way not to even start
>> optimization if we don't have enough space on drive?
>>
>> regards
>> -siraj
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>>     
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>   


Re: Disk full while optimizing an index

Posted by Jason Rutherglen <ja...@gmail.com>.
Siraj,

You could estimate the maximum size used during optimization at 2.5 (a
sort of rough maximum) times your current index size, and not optimize
if your index (at 2.5 times) would exceed your allowable disk space.

Jason

On Mon, Nov 30, 2009 at 2:50 PM, Siraj Haider <si...@jobdiva.com> wrote:
> Index optimization fails if we don't have enough space on the drive and
> leaves the hard drive almost full.  Is there a way not to even start
> optimization if we don't have enough space on drive?
>
> regards
> -siraj
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org