Re: Disk space used by optimize - non space in disk corrupts index.

[Ernesto De Santis <er...@colaborativa.net>]

Hi all

We have a big index and a little space in disk.
When optimize and all space is consumed, our index is corrupted.
segments file point to nonexistent files.

Enviroment:
java 1.4.2_04
W2000 SP4
Tomat 5.5.4

Bye,
Ernesto.


Yura Smolsky escribió:

>Hello, Otis.
>
>There is a big difference when you use compound index format or
>multiple files. I have tested it on the big index (45 Gb). When I used
>compound file then optimize takes 3 times more space, b/c *.cfs needs
>to be unpacked.
>
>Now I do use non compound file format. It needs like twice as much
>disk space.
>
>OG> Have you tried using the multifile index format?  Now I wonder if there
>OG> is actually a difference in disk space cosumed by optimize() when you
>OG> use multifile and compound index format...
>
>OG> Otis
>
>OG> --- "Kauler, Leto S" <le...@education.tas.gov.au> wrote:
>
>  
>
>>>Our copy of LIA is "in the mail" ;)
>>>
>>>Yes the final three files are: the .cfs (46.8MB), deletable (4
>>>bytes),
>>>and segments (29 bytes).
>>>
>>>--Leto
>>>
>>>
>>>
>>>      
>>>
>>>>-----Original Message-----
>>>>From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com] 
>>>>
>>>>Hello,
>>>>
>>>>Yes, that is how optimize works - copies all existing index 
>>>>segments into one unified index segment, thus optimizing it.
>>>>
>>>>see hit #1:
>>>>        
>>>>
>>>http://www.lucenebook.com/search?query=optimize+disk+space
>>>      
>>>
>>>>However, three times the space sounds a bit too much, or I 
>>>>make a mistake in the book. :)
>>>>
>>>>You said you end up with 3 files - .cfs is one of them, right?
>>>>
>>>>Otis
>>>>
>>>>
>>>>--- "Kauler, Leto S" <le...@education.tas.gov.au> wrote:
>>>>
>>>>        
>>>>
>>>>>Just a quick question:  after writing an index and then calling
>>>>>optimize(), is it normal for the index to expand to about 
>>>>>          
>>>>>
>>>>three times 
>>>>        
>>>>
>>>>>the size before finally compressing?
>>>>>
>>>>>In our case the optimise grinds the disk, expanding the index
>>>>>          
>>>>>
>>>into 
>>>      
>>>
>>>>>many files of about 145MB total, before compressing down to three
>>>>>          
>>>>>
>>>>>files of about 47MB total.  That must be a lot of disk activity
>>>>>          
>>>>>
>>>for 
>>>      
>>>
>>>>>the people with multi-gigabyte indexes!
>>>>>
>>>>>Regards,
>>>>>Leto
>>>>>          
>>>>>
>>>CONFIDENTIALITY NOTICE AND DISCLAIMER
>>>
>>>Information in this transmission is intended only for the person(s)
>>>to whom it is addressed and may contain privileged and/or
>>>confidential information. If you are not the intended recipient, any
>>>disclosure, copying or dissemination of the information is
>>>unauthorised and you should delete/destroy all copies and notify the
>>>sender. No liability is accepted for any unauthorised use of the
>>>information contained in this transmission.
>>>
>>>This disclaimer has been automatically added.
>>>
>>>---------------------------------------------------------------------
>>>To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
>>>For additional commands, e-mail:
>>>lucene-user-help@jakarta.apache.org
>>>
>>>
>>>      
>>>
>
>
>OG> ---------------------------------------------------------------------
>OG> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
>OG> For additional commands, e-mail:
>OG> lucene-user-help@jakarta.apache.org
>
>
>Yura Smolsky,
>
>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
>  
>


-- 
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.300 / Virus Database: 265.8.5 - Release Date: 03/02/2005


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org