expungeDeletes vs optimize

[Bryan Bende <bb...@gmail.com>]

Does calling commit with expungeDeletes=true result in a full rewrite of
the index like an optimize does? or does it only merge away the documents
that were "deleted" by commit?

Every two weeks or so we run a process to rebuild our index from the
original documents resulting in a large amount of deleted docs still on
disk, and basically doubling the amount of disk space used by the index. We
are trying determine if it is best to just run an optimize at the end of
this process, or if there is a better solution. This is with solr 4.3.

RE: expungeDeletes vs optimize

["Petersen, Robert" <ro...@mail.rakuten.com>]

Hi Bryan,

>From what I've seen it will only get rid of the deletes in the segments that the commit merged and there will be some residual deleted docs still in the index.  It doesn't do the full rewrite.  Even if you play with merge factors etc, you'll still have lint.     In your situation I'd probably run the full optimize after your process finishes, then you'll have the smallest and fastest-to-search index possible.  

Cheers,
Robi

-----Original Message-----
From: Bryan Bende [mailto:bbende@gmail.com] 
Sent: Wednesday, February 05, 2014 6:37 AM
To: solr-user@lucene.apache.org
Subject: expungeDeletes vs optimize

Does calling commit with expungeDeletes=true result in a full rewrite of the index like an optimize does? or does it only merge away the documents that were "deleted" by commit?

Every two weeks or so we run a process to rebuild our index from the original documents resulting in a large amount of deleted docs still on disk, and basically doubling the amount of disk space used by the index. We are trying determine if it is best to just run an optimize at the end of this process, or if there is a better solution. This is with solr 4.3.