You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Eric Pugh <ep...@opensourceconnections.com> on 2010/08/18 22:46:41 UTC
Does expungeDeletes need calling during an optimize?
Hi all,
I was digging into what exactly expungeDeletes means, and what it would during during an <optimize/> call. From my reading of the docs: http://lucene.apache.org/java/2_9_3/api/core/org/apache/lucene/index/IndexWriter.html#expungeDeletes() we should either do an optimize or an expungeDeletes, but don't need to do both. While DUH has this behavior:
if(cmd.optimize) writer.optimize(cmd.maxOptimizeSegments);
if(cmd.expungeDeletes) writer.expungeDeletes(cmd.expungeDeletes);
It seems like DUH2 behaves the way I expected:
if (cmd.optimize) {
openWriter();
writer.optimize(cmd.maxOptimizeSegments);
} else if (cmd.expungeDeletes) {
openWriter();
writer.expungeDeletes();
}
So would it make sense to update the wiki page to say the expungeDeletes only makes sense as a commit parameter, not an optimize parameter?
Eric
-----------------------------------------------------
Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com
Co-Author: Solr 1.4 Enterprise Search Server available from http://www.packtpub.com/solr-1-4-enterprise-search-server
Free/Busy: http://tinyurl.com/eric-cal
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Does expungeDeletes need calling during an optimize?
Posted by Eric Pugh <ep...@opensourceconnections.com>.
It is deprecated!
Mostly I was trying to better understand the specifics of what "expungeDeletes" means, and so was looking at the codebase. In terms of the actual impact of doing expungeDeletes, it seems like it's a "mini" optimize that you would use in an environment where you are indexing and deleting documents rapidly, and don't want file size to grow too quickly, yet can't afford the cost of frequent optimize operations?
Eric
On Aug 19, 2010, at 1:17 AM, Lance Norskog wrote:
> Isn't DUH deprecated by now? Should anyone use it? Does it still work?
>
> On Wed, Aug 18, 2010 at 2:15 PM, Eric Pugh
> <ep...@opensourceconnections.com> wrote:
>> Okay, I tried to clarify wiki page.
>>
>> Eric
>>
>> On Aug 18, 2010, at 4:56 PM, Yonik Seeley wrote:
>>
>>> On Wed, Aug 18, 2010 at 4:46 PM, Eric Pugh
>>> <ep...@opensourceconnections.com> wrote:
>>>> So would it make sense to update the wiki page to say the expungeDeletes only makes sense as a commit parameter, not an optimize parameter?
>>>
>>> Perhaps a better way of putting it is that expungeDeletes on an
>>> optimize call is redundant - an optimize always expunges deletes.
>>>
>>> -Yonik
>>> http://www.lucidimagination.com
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>
>> -----------------------------------------------------
>> Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com
>> Co-Author: Solr 1.4 Enterprise Search Server available from http://www.packtpub.com/solr-1-4-enterprise-search-server
>> Free/Busy: http://tinyurl.com/eric-cal
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>
>
>
> --
> Lance Norskog
> goksron@gmail.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
-----------------------------------------------------
Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com
Co-Author: Solr 1.4 Enterprise Search Server available from http://www.packtpub.com/solr-1-4-enterprise-search-server
Free/Busy: http://tinyurl.com/eric-cal
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Does expungeDeletes need calling during an optimize?
Posted by Lance Norskog <go...@gmail.com>.
Isn't DUH deprecated by now? Should anyone use it? Does it still work?
On Wed, Aug 18, 2010 at 2:15 PM, Eric Pugh
<ep...@opensourceconnections.com> wrote:
> Okay, I tried to clarify wiki page.
>
> Eric
>
> On Aug 18, 2010, at 4:56 PM, Yonik Seeley wrote:
>
>> On Wed, Aug 18, 2010 at 4:46 PM, Eric Pugh
>> <ep...@opensourceconnections.com> wrote:
>>> So would it make sense to update the wiki page to say the expungeDeletes only makes sense as a commit parameter, not an optimize parameter?
>>
>> Perhaps a better way of putting it is that expungeDeletes on an
>> optimize call is redundant - an optimize always expunges deletes.
>>
>> -Yonik
>> http://www.lucidimagination.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> -----------------------------------------------------
> Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com
> Co-Author: Solr 1.4 Enterprise Search Server available from http://www.packtpub.com/solr-1-4-enterprise-search-server
> Free/Busy: http://tinyurl.com/eric-cal
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
--
Lance Norskog
goksron@gmail.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Does expungeDeletes need calling during an optimize?
Posted by Eric Pugh <ep...@opensourceconnections.com>.
Okay, I tried to clarify wiki page.
Eric
On Aug 18, 2010, at 4:56 PM, Yonik Seeley wrote:
> On Wed, Aug 18, 2010 at 4:46 PM, Eric Pugh
> <ep...@opensourceconnections.com> wrote:
>> So would it make sense to update the wiki page to say the expungeDeletes only makes sense as a commit parameter, not an optimize parameter?
>
> Perhaps a better way of putting it is that expungeDeletes on an
> optimize call is redundant - an optimize always expunges deletes.
>
> -Yonik
> http://www.lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
-----------------------------------------------------
Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com
Co-Author: Solr 1.4 Enterprise Search Server available from http://www.packtpub.com/solr-1-4-enterprise-search-server
Free/Busy: http://tinyurl.com/eric-cal
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Does expungeDeletes need calling during an optimize?
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Wed, Aug 18, 2010 at 4:46 PM, Eric Pugh
<ep...@opensourceconnections.com> wrote:
> So would it make sense to update the wiki page to say the expungeDeletes only makes sense as a commit parameter, not an optimize parameter?
Perhaps a better way of putting it is that expungeDeletes on an
optimize call is redundant - an optimize always expunges deletes.
-Yonik
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org