You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Martijn van Groningen (JIRA)" <ji...@apache.org> on 2011/08/20 16:31:27 UTC
[jira] [Updated] (SOLR-2725) TieredMergePolicy and expungeDeletes
behaviour
[ https://issues.apache.org/jira/browse/SOLR-2725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Martijn van Groningen updated SOLR-2725:
----------------------------------------
Description:
During executing a commit with expungeDeletes I noticed there were still a lot of segments left.
However there were still ~30 segments left with deletes after the commit finished.
After looking in SolrIndexConfig class I noticed that TieredMergePolicy#setExpungeDeletesPctAllowed isn't invoked.
I think the following statements in SolrIndexConfig#buildMergePolicy method will purge all deletes:
{code}
tieredMergePolicy.setExpungeDeletesPctAllowed(0);
{code}
This also reflects the behavior of Solr 3.1 / 3.2
After some discussion on IRC setting expungeDeletesPctAllowed always to zero isn't best for performance:
http://colabti.org/irclogger/irclogger_log/lucene-dev?date=2011-08-20#l120
I think we should add an option to solrconfig.xml that allows users to set this option to whatever value is best for them:
{code:xml}
<expungeDeletesPctAllowed>0</expungeDeletesPctAllowed>
{code}
Also having a expungeDeletesPctAllowed per commit command would be great:
{code:xml}
<commit waitFlush="false" waitSearcher="false" expungeDeletes="true" expungeDeletesPctAllowed="0"/>
{code}
was:
During executing a commit with expungeDeletes I noticed there were still a lot of segments left.
My solconfig.xml (that uses TieredMergePolicy, b/c luceneMatchVersion=3.3) contained:
{code:xml}
<indexDefaults>
<mergeFactor>5</mergeFactor>
{code}
However there were still ~30 segments left with deletes after the commit finished.
After looking in SolrIndexConfig class I noticed that TieredMergePolicy#setMaxMergeAtOnceExplicit and
TieredMergePolicy#setExpungeDeletesPctAllowed aren't invoked.
I think the following statements should be added to the SolrIndexConfig#buildMergePolicy method:
{code}
tieredMergePolicy.setMaxMergeAtOnceExplicit(mergeFactor);
tieredMergePolicy.setExpungeDeletesPctAllowed(0);
{code}
I think these changes should reflect the behavior of Solr 3.1 / 3.2
> TieredMergePolicy and expungeDeletes behaviour
> ----------------------------------------------
>
> Key: SOLR-2725
> URL: https://issues.apache.org/jira/browse/SOLR-2725
> Project: Solr
> Issue Type: Bug
> Affects Versions: 3.3
> Reporter: Martijn van Groningen
> Fix For: 3.4, 4.0
>
>
> During executing a commit with expungeDeletes I noticed there were still a lot of segments left.
> However there were still ~30 segments left with deletes after the commit finished.
> After looking in SolrIndexConfig class I noticed that TieredMergePolicy#setExpungeDeletesPctAllowed isn't invoked.
> I think the following statements in SolrIndexConfig#buildMergePolicy method will purge all deletes:
> {code}
> tieredMergePolicy.setExpungeDeletesPctAllowed(0);
> {code}
> This also reflects the behavior of Solr 3.1 / 3.2
> After some discussion on IRC setting expungeDeletesPctAllowed always to zero isn't best for performance:
> http://colabti.org/irclogger/irclogger_log/lucene-dev?date=2011-08-20#l120
> I think we should add an option to solrconfig.xml that allows users to set this option to whatever value is best for them:
> {code:xml}
> <expungeDeletesPctAllowed>0</expungeDeletesPctAllowed>
> {code}
> Also having a expungeDeletesPctAllowed per commit command would be great:
> {code:xml}
> <commit waitFlush="false" waitSearcher="false" expungeDeletes="true" expungeDeletesPctAllowed="0"/>
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org