You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Uwe Schindler (JIRA)" <ji...@apache.org> on 2011/05/12 09:21:47 UTC
[jira] [Issue Comment Edited] (LUCENE-1076) Allow MergePolicy to
select non-contiguous merges
[ https://issues.apache.org/jira/browse/LUCENE-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13032282#comment-13032282 ]
Uwe Schindler edited comment on LUCENE-1076 at 5/12/11 7:21 AM:
----------------------------------------------------------------
{quote}
bq. This means docIDs may be reordered, since Tiered MP can merge out-of-order segments.
I think this is a very hard break and it should depend on the Version you pass to IWC. Stuff like that is really a good usecase for Version. I had customers in the past that heavily depend on the lucene doc ID while it is not recommended but with this change their app will suddenly not work anymore. so we should make sure that they can upgrade seamlessly!
{quote}
I think we should also warn people that have this problem to use IndexUpgrader (LUCENE-3082), because it has the same problem. Segments are reordered (segments that were upgraded before a call to MP's optimize come first, then the upgraded ones). Maybe we should add this to JavaDocs in 3.x.
I'll reopen LUCENE-3082.
was (Author: thetaphi):
{quote}
bq. This means docIDs may be reordered, since Tiered MP can merge out-of-order segments.
I think this is a very hard break and it should depend on the Version you pass to IWC. Stuff like that is really a good usecase for Version. I had customers in the past that heavily depend on the lucene doc ID while it is not recommended but with this change their app will suddenly not work anymore. so we should make sure that they can upgrade seamlessly!
{quote}
I think we should also warn people that have this problem to use IndexUpgrader, because it has the same problem. Segments are reordered (segments that were upgraded before a call to MP's optimize come first, then the upgraded ones). Maybe we should add this to JavaDocs in 3.x.
I'll reopen.
> Allow MergePolicy to select non-contiguous merges
> -------------------------------------------------
>
> Key: LUCENE-1076
> URL: https://issues.apache.org/jira/browse/LUCENE-1076
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.3
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-1076-3x.patch, LUCENE-1076.patch, LUCENE-1076.patch, LUCENE-1076.patch
>
>
> I started work on this but with LUCENE-1044 I won't make much progress
> on it for a while, so I want to checkpoint my current state/patch.
> For backwards compatibility we must leave the default MergePolicy as
> selecting contiguous merges. This is necessary because some
> applications rely on "temporal monotonicity" of doc IDs, which means
> even though merges can re-number documents, the renumbering will
> always reflect the order in which the documents were added to the
> index.
> Still, for those apps that do not rely on this, we should offer a
> MergePolicy that is free to select the best merges regardless of
> whether they are continuguous. This requires fixing IndexWriter to
> accept such a merge, and, fixing LogMergePolicy to optionally allow
> it the freedom to do so.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org