You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Michael McCandless (JIRA)" <ji...@apache.org> on 2008/02/12 14:31:10 UTC

[jira] Created: (LUCENE-1177) IW.optimize() can do too many merges at the very end

IW.optimize() can do too many merges at the very end
----------------------------------------------------

                 Key: LUCENE-1177
                 URL: https://issues.apache.org/jira/browse/LUCENE-1177
             Project: Lucene - Java
          Issue Type: Bug
          Components: Index
    Affects Versions: 2.3
            Reporter: Michael McCandless
            Assignee: Michael McCandless
            Priority: Minor
             Fix For: 2.4


This was fixed on trunk in LUCENE-1044 but I'd like to separately
backport it to 2.3.

With ConcurrentMergeScheduler there is a bug, only when CFS is on,
whereby after the final merge of an optimize has finished and while
it's building its CFS, the merge policy may incorrectly ask for
another merge to collapse that segment into a compound file.  The net
effect is optimize can spend many extra iterations unecessarily
merging a single segment to collapse it to compound file.

I believe the case is rare (hard to hit), and maybe only if you have
multiple threads calling optimize at once (the TestThreadedOptimize
test can hit it), but it's a low-risk fix so I plan to commit to 2.3
shortly.



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Resolved: (LUCENE-1177) IW.optimize() can do too many merges at the very end

Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-1177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless resolved LUCENE-1177.
----------------------------------------

    Resolution: Fixed

Committed to 2.3 (and was already fixed on trunk).

> IW.optimize() can do too many merges at the very end
> ----------------------------------------------------
>
>                 Key: LUCENE-1177
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1177
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.3
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 2.4
>
>         Attachments: LUCENE-1177.patch
>
>
> This was fixed on trunk in LUCENE-1044 but I'd like to separately
> backport it to 2.3.
> With ConcurrentMergeScheduler there is a bug, only when CFS is on,
> whereby after the final merge of an optimize has finished and while
> it's building its CFS, the merge policy may incorrectly ask for
> another merge to collapse that segment into a compound file.  The net
> effect is optimize can spend many extra iterations unecessarily
> merging a single segment to collapse it to compound file.
> I believe the case is rare (hard to hit), and maybe only if you have
> multiple threads calling optimize at once (the TestThreadedOptimize
> test can hit it), but it's a low-risk fix so I plan to commit to 2.3
> shortly.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Updated: (LUCENE-1177) IW.optimize() can do too many merges at the very end

Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-1177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless updated LUCENE-1177:
---------------------------------------

    Attachment: LUCENE-1177.patch

Attached patch.  Will commit shortly to 2.3.

> IW.optimize() can do too many merges at the very end
> ----------------------------------------------------
>
>                 Key: LUCENE-1177
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1177
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.3
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 2.4
>
>         Attachments: LUCENE-1177.patch
>
>
> This was fixed on trunk in LUCENE-1044 but I'd like to separately
> backport it to 2.3.
> With ConcurrentMergeScheduler there is a bug, only when CFS is on,
> whereby after the final merge of an optimize has finished and while
> it's building its CFS, the merge policy may incorrectly ask for
> another merge to collapse that segment into a compound file.  The net
> effect is optimize can spend many extra iterations unecessarily
> merging a single segment to collapse it to compound file.
> I believe the case is rare (hard to hit), and maybe only if you have
> multiple threads calling optimize at once (the TestThreadedOptimize
> test can hit it), but it's a low-risk fix so I plan to commit to 2.3
> shortly.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


[jira] Updated: (LUCENE-1177) IW.optimize() can do too many merges at the very end

Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-1177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hoss Man updated LUCENE-1177:
-----------------------------

    Fix Version/s: 2.3.1

> IW.optimize() can do too many merges at the very end
> ----------------------------------------------------
>
>                 Key: LUCENE-1177
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1177
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.3
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 2.3.1, 2.4
>
>         Attachments: LUCENE-1177.patch
>
>
> This was fixed on trunk in LUCENE-1044 but I'd like to separately
> backport it to 2.3.
> With ConcurrentMergeScheduler there is a bug, only when CFS is on,
> whereby after the final merge of an optimize has finished and while
> it's building its CFS, the merge policy may incorrectly ask for
> another merge to collapse that segment into a compound file.  The net
> effect is optimize can spend many extra iterations unecessarily
> merging a single segment to collapse it to compound file.
> I believe the case is rare (hard to hit), and maybe only if you have
> multiple threads calling optimize at once (the TestThreadedOptimize
> test can hit it), but it's a low-risk fix so I plan to commit to 2.3
> shortly.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org