[jira] [Reopened] (LUCENE-4752) Merge segments to sort them

["Adrien Grand (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LUCENE-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adrien Grand reopened LUCENE-4752:
----------------------------------


I just found what caused the last Jenkins failures: sometimes deletions happen concurrently with a merge. In this case, deletes are still applied to the old ReaderAndLiveDocs and once the merge is finished, IndexWriter runs commitMergedDeletes to apply deletes to the new segment too, but since it assumes doc IDs are assigned sequentially, it doesn't work with SortingMergePolicy. (This explains why the bug was hard to reproduce too.)
                
> Merge segments to sort them
> ---------------------------
>
>                 Key: LUCENE-4752
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4752
>             Project: Lucene - Core
>          Issue Type: New Feature
>          Components: core/index
>            Reporter: David Smiley
>            Assignee: Adrien Grand
>         Attachments: LUCENE-4752.patch, LUCENE-4752.patch, LUCENE-4752.patch, LUCENE-4752.patch, LUCENE-4752.patch, LUCENE-4752.patch, LUCENE-4752.patch, natural_10M_ingestion.log, sorting_10M_ingestion.log
>
>
> It would be awesome if Lucene could write the documents out in a segment based on a configurable order.  This of course applies to merging segments to. The benefit is increased locality on disk of documents that are likely to be accessed together.  This often applies to documents near each other in time, but also spatially.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org