You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Yasuhiro Matsuda (JIRA)" <ji...@apache.org> on 2009/05/12 18:41:46 UTC
[jira] Updated: (LUCENE-1634) LogMergePolicy should use the number
of deleted docs when deciding which segments to merge
[ https://issues.apache.org/jira/browse/LUCENE-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yasuhiro Matsuda updated LUCENE-1634:
-------------------------------------
Attachment: LUCENE-1634.patch
I posted a patch.
> LogMergePolicy should use the number of deleted docs when deciding which segments to merge
> ------------------------------------------------------------------------------------------
>
> Key: LUCENE-1634
> URL: https://issues.apache.org/jira/browse/LUCENE-1634
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Yasuhiro Matsuda
> Attachments: LUCENE-1634.patch
>
>
> I found that IndexWriter.optimize(int) method does not pick up large segments with a lot of deletes even when most of the docs are deleted. And the existence of such segments affected the query performance significantly.
> I created an index with 1 million docs, then went over all docs and updated a few thousand at a time. I ran optimize(20) occasionally. What saw were large segments with most of docs deleted. Although these segments did not have valid docs they remained in the directory for a very long time until more segments with comparable or bigger sizes were created.
> This is because LogMergePolicy.findMergeForOptimize uses the size of segments but does not take the number of deleted documents into consideration when it decides which segments to merge. So, a simple fix is to use the delete count to calibrate the segment size. I can create a patch for this.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org