You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by vo...@apache.org on 2025/08/19 13:29:38 UTC
(lucene) branch branch_10x updated (336b8692f7c -> 3ab1f03d18c)
This is an automated email from the ASF dual-hosted git repository.
voditas pushed a change to branch branch_10x
in repository https://gitbox.apache.org/repos/asf/lucene.git
from 336b8692f7c Hunspell suggestions: Cherry-pick latest changes from `main` to `branch_10x` (#15086)
new 57ae79a34b2 Improve set deletions percentage javadoc (#12828)
new 3ab1f03d18c Decrease minimum deletes percentage in TMP (#14893)
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
lucene/CHANGES.txt | 3 +++
.../org/apache/lucene/index/TieredMergePolicy.java | 18 +++++++++++++-----
2 files changed, 16 insertions(+), 5 deletions(-)
(lucene) 02/02: Decrease minimum deletes percentage in TMP (#14893)
Posted by vo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
voditas pushed a commit to branch branch_10x
in repository https://gitbox.apache.org/repos/asf/lucene.git
commit 3ab1f03d18c3c1ab39d4054673bd9a951677699a
Author: Stefan Vodita <41...@users.noreply.github.com>
AuthorDate: Tue Aug 19 14:02:22 2025 +0100
Decrease minimum deletes percentage in TMP (#14893)
---
lucene/CHANGES.txt | 3 +++
.../src/java/org/apache/lucene/index/TieredMergePolicy.java | 13 ++++++++++---
2 files changed, 13 insertions(+), 3 deletions(-)
diff --git a/lucene/CHANGES.txt b/lucene/CHANGES.txt
index 71573c5db68..cb495bf65c8 100644
--- a/lucene/CHANGES.txt
+++ b/lucene/CHANGES.txt
@@ -15,6 +15,9 @@ API Changes
* GITHUB#14426: Support determining desired off-heap memory requirements through
KnnVectorsReader::getOffHeapByteSize (Chris Hegarty)
+* GITHUB#14893: TieredMergePolicy minimum deletes percentage decreased from 5% inclusive to 0% exclusive.
+ (Stefan Vodita)
+
* GITHUB#14899: Deprecate MergeSpecification#segString(Directory) (kitoha)
* GITHUB#14978: Add a bulk scoring interface to RandomVectorScorer
diff --git a/lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java b/lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java
index accab76d699..455285482c9 100644
--- a/lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java
+++ b/lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java
@@ -159,18 +159,25 @@ public class TieredMergePolicy extends MergePolicy {
/**
* Sets the maximum percentage of doc id space taken by deleted docs. The denominator includes
* both active and deleted documents. Lower values make the index more space efficient at the
- * expense of increased CPU and I/O activity. Values must be between 5 and 50. Default value is
+ * expense of increased CPU and I/O activity. Values must be between 0 and 50. Default value is
* 20.
*
* <p>When the maximum delete percentage is lowered, the indexing thread will call for merges more
* often, meaning that write amplification factor will be increased. Write amplification factor
* measures the number of times each document in the index is written. A higher write
* amplification factor will lead to higher CPU and I/O activity as indicated above.
+ *
+ * <p>Values below 5% can lead to exceptionally high merge cost where indexing will continuously
+ * merge nearly all segments, and select newly merged segments immediately for merging again,
+ * often forcing degenerate merge selection like singleton merges. If you venture into this dark
+ * forest, consider limiting the maximum number of concurrent merges and threads (see {@link
+ * ConcurrentMergeScheduler#setMaxMergesAndThreads}) as a coarse attempt to bound the otherwise
+ * pathological indexing behavior.
*/
public TieredMergePolicy setDeletesPctAllowed(double v) {
- if (v < 5 || v > 50) {
+ if (v <= 0 || v > 50) {
throw new IllegalArgumentException(
- "indexPctDeletedTarget must be >= 5.0 and <= 50 (got " + v + ")");
+ "indexPctDeletedTarget must be > 0 and <= 50 (got " + v + ")");
}
deletesPctAllowed = v;
return this;
(lucene) 01/02: Improve set deletions percentage javadoc (#12828)
Posted by vo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
voditas pushed a commit to branch branch_10x
in repository https://gitbox.apache.org/repos/asf/lucene.git
commit 57ae79a34b2cf7ce9fc2533784406e6d83d44b31
Author: Tinker Xiao <yu...@gmail.com>
AuthorDate: Wed Jan 22 03:48:40 2025 -0800
Improve set deletions percentage javadoc (#12828)
Co-authored-by: Yuan,Xiao <yu...@amazon.com>
---
.../core/src/java/org/apache/lucene/index/TieredMergePolicy.java | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java b/lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java
index 18d59f182db..accab76d699 100644
--- a/lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java
+++ b/lucene/core/src/java/org/apache/lucene/index/TieredMergePolicy.java
@@ -157,9 +157,10 @@ public class TieredMergePolicy extends MergePolicy {
}
/**
- * Controls the maximum percentage of deleted documents that is tolerated in the index. Lower
- * values make the index more space efficient at the expense of increased CPU and I/O activity.
- * Values must be between 5 and 50. Default value is 20.
+ * Sets the maximum percentage of doc id space taken by deleted docs. The denominator includes
+ * both active and deleted documents. Lower values make the index more space efficient at the
+ * expense of increased CPU and I/O activity. Values must be between 5 and 50. Default value is
+ * 20.
*
* <p>When the maximum delete percentage is lowered, the indexing thread will call for merges more
* often, meaning that write amplification factor will be increased. Write amplification factor