You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by mi...@apache.org on 2026/01/09 20:33:14 UTC
(lucene) branch branch_10x updated: Expose MergeRateLimiter in runOnMergeFinished hook (#15507)
This is an automated email from the ASF dual-hosted git repository.
mikemccand pushed a commit to branch branch_10x
in repository https://gitbox.apache.org/repos/asf/lucene.git
The following commit(s) were added to refs/heads/branch_10x by this push:
new 1fdcbaea4ec Expose MergeRateLimiter in runOnMergeFinished hook (#15507)
1fdcbaea4ec is described below
commit 1fdcbaea4ec9697bfc5a8b3938a540b2dcc708d0
Author: Zihan Xu <zi...@andrew.cmu.edu>
AuthorDate: Fri Jan 9 11:55:20 2026 -0800
Expose MergeRateLimiter in runOnMergeFinished hook (#15507)
* Expose MergeRateLimiter in runOnMergeFinished hook
Changes runOnMergeFinished method signature in ConcurrentMergeScheduler
to provide access to OneMerge and MergeRateLimiter parameters. This
allows applications to monitor merge performance metrics like pause time,
throttling behavior, and throughput.
- Changed runOnMergeFinished from package-private to protected
- Added OneMerge merge parameter for merge context
- Added MergeRateLimiter rateLimiter parameter for metrics access
This enables custom monitoring and observability for merge operations
without requiring internal API access.
* Fomat code
* Include Javadocs for runOnMergeFinished
* Add @lucene.experimental to the API javadocs
* Add an entry to CHANGES.txt
---------
Co-authored-by: Zihan Xu <za...@amazon.com>
Co-authored-by: Michael McCandless <mi...@apache.org>
---
lucene/CHANGES.txt | 6 ++++++
.../java/org/apache/lucene/index/ConcurrentMergeScheduler.java | 10 ++++++++--
2 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/lucene/CHANGES.txt b/lucene/CHANGES.txt
index 66ec78f553d..dcfc576d19c 100644
--- a/lucene/CHANGES.txt
+++ b/lucene/CHANGES.txt
@@ -39,6 +39,12 @@ API Changes
* GITHUB#15559: Deprecate some public FuzzySet methods ahead of removing them in the future (Greg Miller)
+* GITHUB#15507: Expose MergeRateLimiter in ConcurrentMergeScheduler.runOnMergeFinished hook.
+ The runOnMergeFinished method signature now includes OneMerge and MergeRateLimiter parameters,
+ enabling applications to monitor merge performance metrics like pause time, throttling behavior,
+ and throughput. This provides visibility into Lucene's merge backpressure mechanisms that can
+ stall indexing when merge debt accumulates. (Zihan Xu)
+
New Features
---------------------
* GITHUB#15328: VectorSimilarityFunction.getValues() now implements doubleVal allowing its
diff --git a/lucene/core/src/java/org/apache/lucene/index/ConcurrentMergeScheduler.java b/lucene/core/src/java/org/apache/lucene/index/ConcurrentMergeScheduler.java
index 86a71efd8f4..3f235b93cd7 100644
--- a/lucene/core/src/java/org/apache/lucene/index/ConcurrentMergeScheduler.java
+++ b/lucene/core/src/java/org/apache/lucene/index/ConcurrentMergeScheduler.java
@@ -677,7 +677,13 @@ public class ConcurrentMergeScheduler extends MergeScheduler {
return thread;
}
- synchronized void runOnMergeFinished(MergeSource mergeSource) {
+ /**
+ * Called when a merge thread finishes.
+ *
+ * @lucene.experimental
+ */
+ protected synchronized void runOnMergeFinished(
+ MergeSource mergeSource, OneMerge merge, MergeRateLimiter rateLimiter) {
// the merge call as well as the merge thread handling in the finally
// block must be sync'd on CMS otherwise stalling decisions might cause
// us to miss pending merges
@@ -742,7 +748,7 @@ public class ConcurrentMergeScheduler extends MergeScheduler {
rateToString(rateLimiter.getMBPerSec())));
}
- runOnMergeFinished(mergeSource);
+ runOnMergeFinished(mergeSource, merge, rateLimiter);
if (verbose()) {
message(String.format(Locale.ROOT, "merge thread %s end", this.getName()));
Re: (lucene) branch branch_10x updated: Expose MergeRateLimiter in runOnMergeFinished hook (#15507)
Posted by Michael Sokolov <ms...@gmail.com>.
instructions here: https://www.apache.org/foundation/mailinglists
On Fri, Jan 9, 2026 at 6:07 PM Heitor Peralles <he...@gmail.com> wrote:
>
> How can I unsubscribe this list?
>
> On Fri, Jan 9, 2026 at 5:33 PM <mi...@apache.org> wrote:
>>
>> This is an automated email from the ASF dual-hosted git repository.
>>
>> mikemccand pushed a commit to branch branch_10x
>> in repository https://gitbox.apache.org/repos/asf/lucene.git
>>
>>
>> The following commit(s) were added to refs/heads/branch_10x by this push:
>> new 1fdcbaea4ec Expose MergeRateLimiter in runOnMergeFinished hook (#15507)
>> 1fdcbaea4ec is described below
>>
>> commit 1fdcbaea4ec9697bfc5a8b3938a540b2dcc708d0
>> Author: Zihan Xu <zi...@andrew.cmu.edu>
>> AuthorDate: Fri Jan 9 11:55:20 2026 -0800
>>
>> Expose MergeRateLimiter in runOnMergeFinished hook (#15507)
>>
>> * Expose MergeRateLimiter in runOnMergeFinished hook
>>
>> Changes runOnMergeFinished method signature in ConcurrentMergeScheduler
>> to provide access to OneMerge and MergeRateLimiter parameters. This
>> allows applications to monitor merge performance metrics like pause time,
>> throttling behavior, and throughput.
>>
>> - Changed runOnMergeFinished from package-private to protected
>> - Added OneMerge merge parameter for merge context
>> - Added MergeRateLimiter rateLimiter parameter for metrics access
>>
>> This enables custom monitoring and observability for merge operations
>> without requiring internal API access.
>>
>> * Fomat code
>>
>> * Include Javadocs for runOnMergeFinished
>>
>> * Add @lucene.experimental to the API javadocs
>>
>> * Add an entry to CHANGES.txt
>>
>> ---------
>>
>> Co-authored-by: Zihan Xu <za...@amazon.com>
>> Co-authored-by: Michael McCandless <mi...@apache.org>
>> ---
>> lucene/CHANGES.txt | 6 ++++++
>> .../java/org/apache/lucene/index/ConcurrentMergeScheduler.java | 10 ++++++++--
>> 2 files changed, 14 insertions(+), 2 deletions(-)
>>
>> diff --git a/lucene/CHANGES.txt b/lucene/CHANGES.txt
>> index 66ec78f553d..dcfc576d19c 100644
>> --- a/lucene/CHANGES.txt
>> +++ b/lucene/CHANGES.txt
>> @@ -39,6 +39,12 @@ API Changes
>>
>> * GITHUB#15559: Deprecate some public FuzzySet methods ahead of removing them in the future (Greg Miller)
>>
>> +* GITHUB#15507: Expose MergeRateLimiter in ConcurrentMergeScheduler.runOnMergeFinished hook.
>> + The runOnMergeFinished method signature now includes OneMerge and MergeRateLimiter parameters,
>> + enabling applications to monitor merge performance metrics like pause time, throttling behavior,
>> + and throughput. This provides visibility into Lucene's merge backpressure mechanisms that can
>> + stall indexing when merge debt accumulates. (Zihan Xu)
>> +
>> New Features
>> ---------------------
>> * GITHUB#15328: VectorSimilarityFunction.getValues() now implements doubleVal allowing its
>> diff --git a/lucene/core/src/java/org/apache/lucene/index/ConcurrentMergeScheduler.java b/lucene/core/src/java/org/apache/lucene/index/ConcurrentMergeScheduler.java
>> index 86a71efd8f4..3f235b93cd7 100644
>> --- a/lucene/core/src/java/org/apache/lucene/index/ConcurrentMergeScheduler.java
>> +++ b/lucene/core/src/java/org/apache/lucene/index/ConcurrentMergeScheduler.java
>> @@ -677,7 +677,13 @@ public class ConcurrentMergeScheduler extends MergeScheduler {
>> return thread;
>> }
>>
>> - synchronized void runOnMergeFinished(MergeSource mergeSource) {
>> + /**
>> + * Called when a merge thread finishes.
>> + *
>> + * @lucene.experimental
>> + */
>> + protected synchronized void runOnMergeFinished(
>> + MergeSource mergeSource, OneMerge merge, MergeRateLimiter rateLimiter) {
>> // the merge call as well as the merge thread handling in the finally
>> // block must be sync'd on CMS otherwise stalling decisions might cause
>> // us to miss pending merges
>> @@ -742,7 +748,7 @@ public class ConcurrentMergeScheduler extends MergeScheduler {
>> rateToString(rateLimiter.getMBPerSec())));
>> }
>>
>> - runOnMergeFinished(mergeSource);
>> + runOnMergeFinished(mergeSource, merge, rateLimiter);
>>
>> if (verbose()) {
>> message(String.format(Locale.ROOT, "merge thread %s end", this.getName()));
>>
>
>
> --
>
> Heitor G. Peralles
> Direct: +55 21 979 661 187
> Email: heitorgp@gmail.com
> LinkedIn: linkedin.com/in/heitorperalles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: (lucene) branch branch_10x updated: Expose MergeRateLimiter in runOnMergeFinished hook (#15507)
Posted by Heitor Peralles <he...@gmail.com>.
How can I unsubscribe this list?
On Fri, Jan 9, 2026 at 5:33 PM <mi...@apache.org> wrote:
> This is an automated email from the ASF dual-hosted git repository.
>
> mikemccand pushed a commit to branch branch_10x
> in repository https://gitbox.apache.org/repos/asf/lucene.git
>
>
> The following commit(s) were added to refs/heads/branch_10x by this push:
> new 1fdcbaea4ec Expose MergeRateLimiter in runOnMergeFinished hook
> (#15507)
> 1fdcbaea4ec is described below
>
> commit 1fdcbaea4ec9697bfc5a8b3938a540b2dcc708d0
> Author: Zihan Xu <zi...@andrew.cmu.edu>
> AuthorDate: Fri Jan 9 11:55:20 2026 -0800
>
> Expose MergeRateLimiter in runOnMergeFinished hook (#15507)
>
> * Expose MergeRateLimiter in runOnMergeFinished hook
>
> Changes runOnMergeFinished method signature in ConcurrentMergeScheduler
> to provide access to OneMerge and MergeRateLimiter parameters. This
> allows applications to monitor merge performance metrics like pause
> time,
> throttling behavior, and throughput.
>
> - Changed runOnMergeFinished from package-private to protected
> - Added OneMerge merge parameter for merge context
> - Added MergeRateLimiter rateLimiter parameter for metrics access
>
> This enables custom monitoring and observability for merge operations
> without requiring internal API access.
>
> * Fomat code
>
> * Include Javadocs for runOnMergeFinished
>
> * Add @lucene.experimental to the API javadocs
>
> * Add an entry to CHANGES.txt
>
> ---------
>
> Co-authored-by: Zihan Xu <za...@amazon.com>
> Co-authored-by: Michael McCandless <mi...@apache.org>
> ---
> lucene/CHANGES.txt | 6 ++++++
> .../java/org/apache/lucene/index/ConcurrentMergeScheduler.java | 10
> ++++++++--
> 2 files changed, 14 insertions(+), 2 deletions(-)
>
> diff --git a/lucene/CHANGES.txt b/lucene/CHANGES.txt
> index 66ec78f553d..dcfc576d19c 100644
> --- a/lucene/CHANGES.txt
> +++ b/lucene/CHANGES.txt
> @@ -39,6 +39,12 @@ API Changes
>
> * GITHUB#15559: Deprecate some public FuzzySet methods ahead of removing
> them in the future (Greg Miller)
>
> +* GITHUB#15507: Expose MergeRateLimiter in
> ConcurrentMergeScheduler.runOnMergeFinished hook.
> + The runOnMergeFinished method signature now includes OneMerge and
> MergeRateLimiter parameters,
> + enabling applications to monitor merge performance metrics like pause
> time, throttling behavior,
> + and throughput. This provides visibility into Lucene's merge
> backpressure mechanisms that can
> + stall indexing when merge debt accumulates. (Zihan Xu)
> +
> New Features
> ---------------------
> * GITHUB#15328: VectorSimilarityFunction.getValues() now implements
> doubleVal allowing its
> diff --git
> a/lucene/core/src/java/org/apache/lucene/index/ConcurrentMergeScheduler.java
> b/lucene/core/src/java/org/apache/lucene/index/ConcurrentMergeScheduler.java
> index 86a71efd8f4..3f235b93cd7 100644
> ---
> a/lucene/core/src/java/org/apache/lucene/index/ConcurrentMergeScheduler.java
> +++
> b/lucene/core/src/java/org/apache/lucene/index/ConcurrentMergeScheduler.java
> @@ -677,7 +677,13 @@ public class ConcurrentMergeScheduler extends
> MergeScheduler {
> return thread;
> }
>
> - synchronized void runOnMergeFinished(MergeSource mergeSource) {
> + /**
> + * Called when a merge thread finishes.
> + *
> + * @lucene.experimental
> + */
> + protected synchronized void runOnMergeFinished(
> + MergeSource mergeSource, OneMerge merge, MergeRateLimiter
> rateLimiter) {
> // the merge call as well as the merge thread handling in the finally
> // block must be sync'd on CMS otherwise stalling decisions might
> cause
> // us to miss pending merges
> @@ -742,7 +748,7 @@ public class ConcurrentMergeScheduler extends
> MergeScheduler {
> rateToString(rateLimiter.getMBPerSec())));
> }
>
> - runOnMergeFinished(mergeSource);
> + runOnMergeFinished(mergeSource, merge, rateLimiter);
>
> if (verbose()) {
> message(String.format(Locale.ROOT, "merge thread %s end",
> this.getName()));
>
>
--
*Heitor G. Peralles*
*Direct**:* +55 21 979 661 187 <https://wa.me/5521979661187>
*Email:* heitorgp@gmail.com
*LinkedIn:* linkedin.com/in/heitorperalles
<https://www.linkedin.com/in/heitorperalles/>