You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "imzs (via GitHub)" <gi...@apache.org> on 2026/09/08 08:00:39 UTC
[I] [Enhancement] Support suppressing minimal-value metrics to reduce export payload [rocketmq]
imzs opened a new issue, #11078:
URL: https://github.com/apache/rocketmq/issues/11078
### Before Creating the Enhancement Request
- [x] I have confirmed that this should be classified as an enhancement rather than a bug/feature.
### Summary
This PR introduces an opt-in broker config `suppressMinValueMetrics` (default `false`, backward compatible) that suppresses exporting data points whose value is not greater than a
minimal threshold.
### Motivation
The consumer lag gauges are asynchronous ObservableGauges that emit one data point per consumer_group × topic × is_retry combination. In deployments with many consumer groups and topics, the vast majority of these combinations are idle at any given sampling instant, so their values are 0 — yet they are still recorded and exported on every collection cycle. This inflates both metric cardinality and the OTLP export payload.
In production, we observed zero-valued points accounting for ~95% of gauge data points and the bulk of the payload. Suppressing them is safe as long as the monitoring pipeline tolerates discontinuous series.
### Describe the Solution You'd Like
- `BrokerConfig`: add `suppressMinValueMetrics` (default `false`).
- `BrokerMetricsManager`: add a general guard `shouldRecordValue(currentValue, minValue)`.
When suppression is enabled, a data point is recorded only if `currentValue > minValue`.
- Apply the guard at the record site of the 5 consumer lag gauges, all passing `minValue = 0` for now, i.e. zero-valued points are dropped.
### Describe Alternatives You've Considered
.
### Additional Context
When enabled, threshold-based dashboards/alerts (e.g. `lag > N`) are unaffected; the only difference is
that idle series appear as gaps instead of a flat zero line. Therefore, before turning this on, make sure
there are **no alert rules based on `absent()` / `absent_over_time()`** (or any equivalent "no data" detection) targeting these metrics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Enhancement] Support suppressing minimal-value metrics to reduce export payload [rocketmq]
Posted by "imzs (via GitHub)" <gi...@apache.org>.
imzs commented on issue #11078:
URL: https://github.com/apache/rocketmq/issues/11078#issuecomment-5581795609
> **Issue Evaluation**
>
> Category: `bug` | Status: **Under Review**
>
> Thank you for reporting this issue. It has been classified as a potential bug and flagged for code-level verification.
>
> **Next steps:**
>
> * Code-level analysis will be performed to verify reproducibility
>
> * If confirmed, a fix proposal may be generated automatically
>
>
> If you have additional context (version, reproduction steps, logs), please share it in the comments.
>
> _Automated evaluation by RockteMQ-AI_
One correction on the category: this is not a bug — the current behavior is correct and intentional. Async ObservableGauges are expected to report the full set of series on every collection cycle, including zero values, and no invariant is violated.
What we're asking for is an enhancement: an opt-in way to suppress zero-valued data points on the consumer lag gauges, to reduce metric cardinality and export payload.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Enhancement] Support suppressing minimal-value metrics to reduce export payload [rocketmq]
Posted by "RockteMQ-AI (via GitHub)" <gi...@apache.org>.
RockteMQ-AI commented on issue #11078:
URL: https://github.com/apache/rocketmq/issues/11078#issuecomment-5711089183
**Issue Evaluation**
Category: `enhancement` | Status: **Evaluated**
This is a reasonable enhancement request to support suppressing minimal-value metrics to reduce export payload.
**Feasibility:** Feasible - can be implemented as a configurable threshold filter in the metrics export pipeline.
**Scope:** Metrics collection and export subsystem. Affects all metric exporters (Prometheus, OpenTelemetry, etc.).
**Compatibility:** No breaking changes expected. Should be opt-in via configuration to preserve existing behavior.
**Assessment:** This is a valuable optimization for large-scale deployments where metrics cardinality can impact monitoring system performance. The implementation should:
1. Add a configurable threshold (e.g., `metrics.min.value.threshold`)
2. Apply filtering at the export stage, not collection stage
3. Preserve all metrics when threshold is not configured (default behavior)
This would benefit operations teams running large RocketMQ clusters with high metric cardinality.
---
*Automated evaluation by RockteMQ-AI*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Enhancement] Support suppressing minimal-value metrics to reduce export payload [rocketmq]
Posted by "RockteMQ-AI (via GitHub)" <gi...@apache.org>.
RockteMQ-AI commented on issue #11078:
URL: https://github.com/apache/rocketmq/issues/11078#issuecomment-5655077471
**Issue Evaluation**
Category: `type/enhancement` | Status: **Evaluated**
**Feasibility:** Feasible
**Scope:** `broker` metrics — `BrokerMetricsManager`, consumer lag gauges
**Compatibility:** Backward compatible (opt-in, default `false`)
The enhancement addresses a real production pain point: zero-valued gauge data points dominating OTLP export payload in large deployments. The proposed approach is clean:
- Opt-in config with safe default (`false`) ensures no behavior change for existing users.
- The `shouldRecordValue` guard is general and can be extended to other metrics later.
- The caveat about `absent()` / `absent_over_time()` alerts is important and well-documented.
One suggestion: consider adding a counter metric for suppressed data points, so operators can monitor how much payload reduction they're getting.
---
*Automated evaluation by github-manager*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Enhancement] Support suppressing minimal-value metrics to reduce export payload [rocketmq]
Posted by "RockteMQ-AI (via GitHub)" <gi...@apache.org>.
RockteMQ-AI commented on issue #11078:
URL: https://github.com/apache/rocketmq/issues/11078#issuecomment-5581469272
**Issue Evaluation**
Category: `bug` | Status: **Under Review**
Thank you for reporting this issue. It has been classified as a potential bug and flagged for code-level verification.
**Next steps:**
- Code-level analysis will be performed to verify reproducibility
- If confirmed, a fix proposal may be generated automatically
If you have additional context (version, reproduction steps, logs), please share it in the comments.
---
*Automated evaluation by RockteMQ-AI*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org