You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "lhotari (via GitHub)" <gi...@apache.org> on 2024/08/26 10:49:07 UTC
[PR] [improve][broker] Add msgInReplay subscription stat and metric to improve Key_Shared observability [pulsar]
lhotari opened a new pull request, #23224:
URL: https://github.com/apache/pulsar/pull/23224
Main Issue: #23205
### Motivation
Currently there aren't ways to observe the number of messages that are "in replay" with Key_Shared or Shared subscriptions. Having these metrics would make it easier to understand the behavior of issues such as #23200 .
Please check #23205 for more reasons.
### Modifications
- add `msgInReplay` in the similar way as the existing `msgDelayed` subscription stat and metric.
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [ ] `doc` <!-- Your PR contains doc changes. -->
- [x] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [improve][broker] Add msgInReplay subscription stat and metric to improve Key_Shared observability [pulsar]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on PR #23224:
URL: https://github.com/apache/pulsar/pull/23224#issuecomment-2316674673
> +1 for having a PIP for this change since it actually changed the Public API output and metrics.
>
> Another point is can we only have new the field in the topic stats? Since we have a lot of metrics on the topic level. Finally, we can't add everything to the prometheus metrics. The stats, the logs are also the part of observability.
@codelipenghui This is not a breaking change in metrics and this change isn't adding significantly more metrics than what currently exists. It's currently a gap in observability of all released versions of Pulsar when msgInReplay is missing. That's why everyone needs this and making it configurable with a feature flag wouldn't be sensible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [improve][broker] Add msgInReplay subscription stat and metric to improve Key_Shared observability [pulsar]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari merged PR #23224:
URL: https://github.com/apache/pulsar/pull/23224
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [improve][broker] Add msgInReplay subscription stat and metric to improve Key_Shared observability [pulsar]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #23224:
URL: https://github.com/apache/pulsar/pull/23224#issuecomment-2310738990
## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/23224?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 74.54%. Comparing base [(`bbc6224`)](https://app.codecov.io/gh/apache/pulsar/commit/bbc62245c5ddba1de4b1e7cee4ab49334bc36277?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`d6d08be`)](https://app.codecov.io/gh/apache/pulsar/commit/d6d08bed0e0a21910133e0b79a2bb69a981ba9db?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 545 commits behind head on master.
<details><summary>Additional details and impacted files</summary>
[](https://app.codecov.io/gh/apache/pulsar/pull/23224?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
```diff
@@ Coverage Diff @@
## master #23224 +/- ##
============================================
+ Coverage 73.57% 74.54% +0.96%
- Complexity 32624 34226 +1602
============================================
Files 1877 1922 +45
Lines 139502 144768 +5266
Branches 15299 15832 +533
============================================
+ Hits 102638 107916 +5278
+ Misses 28908 28578 -330
- Partials 7956 8274 +318
```
| [Flag](https://app.codecov.io/gh/apache/pulsar/pull/23224/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [inttests](https://app.codecov.io/gh/apache/pulsar/pull/23224/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `27.66% <28.57%> (+3.08%)` | :arrow_up: |
| [systests](https://app.codecov.io/gh/apache/pulsar/pull/23224/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `24.69% <28.57%> (+0.36%)` | :arrow_up: |
| [unittests](https://app.codecov.io/gh/apache/pulsar/pull/23224/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `73.90% <100.00%> (+1.05%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files](https://app.codecov.io/gh/apache/pulsar/pull/23224?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...ervice/persistent/MessageRedeliveryController.java](https://app.codecov.io/gh/apache/pulsar/pull/23224?src=pr&el=tree&filepath=pulsar-broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpulsar%2Fbroker%2Fservice%2Fpersistent%2FMessageRedeliveryController.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL3BlcnNpc3RlbnQvTWVzc2FnZVJlZGVsaXZlcnlDb250cm9sbGVyLmphdmE=) | `95.38% <100.00%> (+0.38%)` | :arrow_up: |
| [...sistent/PersistentDispatcherMultipleConsumers.java](https://app.codecov.io/gh/apache/pulsar/pull/23224?src=pr&el=tree&filepath=pulsar-broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpulsar%2Fbroker%2Fservice%2Fpersistent%2FPersistentDispatcherMultipleConsumers.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL3BlcnNpc3RlbnQvUGVyc2lzdGVudERpc3BhdGNoZXJNdWx0aXBsZUNvbnN1bWVycy5qYXZh) | `73.37% <100.00%> (-0.96%)` | :arrow_down: |
| [...ker/service/persistent/PersistentSubscription.java](https://app.codecov.io/gh/apache/pulsar/pull/23224?src=pr&el=tree&filepath=pulsar-broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpulsar%2Fbroker%2Fservice%2Fpersistent%2FPersistentSubscription.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL3BlcnNpc3RlbnQvUGVyc2lzdGVudFN1YnNjcmlwdGlvbi5qYXZh) | `77.54% <100.00%> (+0.85%)` | :arrow_up: |
| [...ker/stats/prometheus/AggregatedNamespaceStats.java](https://app.codecov.io/gh/apache/pulsar/pull/23224?src=pr&el=tree&filepath=pulsar-broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpulsar%2Fbroker%2Fstats%2Fprometheus%2FAggregatedNamespaceStats.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zdGF0cy9wcm9tZXRoZXVzL0FnZ3JlZ2F0ZWROYW1lc3BhY2VTdGF0cy5qYXZh) | `100.00% <100.00%> (ø)` | |
| [.../stats/prometheus/AggregatedSubscriptionStats.java](https://app.codecov.io/gh/apache/pulsar/pull/23224?src=pr&el=tree&filepath=pulsar-broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpulsar%2Fbroker%2Fstats%2Fprometheus%2FAggregatedSubscriptionStats.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zdGF0cy9wcm9tZXRoZXVzL0FnZ3JlZ2F0ZWRTdWJzY3JpcHRpb25TdGF0cy5qYXZh) | `100.00% <ø> (ø)` | |
| [...ker/stats/prometheus/NamespaceStatsAggregator.java](https://app.codecov.io/gh/apache/pulsar/pull/23224?src=pr&el=tree&filepath=pulsar-broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpulsar%2Fbroker%2Fstats%2Fprometheus%2FNamespaceStatsAggregator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zdGF0cy9wcm9tZXRoZXVzL05hbWVzcGFjZVN0YXRzQWdncmVnYXRvci5qYXZh) | `94.24% <100.00%> (+3.83%)` | :arrow_up: |
| [...che/pulsar/broker/stats/prometheus/TopicStats.java](https://app.codecov.io/gh/apache/pulsar/pull/23224?src=pr&el=tree&filepath=pulsar-broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpulsar%2Fbroker%2Fstats%2Fprometheus%2FTopicStats.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zdGF0cy9wcm9tZXRoZXVzL1RvcGljU3RhdHMuamF2YQ==) | `98.42% <100.00%> (+4.22%)` | :arrow_up: |
| [...mon/policies/data/stats/SubscriptionStatsImpl.java](https://app.codecov.io/gh/apache/pulsar/pull/23224?src=pr&el=tree&filepath=pulsar-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpulsar%2Fcommon%2Fpolicies%2Fdata%2Fstats%2FSubscriptionStatsImpl.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cHVsc2FyLWNvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NvbW1vbi9wb2xpY2llcy9kYXRhL3N0YXRzL1N1YnNjcmlwdGlvblN0YXRzSW1wbC5qYXZh) | `94.40% <100.00%> (+0.28%)` | :arrow_up: |
... and [534 files with indirect coverage changes](https://app.codecov.io/gh/apache/pulsar/pull/23224/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [improve][broker] Add msgInReplay subscription stat and metric to improve Key_Shared observability [pulsar]
Posted by "dao-jun (via GitHub)" <gi...@apache.org>.
dao-jun commented on PR #23224:
URL: https://github.com/apache/pulsar/pull/23224#issuecomment-2311412560
LGTM, but maybe the change needs a PIP?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org