You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@uniffle.apache.org by "maobaolong (via GitHub)" <gi...@apache.org> on 2024/09/13 02:33:11 UTC
[PR] [#2117] feat(server): Introduce block count in buffer pool metrics [incubator-uniffle]
maobaolong opened a new pull request, #2118:
URL: https://github.com/apache/incubator-uniffle/pull/2118
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
Examples:
- "[#123] feat(operator): support xxx"
- "[#233] fix: check null before access result in xxx"
- "[MINOR] refactor: fix typo in variable name"
- "[MINOR] docs: fix typo in README"
- "[#255] test: fix flaky test NameOfTheTest"
Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. Contributor guidelines:
https://github.com/apache/incubator-uniffle/blob/master/CONTRIBUTING.md
3. If the PR is unfinished, please mark this PR as draft.
-->
### What changes were proposed in this pull request?
Introduce block count in buffer pool metrics.
### Why are the changes needed?
(Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, describe the bug.)
Fix: #2117
### Does this PR introduce _any_ user-facing change?
Yes. `block_count_in_buffer_pool` metrics introduced.
### How was this patch tested?
By dashboard server metrics popup page.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2117] feat(server): Introduce bufferPool related metrics(block/buffer/shuffle count) [incubator-uniffle]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #2118:
URL: https://github.com/apache/incubator-uniffle/pull/2118#issuecomment-2370124365
## [Codecov](https://app.codecov.io/gh/apache/incubator-uniffle/pull/2118?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: Patch coverage is `51.72414%` with `14 lines` in your changes missing coverage. Please review.
> Project coverage is 52.37%. Comparing base [(`7e84a1a`)](https://app.codecov.io/gh/apache/incubator-uniffle/commit/7e84a1a6d826a3c705b7896fde18e340022b4bff?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`79df183`)](https://app.codecov.io/gh/apache/incubator-uniffle/commit/79df183623331e28073e54e697d3ec0b6eba089e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/incubator-uniffle/pull/2118?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...g/apache/uniffle/common/metrics/SupplierGauge.java](https://app.codecov.io/gh/apache/incubator-uniffle/pull/2118?src=pr&el=tree&filepath=common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Funiffle%2Fcommon%2Fmetrics%2FSupplierGauge.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS91bmlmZmxlL2NvbW1vbi9tZXRyaWNzL1N1cHBsaWVyR2F1Z2UuamF2YQ==) | 0.00% | [10 Missing :warning: ](https://app.codecov.io/gh/apache/incubator-uniffle/pull/2118?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [.../apache/uniffle/common/metrics/MetricsManager.java](https://app.codecov.io/gh/apache/incubator-uniffle/pull/2118?src=pr&el=tree&filepath=common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Funiffle%2Fcommon%2Fmetrics%2FMetricsManager.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS91bmlmZmxlL2NvbW1vbi9tZXRyaWNzL01ldHJpY3NNYW5hZ2VyLmphdmE=) | 0.00% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/incubator-uniffle/pull/2118?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...rg/apache/uniffle/server/ShuffleServerMetrics.java](https://app.codecov.io/gh/apache/incubator-uniffle/pull/2118?src=pr&el=tree&filepath=server%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Funiffle%2Fserver%2FShuffleServerMetrics.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS91bmlmZmxlL3NlcnZlci9TaHVmZmxlU2VydmVyTWV0cmljcy5qYXZh) | 50.00% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/incubator-uniffle/pull/2118?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #2118 +/- ##
============================================
+ Coverage 43.83% 52.37% +8.53%
- Complexity 0 2730 +2730
============================================
Files 21 444 +423
Lines 2537 20693 +18156
Branches 0 1896 +1896
============================================
+ Hits 1112 10837 +9725
- Misses 1352 9169 +7817
- Partials 73 687 +614
```
| [Flag](https://app.codecov.io/gh/apache/incubator-uniffle/pull/2118/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [](https://app.codecov.io/gh/apache/incubator-uniffle/pull/2118/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `52.37% <51.72%> (+8.53%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/incubator-uniffle/pull/2118?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2117] feat(server): Introduce block count in buffer pool metrics [incubator-uniffle]
Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong closed pull request #2118: [#2117] feat(server): Introduce block count in buffer pool metrics
URL: https://github.com/apache/incubator-uniffle/pull/2118
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2117] feat(server): Introduce bufferPool related metrics(block/buffer/shuffle count) [incubator-uniffle]
Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on PR #2118:
URL: https://github.com/apache/incubator-uniffle/pull/2118#issuecomment-2371487609
I countered the flaky test `AccessClusterTest`, same with https://github.com/apache/incubator-uniffle/actions/runs/11008932091/job/30567534037?pr=2138
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2117] feat(server): Introduce bufferPool related metrics(block/buffer/shuffle count) [incubator-uniffle]
Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston merged PR #2118:
URL: https://github.com/apache/incubator-uniffle/pull/2118
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2117] feat(server): Introduce bufferPool related metrics(block/buffer/shuffle count) [incubator-uniffle]
Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong closed pull request #2118: [#2117] feat(server): Introduce bufferPool related metrics(block/buffer/shuffle count)
URL: https://github.com/apache/incubator-uniffle/pull/2118
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2117] feat(server): Introduce bufferPool related metrics(block/buffer/shuffle count) [incubator-uniffle]
Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong closed pull request #2118: [#2117] feat(server): Introduce bufferPool related metrics(block/buffer/shuffle count)
URL: https://github.com/apache/incubator-uniffle/pull/2118
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2117] feat(server): Introduce bufferPool related metrics(block/buffer/shuffle count) [incubator-uniffle]
Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong closed pull request #2118: [#2117] feat(server): Introduce bufferPool related metrics(block/buffer/shuffle count)
URL: https://github.com/apache/incubator-uniffle/pull/2118
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2117] feat(server): Introduce block count in buffer pool metrics [incubator-uniffle]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #2118:
URL: https://github.com/apache/incubator-uniffle/pull/2118#issuecomment-2347942400
## Test Results
2 881 files +20 2 881 suites +20 5h 55m 52s :stopwatch: + 8m 41s
1 019 tests ± 0 1 017 :white_check_mark: ± 0 1 :zzz: ±0 1 :x: ±0
12 867 runs +49 12 851 :white_check_mark: +49 15 :zzz: ±0 1 :x: ±0
For more details on these failures, see [this check](https://github.com/apache/incubator-uniffle/runs/30087575481).
Results for commit a7c9335a. ± Comparison against base commit c36d8c8d.
[test-results]:data:application/gzip;base64,H4sIAESq42YC/1XMyQ7CIBSF4VdpWLsAKpMvY24ZEmJbDMPK+O7S2iIuz3dv/hdyfrYJ3QYqJbkMKBWf+21KhOzDugkZGa1U73n7IJioc95T0fpr4mcP/9ysgQM//4GNMcQquEos616lkotjtiqVjDQ7quyErrrvPqrDsvhcBwKh1TgyMGCuzsFkNQelnJkUFcoyjJ2wduIcvT8kTx5/FAEAAA==
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2117] feat(server): Introduce block count in buffer pool metrics [incubator-uniffle]
Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on PR #2118:
URL: https://github.com/apache/incubator-uniffle/pull/2118#issuecomment-2347925832
@kuszz This PR base on the `addLabeledCacheGauge` API finished by you, you can create a new PR about that code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2117] feat(server): Introduce bufferPool related metrics(block/buffer/shuffle count) [incubator-uniffle]
Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston commented on code in PR #2118:
URL: https://github.com/apache/incubator-uniffle/pull/2118#discussion_r1776474126
##########
common/src/main/java/org/apache/uniffle/common/metrics/SupplierGauge.java:
##########
@@ -36,27 +36,46 @@ class SupplierGauge<T extends Number> extends Collector implements Collector.Des
private Supplier<T> supplier;
private List<String> labelNames;
private List<String> labelValues;
+ private long updateInterval;
+ private long lastUpdateTime;
+ private T lastValue;
SupplierGauge(
Review Comment:
Oh, I like this gauge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org