You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "rajagopr (via GitHub)" <gi...@apache.org> on 2025/03/10 21:57:18 UTC
[PR] Use instance data dir path to compute disk utilization info [pinot]
rajagopr opened a new pull request, #15236:
URL: https://github.com/apache/pinot/pull/15236
Changes are to use the `instanceDataDir` path to compute the disk utilization as opposed to a default path sent from the controller. The default path sent from the controller is unused as of now. The disk utilization results are also leveraged by the enhancements being made to the `rebalancer` and as a result we can not disable the `ResourceUtilizationChecker` periodic task by default.
Running the `ResourceUtilizationChecker` periodic task by default resulted in errors for the `QuickStart` apps which relied on the default path `/home/pinot/data` which maynot be available on all deployments. The changes in this PR are to rely on the `instanceDataDir` instead.
## Testing
Testing results from an internal test cluser.
```
2025/03/10 21:41:21.240 INFO [DiskUtilizationChecker] [pool-20-thread-3] Service response: {Server_pinot-server-server-0-0.pinot-pinot-server-headless.cell-q355j0-managed.svc.cluster.local_8098={
"instanceId" : "Server_pinot-server-server-0-0.pinot-pinot-server-headless.cell-q355j0-managed.svc.cluster.local_8098",
"path" : "/home/pinot/data/index",
"totalSpaceBytes" : 539978293248,
"usedSpaceBytes" : 531307618304,
"lastUpdatedTimeInEpochMs" : 1741642881240
}}
```
Verified that the `QuickStart` apps does not throw any errors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Use instance data dir path to compute disk utilization info [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #15236:
URL: https://github.com/apache/pinot/pull/15236#issuecomment-2712003809
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15236?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 56.03%. Comparing base [(`59551e4`)](https://app.codecov.io/gh/apache/pinot/commit/59551e45224f1535c4863fd577622b37366ccc97?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`e05c115`)](https://app.codecov.io/gh/apache/pinot/commit/e05c1155610be79ad50ea2c9c1b4be73c7ab1521?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 1831 commits behind head on master.
> :exclamation: There is a different number of reports uploaded between BASE (59551e4) and HEAD (e05c115). Click for more details.
>
> <details><summary>HEAD has 48 uploads less than BASE</summary>
>
>| Flag | BASE (59551e4) | HEAD (e05c115) |
>|------|------|------|
>|integration|7|0|
>|integration2|3|0|
>|temurin|12|2|
>|java-21|7|2|
>|skip-bytebuffers-true|3|1|
>|skip-bytebuffers-false|7|1|
>|unittests|5|2|
>|java-11|5|0|
>|unittests2|3|0|
>|integration1|2|0|
>|custom-integration1|2|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #15236 +/- ##
============================================
- Coverage 61.75% 56.03% -5.72%
- Complexity 207 685 +478
============================================
Files 2436 2165 -271
Lines 133233 116809 -16424
Branches 20636 18695 -1941
============================================
- Hits 82274 65452 -16822
- Misses 44911 45979 +1068
+ Partials 6048 5378 -670
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/15236/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/15236/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/15236/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/15236/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/15236/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/15236/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/15236/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.03% <ø> (-5.60%)` | :arrow_down: |
| [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/15236/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.99% <ø> (-5.76%)` | :arrow_down: |
| [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/15236/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.99% <ø> (+28.26%)` | :arrow_up: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/15236/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.03% <ø> (-5.72%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/15236/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.03% <ø> (-5.72%)` | :arrow_down: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/15236/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.03% <ø> (+9.14%)` | :arrow_up: |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/15236/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/15236?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary>🚀 New features to boost your workflow: </summary>
- ❄ [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- 📦 [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Use instance data dir path to compute disk utilization info [pinot]
Posted by "swaminathanmanish (via GitHub)" <gi...@apache.org>.
swaminathanmanish merged PR #15236:
URL: https://github.com/apache/pinot/pull/15236
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org