You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "tarun11Mavani (via GitHub)" <gi...@apache.org> on 2025/11/15 19:21:06 UTC
[PR] For multistream tables convert partition ID (which includes padding o… [pinot]
tarun11Mavani opened a new pull request, #17217:
URL: https://github.com/apache/pinot/pull/17217
For multi-stream tables, convert Pinot partition ID (which includes padding offset) to stream partition ID. This ensures the partition metadata stored in ZK matches what the broker's partition function computes during query pruning.
For example, stream 1 partition 5 has Pinot partition ID 10005, but should store 5 to use it during broker side pruning.
What This Fixes:
For multi-stream tables with partition-based segment pruning enabled:
Stream 1, Partition 5:
- Segment name: myTable__10005__0__timestamp
Before (Broken):
- ZK metadata stores: partitionId = 10005
- Query computes: hash("CA") % 10 = 5 - Comparison: 10005 ≠ 5
- Result: Segment INCORRECTLY PRUNED (data loss!)
After (Fixed):
- ZK metadata stores: streamPartitionId = 5
- Query computes: hash("CA") % 10 = 5 - Comparison: 5 = 5
- Result: Segment CORRECTLY INCLUDED
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] For multistream tables convert partition ID (which includes padding o… [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #17217:
URL: https://github.com/apache/pinot/pull/17217#issuecomment-3536837737
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/17217?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 55.92%. Comparing base ([`c6d4f1c`](https://app.codecov.io/gh/apache/pinot/commit/c6d4f1c5aab5bf64c4b351f479d9ae95e3483f81?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`8b357ec`](https://app.codecov.io/gh/apache/pinot/commit/8b357eccbd4b2b8101bc20426cd4e3d5b01879ef?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
> :exclamation: There is a different number of reports uploaded between BASE (c6d4f1c) and HEAD (8b357ec). Click for more details.
>
> <details><summary>HEAD has 28 uploads less than BASE</summary>
>
>| Flag | BASE (c6d4f1c) | HEAD (8b357ec) |
>|------|------|------|
>|java-21|4|1|
>|unittests1|2|1|
>|unittests|3|1|
>|temurin|8|1|
>|unittests2|1|0|
>|java-11|4|0|
>|integration|5|0|
>|integration2|1|0|
>|integration1|2|0|
>|custom-integration1|2|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #17217 +/- ##
============================================
- Coverage 63.17% 55.92% -7.25%
+ Complexity 1428 701 -727
============================================
Files 3121 2434 -687
Lines 184760 136538 -48222
Branches 28327 21813 -6514
============================================
- Hits 116723 76362 -40361
+ Misses 59021 53770 -5251
+ Partials 9016 6406 -2610
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/17217/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/17217/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/17217/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/17217/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/17217/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/17217/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/17217/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.92% <ø> (-0.02%)` | :arrow_down: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/17217/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.92% <ø> (-7.25%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/17217/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.92% <ø> (-7.25%)` | :arrow_down: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/17217/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.92% <ø> (-0.06%)` | :arrow_down: |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/17217/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/17217?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Fix Segment Partition Tracking for Multi-Topic Ingestion [pinot]
Posted by "ankitsultana (via GitHub)" <gi...@apache.org>.
ankitsultana merged PR #17217:
URL: https://github.com/apache/pinot/pull/17217
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org