You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "Vamsi-klu (via GitHub)" <gi...@apache.org> on 2026/07/25 06:18:32 UTC
[PR] Auto force-commit partial OFFLINE consuming segments [pinot]
Vamsi-klu opened a new pull request, #19083:
URL: https://github.com/apache/pinot/pull/19083
## Why
When a server hits fatal consumption errors it marks its replica **OFFLINE** in IdealState. RSVM already recreates a consuming segment when **all** replicas are OFFLINE, but **partial** OFFLINE (common after transient stream blips) was left alone. Queries then fan in to the remaining CONSUMING replicas indefinitely — imbalance and partial-result risk.
There is a separate flag that only flips OFFLINE→CONSUMING (#11314 path). This PR implements the issue’s requested approach: **partition-scoped force-commit** when enough progress is likely, behind an **opt-in** config.
## Impact
- **Heals chronic under-replication** of consuming segments without manual force-commit.
- **Rebalances** sealed data onto a new CONSUMING generation for all replicas.
- **Safe by default**: feature flag off; age gate avoids force-committing brand-new empty segments; once-per-segment tracking avoids RSVM storms; reuses `validateForceCommitAllowed` (partial upsert / drop-OOO guards).
- New meters: success / failed / skipped auto force-commit.
## How
- Config:
- `controller.realtime.segment.autoForceCommitOnPartialOfflineEnabled` (default **false**)
- `controller.realtime.segment.autoForceCommitOnPartialOfflineMinAgeMs` (default 5 minutes)
- In `ensureAllPartitionsConsuming`, when IdealState shows mixed CONSUMING+OFFLINE for an IN_PROGRESS segment and age gate passes, call existing partition-scoped `forceCommit`.
- Do not combine with OFFLINE→CONSUMING repair in the same tick when force-commit fires.
## Test plan
- [x] `PinotLLCRealtimeSegmentManagerTest`: force-commit when partial OFFLINE + age ok; age gate skip; all-OFFLINE still recreates; paused table skip; validateForceCommit failure skip.
- [ ] `./mvnw -pl pinot-controller -am -Dtest=PinotLLCRealtimeSegmentManagerTest -Dsurefire.failIfNoSpecifiedTests=false test`
- [ ] Staging: enable flag on a non-prod realtime table, mark one replica OFFLINE after the segment is aged, confirm force-commit + new CONSUMING.
## Related
fixes: #15897
## Reviewers
Suggested: sajjad-moradi (issue author), maintainers of LLC realtime
## Config / release notes
New controller configs (default off) — consider `release-notes` label.
---
##### Was generative AI tooling used to co-author this PR?
- [x] Yes — Grok Build (xAI)
Generated-by: Grok Build (xAI)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Auto force-commit partial OFFLINE consuming segments [pinot]
Posted by "Vamsi-klu (via GitHub)" <gi...@apache.org>.
Vamsi-klu commented on PR #19083:
URL: https://github.com/apache/pinot/pull/19083#issuecomment-5077582092
**Ready for review** — all required CI checks are green on this PR.
Issue: #15897
Could the following folks take a look when convenient?
@Jackie-Jiang
@sajjad-moradi
Formal GitHub "Request review" is unavailable from a fork contributor account on `apache/pinot` (API returns 404/403), so this is the review ping.
Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Auto force-commit partial OFFLINE consuming segments [pinot]
Posted by "noob-se7en (via GitHub)" <gi...@apache.org>.
noob-se7en commented on PR #19083:
URL: https://github.com/apache/pinot/pull/19083#issuecomment-5236332991
> Pushed `80cc4f3`. All of the red CI came from one root cause: the five doc blocks this PR adds used `/** */` and master now requires the `///` JEP 467 form, so checkstyle failed the `pinot-controller` build and took Linter, Quickstart and the four compat jobs down with it. Converted all five and merged latest master.
>
> @noob-se7en on your point about force-committing partial offline segments, that is a fair concern and I do not want to paper over a genuinely bad server. What is there today: the feature is off by default, the commit is partition scoped so only the partition holding the OFFLINE replica commits rather than the whole table, there is an age gate of five minutes so young segments are never touched, and a once-per-segment guard stops repeats. The case I am targeting is the one from #15897, where a transient stream blip leaves one replica OFFLINE indefinitely and the partition runs under-replicated until somebody force commits by hand.
>
> Would it address your concern if the trigger also required the partial OFFLINE state to persist across N consecutive validation runs? Or would you rather this repair used the existing OFFLINE to CONSUMING reset instead of a force commit? I am happy to go either way. One thing worth flagging whichever way we go: a controller leadership change resets the in-memory once-per-segment tracking, so a ZK visible guard would be stronger than what I have now.
We never/rarely ran into that issue after https://github.com/apache/pinot/pull/17062 was merged. I still don't think there is a strong need for this fix.
And I am also not sure if forceCommitting is the best design here, adding new code behind default off config adds more to the config burden.
If we can find out strong reason/latest production incident which re-surfaces a problem this PR addresses that will help.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Auto force-commit partial OFFLINE consuming segments [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #19083:
URL: https://github.com/apache/pinot/pull/19083#issuecomment-5077451040
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/19083?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 56.85%. Comparing base ([`2bcbdfe`](https://app.codecov.io/gh/apache/pinot/commit/2bcbdfed0af40edde0d4c12ebdff2a2463338d67?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5eb27dd`](https://app.codecov.io/gh/apache/pinot/commit/5eb27dd7af645952da924ae8db942eb9920bae84?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
> :exclamation: There is a different number of reports uploaded between BASE (2bcbdfe) and HEAD (5eb27dd). Click for more details.
>
> <details><summary>HEAD has 16 uploads less than BASE</summary>
>
>| Flag | BASE (2bcbdfe) | HEAD (5eb27dd) |
>|------|------|------|
>|unittests|2|1|
>|java-25|5|1|
>|temurin|5|1|
>|unittests2|1|0|
>|integration|3|0|
>|integration1|1|0|
>|integration2|1|0|
>|custom-integration1|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #19083 +/- ##
============================================
- Coverage 65.47% 56.85% -8.63%
+ Complexity 1421 1 -1420
============================================
Files 3426 2629 -797
Lines 217315 157634 -59681
Branches 34509 25850 -8659
============================================
- Hits 142283 89618 -52665
+ Misses 63513 60296 -3217
+ Partials 11519 7720 -3799
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/19083/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/19083/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/19083/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/19083/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/19083/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-25](https://app.codecov.io/gh/apache/pinot/pull/19083/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.85% <100.00%> (-8.63%)` | :arrow_down: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/19083/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.85% <100.00%> (-8.63%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/19083/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.85% <100.00%> (-8.62%)` | :arrow_down: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/19083/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.85% <100.00%> (+0.02%)` | :arrow_up: |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/19083/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/pinot/pull/19083?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Auto force-commit partial OFFLINE consuming segments [pinot]
Posted by "Vamsi-klu (via GitHub)" <gi...@apache.org>.
Vamsi-klu commented on PR #19083:
URL: https://github.com/apache/pinot/pull/19083#issuecomment-5642791805
Thanks @noob-se7en. I agree, and I am closing this.
There is no post-#17062 production incident to justify an auto force-commit path. #17062 already covers the transient Kafka recreate case that used to mark a replica OFFLINE. Partial OFFLINE that still happens is either a sick server (force-commit would punish the healthy replicas) or the short Helix-thread init retry, which is a server-side problem.
The safer controller repair, OFFLINE to CONSUMING for mixed IN_PROGRESS replicas, already shipped in #17754 as controller.realtime.segment.partialOfflineReplicaRepairEnabled (default off). A second default-off flag is more config than it is worth.
I am not taking the N-run gate or rewriting this PR as a reset. The reset exists. If we still want to harden first consumer create, that belongs on the server (#19370), not as RSVM force-commit.
Happy to reopen if someone has a concrete post-#17062 incident that neither #17062 nor #17754 covers.
cc @Jackie-Jiang @sajjad-moradi @swaminathanmanish
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Auto force-commit partial OFFLINE consuming segments [pinot]
Posted by "Vamsi-klu (via GitHub)" <gi...@apache.org>.
Vamsi-klu commented on PR #19083:
URL: https://github.com/apache/pinot/pull/19083#issuecomment-5642791992
Closing per the PR comment. No post-#17062 incident, and the reset already exists in #17754.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Auto force-commit partial OFFLINE consuming segments [pinot]
Posted by "Vamsi-klu (via GitHub)" <gi...@apache.org>.
Vamsi-klu commented on PR #19083:
URL: https://github.com/apache/pinot/pull/19083#issuecomment-5235777533
Pushed `80cc4f3`. All of the red CI came from one root cause: the five doc blocks this PR adds used `/** */` and master now requires the `///` JEP 467 form, so checkstyle failed the `pinot-controller` build and took Linter, Quickstart and the four compat jobs down with it. Converted all five and merged latest master.
@noob-se7en on your point about force-committing partial offline segments, that is a fair concern and I do not want to paper over a genuinely bad server. What is there today: the feature is off by default, the commit is partition scoped so only the partition holding the OFFLINE replica commits rather than the whole table, there is an age gate of five minutes so young segments are never touched, and a once-per-segment guard stops repeats. The case I am targeting is the one from #15897, where a transient stream blip leaves one replica OFFLINE indefinitely and the partition runs under-replicated until somebody force commits by hand.
Would it address your concern if the trigger also required the partial OFFLINE state to persist across N consecutive validation runs? Or would you rather this repair used the existing OFFLINE to CONSUMING reset instead of a force commit? I am happy to go either way. One thing worth flagging whichever way we go: a controller leadership change resets the in-memory once-per-segment tracking, so a ZK visible guard would be stronger than what I have now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Auto force-commit partial OFFLINE consuming segments [pinot]
Posted by "Vamsi-klu (via GitHub)" <gi...@apache.org>.
Vamsi-klu commented on code in PR #19083:
URL: https://github.com/apache/pinot/pull/19083#discussion_r3725703100
##########
pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManagerTest.java:
##########
@@ -178,6 +178,18 @@ private CommittingSegmentDescriptor createCommittingSegmentDescriptor(String seg
return createCommittingSegmentDescriptor(segmentName, NEXT_OFFSET);
}
+ /**
+ * Test cases for new table being created, and initial segments setup that follows.
+ */
+
+ /**
+ * Test cases for new table being created, and initial segments setup that follows.
+ */
+
+ /**
+ * Test cases for new table being created, and initial segments setup that follows.
+ */
+
Review Comment:
Good catch, thanks. These duplicate Javadoc blocks were accidentally introduced in this PR at all six spots (the same section header ends up repeated four times in each place). I will remove the extra copies and keep a single Javadoc per test section in the next push.
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java:
##########
@@ -2535,6 +2647,21 @@ private boolean isTmpAndCanDelete(String filePath, Set<String> downloadUrls, Pin
return getCurrentTimeMs() - lastModified > _controllerConf.getTmpSegmentRetentionInSeconds() * 1000L;
}
+ /**
+ * Reset consumption start offsets for selected partition groups (issue #6637).
+ * For each partition: OFFLINE the latest CONSUMING segment if needed, then create a new IN_PROGRESS
+ * consuming segment at the requested offset. Does not attempt to seal/commit poisoned segments.
+ *
+ * <p>Best-effort per partition: successes and failures are returned independently. Concurrent RSVM/commit
+ * races and pauseless edge cases are not fully serialized yet (IdealState is written via {@link #setIdealState},
+ * not Helix CAS — callers must avoid concurrent commits on the same partitions).
+ *
+ * @param tableNameWithType realtime table name with type
+ * @param partitionToOffsetSerialized map of partition group id → serialized {@link StreamPartitionMsgOffset}
+ * @param comment optional operator comment for logs (newlines stripped)
+ * @return per-partition result maps with keys status/oldSegment/newSegment/offset/message
+ */
+
Review Comment:
Thanks, agreed. That reset-offsets Javadoc block does not belong to any method in this file and was added by mistake right above forceCommit, which already has its own Javadoc. I will delete the stray block in the next push.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Auto force-commit partial OFFLINE consuming segments [pinot]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #19083:
URL: https://github.com/apache/pinot/pull/19083#discussion_r3669269554
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java:
##########
@@ -2535,6 +2647,21 @@ private boolean isTmpAndCanDelete(String filePath, Set<String> downloadUrls, Pin
return getCurrentTimeMs() - lastModified > _controllerConf.getTmpSegmentRetentionInSeconds() * 1000L;
}
+ /**
+ * Reset consumption start offsets for selected partition groups (issue #6637).
+ * For each partition: OFFLINE the latest CONSUMING segment if needed, then create a new IN_PROGRESS
+ * consuming segment at the requested offset. Does not attempt to seal/commit poisoned segments.
+ *
+ * <p>Best-effort per partition: successes and failures are returned independently. Concurrent RSVM/commit
+ * races and pauseless edge cases are not fully serialized yet (IdealState is written via {@link #setIdealState},
+ * not Helix CAS — callers must avoid concurrent commits on the same partitions).
+ *
+ * @param tableNameWithType realtime table name with type
+ * @param partitionToOffsetSerialized map of partition group id → serialized {@link StreamPartitionMsgOffset}
+ * @param comment optional operator comment for logs (newlines stripped)
+ * @return per-partition result maps with keys status/oldSegment/newSegment/offset/message
+ */
+
Review Comment:
This Javadoc describes the "reset consumption start offsets" API, but it is placed immediately above the force-commit API Javadoc, making the documentation misleading. Remove this stray Javadoc block (the force-commit method already has its own Javadoc).
##########
pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManagerTest.java:
##########
@@ -178,6 +178,18 @@ private CommittingSegmentDescriptor createCommittingSegmentDescriptor(String seg
return createCommittingSegmentDescriptor(segmentName, NEXT_OFFSET);
}
+ /**
+ * Test cases for new table being created, and initial segments setup that follows.
+ */
+
+ /**
+ * Test cases for new table being created, and initial segments setup that follows.
+ */
+
+ /**
+ * Test cases for new table being created, and initial segments setup that follows.
+ */
+
Review Comment:
Redundant duplicate Javadoc blocks were added; this repeats the same section header three extra times and adds noise. Remove the duplicates and keep a single Javadoc for this test section.
This issue also appears in the following locations of the same file:
- line 739
- line 892
- line 1876
- line 2015
- line 2383
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Auto force-commit partial OFFLINE consuming segments [pinot]
Posted by "Vamsi-klu (via GitHub)" <gi...@apache.org>.
Vamsi-klu closed pull request #19083: Auto force-commit partial OFFLINE consuming segments
URL: https://github.com/apache/pinot/pull/19083
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Auto force-commit partial OFFLINE consuming segments [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on PR #19083:
URL: https://github.com/apache/pinot/pull/19083#issuecomment-5109794193
cc @swaminathanmanish @noob-se7en
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Auto force-commit partial OFFLINE consuming segments [pinot]
Posted by "Vamsi-klu (via GitHub)" <gi...@apache.org>.
Vamsi-klu commented on code in PR #19083:
URL: https://github.com/apache/pinot/pull/19083#discussion_r3746565157
##########
pinot-controller/src/test/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManagerTest.java:
##########
@@ -178,6 +178,18 @@ private CommittingSegmentDescriptor createCommittingSegmentDescriptor(String seg
return createCommittingSegmentDescriptor(segmentName, NEXT_OFFSET);
}
+ /**
+ * Test cases for new table being created, and initial segments setup that follows.
+ */
+
+ /**
+ * Test cases for new table being created, and initial segments setup that follows.
+ */
+
+ /**
+ * Test cases for new table being created, and initial segments setup that follows.
+ */
+
Review Comment:
Fixed in `225357a`, the duplicated section headers are gone.
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java:
##########
@@ -2535,6 +2647,21 @@ private boolean isTmpAndCanDelete(String filePath, Set<String> downloadUrls, Pin
return getCurrentTimeMs() - lastModified > _controllerConf.getTmpSegmentRetentionInSeconds() * 1000L;
}
+ /**
+ * Reset consumption start offsets for selected partition groups (issue #6637).
+ * For each partition: OFFLINE the latest CONSUMING segment if needed, then create a new IN_PROGRESS
+ * consuming segment at the requested offset. Does not attempt to seal/commit poisoned segments.
+ *
+ * <p>Best-effort per partition: successes and failures are returned independently. Concurrent RSVM/commit
+ * races and pauseless edge cases are not fully serialized yet (IdealState is written via {@link #setIdealState},
+ * not Helix CAS — callers must avoid concurrent commits on the same partitions).
+ *
+ * @param tableNameWithType realtime table name with type
+ * @param partitionToOffsetSerialized map of partition group id → serialized {@link StreamPartitionMsgOffset}
+ * @param comment optional operator comment for logs (newlines stripped)
+ * @return per-partition result maps with keys status/oldSegment/newSegment/offset/message
+ */
+
Review Comment:
Fixed in `225357a`, that stray block is removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org