You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "dybyte (via GitHub)" <gi...@apache.org> on 2026/06/30 08:22:33 UTC
[PR] [Improve][Zeta] Abstract checkpoint id counter state store [seatunnel]
dybyte opened a new pull request, #11226:
URL: https://github.com/apache/seatunnel/pull/11226
Related: #10804
### Purpose of this pull request
Abstract the checkpoint id counter map behind a dedicated `CounterStateStore` interface and wire it through `EngineStateStores`.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
### Check list
* [ ] If any new Jar binary package adding in your PR, please add License Notice according
[New License Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
* [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
* [ ] If necessary, please update `incompatible-changes.md` to describe the incompatibility caused by this PR.
* [ ] If you are contributing the connector code, please check that the following files are updated:
1. Update [plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties) and add new connector information in it
2. Update the pom file of [seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
3. Add ci label in [label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml)
4. Add e2e testcase in [seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/)
5. Update connector [plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improve][Zeta] Abstract checkpoint id counter state store [seatunnel]
Posted by "dybyte (via GitHub)" <gi...@apache.org>.
dybyte closed pull request #11226: [Improve][Zeta] Abstract checkpoint id counter state store
URL: https://github.com/apache/seatunnel/pull/11226
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improve][Zeta] Abstract checkpoint id counter state store [seatunnel]
Posted by "dybyte (via GitHub)" <gi...@apache.org>.
dybyte commented on code in PR #11226:
URL: https://github.com/apache/seatunnel/pull/11226#discussion_r3497710818
##########
seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/checkpoint/StateStoreCheckpointIDCounter.java:
##########
@@ -57,26 +57,26 @@ public void start() throws Exception {
@Override
public CompletableFuture<Void> shutdown(PipelineStatus pipelineStatus) {
if (pipelineStatus.isEndState()) {
- checkpointIdMap.remove(key);
+ checkpointCounterStore.remove(key);
}
return CompletableFuture.completedFuture(null);
}
@Override
public long getAndIncrement() throws Exception {
- Long nextId = checkpointIdMap.compute(key, (k, v) -> v == null ? null : v + 1);
+ long nextId = checkpointCounterStore.incrementAndGet(key);
checkNotNull(nextId);
Review Comment:
Closing this PR because the source branch link was broken after I renamed the branch.
I will reopen a new PR with the corrected branch name. Sorry for the inconvenience.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Improve][Zeta] Abstract checkpoint id counter state store [seatunnel]
Posted by "CosmosNi (via GitHub)" <gi...@apache.org>.
CosmosNi commented on code in PR #11226:
URL: https://github.com/apache/seatunnel/pull/11226#discussion_r3497462354
##########
seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/checkpoint/StateStoreCheckpointIDCounter.java:
##########
@@ -57,26 +57,26 @@ public void start() throws Exception {
@Override
public CompletableFuture<Void> shutdown(PipelineStatus pipelineStatus) {
if (pipelineStatus.isEndState()) {
- checkpointIdMap.remove(key);
+ checkpointCounterStore.remove(key);
}
return CompletableFuture.completedFuture(null);
}
@Override
public long getAndIncrement() throws Exception {
- Long nextId = checkpointIdMap.compute(key, (k, v) -> v == null ? null : v + 1);
+ long nextId = checkpointCounterStore.incrementAndGet(key);
checkNotNull(nextId);
Review Comment:
The default value of nextId is 1L, so the checkNotNull check has no effect. Alternatively, should incrementAndGet return -1 to throw an exception? Since the start method will initialize it, if it is null, that indicates the state has been lost.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org