You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Efrat19 (via GitHub)" <gi...@apache.org> on 2026/10/18 17:35:39 UTC
[PR] [FLINK-33186][tests] Disable periodic checkpoints in the restored job of CheckpointAfterAllTasksFinishedITCase [flink]
Efrat19 opened a new pull request, #29517:
URL: https://github.com/apache/flink/pull/29517
The test was originally meant to enable checkpointing with a large interval, so no checkpoint is effectively triggered.
Since FLINK-38939 the coordinator re-schedules the periodic trigger based on mean pause. With the default min pause of 0 a periodic checkpoint fires as soon as all tasks are running.
That early checkpoint lets the subtasks that were finished on restore (sourceA, passA -> sinkA) complete and exit immediately. The end-of-data checkpoint is queued behind it and triggered against a plan that still lists those subtasks as RUNNING. If one of them reaches FINISHED on the TaskManager before the trigger arrives, it declines with CHECKPOINT_DECLINED_TASK_NOT_READY. That reason is ignored by the CheckpointFailureManager, the end-of-data trigger is not retried, and no periodic checkpoint follows, so sourceB waits forever in StreamTask.afterInvoke for a final checkpoint and the passB -> sinkB subtasks wait for its end-of-partition.
Set the min pause of the restored job to one year, the maximum the coordinator accepts. The pause gates periodic triggers only, so the end-of-data checkpoint still fires at once, while every scheduler start or restart now waits longer than the test can run. All tasks then wait for the same final checkpoint and none can finish before it is taken, which removes the race instead of recovering from it.
Generated-by: Claude Code (Claude Fable 5.1)
## What is the purpose of the change
Fix flakiness in `testRestoreAfterSomeTasksFinished` caused by periodic checkpointing
## Brief change log
- Effectively re-disable periodic checkpointing for the test by increasing `minPauseBetweenCheckpoints`
## Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Co-Generated-by: Claude Code (Claude Fable 5.1)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33186][tests] Re-disable periodic checkpoints in CheckpointAfterAllTasksFinishedITCase tests [flink]
Posted by "MartijnVisser (via GitHub)" <gi...@apache.org>.
MartijnVisser merged PR #29517:
URL: https://github.com/apache/flink/pull/29517
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33186][tests] Disable periodic checkpoints in the restored job of CheckpointAfterAllTasksFinishedITCase [flink]
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #29517:
URL: https://github.com/apache/flink/pull/29517#issuecomment-6253171004
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "71a3f2a6414dc0da2c931f1c05a18cba6a8f887f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "71a3f2a6414dc0da2c931f1c05a18cba6a8f887f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 71a3f2a6414dc0da2c931f1c05a18cba6a8f887f UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33186][tests] Re-disable periodic checkpoints in the restored job of CheckpointAfterAllTasksFinishedITCase [flink]
Posted by "MartijnVisser (via GitHub)" <gi...@apache.org>.
MartijnVisser commented on code in PR #29517:
URL: https://github.com/apache/flink/pull/29517#discussion_r4337166922
##########
flink-tests/src/test/java/org/apache/flink/test/checkpointing/CheckpointAfterAllTasksFinishedITCase.java:
##########
@@ -128,6 +128,9 @@ void testRestoreAfterSomeTasksFinished() throws Exception {
env.enableCheckpointing(
Duration.ofNanos(Long.MAX_VALUE) /* max duration allowed by FLINK */
.toMillis());
+ // one year is the maximum min pause the coordinator accepts
+ env.getCheckpointConfig()
+ .setMinPauseBetweenCheckpoints(Duration.ofDays(365).toMillis());
Review Comment:
I think `testImmediateCheckpointing` needs the same min pause. Locally it took a periodic checkpoint in 20 of 20 runs, and in 12 of them the end-of-data checkpoint was declined.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33186][tests] Re-disable periodic checkpoints in CheckpointAfterAllTasksFinishedITCase tests [flink]
Posted by "Efrat19 (via GitHub)" <gi...@apache.org>.
Efrat19 commented on code in PR #29517:
URL: https://github.com/apache/flink/pull/29517#discussion_r4345392865
##########
flink-tests/src/test/java/org/apache/flink/test/checkpointing/CheckpointAfterAllTasksFinishedITCase.java:
##########
@@ -128,6 +128,9 @@ void testRestoreAfterSomeTasksFinished() throws Exception {
env.enableCheckpointing(
Duration.ofNanos(Long.MAX_VALUE) /* max duration allowed by FLINK */
.toMillis());
+ // one year is the maximum min pause the coordinator accepts
+ env.getCheckpointConfig()
+ .setMinPauseBetweenCheckpoints(Duration.ofDays(365).toMillis());
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org