You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by zentol <gi...@git.apache.org> on 2018/03/21 13:35:50 UTC
[GitHub] flink pull request #5736: [FLINK-9042][tests] Port ResumeCheckpointManuallyI...
GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/5736
[FLINK-9042][tests] Port ResumeCheckpointManuallyITCase to flip6
## What is the purpose of the change
Ports the `ResumeCheckpointManuallyITCase` to use `MiniClusterResource`.
This port was much more involved as others and effectively required a full rewrite.
Previously, a job was started without checkpointing enabled, and we used internal APIs to trigger an externalized checkpoint, cancel the job, and resume from the created checkpoint.
Now, checkpointing is enabled (with RETAIN_ON_CANCELLATION), we poll the checkpoint directory for finished checkpoints, cancel the job, scan again for a finished checkpoint and resume from it.
## Verifying this change
Run `ResumeCheckpointManuallyITCase` with `flip6` profile enabled/disabled.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/flink 9042
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5736.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5736
----
commit 2ce5353d879bc93dc644f34adaa4fc69dec90263
Author: zentol <ch...@...>
Date: 2018-03-21T12:31:56Z
[FLINK-9042][tests] Port ResumeCheckpointManuallyITCase to flip6
----
---
[GitHub] flink pull request #5736: [FLINK-9042][tests] Port ResumeCheckpointManuallyI...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/5736
---
[GitHub] flink pull request #5736: [FLINK-9042][tests] Port ResumeCheckpointManuallyI...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5736#discussion_r176733032
--- Diff: flink-tests/src/test/java/org/apache/flink/test/checkpointing/ResumeCheckpointManuallyITCase.java ---
@@ -263,56 +268,113 @@ private void testExternalizedCheckpoints(
config.setString(HighAvailabilityOptions.HA_STORAGE_PATH, haDir.toURI().toString());
}
- TestingCluster cluster = new TestingCluster(config);
- cluster.start();
+ MiniClusterResource cluster = new MiniClusterResource(
+ new MiniClusterResource.MiniClusterResourceConfiguration(
+ new Configuration(),
+ NUM_TASK_MANAGERS,
+ SLOTS_PER_TASK_MANAGER),
+ true);
--- End diff --
ha. It _looks_ like it could be, but only because I did a mistake. We're passing a blank configuration to the cluster, and not the `config` we set up above.
With that fixed the can't be a class rule since each test requires it's own configuration. It also cannot be turned into a `Rule` easily as the config parameters are passed through the methods, and we would have to restructure the tests rather significantly.
---
[GitHub] flink issue #5736: [FLINK-9042][tests] Port ResumeCheckpointManuallyITCase t...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/5736
merging,
---
[GitHub] flink pull request #5736: [FLINK-9042][tests] Port ResumeCheckpointManuallyI...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/5736#discussion_r176686071
--- Diff: flink-tests/src/test/java/org/apache/flink/test/checkpointing/ResumeCheckpointManuallyITCase.java ---
@@ -263,56 +268,113 @@ private void testExternalizedCheckpoints(
config.setString(HighAvailabilityOptions.HA_STORAGE_PATH, haDir.toURI().toString());
}
- TestingCluster cluster = new TestingCluster(config);
- cluster.start();
+ MiniClusterResource cluster = new MiniClusterResource(
+ new MiniClusterResource.MiniClusterResourceConfiguration(
+ new Configuration(),
+ NUM_TASK_MANAGERS,
+ SLOTS_PER_TASK_MANAGER),
+ true);
--- End diff --
Can we make this a `@ClassRule`?
---
[GitHub] flink issue #5736: [FLINK-9042][tests] Port ResumeCheckpointManuallyITCase t...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/5736
Checkstyle violation:
```
14:50:10.329 [ERROR] src/test/java/org/apache/flink/test/checkpointing/ResumeCheckpointManuallyITCase.java:[297] (regexp) RegexpSingleline: Trailing whitespace
14:50:10.329 [ERROR] src/test/java/org/apache/flink/test/checkpointing/ResumeCheckpointManuallyITCase.java:[332] (regexp) RegexpSingleline: Trailing whitespace
```
---