You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/12/03 13:42:16 UTC
[GitHub] [flink] azagrebin opened a new pull request #14301: [FLINK-20468][minicluster] Enable leadership control in MiniCluster to test JM failover
azagrebin opened a new pull request #14301:
URL: https://github.com/apache/flink/pull/14301
Currently, there is no easy way to test how JM failover (revoke and grant leadership) affects other features with the MiniCluster and its testing resource rule. The custom HA services can be provided to the `TestingMiniCluster` but there is no simple HA services to support revoking and granting leadership with a valid in-memory checkpoint store. Providing a way to enable such embedded HA services for the `MiniCluster` out of the box allows to implement IT cases similar to E2E tests.
This PR modifies `TestingCheckpointRecoveryFactory` to create and keep checkpoint store and counter per job to share the factory in cluster for multiple jobs. Uses in-memory `RecoverableCompletedCheckpointStore` (renamed to `EmbeddedCompletedCheckpointStore`) in `TestingEmbeddedHaServices` (renamed to `EmbeddedHaServicesWithLeadershipControl`). This allows using `EmbeddedHaServicesWithLeadershipControl` to revoke and grant leadership with valid checkpoint store.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] azagrebin commented on pull request #14301: [FLINK-20468][minicluster] Enable leadership control in MiniCluster to test JM failover
Posted by GitBox <gi...@apache.org>.
azagrebin commented on pull request #14301:
URL: https://github.com/apache/flink/pull/14301#issuecomment-739833708
Thanks for the review @tillrohrmann
I have addressed comments, the PR will be merged after the CI is green
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14301: [FLINK-20468][minicluster] Enable leadership control in MiniCluster to test JM failover
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #14301:
URL: https://github.com/apache/flink/pull/14301#issuecomment-738011652
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "20f56d1b5af948003d4eade6dd5a9482d54b48dc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "20f56d1b5af948003d4eade6dd5a9482d54b48dc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 20f56d1b5af948003d4eade6dd5a9482d54b48dc UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] tillrohrmann commented on a change in pull request #14301: [FLINK-20468][minicluster] Enable leadership control in MiniCluster to test JM failover
Posted by GitBox <gi...@apache.org>.
tillrohrmann commented on a change in pull request #14301:
URL: https://github.com/apache/flink/pull/14301#discussion_r536125712
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/PerJobCheckpointRecoveryFactory.java
##########
@@ -29,13 +30,13 @@
* Simple {@link CheckpointRecoveryFactory} which creates a
* {@link CompletedCheckpointStore} and a {@link CheckpointIDCounter} per {@link JobID}.
*/
-public class TestingCheckpointRecoveryFactory implements CheckpointRecoveryFactory {
+public class PerJobCheckpointRecoveryFactory implements CheckpointRecoveryFactory {
Review comment:
Why did you call it `PerJobCheckpointRecoveryFactory`? Is it because it stores for different jobs the services? To me it sounds that this factory only works for a single job.
##########
File path: flink-connectors/flink-connector-files/src/test/java/org/apache/flink/connector/file/src/FileSourceTextLinesITCase.java
##########
@@ -76,46 +68,15 @@
@ClassRule
public static final TemporaryFolder TMP_FOLDER = new TemporaryFolder();
- private static TestingMiniCluster miniCluster;
-
- private static EmbeddedHaServicesWithLeadershipControl highAvailabilityServices;
-
- @BeforeClass
- public static void setupMiniCluster() throws Exception {
- highAvailabilityServices =
- new EmbeddedHaServicesWithLeadershipControl(TestingUtils.defaultExecutor());
-
- final Configuration configuration = createConfiguration();
-
- miniCluster = new TestingMiniCluster(
- new TestingMiniClusterConfiguration.Builder()
- .setConfiguration(configuration)
- .setNumTaskManagers(1)
- .setNumSlotsPerTaskManager(PARALLELISM)
- .setRpcServiceSharing(RpcServiceSharing.DEDICATED)
- .build(),
- () -> highAvailabilityServices);
-
- miniCluster.start();
- }
-
- private static Configuration createConfiguration() throws IOException {
- final Configuration configuration = new Configuration();
- final String checkPointDir = Path.fromLocalFile(TMP_FOLDER.newFolder()).toUri().toString();
- configuration.set(CheckpointingOptions.CHECKPOINTS_DIRECTORY, checkPointDir);
- return configuration;
- }
-
- @AfterClass
- public static void shutdownMiniCluster() throws Exception {
- if (miniCluster != null) {
- miniCluster.close();
- }
- if (highAvailabilityServices != null) {
- highAvailabilityServices.closeAndCleanupAllData();
- highAvailabilityServices = null;
- }
- }
+ @ClassRule
+ public static final MiniClusterWithClientResource MINI_CLUSTER_RESOURCE = new MiniClusterWithClientResource(
+ new MiniClusterResourceConfiguration
+ .Builder()
+ .setNumberTaskManagers(1)
+ .setNumberSlotsPerTaskManager(PARALLELISM)
+ .setRpcServiceSharing(RpcServiceSharing.DEDICATED)
+ .enableEmbeddedHaLeadershipControl()
Review comment:
maybe call `withHaLeadershipControl()`.
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniCluster.java
##########
@@ -938,6 +962,13 @@ private void terminateMiniClusterServices() throws Exception {
}
}
+ @Nullable
+ private static BiFunction<Configuration, Executor, HighAvailabilityServices> createHighAvailabilityServicesFactory(
+ boolean enableEmbeddedHaLeadershipControl) {
+ return enableEmbeddedHaLeadershipControl ?
+ (conf, executor) -> new EmbeddedHaServicesWithLeadershipControl(executor) : null;
+ }
Review comment:
For what do we need this method here?
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniClusterConfiguration.java
##########
@@ -176,6 +184,11 @@ public Builder setCommonBindAddress(String commonBindAddress) {
return this;
}
+ public Builder enableEmbeddedHaLeadershipControl(boolean enableEmbeddedHaLeadershipControl) {
Review comment:
maybe rename into `withHaLeadershipControl` and add a description that this overrides the HA config option.
##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/TestingCheckpointRecoveryFactory.java
##########
@@ -20,27 +20,47 @@
import org.apache.flink.api.common.JobID;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.function.Function;
+import java.util.function.Supplier;
+
/**
- * Simple {@link CheckpointRecoveryFactory} which is initialized with a
- * {@link CompletedCheckpointStore} and a {@link CheckpointIDCounter}.
+ * Simple {@link CheckpointRecoveryFactory} which creates a
+ * {@link CompletedCheckpointStore} and a {@link CheckpointIDCounter} per {@link JobID}.
*/
public class TestingCheckpointRecoveryFactory implements CheckpointRecoveryFactory {
+ private final Function<Integer, CompletedCheckpointStore> completedCheckpointStorePerJobFactory;
+ private final Supplier<CheckpointIDCounter> checkpointIDCounterPerJobFactory;
+ private final Map<JobID, CompletedCheckpointStore> store;
+ private final Map<JobID, CheckpointIDCounter> counter;
- private final CompletedCheckpointStore store;
- private final CheckpointIDCounter counter;
-
- public TestingCheckpointRecoveryFactory(CompletedCheckpointStore store, CheckpointIDCounter counter) {
- this.store = store;
- this.counter = counter;
+ public TestingCheckpointRecoveryFactory(
+ Function<Integer, CompletedCheckpointStore> completedCheckpointStorePerJobFactory,
+ Supplier<CheckpointIDCounter> checkpointIDCounterPerJobFactory) {
+ this.completedCheckpointStorePerJobFactory = completedCheckpointStorePerJobFactory;
+ this.checkpointIDCounterPerJobFactory = checkpointIDCounterPerJobFactory;
+ this.store = new HashMap<>();
+ this.counter = new HashMap<>();
}
@Override
- public CompletedCheckpointStore createCheckpointStore(JobID jobId, int maxNumberOfCheckpointsToRetain, ClassLoader userClassLoader) throws Exception {
- return store;
+ public CompletedCheckpointStore createCheckpointStore(
+ JobID jobId,
+ int maxNumberOfCheckpointsToRetain,
+ ClassLoader userClassLoader) {
+ return store.computeIfAbsent(jobId, jId ->
+ completedCheckpointStorePerJobFactory.apply(maxNumberOfCheckpointsToRetain));
}
@Override
- public CheckpointIDCounter createCheckpointIDCounter(JobID jobId) throws Exception {
- return counter;
+ public CheckpointIDCounter createCheckpointIDCounter(JobID jobId) {
+ return counter.computeIfAbsent(jobId, jId -> checkpointIDCounterPerJobFactory.get());
+ }
+
+ public static CheckpointRecoveryFactory createSamePerJob(
Review comment:
Maybe rename into `useSameServicesForAllJobs()`.
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniClusterConfiguration.java
##########
@@ -54,12 +56,13 @@ public MiniClusterConfiguration(
Configuration configuration,
int numTaskManagers,
RpcServiceSharing rpcServiceSharing,
- @Nullable String commonBindAddress) {
-
+ @Nullable String commonBindAddress,
+ boolean enableEmbeddedHaLeadershipControl) {
Review comment:
I am always a bit more in favour of passing enum instead of booleans because enum values are more descriptive than `true` and `false`.
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniCluster.java
##########
@@ -428,11 +434,29 @@ DispatcherResourceManagerComponentFactory createDispatcherResourceManagerCompone
}
@VisibleForTesting
- protected HighAvailabilityServices createHighAvailabilityServices(Configuration configuration, Executor executor) throws Exception {
+ protected HighAvailabilityServices createHighAvailabilityServices(
+ Configuration configuration,
+ Executor executor) throws Exception {
LOG.info("Starting high-availability services");
- return HighAvailabilityServicesUtils.createAvailableOrEmbeddedServices(
- configuration,
- executor);
+ return miniClusterConfiguration.embeddedHaLeadershipControlEnabled() ?
+ new EmbeddedHaServicesWithLeadershipControl(executor) :
+ HighAvailabilityServicesUtils.createAvailableOrEmbeddedServices(configuration, executor);
+ }
+
+ /**
+ * Returns {@link HaLeadershipControl} if enabled.
+ *
+ * <p>{@link HaLeadershipControl} allows granting and revoking leadership of HA components,
+ * e.g. JobManager. The method return {@link Optional#empty()} if the control is not enabled in
+ * {@link MiniClusterConfiguration}.
+ *
+ * <p>Enabling this feature disables {@link HighAvailabilityOptions#HA_MODE} option.
Review comment:
This paragraph should be added to `enableHaLeadershipControl` method on the builder, I guess.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] azagrebin commented on a change in pull request #14301: [FLINK-20468][minicluster] Enable leadership control in MiniCluster to test JM failover
Posted by GitBox <gi...@apache.org>.
azagrebin commented on a change in pull request #14301:
URL: https://github.com/apache/flink/pull/14301#discussion_r537325089
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/PerJobCheckpointRecoveryFactory.java
##########
@@ -29,13 +30,13 @@
* Simple {@link CheckpointRecoveryFactory} which creates a
* {@link CompletedCheckpointStore} and a {@link CheckpointIDCounter} per {@link JobID}.
*/
-public class TestingCheckpointRecoveryFactory implements CheckpointRecoveryFactory {
+public class PerJobCheckpointRecoveryFactory implements CheckpointRecoveryFactory {
Review comment:
yes, would `MockCheckpointRecoveryFactory` be more clear?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14301: [FLINK-20468][minicluster] Enable leadership control in MiniCluster to test JM failover
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14301:
URL: https://github.com/apache/flink/pull/14301#issuecomment-738011652
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "20f56d1b5af948003d4eade6dd5a9482d54b48dc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10490",
"triggerID" : "20f56d1b5af948003d4eade6dd5a9482d54b48dc",
"triggerType" : "PUSH"
}, {
"hash" : "7170ccf1ed3381fd6ffe951fa5de2b16f0032866",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10519",
"triggerID" : "7170ccf1ed3381fd6ffe951fa5de2b16f0032866",
"triggerType" : "PUSH"
}, {
"hash" : "553a5f351ec34f9c157024345d4f31d89c8f091f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "553a5f351ec34f9c157024345d4f31d89c8f091f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7170ccf1ed3381fd6ffe951fa5de2b16f0032866 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10519)
* 553a5f351ec34f9c157024345d4f31d89c8f091f UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14301: [FLINK-20468][minicluster] Enable leadership control in MiniCluster to test JM failover
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #14301:
URL: https://github.com/apache/flink/pull/14301#issuecomment-738003180
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 2e3820b1dda6ac876db915d46a727722e4158bea (Thu Dec 03 13:45:39 UTC 2020)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14301: [FLINK-20468][minicluster] Enable leadership control in MiniCluster to test JM failover
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14301:
URL: https://github.com/apache/flink/pull/14301#issuecomment-738011652
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "20f56d1b5af948003d4eade6dd5a9482d54b48dc",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10490",
"triggerID" : "20f56d1b5af948003d4eade6dd5a9482d54b48dc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 20f56d1b5af948003d4eade6dd5a9482d54b48dc Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10490)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14301: [FLINK-20468][minicluster] Enable leadership control in MiniCluster to test JM failover
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14301:
URL: https://github.com/apache/flink/pull/14301#issuecomment-738011652
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "20f56d1b5af948003d4eade6dd5a9482d54b48dc",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10490",
"triggerID" : "20f56d1b5af948003d4eade6dd5a9482d54b48dc",
"triggerType" : "PUSH"
}, {
"hash" : "7170ccf1ed3381fd6ffe951fa5de2b16f0032866",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7170ccf1ed3381fd6ffe951fa5de2b16f0032866",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 20f56d1b5af948003d4eade6dd5a9482d54b48dc Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10490)
* 7170ccf1ed3381fd6ffe951fa5de2b16f0032866 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] azagrebin commented on a change in pull request #14301: [FLINK-20468][minicluster] Enable leadership control in MiniCluster to test JM failover
Posted by GitBox <gi...@apache.org>.
azagrebin commented on a change in pull request #14301:
URL: https://github.com/apache/flink/pull/14301#discussion_r537325089
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/PerJobCheckpointRecoveryFactory.java
##########
@@ -29,13 +30,13 @@
* Simple {@link CheckpointRecoveryFactory} which creates a
* {@link CompletedCheckpointStore} and a {@link CheckpointIDCounter} per {@link JobID}.
*/
-public class TestingCheckpointRecoveryFactory implements CheckpointRecoveryFactory {
+public class PerJobCheckpointRecoveryFactory implements CheckpointRecoveryFactory {
Review comment:
yes, would `MockCheckpointRecoveryFactory` be more clear?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] azagrebin closed pull request #14301: [FLINK-20468][minicluster] Enable leadership control in MiniCluster to test JM failover
Posted by GitBox <gi...@apache.org>.
azagrebin closed pull request #14301:
URL: https://github.com/apache/flink/pull/14301
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] azagrebin commented on pull request #14301: [FLINK-20468][minicluster] Enable leadership control in MiniCluster to test JM failover
Posted by GitBox <gi...@apache.org>.
azagrebin commented on pull request #14301:
URL: https://github.com/apache/flink/pull/14301#issuecomment-738003090
The PR is currently based on #14300 (first commit).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14301: [FLINK-20468][minicluster] Enable leadership control in MiniCluster to test JM failover
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14301:
URL: https://github.com/apache/flink/pull/14301#issuecomment-738011652
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "20f56d1b5af948003d4eade6dd5a9482d54b48dc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10490",
"triggerID" : "20f56d1b5af948003d4eade6dd5a9482d54b48dc",
"triggerType" : "PUSH"
}, {
"hash" : "7170ccf1ed3381fd6ffe951fa5de2b16f0032866",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10519",
"triggerID" : "7170ccf1ed3381fd6ffe951fa5de2b16f0032866",
"triggerType" : "PUSH"
}, {
"hash" : "553a5f351ec34f9c157024345d4f31d89c8f091f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10578",
"triggerID" : "553a5f351ec34f9c157024345d4f31d89c8f091f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7170ccf1ed3381fd6ffe951fa5de2b16f0032866 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10519)
* 553a5f351ec34f9c157024345d4f31d89c8f091f Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10578)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14301: [FLINK-20468][minicluster] Enable leadership control in MiniCluster to test JM failover
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14301:
URL: https://github.com/apache/flink/pull/14301#issuecomment-738011652
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "20f56d1b5af948003d4eade6dd5a9482d54b48dc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10490",
"triggerID" : "20f56d1b5af948003d4eade6dd5a9482d54b48dc",
"triggerType" : "PUSH"
}, {
"hash" : "7170ccf1ed3381fd6ffe951fa5de2b16f0032866",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10519",
"triggerID" : "7170ccf1ed3381fd6ffe951fa5de2b16f0032866",
"triggerType" : "PUSH"
}, {
"hash" : "553a5f351ec34f9c157024345d4f31d89c8f091f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10578",
"triggerID" : "553a5f351ec34f9c157024345d4f31d89c8f091f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 553a5f351ec34f9c157024345d4f31d89c8f091f Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10578)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14301: [FLINK-20468][minicluster] Enable leadership control in MiniCluster to test JM failover
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14301:
URL: https://github.com/apache/flink/pull/14301#issuecomment-738011652
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "20f56d1b5af948003d4eade6dd5a9482d54b48dc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10490",
"triggerID" : "20f56d1b5af948003d4eade6dd5a9482d54b48dc",
"triggerType" : "PUSH"
}, {
"hash" : "7170ccf1ed3381fd6ffe951fa5de2b16f0032866",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10519",
"triggerID" : "7170ccf1ed3381fd6ffe951fa5de2b16f0032866",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7170ccf1ed3381fd6ffe951fa5de2b16f0032866 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10519)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14301: [FLINK-20468][minicluster] Enable leadership control in MiniCluster to test JM failover
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #14301:
URL: https://github.com/apache/flink/pull/14301#issuecomment-738011652
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "20f56d1b5af948003d4eade6dd5a9482d54b48dc",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10490",
"triggerID" : "20f56d1b5af948003d4eade6dd5a9482d54b48dc",
"triggerType" : "PUSH"
}, {
"hash" : "7170ccf1ed3381fd6ffe951fa5de2b16f0032866",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10519",
"triggerID" : "7170ccf1ed3381fd6ffe951fa5de2b16f0032866",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 20f56d1b5af948003d4eade6dd5a9482d54b48dc Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10490)
* 7170ccf1ed3381fd6ffe951fa5de2b16f0032866 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=10519)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] azagrebin commented on a change in pull request #14301: [FLINK-20468][minicluster] Enable leadership control in MiniCluster to test JM failover
Posted by GitBox <gi...@apache.org>.
azagrebin commented on a change in pull request #14301:
URL: https://github.com/apache/flink/pull/14301#discussion_r537330059
##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniCluster.java
##########
@@ -938,6 +962,13 @@ private void terminateMiniClusterServices() throws Exception {
}
}
+ @Nullable
+ private static BiFunction<Configuration, Executor, HighAvailabilityServices> createHighAvailabilityServicesFactory(
+ boolean enableEmbeddedHaLeadershipControl) {
+ return enableEmbeddedHaLeadershipControl ?
+ (conf, executor) -> new EmbeddedHaServicesWithLeadershipControl(executor) : null;
+ }
Review comment:
leftover
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org