You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/03/21 14:03:22 UTC

[GitHub] [flink] XComp opened a new pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

XComp opened a new pull request #19191:
URL: https://github.com/apache/flink/pull/19191


   ## What is the purpose of the change
   
   We should make sure that the shutdown mechanism works idempotently. Additionally, errors should be propagated. Returning a CompletableFuture as the result object of the shutdown matches the general pattern we're using for cleanup tasks that are then handled downstream.
   
   ## Brief change log
   
   * Modified `CheckpointIDCounter.shutdown` method signature
   * Refactored `ZooKeeperIDCounter` to make cleanup being handled asynchronously properly
   
   ## Verifying this change
   
   * Tests were added to `ZooKeeperIDCounterITCase` and 
   `KubernetesCheckpointIDCounterTest`
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): no
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
     - The serializers: no
     - The runtime per-record code paths (performance sensitive): no
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: yes
     - The S3 file system connector: no
   
   ## Documentation
   
     - Does this pull request introduce a new feature? no
     - If yes, how is the feature documented? not applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1074244974


   I applied all comments...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537",
       "triggerID" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * e1a756ad02230b7eedd830117b259de7ccf3436e Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537) 
   * 02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 811d3062a88ea1bc58b361e8873944479bcf66a5 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 811d3062a88ea1bc58b361e8873944479bcf66a5 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 7b41fe3f05fe2691482af81f947ae0a919bbb57c Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] zentol commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
zentol commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831163556



##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounter.java
##########
@@ -118,11 +122,46 @@ public void start() throws Exception {
 
                 if (jobStatus.isGloballyTerminalState()) {
                     LOG.info("Removing {} from ZooKeeper", counterPath);
+                    final CompletableFuture<Void> deletionFuture = new CompletableFuture<>();
                     try {
-                        client.delete().inBackground().forPath(counterPath);
+                        client.delete()
+                                .inBackground(
+                                        (curatorFramework, curatorEvent) -> {
+                                            Preconditions.checkArgument(
+                                                    curatorEvent.getType()
+                                                            == CuratorEventType.DELETE,
+                                                    "An unexpected CuratorEvent was monitored: "
+                                                            + curatorEvent.getType());
+                                            Preconditions.checkArgument(
+                                                    counterPath.equals(curatorEvent.getPath()),
+                                                    "An unexpected path was selected for deletion: "
+                                                            + curatorEvent.getPath());
+
+                                            final KeeperException.Code eventCode =
+                                                    KeeperException.Code.get(
+                                                            curatorEvent.getResultCode());
+                                            if (Sets.immutableEnumSet(
+                                                            KeeperException.Code.OK,
+                                                            KeeperException.Code.NONODE)
+                                                    .contains(eventCode)) {
+                                                deletionFuture.complete(null);
+                                            } else {
+                                                deletionFuture.completeExceptionally(
+                                                        KeeperException.create(
+                                                                KeeperException.Code.get(
+                                                                        curatorEvent
+                                                                                .getResultCode())));

Review comment:
       Can you give an example for how this will look like when logged in practice?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] zentol commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
zentol commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831163556



##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounter.java
##########
@@ -118,11 +122,46 @@ public void start() throws Exception {
 
                 if (jobStatus.isGloballyTerminalState()) {
                     LOG.info("Removing {} from ZooKeeper", counterPath);
+                    final CompletableFuture<Void> deletionFuture = new CompletableFuture<>();
                     try {
-                        client.delete().inBackground().forPath(counterPath);
+                        client.delete()
+                                .inBackground(
+                                        (curatorFramework, curatorEvent) -> {
+                                            Preconditions.checkArgument(
+                                                    curatorEvent.getType()
+                                                            == CuratorEventType.DELETE,
+                                                    "An unexpected CuratorEvent was monitored: "
+                                                            + curatorEvent.getType());
+                                            Preconditions.checkArgument(
+                                                    counterPath.equals(curatorEvent.getPath()),
+                                                    "An unexpected path was selected for deletion: "
+                                                            + curatorEvent.getPath());
+
+                                            final KeeperException.Code eventCode =
+                                                    KeeperException.Code.get(
+                                                            curatorEvent.getResultCode());
+                                            if (Sets.immutableEnumSet(
+                                                            KeeperException.Code.OK,
+                                                            KeeperException.Code.NONODE)
+                                                    .contains(eventCode)) {
+                                                deletionFuture.complete(null);
+                                            } else {
+                                                deletionFuture.completeExceptionally(
+                                                        KeeperException.create(
+                                                                KeeperException.Code.get(
+                                                                        curatorEvent
+                                                                                .getResultCode())));

Review comment:
       Can you give an example for how this will look like in practice?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] zentol commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
zentol commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831166409



##########
File path: flink-kubernetes/src/test/java/org/apache/flink/kubernetes/highavailability/KubernetesCheckpointIDCounterTest.java
##########
@@ -50,6 +55,126 @@ public void testGetAndIncrement() throws Exception {
         };
     }
 
+    @Test
+    public void testShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownForLocallyTerminatedJobStatus() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.SUSPENDED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(true));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownFailureDueToMissingConfigMap() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            flinkKubeClient.deleteConfigMap(LEADER_CONFIGMAP_NAME);
+
+                            assertThrows(
+                                    CompletionException.class,
+                                    () -> checkpointIDCounter.shutdown(JobStatus.FINISHED).get());

Review comment:
       Why do we consider this as idempotent? Is it because on retry we end up recreating the configmap?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "210c4fde65f58fea333da139ba4ad4a2701b5ef4",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "210c4fde65f58fea333da139ba4ad4a2701b5ef4",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 20eee7aa5c055ce10681585e1b1e3311520819eb Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527) 
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 811d3062a88ea1bc58b361e8873944479bcf66a5 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529) 
   * 210c4fde65f58fea333da139ba4ad4a2701b5ef4 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "210c4fde65f58fea333da139ba4ad4a2701b5ef4",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "210c4fde65f58fea333da139ba4ad4a2701b5ef4",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 811d3062a88ea1bc58b361e8873944479bcf66a5 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529) 
   * 210c4fde65f58fea333da139ba4ad4a2701b5ef4 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "210c4fde65f58fea333da139ba4ad4a2701b5ef4",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "210c4fde65f58fea333da139ba4ad4a2701b5ef4",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 811d3062a88ea1bc58b361e8873944479bcf66a5 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529) 
   * 210c4fde65f58fea333da139ba4ad4a2701b5ef4 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1074014269


   > Why is the recursive ZK deletion not necessary?
   
   The `ZooKeeperCheckpointRecoveryFactory` is instantiated in [AbstractZooKeeperHaServices:67](https://github.com/apache/flink/blob/8ddfd590ebba7fc727e79db41b82d3d40a02b56a/flink-runtime/src/main/java/org/apache/flink/runtime/highavailability/zookeeper/AbstractZooKeeperHaServices.java#L67) and specifies `/jobs` as the "namespace" under the Flink cluster node.
   
   The `ZooKeeperCheckpointRecoveryFactory` instantiates the `ZooKeeperCheckpointIDCounter` in [ZooKeeperCheckpointRecoveryFactory:71](https://github.com/apache/flink/blob/50be2480b29da6fb21d7033a86bef08b6a4a2ea5/flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointRecoveryFactory.java#L71) with `/<job-id>` as the subpath on top of `/jobs`. This leaves the namespace of the client used in `ZooKeeperCheckpointIDCounter` with `/flink/<cluster-id>/jobs/<job-id>/` and the path for the actual counter being `/checkpoint_id_counter` in [ZooKeeperCheckpointIDCounter:92](https://github.com/apache/flink/blob/e85acd0bdb48c89f65bfa0287b21b0286a2afacb/flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounter.java#L92).
   
   The ZooKeeperCheckpointIDCounter uses a [SharedCount](https://github.com/apache/curator/blob/master/curator-recipes/src/main/java/org/apache/curator/framework/recipes/shared/SharedCount.java) which internally uses a [SharedValue](https://github.com/apache/curator/blob/master/curator-recipes/src/main/java/org/apache/curator/framework/recipes/shared/SharedValue.java). The `SharedValue` doesn't add any children, either.
   
   But I agree that there's a risk that I overlooked something. It just felt consistent and enabled me to test the shutdown mechanism with an actual failure cause.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 7b41fe3f05fe2691482af81f947ae0a919bbb57c Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533) 
   * e1a756ad02230b7eedd830117b259de7ccf3436e UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1074014269


   > Why is the recursive ZK deletion not necessary?
   
   The `ZooKeeperCheckpointRecoveryFactory` is instantiated in [AbstractZooKeeperHaServices:67](https://github.com/apache/flink/blob/8ddfd590ebba7fc727e79db41b82d3d40a02b56a/flink-runtime/src/main/java/org/apache/flink/runtime/highavailability/zookeeper/AbstractZooKeeperHaServices.java#L67) and specifies `/jobs` as the "namespace" under the Flink cluster node.
   
   The `ZooKeeperCheckpointRecoveryFactory` instantiates the `ZooKeeperCheckpointIDCounter` in [ZooKeeperCheckpointRecoveryFactory:71](https://github.com/apache/flink/blob/50be2480b29da6fb21d7033a86bef08b6a4a2ea5/flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointRecoveryFactory.java#L71) with `/<job-id>` as the subpath on top of `/jobs`. This leaves the namespace of the client used in `ZooKeeperCheckpointIDCounter` with `/flink/<cluster-id>/jobs/<job-id>/` and the path for the actual counter being `/checkpoint_id_counter` in [ZooKeeperCheckpointIDCounter:92](https://github.com/apache/flink/blob/e85acd0bdb48c89f65bfa0287b21b0286a2afacb/flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounter.java#L92).
   
   The ZooKeeperCheckpointIDCounter uses a [SharedCount](https://github.com/apache/curator/blob/master/curator-recipes/src/main/java/org/apache/curator/framework/recipes/shared/SharedCount.java) which internally uses a [SharedValue](https://github.com/apache/curator/blob/master/curator-recipes/src/main/java/org/apache/curator/framework/recipes/shared/SharedValue.java). The `SharedValue` doesn't add any children, either.
   
   But I agree, that there's a risk that I overlooked something. It just felt consistent and enabled me to test the shutdown mechanism with an actual failure cause.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537",
       "triggerID" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33542",
       "triggerID" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * e1a756ad02230b7eedd830117b259de7ccf3436e Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537) 
   * 02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33542) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831228638



##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounter.java
##########
@@ -118,11 +122,46 @@ public void start() throws Exception {
 
                 if (jobStatus.isGloballyTerminalState()) {
                     LOG.info("Removing {} from ZooKeeper", counterPath);
+                    final CompletableFuture<Void> deletionFuture = new CompletableFuture<>();
                     try {
-                        client.delete().inBackground().forPath(counterPath);
+                        client.delete()
+                                .inBackground(
+                                        (curatorFramework, curatorEvent) -> {
+                                            Preconditions.checkArgument(
+                                                    curatorEvent.getType()
+                                                            == CuratorEventType.DELETE,
+                                                    "An unexpected CuratorEvent was monitored: "
+                                                            + curatorEvent.getType());
+                                            Preconditions.checkArgument(
+                                                    counterPath.equals(curatorEvent.getPath()),
+                                                    "An unexpected path was selected for deletion: "
+                                                            + curatorEvent.getPath());
+
+                                            final KeeperException.Code eventCode =
+                                                    KeeperException.Code.get(
+                                                            curatorEvent.getResultCode());
+                                            if (Sets.immutableEnumSet(
+                                                            KeeperException.Code.OK,
+                                                            KeeperException.Code.NONODE)
+                                                    .contains(eventCode)) {
+                                                deletionFuture.complete(null);
+                                            } else {
+                                                deletionFuture.completeExceptionally(
+                                                        KeeperException.create(
+                                                                KeeperException.Code.get(
+                                                                        curatorEvent
+                                                                                .getResultCode())));

Review comment:
       Or what do you mean by "logged in practice"?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 20eee7aa5c055ce10681585e1b1e3311520819eb Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527) 
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 20eee7aa5c055ce10681585e1b1e3311520819eb Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 20eee7aa5c055ce10681585e1b1e3311520819eb Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831279324



##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounter.java
##########
@@ -118,11 +122,46 @@ public void start() throws Exception {
 
                 if (jobStatus.isGloballyTerminalState()) {
                     LOG.info("Removing {} from ZooKeeper", counterPath);
+                    final CompletableFuture<Void> deletionFuture = new CompletableFuture<>();
                     try {
-                        client.delete().inBackground().forPath(counterPath);
+                        client.delete()
+                                .inBackground(
+                                        (curatorFramework, curatorEvent) -> {
+                                            Preconditions.checkArgument(
+                                                    curatorEvent.getType()
+                                                            == CuratorEventType.DELETE,
+                                                    "An unexpected CuratorEvent was monitored: "
+                                                            + curatorEvent.getType());
+                                            Preconditions.checkArgument(
+                                                    counterPath.equals(curatorEvent.getPath()),
+                                                    "An unexpected path was selected for deletion: "
+                                                            + curatorEvent.getPath());
+
+                                            final KeeperException.Code eventCode =
+                                                    KeeperException.Code.get(
+                                                            curatorEvent.getResultCode());
+                                            if (Sets.immutableEnumSet(
+                                                            KeeperException.Code.OK,
+                                                            KeeperException.Code.NONODE)
+                                                    .contains(eventCode)) {
+                                                deletionFuture.complete(null);
+                                            } else {
+                                                deletionFuture.completeExceptionally(
+                                                        KeeperException.create(
+                                                                KeeperException.Code.get(
+                                                                        curatorEvent
+                                                                                .getResultCode())));

Review comment:
       I updated the test code once more. Essentially, it will show the stacktrace of a generic error-case-specific `KeeperException`. For the `testShutdownWithFailureDueToExistingChildNodes` generates the following stacktrace:
   ```
   java.util.concurrent.CompletionException: org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /checkpoint_id_counter
   
   	at java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:375)
   	at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1947)
   	at org.apache.flink.runtime.checkpoint.ZooKeeperCheckpointIDCounterITCase.testShutdownWithFailureDueToExistingChildNodes(ZooKeeperCheckpointIDCounterITCase.java:114)
   	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   	at java.lang.reflect.Method.invoke(Method.java:498)
   	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
   	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
   	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
   	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
   	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
   	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
   	at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
   	at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
   	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
   	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
   	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
   	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
   	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
   	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
   	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
   	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
   	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
   	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
   	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
   	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
   	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
   	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
   	at java.util.ArrayList.forEach(ArrayList.java:1259)
   	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
   	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
   	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
   	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
   	at java.util.ArrayList.forEach(ArrayList.java:1259)
   	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
   	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
   	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
   	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
   	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
   	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
   	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
   	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
   	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
   	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
   	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
   	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
   	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
   	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
   	at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
   	at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
   	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71)
   	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
   	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:221)
   	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
   Caused by: org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /checkpoint_id_counter
   	at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException.create(KeeperException.java:132)
   	at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   	at org.apache.flink.runtime.checkpoint.ZooKeeperCheckpointIDCounter.handleDeletionOfCounterPath(ZooKeeperCheckpointIDCounter.java:167)
   	at org.apache.flink.runtime.checkpoint.ZooKeeperCheckpointIDCounter.lambda$shutdown$0(ZooKeeperCheckpointIDCounter.java:131)
   	at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:926)
   	at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:683)
   	at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.DeleteBuilderImpl$2.processResult(DeleteBuilderImpl.java:207)
   	at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:675)
   	at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510)
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831279324



##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounter.java
##########
@@ -118,11 +122,46 @@ public void start() throws Exception {
 
                 if (jobStatus.isGloballyTerminalState()) {
                     LOG.info("Removing {} from ZooKeeper", counterPath);
+                    final CompletableFuture<Void> deletionFuture = new CompletableFuture<>();
                     try {
-                        client.delete().inBackground().forPath(counterPath);
+                        client.delete()
+                                .inBackground(
+                                        (curatorFramework, curatorEvent) -> {
+                                            Preconditions.checkArgument(
+                                                    curatorEvent.getType()
+                                                            == CuratorEventType.DELETE,
+                                                    "An unexpected CuratorEvent was monitored: "
+                                                            + curatorEvent.getType());
+                                            Preconditions.checkArgument(
+                                                    counterPath.equals(curatorEvent.getPath()),
+                                                    "An unexpected path was selected for deletion: "
+                                                            + curatorEvent.getPath());
+
+                                            final KeeperException.Code eventCode =
+                                                    KeeperException.Code.get(
+                                                            curatorEvent.getResultCode());
+                                            if (Sets.immutableEnumSet(
+                                                            KeeperException.Code.OK,
+                                                            KeeperException.Code.NONODE)
+                                                    .contains(eventCode)) {
+                                                deletionFuture.complete(null);
+                                            } else {
+                                                deletionFuture.completeExceptionally(
+                                                        KeeperException.create(
+                                                                KeeperException.Code.get(
+                                                                        curatorEvent
+                                                                                .getResultCode())));

Review comment:
       I updated the test code once more. Essentially, it will show the stacktrace of a generic error-case-specific `KeeperException`. For the `testShutdownWithFailureDueToExistingChildNodes` generates the following stacktrace:
   ```
   java.util.concurrent.CompletionException: org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /checkpoint_id_counter
   
   	at java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:375)
   	at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1947)
   	at org.apache.flink.runtime.checkpoint.ZooKeeperCheckpointIDCounterITCase.testShutdownWithFailureDueToExistingChildNodes(ZooKeeperCheckpointIDCounterITCase.java:114)
   	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [...]
   	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
   	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:221)
   	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
   Caused by: org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /checkpoint_id_counter
   	at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException.create(KeeperException.java:132)
   	at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   	at org.apache.flink.runtime.checkpoint.ZooKeeperCheckpointIDCounter.handleDeletionOfCounterPath(ZooKeeperCheckpointIDCounter.java:167)
   	at org.apache.flink.runtime.checkpoint.ZooKeeperCheckpointIDCounter.lambda$shutdown$0(ZooKeeperCheckpointIDCounter.java:131)
   	at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:926)
   	at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:683)
   	at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.DeleteBuilderImpl$2.processResult(DeleteBuilderImpl.java:207)
   	at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:675)
   	at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510)
   ```
   
   There's a log message printed on INFO beforehand in [ZooKeeperCheckpointIDCounter:L113](https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounter.java#L113)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 811d3062a88ea1bc58b361e8873944479bcf66a5 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529) 
   * 7b41fe3f05fe2691482af81f947ae0a919bbb57c UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1074931227


   I did a slight change to the cleanup log message hotfix commit changing the wording a bit...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537",
       "triggerID" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33542",
       "triggerID" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33542) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp merged pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp merged pull request #19191:
URL: https://github.com/apache/flink/pull/19191


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537",
       "triggerID" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33542",
       "triggerID" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9d5feafa7d0d652ff58e9c067afd914fbee9abd0",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33590",
       "triggerID" : "9d5feafa7d0d652ff58e9c067afd914fbee9abd0",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33542) 
   * 9d5feafa7d0d652ff58e9c067afd914fbee9abd0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33590) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1074014269


   > Why is the recursive ZK deletion not necessary?
   
   The `ZooKeeperCheckpointRecoveryFactory` is instantiated in [AbstractZooKeeperHaServices:67](https://github.com/apache/flink/blob/8ddfd590ebba7fc727e79db41b82d3d40a02b56a/flink-runtime/src/main/java/org/apache/flink/runtime/highavailability/zookeeper/AbstractZooKeeperHaServices.java#L67) and specifies `/jobs` as the "namespace" under the Flink cluster node.
   
   The `ZooKeeperCheckpointRecoveryFactory` instantiates the `ZooKeeperCheckpointIDCounter` in [ZooKeeperCheckpointRecoveryFactory:71](https://github.com/apache/flink/blob/50be2480b29da6fb21d7033a86bef08b6a4a2ea5/flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointRecoveryFactory.java#L71) with `/<job-id>` as the subpath on top of `/jobs`. This leaves the namespace of the client used in `ZooKeeperCheckpointIDCounter` with `/flink/<cluster-id>/jobs/<job-id>/` and the path for the actual counter being `/checkpoint_id_counter` in [ZooKeeperCheckpointIDCounter:92](https://github.com/apache/flink/blob/e85acd0bdb48c89f65bfa0287b21b0286a2afacb/flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounter.java#L92).
   
   The `ZooKeeperCheckpointIDCounter` uses a [SharedCount](https://github.com/apache/curator/blob/master/curator-recipes/src/main/java/org/apache/curator/framework/recipes/shared/SharedCount.java) which internally uses a [SharedValue](https://github.com/apache/curator/blob/master/curator-recipes/src/main/java/org/apache/curator/framework/recipes/shared/SharedValue.java). The `SharedValue` doesn't add any children, either.
   
   But I agree that there's a risk that I overlooked something. It just felt consistent and enabled me to test the shutdown mechanism with an actual failure cause.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] zentol commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
zentol commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831233565



##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounter.java
##########
@@ -118,11 +122,46 @@ public void start() throws Exception {
 
                 if (jobStatus.isGloballyTerminalState()) {
                     LOG.info("Removing {} from ZooKeeper", counterPath);
+                    final CompletableFuture<Void> deletionFuture = new CompletableFuture<>();
                     try {
-                        client.delete().inBackground().forPath(counterPath);
+                        client.delete()
+                                .inBackground(
+                                        (curatorFramework, curatorEvent) -> {
+                                            Preconditions.checkArgument(
+                                                    curatorEvent.getType()
+                                                            == CuratorEventType.DELETE,
+                                                    "An unexpected CuratorEvent was monitored: "
+                                                            + curatorEvent.getType());
+                                            Preconditions.checkArgument(
+                                                    counterPath.equals(curatorEvent.getPath()),
+                                                    "An unexpected path was selected for deletion: "
+                                                            + curatorEvent.getPath());
+
+                                            final KeeperException.Code eventCode =
+                                                    KeeperException.Code.get(
+                                                            curatorEvent.getResultCode());
+                                            if (Sets.immutableEnumSet(
+                                                            KeeperException.Code.OK,
+                                                            KeeperException.Code.NONODE)
+                                                    .contains(eventCode)) {
+                                                deletionFuture.complete(null);
+                                            } else {
+                                                deletionFuture.completeExceptionally(
+                                                        KeeperException.create(
+                                                                KeeperException.Code.get(
+                                                                        curatorEvent
+                                                                                .getResultCode())));

Review comment:
       What does it look like in a log file. I want to know what and how much information is conveyed to the user, and whether that is sufficient to diagnose the issue.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537",
       "triggerID" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 7b41fe3f05fe2691482af81f947ae0a919bbb57c Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533) 
   * e1a756ad02230b7eedd830117b259de7ccf3436e Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] zentol commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
zentol commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831239764



##########
File path: flink-kubernetes/src/test/java/org/apache/flink/kubernetes/highavailability/KubernetesCheckpointIDCounterTest.java
##########
@@ -50,6 +55,130 @@ public void testGetAndIncrement() throws Exception {
         };
     }
 
+    @Test
+    public void testShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownForLocallyTerminatedJobStatus() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.SUSPENDED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(true));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownFailureDueToMissingConfigMap() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            // deleting the ConfigMap from outside of the CheckpointIDCounter while

Review comment:
       How is this case different to a delete being retried? (e.g., the delete went through but the ack didn't come back?)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831163056



##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounterITCase.java
##########
@@ -62,7 +62,7 @@ public void testShutdownRemovesState() throws Exception {
         CuratorFramework client = zookeeper.getClient();
         assertThat(client.checkExists().forPath(counter.getPath())).isNotNull();
 
-        counter.shutdown(JobStatus.FINISHED);
+        counter.shutdown(JobStatus.FINISHED).join();

Review comment:
       Yeah, I'm not 100% sure why it worked without problems beforehand with the `inBackground`. Maybe, the requests are queued up on the ZooKeeper's side... 🤔 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] zentol commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
zentol commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831157637



##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounterITCase.java
##########
@@ -62,7 +62,7 @@ public void testShutdownRemovesState() throws Exception {
         CuratorFramework client = zookeeper.getClient();
         assertThat(client.checkExists().forPath(counter.getPath())).isNotNull();
 
-        counter.shutdown(JobStatus.FINISHED);
+        counter.shutdown(JobStatus.FINISHED).join();

Review comment:
       surprised this didn't cause test instabilities before...




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 811d3062a88ea1bc58b361e8873944479bcf66a5 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529) 
   * 7b41fe3f05fe2691482af81f947ae0a919bbb57c UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831221407



##########
File path: flink-kubernetes/src/test/java/org/apache/flink/kubernetes/highavailability/KubernetesCheckpointIDCounterTest.java
##########
@@ -50,6 +55,126 @@ public void testGetAndIncrement() throws Exception {
         };
     }
 
+    @Test
+    public void testShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownForLocallyTerminatedJobStatus() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.SUSPENDED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(true));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownFailureDueToMissingConfigMap() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            flinkKubeClient.deleteConfigMap(LEADER_CONFIGMAP_NAME);
+
+                            assertThrows(
+                                    CompletionException.class,
+                                    () -> checkpointIDCounter.shutdown(JobStatus.FINISHED).get());

Review comment:
       We're not considering that as idempotent. I just wanted to add a test case to see that k8s errors are properly propagated through the `CompletableFuture`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831251475



##########
File path: flink-kubernetes/src/test/java/org/apache/flink/kubernetes/highavailability/KubernetesCheckpointIDCounterTest.java
##########
@@ -50,6 +55,130 @@ public void testGetAndIncrement() throws Exception {
         };
     }
 
+    @Test
+    public void testShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownForLocallyTerminatedJobStatus() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.SUSPENDED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(true));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownFailureDueToMissingConfigMap() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            // deleting the ConfigMap from outside of the CheckpointIDCounter while

Review comment:
       retrying the deletion will cause an entry being removed from the (existing) ConfigMap. This would not cause any errors. The ConfigMap itself is removed in the `Kubernetes(MultipleComponentLeaderElection)HaServices.internalCleanupJobData(JobID)` after the `JobMaster` is closed. That's why, there's the `JobManagerRunner`/`JobMaster` cleanup is prioritized over the other cleanups in [DispatcherResourceCleanerFactory:110](https://github.com/apache/flink/blob/c5352fc55972420ed5bf1afdfd97834540b1407a/flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/cleanup/DispatcherResourceCleanerFactory.java#L110)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537",
       "triggerID" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33542",
       "triggerID" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33542) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537",
       "triggerID" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33542",
       "triggerID" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9d5feafa7d0d652ff58e9c067afd914fbee9abd0",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "9d5feafa7d0d652ff58e9c067afd914fbee9abd0",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33542) 
   * 9d5feafa7d0d652ff58e9c067afd914fbee9abd0 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831279324



##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounter.java
##########
@@ -118,11 +122,46 @@ public void start() throws Exception {
 
                 if (jobStatus.isGloballyTerminalState()) {
                     LOG.info("Removing {} from ZooKeeper", counterPath);
+                    final CompletableFuture<Void> deletionFuture = new CompletableFuture<>();
                     try {
-                        client.delete().inBackground().forPath(counterPath);
+                        client.delete()
+                                .inBackground(
+                                        (curatorFramework, curatorEvent) -> {
+                                            Preconditions.checkArgument(
+                                                    curatorEvent.getType()
+                                                            == CuratorEventType.DELETE,
+                                                    "An unexpected CuratorEvent was monitored: "
+                                                            + curatorEvent.getType());
+                                            Preconditions.checkArgument(
+                                                    counterPath.equals(curatorEvent.getPath()),
+                                                    "An unexpected path was selected for deletion: "
+                                                            + curatorEvent.getPath());
+
+                                            final KeeperException.Code eventCode =
+                                                    KeeperException.Code.get(
+                                                            curatorEvent.getResultCode());
+                                            if (Sets.immutableEnumSet(
+                                                            KeeperException.Code.OK,
+                                                            KeeperException.Code.NONODE)
+                                                    .contains(eventCode)) {
+                                                deletionFuture.complete(null);
+                                            } else {
+                                                deletionFuture.completeExceptionally(
+                                                        KeeperException.create(
+                                                                KeeperException.Code.get(
+                                                                        curatorEvent
+                                                                                .getResultCode())));

Review comment:
       I updated the test code once more. Essentially, it will show the stacktrace of a generic error-case-specific `KeeperException`. For the `testShutdownWithFailureDueToExistingChildNodes` generates the following stacktrace:
   ```
   java.util.concurrent.CompletionException: org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /checkpoint_id_counter
   
   	at java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:375)
   	at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1947)
   	at org.apache.flink.runtime.checkpoint.ZooKeeperCheckpointIDCounterITCase.testShutdownWithFailureDueToExistingChildNodes(ZooKeeperCheckpointIDCounterITCase.java:114)
   	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   	at java.lang.reflect.Method.invoke(Method.java:498)
   	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
   	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
   	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
   	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
   	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
   	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
   	at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
   	at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
   	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
   	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
   	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
   	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
   	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
   	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
   	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
   	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
   	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
   	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
   	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
   	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
   	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
   	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
   	at java.util.ArrayList.forEach(ArrayList.java:1259)
   	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
   	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
   	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
   	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
   	at java.util.ArrayList.forEach(ArrayList.java:1259)
   	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
   	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
   	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
   	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
   	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
   	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
   	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
   	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
   	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
   	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
   	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
   	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
   	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
   	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
   	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
   	at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
   	at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
   	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71)
   	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
   	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:221)
   	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
   Caused by: org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /checkpoint_id_counter
   	at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException.create(KeeperException.java:132)
   	at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   	at org.apache.flink.runtime.checkpoint.ZooKeeperCheckpointIDCounter.handleDeletionOfCounterPath(ZooKeeperCheckpointIDCounter.java:167)
   	at org.apache.flink.runtime.checkpoint.ZooKeeperCheckpointIDCounter.lambda$shutdown$0(ZooKeeperCheckpointIDCounter.java:131)
   	at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:926)
   	at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:683)
   	at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.DeleteBuilderImpl$2.processResult(DeleteBuilderImpl.java:207)
   	at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:675)
   	at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510)
   ```
   
   There's a log message printed on INFO beforehand in [ZooKeeperCheckpointIDCounter:L113](https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounter.java#L113)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] zentol commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
zentol commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831235009



##########
File path: flink-kubernetes/src/test/java/org/apache/flink/kubernetes/highavailability/KubernetesCheckpointIDCounterTest.java
##########
@@ -50,6 +55,126 @@ public void testGetAndIncrement() throws Exception {
         };
     }
 
+    @Test
+    public void testShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownForLocallyTerminatedJobStatus() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.SUSPENDED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(true));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownFailureDueToMissingConfigMap() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            flinkKubeClient.deleteConfigMap(LEADER_CONFIGMAP_NAME);
+
+                            assertThrows(
+                                    CompletionException.class,
+                                    () -> checkpointIDCounter.shutdown(JobStatus.FINISHED).get());

Review comment:
       Well now I'm confused; why is it ok for this shutdown call to fail because of a missing config map?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 811d3062a88ea1bc58b361e8873944479bcf66a5 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] zentol commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
zentol commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831165005



##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounterITCase.java
##########
@@ -66,6 +70,56 @@ public void testShutdownRemovesState() throws Exception {
         assertThat(client.checkExists().forPath(counter.getPath())).isNull();
     }
 
+    @Test
+    public void testShutdownForLocallyTerminatedJobStatus() throws Exception {
+        ZooKeeperCheckpointIDCounter counter = createCheckpointIdCounter();
+        counter.start();
+
+        CuratorFramework client = zookeeper.getClient();
+        assertThat(client.checkExists().forPath(counter.getPath())).isNotNull();
+
+        counter.shutdown(JobStatus.SUSPENDED).join();
+        assertThat(client.checkExists().forPath(counter.getPath())).isNotNull();
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        ZooKeeperCheckpointIDCounter counter = createCheckpointIdCounter();
+        counter.start();
+
+        CuratorFramework client = zookeeper.getClient();
+        client.delete().forPath(counter.getPath());
+
+        // shutdown shouldn't fail due to missing path
+        counter.shutdown(JobStatus.FINISHED).join();
+        assertThat(client.checkExists().forPath(counter.getPath())).isNull();
+    }
+
+    @Test
+    public void testShutdownWithFailureDueToMissingConnection() throws Exception {
+        ZooKeeperCheckpointIDCounter counter = createCheckpointIdCounter();
+        counter.start();
+
+        cleanAndStopZooKeeperIfRunning();
+
+        assertThat(counter.shutdown(JobStatus.FINISHED)).failsWithin(Duration.ofHours(1));
+    }
+
+    @Test
+    public void testShutdownWithFailureDueToChildNodes() throws Exception {

Review comment:
       DueTo**Missing**ChildNode




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831241857



##########
File path: flink-kubernetes/src/test/java/org/apache/flink/kubernetes/highavailability/KubernetesCheckpointIDCounterTest.java
##########
@@ -50,6 +55,126 @@ public void testGetAndIncrement() throws Exception {
         };
     }
 
+    @Test
+    public void testShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownForLocallyTerminatedJobStatus() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.SUSPENDED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(true));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownFailureDueToMissingConfigMap() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            flinkKubeClient.deleteConfigMap(LEADER_CONFIGMAP_NAME);
+
+                            assertThrows(
+                                    CompletionException.class,
+                                    () -> checkpointIDCounter.shutdown(JobStatus.FINISHED).get());

Review comment:
       Deleting the `ConfigMap` is a tool for me to create an inconsistent state, that, as a consequence, causes failures accessing it. I want to check that we do proper failure handling here, i.e. propagating the actual failure instead of just "swallowing" it (like we did before; as far as I understand).
   
   It's not ok to have the `ConfigMap` being removed in the general case. It's just the only way, I could think of, to generate an error on the ZK side. ...as an alternative to mocking the curator-internal ZK call.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831243121



##########
File path: flink-kubernetes/src/test/java/org/apache/flink/kubernetes/highavailability/KubernetesCheckpointIDCounterTest.java
##########
@@ -50,6 +55,126 @@ public void testGetAndIncrement() throws Exception {
         };
     }
 
+    @Test
+    public void testShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownForLocallyTerminatedJobStatus() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.SUSPENDED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(true));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownFailureDueToMissingConfigMap() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            flinkKubeClient.deleteConfigMap(LEADER_CONFIGMAP_NAME);
+
+                            assertThrows(
+                                    CompletionException.class,
+                                    () -> checkpointIDCounter.shutdown(JobStatus.FINISHED).get());

Review comment:
       ...similar to stopping the ZK instance before shutting down the `ZooKeeperIDCounter` instance in `testShutdownWithFailureDueToMissingConnection` where it goes into another codepath (client side error)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 20eee7aa5c055ce10681585e1b1e3311520819eb Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527) 
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831165067



##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounter.java
##########
@@ -118,11 +122,46 @@ public void start() throws Exception {
 
                 if (jobStatus.isGloballyTerminalState()) {
                     LOG.info("Removing {} from ZooKeeper", counterPath);
+                    final CompletableFuture<Void> deletionFuture = new CompletableFuture<>();
                     try {
-                        client.delete().inBackground().forPath(counterPath);
+                        client.delete()
+                                .inBackground(
+                                        (curatorFramework, curatorEvent) -> {
+                                            Preconditions.checkArgument(
+                                                    curatorEvent.getType()
+                                                            == CuratorEventType.DELETE,
+                                                    "An unexpected CuratorEvent was monitored: "
+                                                            + curatorEvent.getType());
+                                            Preconditions.checkArgument(
+                                                    counterPath.equals(curatorEvent.getPath()),
+                                                    "An unexpected path was selected for deletion: "
+                                                            + curatorEvent.getPath());
+
+                                            final KeeperException.Code eventCode =
+                                                    KeeperException.Code.get(
+                                                            curatorEvent.getResultCode());
+                                            if (Sets.immutableEnumSet(
+                                                            KeeperException.Code.OK,
+                                                            KeeperException.Code.NONODE)
+                                                    .contains(eventCode)) {
+                                                deletionFuture.complete(null);
+                                            } else {
+                                                deletionFuture.completeExceptionally(
+                                                        KeeperException.create(
+                                                                KeeperException.Code.get(
+                                                                        curatorEvent
+                                                                                .getResultCode())));

Review comment:
       There are two test cases added to `ZooKeeperCheckpointIDCounterITCase` covering different error types.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 20eee7aa5c055ce10681585e1b1e3311520819eb Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1074168896


   Thanks for the review, @zentol . I addressed all your comments. PTAL


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 20eee7aa5c055ce10681585e1b1e3311520819eb Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527) 
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831398672



##########
File path: flink-kubernetes/src/test/java/org/apache/flink/kubernetes/highavailability/KubernetesCheckpointIDCounterTest.java
##########
@@ -50,6 +55,131 @@ public void testGetAndIncrement() throws Exception {
         };
     }
 
+    @Test
+    public void testShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownForLocallyTerminatedJobStatus() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.SUSPENDED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(true));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));

Review comment:
       I also added a second shutdown call to `testShutdownFailureDueToMissingConfigMap`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1074924947


   CI failure caused by FLINK-26621. I will go ahead squashing everything and create a backport


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 20eee7aa5c055ce10681585e1b1e3311520819eb Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537",
       "triggerID" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33542",
       "triggerID" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33542) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 20eee7aa5c055ce10681585e1b1e3311520819eb Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527) 
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 811d3062a88ea1bc58b361e8873944479bcf66a5 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 811d3062a88ea1bc58b361e8873944479bcf66a5 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529) 
   * 7b41fe3f05fe2691482af81f947ae0a919bbb57c Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533) 
   * e1a756ad02230b7eedd830117b259de7ccf3436e UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] zentol commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
zentol commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831159253



##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounter.java
##########
@@ -118,11 +122,46 @@ public void start() throws Exception {
 
                 if (jobStatus.isGloballyTerminalState()) {
                     LOG.info("Removing {} from ZooKeeper", counterPath);
+                    final CompletableFuture<Void> deletionFuture = new CompletableFuture<>();
                     try {
-                        client.delete().inBackground().forPath(counterPath);
+                        client.delete()
+                                .inBackground(
+                                        (curatorFramework, curatorEvent) -> {

Review comment:
       we could make this more readable by having the runnable be defined in another method.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 20eee7aa5c055ce10681585e1b1e3311520819eb Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527) 
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537",
       "triggerID" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 7b41fe3f05fe2691482af81f947ae0a919bbb57c Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533) 
   * e1a756ad02230b7eedd830117b259de7ccf3436e Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537) 
   * 02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] zentol commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
zentol commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831170400



##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounterITCase.java
##########
@@ -66,6 +70,56 @@ public void testShutdownRemovesState() throws Exception {
         assertThat(client.checkExists().forPath(counter.getPath())).isNull();
     }
 
+    @Test
+    public void testShutdownForLocallyTerminatedJobStatus() throws Exception {
+        ZooKeeperCheckpointIDCounter counter = createCheckpointIdCounter();
+        counter.start();
+
+        CuratorFramework client = zookeeper.getClient();
+        assertThat(client.checkExists().forPath(counter.getPath())).isNotNull();
+
+        counter.shutdown(JobStatus.SUSPENDED).join();
+        assertThat(client.checkExists().forPath(counter.getPath())).isNotNull();
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        ZooKeeperCheckpointIDCounter counter = createCheckpointIdCounter();
+        counter.start();
+
+        CuratorFramework client = zookeeper.getClient();
+        client.delete().forPath(counter.getPath());
+
+        // shutdown shouldn't fail due to missing path
+        counter.shutdown(JobStatus.FINISHED).join();
+        assertThat(client.checkExists().forPath(counter.getPath())).isNull();
+    }
+
+    @Test
+    public void testShutdownWithFailureDueToMissingConnection() throws Exception {
+        ZooKeeperCheckpointIDCounter counter = createCheckpointIdCounter();
+        counter.start();
+
+        cleanAndStopZooKeeperIfRunning();
+
+        assertThat(counter.shutdown(JobStatus.FINISHED)).failsWithin(Duration.ofHours(1));
+    }
+
+    @Test
+    public void testShutdownWithFailureDueToChildNodes() throws Exception {

Review comment:
       DueTo**Existing**ChildNode then.
   
   Why does the comment say "// shutdown shouldn't fail due to missing path"?
   What is actual ZK behavior? Will it happily delete the unexpected child node as well?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831167459



##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounterITCase.java
##########
@@ -66,6 +70,56 @@ public void testShutdownRemovesState() throws Exception {
         assertThat(client.checkExists().forPath(counter.getPath())).isNull();
     }
 
+    @Test
+    public void testShutdownForLocallyTerminatedJobStatus() throws Exception {
+        ZooKeeperCheckpointIDCounter counter = createCheckpointIdCounter();
+        counter.start();
+
+        CuratorFramework client = zookeeper.getClient();
+        assertThat(client.checkExists().forPath(counter.getPath())).isNotNull();
+
+        counter.shutdown(JobStatus.SUSPENDED).join();
+        assertThat(client.checkExists().forPath(counter.getPath())).isNotNull();
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        ZooKeeperCheckpointIDCounter counter = createCheckpointIdCounter();
+        counter.start();
+
+        CuratorFramework client = zookeeper.getClient();
+        client.delete().forPath(counter.getPath());
+
+        // shutdown shouldn't fail due to missing path
+        counter.shutdown(JobStatus.FINISHED).join();
+        assertThat(client.checkExists().forPath(counter.getPath())).isNull();
+    }
+
+    @Test
+    public void testShutdownWithFailureDueToMissingConnection() throws Exception {
+        ZooKeeperCheckpointIDCounter counter = createCheckpointIdCounter();
+        counter.start();
+
+        cleanAndStopZooKeeperIfRunning();
+
+        assertThat(counter.shutdown(JobStatus.FINISHED)).failsWithin(Duration.ofHours(1));
+    }
+
+    @Test
+    public void testShutdownWithFailureDueToChildNodes() throws Exception {

Review comment:
       No, the deletion is failing because there are child nodes which I added for the test case (and I removed the recursive deletion). The IDCounter shouldn't have child nodes in general...




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831221722



##########
File path: flink-kubernetes/src/test/java/org/apache/flink/kubernetes/highavailability/KubernetesCheckpointIDCounterTest.java
##########
@@ -50,6 +55,126 @@ public void testGetAndIncrement() throws Exception {
         };
     }
 
+    @Test
+    public void testShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownForLocallyTerminatedJobStatus() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.SUSPENDED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(true));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownFailureDueToMissingConfigMap() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            flinkKubeClient.deleteConfigMap(LEADER_CONFIGMAP_NAME);
+
+                            assertThrows(
+                                    CompletionException.class,
+                                    () -> checkpointIDCounter.shutdown(JobStatus.FINISHED).get());

Review comment:
       I'm gonna add more description/comments, though




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831226297



##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounter.java
##########
@@ -118,11 +122,46 @@ public void start() throws Exception {
 
                 if (jobStatus.isGloballyTerminalState()) {
                     LOG.info("Removing {} from ZooKeeper", counterPath);
+                    final CompletableFuture<Void> deletionFuture = new CompletableFuture<>();
                     try {
-                        client.delete().inBackground().forPath(counterPath);
+                        client.delete()
+                                .inBackground(
+                                        (curatorFramework, curatorEvent) -> {

Review comment:
       That's a good idea 👍 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537",
       "triggerID" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33542",
       "triggerID" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9d5feafa7d0d652ff58e9c067afd914fbee9abd0",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33590",
       "triggerID" : "9d5feafa7d0d652ff58e9c067afd914fbee9abd0",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 9d5feafa7d0d652ff58e9c067afd914fbee9abd0 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33590) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] zentol commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
zentol commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831381056



##########
File path: flink-kubernetes/src/test/java/org/apache/flink/kubernetes/highavailability/KubernetesCheckpointIDCounterTest.java
##########
@@ -50,6 +55,131 @@ public void testGetAndIncrement() throws Exception {
         };
     }
 
+    @Test
+    public void testShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownForLocallyTerminatedJobStatus() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.SUSPENDED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(true));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownFailureDueToMissingConfigMap() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            // deleting the ConfigMap from outside of the CheckpointIDCounter while
+                            // still using the counter (which is stored as an entry in the
+                            // ConfigMap) causes an unexpected failure which we want to simulate
+                            // here
+                            flinkKubeClient.deleteConfigMap(LEADER_CONFIGMAP_NAME);
+
+                            assertThrows(
+                                    "The shutdown should fail due to the illegal state of the test setup.",

Review comment:
       this message doesn't really provide any value. That some part of the test setup is supposed to make this fail is obvious.

##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounterITCase.java
##########
@@ -62,10 +69,70 @@ public void testShutdownRemovesState() throws Exception {
         CuratorFramework client = zookeeper.getClient();
         assertThat(client.checkExists().forPath(counter.getPath())).isNotNull();
 
-        counter.shutdown(JobStatus.FINISHED);
+        counter.shutdown(JobStatus.FINISHED).join();
+        assertThat(client.checkExists().forPath(counter.getPath())).isNull();
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        ZooKeeperCheckpointIDCounter counter = createCheckpointIdCounter();
+        counter.start();
+
+        CuratorFramework client = zookeeper.getClient();
+        counter.shutdown(JobStatus.FINISHED).join();
+
+        // shutdown shouldn't fail due to missing path
+        counter.shutdown(JobStatus.FINISHED).join();
         assertThat(client.checkExists().forPath(counter.getPath())).isNull();
     }
 
+    @Test
+    public void testShutdownWithFailureDueToMissingConnection() throws Exception {
+        ZooKeeperCheckpointIDCounter counter = createCheckpointIdCounter();
+        counter.start();
+
+        cleanAndStopZooKeeperIfRunning();
+
+        assertThat(counter.shutdown(JobStatus.FINISHED))
+                .as("The shutdown should fail because of the client connection being dropped.")
+                .failsWithin(Duration.ofHours(1));

Review comment:
       Not really a fan of using such dummy timeouts. I'd rather call get() on the future and simplify the whole thing. We shouldn't force ourselves to use the CompletableFuture assertions if they are incompatible with our testing philosophy.

##########
File path: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounter.java
##########
@@ -101,22 +110,66 @@ public void start() throws Exception {
     }
 
     @Override
-    public void shutdown(JobStatus jobStatus) throws Exception {
+    public CompletableFuture<Void> shutdown(JobStatus jobStatus) {
         synchronized (startStopLock) {
             if (isStarted) {
                 LOG.info("Shutting down.");
-                sharedCount.close();
+                try {
+                    sharedCount.close();
+                } catch (IOException e) {
+                    return FutureUtils.completedExceptionally(e);
+                }
 
                 client.getConnectionStateListenable().removeListener(connectionStateListener);
 
                 if (jobStatus.isGloballyTerminalState()) {
                     LOG.info("Removing {} from ZooKeeper", counterPath);
-                    client.delete().deletingChildrenIfNeeded().inBackground().forPath(counterPath);
+                    final CompletableFuture<Void> deletionFuture = new CompletableFuture<>();
+                    try {
+                        client.delete()
+                                .inBackground(
+                                        (curatorFramework, curatorEvent) ->
+                                                handleDeletionOfCounterPath(
+                                                        curatorEvent, deletionFuture))
+                                .forPath(counterPath);

Review comment:
       could create and return the deletion future within the try block

##########
File path: flink-kubernetes/src/test/java/org/apache/flink/kubernetes/highavailability/KubernetesCheckpointIDCounterTest.java
##########
@@ -50,6 +55,131 @@ public void testGetAndIncrement() throws Exception {
         };
     }
 
+    @Test
+    public void testShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownForLocallyTerminatedJobStatus() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.SUSPENDED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(true));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));

Review comment:
       I'd prefer this to follow the pattern in the ZK test where we start out with data in the config map, and then initiate the shutdown twice. As is the shutdown method could fail catastrophically if called twice in a row, rendering it non-idempotent for our purposes.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831391930



##########
File path: flink-kubernetes/src/test/java/org/apache/flink/kubernetes/highavailability/KubernetesCheckpointIDCounterTest.java
##########
@@ -50,6 +55,131 @@ public void testGetAndIncrement() throws Exception {
         };
     }
 
+    @Test
+    public void testShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownForLocallyTerminatedJobStatus() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.SUSPENDED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(true));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));

Review comment:
       Good point 👍 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot commented on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 20eee7aa5c055ce10681585e1b1e3311520819eb UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831251475



##########
File path: flink-kubernetes/src/test/java/org/apache/flink/kubernetes/highavailability/KubernetesCheckpointIDCounterTest.java
##########
@@ -50,6 +55,130 @@ public void testGetAndIncrement() throws Exception {
         };
     }
 
+    @Test
+    public void testShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownForLocallyTerminatedJobStatus() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            checkpointIDCounter.setCount(100L);
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .get(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is("100"));
+
+                            checkpointIDCounter.shutdown(JobStatus.SUSPENDED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(true));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+
+                            checkpointIDCounter.shutdown(JobStatus.FINISHED).join();
+
+                            assertThat(
+                                    getLeaderConfigMap()
+                                            .getData()
+                                            .containsKey(Constants.CHECKPOINT_COUNTER_KEY),
+                                    is(false));
+                        });
+            }
+        };
+    }
+
+    @Test
+    public void testShutdownFailureDueToMissingConfigMap() throws Exception {
+        new Context() {
+            {
+                runTest(
+                        () -> {
+                            leaderCallbackGrantLeadership();
+
+                            final KubernetesCheckpointIDCounter checkpointIDCounter =
+                                    new KubernetesCheckpointIDCounter(
+                                            flinkKubeClient, LEADER_CONFIGMAP_NAME, LOCK_IDENTITY);
+                            checkpointIDCounter.start();
+
+                            // deleting the ConfigMap from outside of the CheckpointIDCounter while

Review comment:
       retrying the deletion will cause an entry being removed from the (existing) ConfigMap. This would not cause any errors. The ConfigMap itself is removed in the `Kubernetes(MultipleComponentLeaderElection)HaServices.internalCleanupJobData(JobID)`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537",
       "triggerID" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * e1a756ad02230b7eedd830117b259de7ccf3436e Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on a change in pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on a change in pull request #19191:
URL: https://github.com/apache/flink/pull/19191#discussion_r831215567



##########
File path: flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/ZooKeeperCheckpointIDCounterITCase.java
##########
@@ -66,6 +70,56 @@ public void testShutdownRemovesState() throws Exception {
         assertThat(client.checkExists().forPath(counter.getPath())).isNull();
     }
 
+    @Test
+    public void testShutdownForLocallyTerminatedJobStatus() throws Exception {
+        ZooKeeperCheckpointIDCounter counter = createCheckpointIdCounter();
+        counter.start();
+
+        CuratorFramework client = zookeeper.getClient();
+        assertThat(client.checkExists().forPath(counter.getPath())).isNotNull();
+
+        counter.shutdown(JobStatus.SUSPENDED).join();
+        assertThat(client.checkExists().forPath(counter.getPath())).isNotNull();
+    }
+
+    @Test
+    public void testIdempotentShutdown() throws Exception {
+        ZooKeeperCheckpointIDCounter counter = createCheckpointIdCounter();
+        counter.start();
+
+        CuratorFramework client = zookeeper.getClient();
+        client.delete().forPath(counter.getPath());
+
+        // shutdown shouldn't fail due to missing path
+        counter.shutdown(JobStatus.FINISHED).join();
+        assertThat(client.checkExists().forPath(counter.getPath())).isNull();
+    }
+
+    @Test
+    public void testShutdownWithFailureDueToMissingConnection() throws Exception {
+        ZooKeeperCheckpointIDCounter counter = createCheckpointIdCounter();
+        counter.start();
+
+        cleanAndStopZooKeeperIfRunning();
+
+        assertThat(counter.shutdown(JobStatus.FINISHED)).failsWithin(Duration.ofHours(1));
+    }
+
+    @Test
+    public void testShutdownWithFailureDueToChildNodes() throws Exception {

Review comment:
       True, and valid point about the comment: That was copied over by mistake




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 20eee7aa5c055ce10681585e1b1e3311520819eb Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527) 
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537",
       "triggerID" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33542",
       "triggerID" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9d5feafa7d0d652ff58e9c067afd914fbee9abd0",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "9d5feafa7d0d652ff58e9c067afd914fbee9abd0",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33542) 
   * 9d5feafa7d0d652ff58e9c067afd914fbee9abd0 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] XComp commented on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
XComp commented on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1076023093


   CI failures related to FLINK-26789 and FLINK-26794


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 811d3062a88ea1bc58b361e8873944479bcf66a5 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529) 
   * 7b41fe3f05fe2691482af81f947ae0a919bbb57c Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533) 
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [flink] flinkbot edited a comment on pull request #19191: [FLINK-26741][runtime] The CheckpointIDCounter.shutdown method should work idempotently

Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #19191:
URL: https://github.com/apache/flink/pull/19191#issuecomment-1073952732


   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33527",
       "triggerID" : "20eee7aa5c055ce10681585e1b1e3311520819eb",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5df4f200865d40d4ae1c96cfc992d5b302dc6756",
       "triggerType" : "PUSH"
     }, {
       "hash" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33529",
       "triggerID" : "811d3062a88ea1bc58b361e8873944479bcf66a5",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33533",
       "triggerID" : "7b41fe3f05fe2691482af81f947ae0a919bbb57c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33537",
       "triggerID" : "e1a756ad02230b7eedd830117b259de7ccf3436e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33542",
       "triggerID" : "02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a",
       "triggerType" : "PUSH"
     }, {
       "hash" : "9d5feafa7d0d652ff58e9c067afd914fbee9abd0",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "9d5feafa7d0d652ff58e9c067afd914fbee9abd0",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5df4f200865d40d4ae1c96cfc992d5b302dc6756 UNKNOWN
   * 02db7936cd90dbd4a7a2ac6ddf930ffb37504e6a Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=33542) 
   * 9d5feafa7d0d652ff58e9c067afd914fbee9abd0 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org