[GitHub] [hudi] danny0405 commented on a diff in pull request #5526: [HUDI-4053] Flaky ITTestHoodieDataSource.testStreamWriteBatchReadOpti…

[GitBox <gi...@apache.org>]

danny0405 commented on code in PR #5526:
URL: https://github.com/apache/hudi/pull/5526#discussion_r867305446


##########
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/table/ITTestHoodieDataSource.java:
##########
@@ -1358,7 +1358,11 @@ private List<Row> execSelectSql(TableEnvironment tEnv, String select, String sin
     TableResult tableResult = tEnv.executeSql("insert into sink " + select);
     // wait for the timeout then cancels the job
     TimeUnit.SECONDS.sleep(timeout);
-    tableResult.getJobClient().ifPresent(JobClient::cancel);
+    try {
+      tableResult.getJobClient().ifPresent(JobClient::cancel);
+    } catch (IllegalStateException e) {
+      log.info("MiniCluster has already been shut down, do nothing.");
+    }

Review Comment:
   Thanks for the fix @xicm ! I have applied your patch but it does not solve the problem.
   
   Actually the test is flaky because in recent change:
   
   https://github.com/apache/hudi/blob/52fe1c9faeb83fe51b520e18d0c37b67ad3fcfe4/hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/compact/CompactionPlanOperator.java#L112
   
   We have tweaked the compaction scheduling strategy: when there is a pending compaction, we can not schedule another one, the test is flaky based on whether there is pending compaction when scheduling.
   
   My initial idea is we change the test itself:
   [4053.patch.zip](https://github.com/apache/hudi/files/8644508/4053.patch.zip)
   Hope it helps, or maybe you have better idea :)
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org