[GitHub] [hudi] wulei0302 commented on a diff in pull request #6963: [WIP][HUDI-5040] flink multi writer for bucket index

[GitBox <gi...@apache.org>]

wulei0302 commented on code in PR #6963:
URL: https://github.com/apache/hudi/pull/6963#discussion_r1021110814


##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java:
##########
@@ -164,20 +166,31 @@ private void init(HoodieRecord record) {
       String baseInstantTime;
       String baseFile = "";
       List<String> logFiles = new ArrayList<>();
+      Option<HoodieInstant> maxCompleteInstant = hoodieTable.getMetaClient().getActiveTimeline().getWriteTimeline()
+          .filterMorCompactionInstants().lastInstant();
+      if (maxCompleteInstant.isPresent()) {
+        if (fileSlice.isPresent()) {
+          baseInstantTime = fileSlice.get().getBaseInstantTime();
+        } else {
+          baseInstantTime = instantTime;
+          // Handle log file only case. This is necessary for the concurrent clustering and writer case (e.g., consistent hashing bucket index).
+          // NOTE: flink engine use instantTime to mark operation type, check BaseFlinkCommitActionExecutor::execute
+          if (record.getCurrentLocation() != null && HoodieInstantTimeGenerator.isValidInstantTime(record.getCurrentLocation().getInstantTime())) {
+            baseInstantTime = record.getCurrentLocation().getInstantTime();
+          }
+        }
+      } else {
+        String instantTime = HoodieActiveTimeline.createNewInstantTime();
+        baseInstantTime = instantTime.substring(0, instantTime.length() - 9) + String.format("%09d", 0);

Review Comment:
   What if the instant time for concurrent writing just spans 24:00. For example,                                                    
   instant_ 1 is 20221113235900000,
   instant_ 2 is 20221114000100000,                                                                                                                                    
   and then instant_ 2 Finish first



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org