[GitHub] [hudi] yihua commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer

["yihua (via GitHub)" <gi...@apache.org>]

yihua commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1107668532


##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/WriteMarkers.java:
##########
@@ -52,16 +58,46 @@ public WriteMarkers(String basePath, String markerFolderPath, String instantTime
     this.instantTime = instantTime;
   }
 
+  public Option<Path> create(String partitionPath, String dataFileName, IOType type) {
+    return create(partitionPath, dataFileName, type, Option.empty());
+  }
+
   /**
    * Creates a marker without checking if the marker already exists.
    *
    * @param partitionPath partition path in the table
    * @param dataFileName data file name
    * @param type  write IO type
+   * @param handler could be empty
    * @return the marker path
    */
-  public Option<Path> create(String partitionPath, String dataFileName, IOType type) {
-    return create(partitionPath, dataFileName, type, false);
+  public Option<Path> create(String partitionPath, String dataFileName, IOType type, Option<HoodieWriteHandle> handler) {
+    boolean checkIfExists = false;
+
+    if (handler.isPresent()
+        && handler.get().getConfig().getWriteConcurrencyMode().supportsOptimisticConcurrencyControl()
+        && handler.get().getConfig().isEarlyConflictDetectionEnable()) {
+
+      HoodieTableMetaClient metaClient = handler.get().getHoodieTableMetaClient();
+      HoodieActiveTimeline activeTimeline = metaClient.getActiveTimeline();
+
+      HoodieTimeline pendingCompactionTimeline = activeTimeline.filterPendingCompactionTimeline();
+      HoodieTimeline pendingReplaceTimeline = activeTimeline.filterPendingReplaceTimeline();
+      // TODO if current is compact or clustering then create marker directly without early conflict detection.
+      // Need to support early conflict detection between table service and common writers.
+      if (pendingCompactionTimeline.containsInstant(instantTime) || pendingReplaceTimeline.containsInstant(instantTime)) {

Review Comment:
   The details will be captured by HUDI-5809.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org