[GitHub] [hudi] prashantwason commented on a diff in pull request #8797: [HUDI-6260] Fix the MDT compaction reader with the instant range filt…

["prashantwason (via GitHub)" <gi...@apache.org>]

prashantwason commented on code in PR #8797:
URL: https://github.com/apache/hudi/pull/8797#discussion_r1207224867


##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/compact/HoodieCompactor.java:
##########
@@ -247,6 +256,17 @@ public List<WriteStatus> compact(HoodieCompactionHandler compactionHandler,
     }).collect(toList());
   }
 
+  private InstantRange getMetadataLogReaderInstantRange(HoodieTableMetaClient metadataMetaClient, String metadataBasePath) {

Review Comment:
   @yihua There are these case for C2.commit.inflight to be there:
   1. C2 is still in inflight and not committed to MDT yet
   2. C2 committed successfully to MDT and still in process of completing the commit on MDT (C2's process not exited yet)
   3. C2 committed successfully to MDT but failed to complete  (C2's process failed)
   
   In the multi-writer scenario, the compaction can the initiated by the process which created C3 and that may lead to the case 2 above. In this case leaving C2 out in compaction would be an error as it is going to be a completed commit on dataset. 
   
   This is the reason why we have checks that we will initiate MDT compaction if there are pending inflights on the dataset timeline.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org