[GitHub] [hudi] danny0405 commented on a diff in pull request #8223: [HUDI-5950] Fixing pending instant deduction to trigger compaction in MDT

["danny0405 (via GitHub)" <gi...@apache.org>]

danny0405 commented on code in PR #8223:
URL: https://github.com/apache/hudi/pull/8223#discussion_r1140992117


##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1046,7 +1046,9 @@ protected void compactIfNecessary(BaseHoodieWriteClient writeClient, String inst
         .lastInstant().orElseThrow(() -> new HoodieMetadataException("No completed deltacommit in metadata table"))
         .getTimestamp();
     List<HoodieInstant> pendingInstants = dataMetaClient.reloadActiveTimeline().filterInflightsAndRequested()
-        .findInstantsBefore(latestDeltaCommitTimeInMetadataTable).getInstants();
+        // ignore pending indexing action

Review Comment:
   How about we add some documents to help the understanding of the use case, sth like this:
   
   ```java
   delta_c1 (F3, F4) (MDT)
   delta_c1 (F1, F2) (DT)
         //
   c2.inflight (compaction triggers in DT)
   
   
   c2 (F7, F8) (compaction complete in MDT)
   c2 fails to commit to DT
         //
   delta_c4 (F9, F10) (MDT)
       -- can we trigger MDT compaction here? The answer is no!
       1. we have no instant filtering for HFile reader
       2. the avro log merge reader has instant filtering, but it can only filter out the invalid instants, not rollback
   delta_c4 (F11, F12) (DT)
         //
   r5 (to rollback c2) (MDT)
   -F7, -F8
   r5 (to rollback c2) (DT)
         //
   delta_c6 (F13, F14) (MDT) -- now the compaction can be triggered
   delta_c6 (F15, F16) (DT)
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org