[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #4530: [HUDI-3178] Fixing metadata table compaction so as to not include uncommitted data

[GitBox <gi...@apache.org>]

alexeykudinkin commented on code in PR #4530:
URL: https://github.com/apache/hudi/pull/4530#discussion_r910389115


##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/metadata/SparkHoodieBackedTableMetadataWriter.java:
##########
@@ -128,6 +128,13 @@ protected void commit(HoodieData<HoodieRecord> hoodieDataRecords, String partiti
     JavaRDD<HoodieRecord> recordRDD = prepRecords(records, partitionName, 1);
 
     try (SparkRDDWriteClient writeClient = new SparkRDDWriteClient(engineContext, metadataWriteConfig, true)) {
+      if (canTriggerTableService) {
+        // trigger compaction before doing the delta commit. this is to ensure, if this delta commit succeeds in metadata table, but failed in data table,

Review Comment:
   Why is it the case that MT commit could succeed, while Data Table commit could fail? 
   MT table should only be updated after we're done with the Data Table changes, and right before we complete the txn, right?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org