[GitHub] [hudi] vinothchandar commented on a change in pull request #3900: [HUDI-2595] Fixing metadata table updates such that only regular writes from data table can trigger table services in metadata table

[GitBox <gi...@apache.org>]

vinothchandar commented on a change in pull request #3900:
URL: https://github.com/apache/hudi/pull/3900#discussion_r743985200



##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/BaseActionExecutor.java
##########
@@ -57,7 +57,7 @@ public BaseActionExecutor(HoodieEngineContext context, HoodieWriteConfig config,
    * @param metadata commit metadata of interest.
    */
   protected final void writeTableMetadata(HoodieCommitMetadata metadata) {
-    table.getMetadataWriter().ifPresent(w -> w.update(metadata, instantTime));
+    table.getMetadataWriter().ifPresent(w -> w.update(metadata, instantTime, false));

Review comment:
       hmmm. should we assume it's a table service at this level? The idea is that even if BaseActionExecutor is further extended, they don't end up scheduling this on metadata table.

##########
File path: hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/SparkRDDWriteClient.java
##########
@@ -305,7 +305,8 @@ protected void completeCompaction(HoodieCommitMetadata metadata, JavaRDD<WriteSt
                                     String compactionCommitTime) {
     this.context.setJobStatus(this.getClass().getSimpleName(), "Collect compaction write status and commit compaction");
     List<HoodieWriteStat> writeStats = writeStatuses.map(WriteStatus::getStat).collect();
-    writeTableMetadata(table, metadata, new HoodieInstant(HoodieInstant.State.INFLIGHT, HoodieTimeline.COMPACTION_ACTION, compactionCommitTime));
+    writeTableMetadata(table, metadata, new HoodieInstant(HoodieInstant.State.INFLIGHT, HoodieTimeline.COMPACTION_ACTION, compactionCommitTime),

Review comment:
       Instead of passing a flag everywhere, can't we just limit this based on the action types and table types alone. i.e 
   
   canTriggerTableServices = true iff (table_type = `cow` && action_type in ("commit")) or (table_type = `mor` && action_type in ("deltacommit")). 
   
   won't that be simpler? 
   
   
   
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org