Re: [PR] feat: add graceful handling for post-commit failures with metrics [hudi]

["suryaprasanna (via GitHub)" <gi...@apache.org>]

suryaprasanna commented on code in PR #18196:
URL: https://github.com/apache/hudi/pull/18196#discussion_r2972039811


##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java:
##########
@@ -278,8 +278,23 @@ public boolean commitStats(String instantTime, TableWriteStats tableWriteStats,
 
     // trigger clean and archival.
     // Each internal call should ensure to lock if required.
-    mayBeCleanAndArchive(table);
-    runTableServicesInline(table, metadata, extraMetadata);
+    boolean postCommitStatus = true;
+    HoodieTimer postCommitTimer = HoodieTimer.start();
+    try {
+      postCommit(table, metadata, instantTime, extraMetadata);
+      mayBeCleanAndArchive(table);
+      runTableServicesInline(table, metadata, extraMetadata);
+    } catch (Exception e) {
+      postCommitStatus = false;
+      if (config.postCommitFailuresIngnored()) {
+        LOG.error("Ignoring exception in perform post commit processing", e);
+      } else {
+        throw e;
+      }
+    } finally {
+      long duration = postCommitTimer.endTimer();

Review Comment:
   Adding extra checks, will complicate the code. So, let's keep it simple to track post commit duration in all cases.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org