[GitHub] [hudi] codope commented on a change in pull request #5048: [HUDI-3634] Could read empty or partial HoodieCommitMetaData in downstream if using HDFS

[GitBox <gi...@apache.org>]

codope commented on a change in pull request #5048:
URL: https://github.com/apache/hudi/pull/5048#discussion_r840249394



##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieActiveTimeline.java
##########
@@ -664,9 +664,21 @@ private void createFileInMetaPath(String filename, Option<byte[]> content, boole
    */
   private void createImmutableFileInPath(Path fullPath, Option<byte[]> content) {
     FSDataOutputStream fsout = null;
+    Path tmpPath = null;
     try {
-      fsout = metaClient.getFs().create(fullPath, false);
-      if (content.isPresent()) {
+      if (!content.isPresent()) {
+        fsout = metaClient.getFs().create(fullPath, false);
+      }
+
+      if (content.isPresent() && metaClient.getTableConfig().allowTempCommit()) {
+        Path parent = fullPath.getParent();
+        tmpPath = new Path(parent, fullPath.getName() + ".tmp");

Review comment:
       let's extract ".tmp" to a constant.

##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieActiveTimeline.java
##########
@@ -676,6 +688,9 @@ private void createImmutableFileInPath(Path fullPath, Option<byte[]> content) {
         if (null != fsout) {
           fsout.close();
         }
+        if (null != tmpPath) {
+          metaClient.getFs().rename(tmpPath, fullPath);

Review comment:
       Are renames atomic on S3 or other cloud object storage? I know put and delete in s3 is strongly consistent but what about copy? 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org