[GitHub] [hadoop] simbadzina commented on a diff in pull request #5145: HDFS-16847: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException.

[GitBox <gi...@apache.org>]

simbadzina commented on code in PR #5145:
URL: https://github.com/apache/hadoop/pull/5145#discussion_r1027110257


##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/driver/impl/StateStoreFileSystemImpl.java:
##########
@@ -82,17 +82,8 @@ protected boolean mkdir(String path) {
   @Override
   protected boolean rename(String src, String dst) {
     try {
-      if (fs instanceof DistributedFileSystem) {
-        DistributedFileSystem dfs = (DistributedFileSystem)fs;
-        dfs.rename(new Path(src), new Path(dst), Options.Rename.OVERWRITE);
-        return true;
-      } else {
-        // Replace should be atomic but not available
-        if (fs.exists(new Path(dst))) {
-          fs.delete(new Path(dst), true);
-        }
-        return fs.rename(new Path(src), new Path(dst));
-      }
+      FileUtil.rename(fs, new Path(src), new Path(dst), Options.Rename.OVERWRITE);

Review Comment:
   Hi @tomscut , if we aren't using a DistributedFileSystem, the old code does the replace itself by using a delete (if necessary) and then rename. This code is already available in Filesystem.java.
   
   More importantly, if there is a non DistributedFileSystem that has an atomic rename, the old code prevents us from using the available atomic rename.
   
   My change delegates the implementation of rename to the underlying Filesystem.
   https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java#L1648-L1650



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org