[GitHub] [hadoop] snvijaya commented on a diff in pull request #4331: HADOOP-18242. ABFS Rename Failure when tracking metadata is in an incomplete state

[GitBox <gi...@apache.org>]

snvijaya commented on code in PR #4331:
URL: https://github.com/apache/hadoop/pull/4331#discussion_r889819359


##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##########
@@ -538,6 +544,23 @@ public Pair<AbfsRestOperation, Boolean> renamePath(
         if (!op.hasResult()) {
           throw e;
         }
+
+        // ref: HADOOP-18242. Rename failure occurring due to a rare case of
+        // tracking metadata being in incomplete state.
+        if (op.getResult().getStorageErrorCode()
+            .equals(RENAME_DESTINATION_PARENT_PATH_NOT_FOUND.getErrorCode())
+            && !hasRetriedRenameOnce) {
+          LOG.info("Rename Failure attempting to resolve tracking metadata "
+              + "state and retrying.");
+
+          // Doing a HEAD call resolves the incomplete metadata state and
+          // then we can retry the rename operation.
+          getPathStatus(source, false, tracingContext);
+          hasRetriedRenameOnce = true;
+          renamePath(source, destination, continuation, tracingContext,
+              sourceEtag);

Review Comment:
   retriedFlag needs to be passed through this recursion call itself.



##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##########
@@ -102,6 +103,11 @@ public class AbfsClient implements Closeable {
 
   private final ListeningScheduledExecutorService executorService;
 
+  /**
+   * Has the Rename operation been retried once or not?
+   */
+  private boolean hasRetriedRenameOnce;

Review Comment:
   Having a class member can be an issue. What if 2 different renames are in progress, while one encounters the issue. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org