You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2018/04/24 20:47:06 UTC
[jira] [Commented] (HADOOP-14048) REDO operation of
WASB#AtomicRename should create placeholder blob for destination folder
[ https://issues.apache.org/jira/browse/HADOOP-14048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450582#comment-16450582 ]
Hudson commented on HADOOP-14048:
---------------------------------
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14057 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14057/])
HADOOP-14048. REDO operation of WASB#AtomicRename should create (aengineer: rev e65d8fb3491b328f61ec3b24b837d0249beec4da)
* (edit) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java
> REDO operation of WASB#AtomicRename should create placeholder blob for destination folder
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-14048
> URL: https://issues.apache.org/jira/browse/HADOOP-14048
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs/azure
> Reporter: NITIN VERMA
> Assignee: NITIN VERMA
> Priority: Critical
> Fix For: 2.8.0, 3.0.0-alpha4
>
> Attachments: HADOOP-14048.patch
>
>
> While doing manual testing, I realized that the crash recovery of AtomicRename operation of a folder in AzureNativeFileSystem doesn't create a placeholder property blob for destination folder. Due to this bug, the destination folder can not be renamed again.
> Below is how I tested this:
> 1. Create a test directory as "/test/A"
> 2. Create 15 block blobs in "/test/A" folder.
> 3. Run "hadoop fs -mv /test/A /test/B" command and crash it as soon as /test/A-RenamePending.json file is created.
> 4. Now run "hadoop fs -lsr /test" command, which should complete the pending rename operation (redo) as a part of crash recovery.
> 5. The REDO method copies the pending files from source folder to destination folder (by consulting A-RenamePending.json file), but it doesn't create a 0-byte property blob for /test/B folder, which is a bug as that folder will not be usable for many operations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org