You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "dhruba borthakur (JIRA)" <ji...@apache.org> on 2008/03/24 23:05:24 UTC
[jira] Created: (HADOOP-3083) Remove lease when file is renamed
Remove lease when file is renamed
---------------------------------
Key: HADOOP-3083
URL: https://issues.apache.org/jira/browse/HADOOP-3083
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Reporter: dhruba borthakur
Assignee: dhruba borthakur
When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3083) Remove lease when file is renamed
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12584937#action_12584937 ]
Konstantin Shvachko commented on HADOOP-3083:
---------------------------------------------
+1
I also think that it is safer to postpone the persistent lease feature for the next release.
As it could be seen Dhruba tried several approaches to fix the issue but all of them turned out to be problematic in one aspect or another.
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3083:
-------------------------------------
Status: Patch Available (was: Open)
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3083:
-------------------------------------
Status: Open (was: Patch Available)
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3083:
-------------------------------------
Status: Patch Available (was: Open)
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Attachments: leaseIgnoreExceptions.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3083:
-------------------------------------
Attachment: doNotSaveLease.patch
Changed the unit test that was checking to make sure that leases are persisted across namenode restarts.
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3083:
-------------------------------------
Status: Patch Available (was: Open)
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3083:
-------------------------------------
Attachment: leaseIgnoreExceptions.patch
Here is a patch that does not throw an exception if a dangling lease is found while saving an fsimage. This is a stop-gap solution so that existing hdfs installs continue to work. A better solution to avoid dangling leases have to be worked out.
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Attachments: leaseIgnoreExceptions.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3083) Remove lease when file is renamed
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12583947#action_12583947 ]
Konstantin Shvachko commented on HADOOP-3083:
---------------------------------------------
# FSEditLog. Redundant:
import java.util.List;
import java.util.Iterator;
# FSDirectory.addFile() should return INodeFileUnderConstruction instead of INode.
# INode.getAbsoluteName() is deprecated and should not be used.
The full path for leases is necessary only when writing the INodeUnderConstruction to fsimage.
Otherwise, it is used in the current code and in the patch only for logging.
During saveImage() the full path of an INode is known so it should be remembered
next to the Lease and used in writeINodeUnderConstruction().
# I just noticed that we store each INodeFileUnderConstruction in fsimage twice, first as a regular inode in the
directory tree, and then in the file-under-construction section with additional file-under-construction specific fields.
This is probably for a different issue.
# We also talked about how to efficiently implement the new file-under-construction
collection that replaces current TreeSet of leases. Dhruba has a solution for that.
# File delete() is very inefficient because it uses removeLeases(), which traverses the whole
tree underneath current INode and collects all nodes in one array in order then to sort out those that are being constructed.
In fact the traversal happens two times in collectSubtreeBlocks() and then in removeLeases().
The subtree should be traversed only once. We can introduce a method say
{code}
collectSubtreeBlocksAndLeases(List<Block> v, List<INodeFileUnderConstruction> u)
{code}
which would collect both blocks and files under construction at the same time.
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, leaseIgnoreExceptions.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3083:
-------------------------------------
Status: Open (was: Patch Available)
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Attachments: leaseIgnoreExceptions.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3083:
-------------------------------------
Attachment: doNotSaveLease.patch
Do not save leases into the image. This needs to be fixed in the next release.
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, doNotSaveLease.patch, leaseIgnoreExceptions.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3083:
-------------------------------------
Attachment: doNotSaveLease.patch
Keep the lease section at the end of the fsimage but do not save any records in this section.
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, leaseIgnoreExceptions.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12581788#action_12581788 ]
dhruba borthakur commented on HADOOP-3083:
------------------------------------------
The HDFS client talks to the namenode using the pathname as the identifier. For example, if the client needs to allocate a new block, it sends a block-allocation request to the namenode. The namenode looks up the inode using the pathname and associates a new block with the file. So, a file is renamed, all new request for block allocations using the old pathname fails. So, just fixing the lease is not sufficient to support writing to files across renames.
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3083:
-------------------------------------
Hadoop Flags: [Reviewed]
Status: Patch Available (was: Open)
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3083:
-------------------------------------
Status: Open (was: Patch Available)
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3083:
-------------------------------------
Attachment: (was: leaseIgnoreExceptions.patch)
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3083) Remove lease when file is renamed
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585970#action_12585970 ]
Hudson commented on HADOOP-3083:
--------------------------------
Integrated in Hadoop-trunk #451 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/451/])
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3083) Remove lease when file is renamed
Posted by "Runping Qi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12581701#action_12581701 ]
Runping Qi commented on HADOOP-3083:
------------------------------------
Actually I think you should repair the leases so that the existing lease holders can continue writing to their files.
Here is a use case.
say a client is writing to a file /xyz/parent_dir/dub_dir/file
Somebody performs a command like "hadoop dfs -rename /xyz/parent /xyz/new_parent"
This operation should not cause the client to fail to continue writing the the file.
Otherwise, "hadoop dfs -rename dir1 dir2" will be a very dangerous operations
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3083:
-------------------------------------
Attachment: leaseInode.patch
Each lease record has a pointer to the inode. This is the most effective solution because rename does not require that all leases be searched.
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, leaseIgnoreExceptions.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-3083:
------------------------------------
Priority: Blocker (was: Major)
Fix Version/s: 0.17.0
Promoted after observing a stuck name node.
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, leaseIgnoreExceptions.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3083) Remove lease when file is renamed
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585746#action_12585746 ]
Hadoop QA commented on HADOOP-3083:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12379418/doNotSaveLease.patch
against trunk revision 643282.
@author +1. The patch does not contain any @author tags.
tests included +1. The patch appears to include 3 new or modified tests.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new javac compiler warnings.
release audit +1. The applied patch does not generate any new release audit warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2167/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2167/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2167/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2167/console
This message is automatically generated.
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3083:
-------------------------------------
Attachment: danglingLease.patch
A rename of directory entry traverses all the leases and fixes dangling leases if any. It does a sequential scan of all leases, so there is some overhead in renaming a file.
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Attachments: danglingLease.patch, leaseIgnoreExceptions.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3083) Remove lease when file is renamed
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585481#action_12585481 ]
Hadoop QA commented on HADOOP-3083:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12379326/doNotSaveLease.patch
against trunk revision 643282.
@author +1. The patch does not contain any @author tags.
tests included -1. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new javac compiler warnings.
release audit +1. The applied patch does not generate any new release audit warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests -1. The patch failed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2156/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2156/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2156/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2156/console
This message is automatically generated.
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3083) Remove lease when file is renamed
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585288#action_12585288 ]
Hadoop QA commented on HADOOP-3083:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12379167/doNotSaveLease.patch
against trunk revision 643282.
@author +1. The patch does not contain any @author tags.
tests included -1. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
patch -1. The patch command could not apply the patch.
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2144/console
This message is automatically generated.
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3083:
-------------------------------------
Attachment: doNotSaveLease.patch
merged patch with latest trunk.
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3083:
-------------------------------------
Attachment: doNotSaveLease.patch
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, leaseIgnoreExceptions.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3083) Remove lease when file is renamed
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12581962#action_12581962 ]
Hadoop QA commented on HADOOP-3083:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12378542/leaseIgnoreExceptions.patch
against trunk revision 619744.
@author +1. The patch does not contain any @author tags.
tests included -1. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new javac compiler warnings.
release audit +1. The applied patch does not generate any new release audit warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2046/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2046/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2046/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2046/console
This message is automatically generated.
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Attachments: leaseIgnoreExceptions.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3083) Remove lease when file is renamed
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-3083:
-------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I juts committed this.
> Remove lease when file is renamed
> ---------------------------------
>
> Key: HADOOP-3083
> URL: https://issues.apache.org/jira/browse/HADOOP-3083
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: danglingLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, doNotSaveLease.patch, leaseInode.patch
>
>
> When a file/directory is removed, the namenode should remove lease(s) for files that were affected by the rename.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.