You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org> on 2012/11/29 23:14:58 UTC
[jira] [Created] (HDFS-4243) INodeDirectory.replaceChild(..) does
not update parent
Tsz Wo (Nicholas), SZE created HDFS-4243:
--------------------------------------------
Summary: INodeDirectory.replaceChild(..) does not update parent
Key: HDFS-4243
URL: https://issues.apache.org/jira/browse/HDFS-4243
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Reporter: Tsz Wo (Nicholas), SZE
Assignee: Jing Zhao
The method replace an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4243) INodeDirectory.replaceChild(..) does
not update parent
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509288#comment-13509288 ]
Hudson commented on HDFS-4243:
------------------------------
Integrated in Hadoop-trunk-Commit #3087 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3087/])
HDFS-4243. When replacing an INodeDirectory, the parent pointers of the children of the child have to be updated to the new child. Contributed by Jing Zhao (Revision 1416709)
Result = SUCCESS
szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1416709
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java
> INodeDirectory.replaceChild(..) does not update parent
> ------------------------------------------------------
>
> Key: HDFS-4243
> URL: https://issues.apache.org/jira/browse/HDFS-4243
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Jing Zhao
> Fix For: 3.0.0
>
> Attachments: HDFS-4243.001.patch, HDFS-4243.002.patch, HDFS-4243.003.patch
>
>
> The method replaces an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
> Only FSDirectory.unprotectedSetQuota(..) calls this method so that the child is always an INodeDirectory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4243) INodeDirectory.replaceChild(..) does
not update parent
Posted by "Jing Zhao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jing Zhao updated HDFS-4243:
----------------------------
Affects Version/s: 3.0.0
Status: Patch Available (was: Open)
> INodeDirectory.replaceChild(..) does not update parent
> ------------------------------------------------------
>
> Key: HDFS-4243
> URL: https://issues.apache.org/jira/browse/HDFS-4243
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Jing Zhao
> Attachments: HDFS-4243.001.patch, HDFS-4243.002.patch, HDFS-4243.003.patch
>
>
> The method replaces an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
> Only FSDirectory.unprotectedSetQuota(..) calls this method so that the child is always an INodeDirectory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4243) INodeDirectory.replaceChild(..) does
not update parent
Posted by "Jing Zhao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507418#comment-13507418 ]
Jing Zhao commented on HDFS-4243:
---------------------------------
"by removing unnecessary FSNamesystem locking" -- I mean remove unnecessary fsdir locking. But to provide an unprotected version of replaceNode for the call in applyEditLogOp should be better.
> INodeDirectory.replaceChild(..) does not update parent
> ------------------------------------------------------
>
> Key: HDFS-4243
> URL: https://issues.apache.org/jira/browse/HDFS-4243
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Jing Zhao
> Attachments: HDFS-4243.001.patch
>
>
> The method replaces an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
> Only FSDirectory.unprotectedSetQuota(..) calls this method so that the child is always an INodeDirectory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4243) INodeDirectory.replaceChild(..) does
not update parent
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-4243:
-----------------------------------------
Description:
The method replaces an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
Only FSDirectory.unprotectedSetQuota(..) calls this method so that the child is always an INodeDirectory.
was:
The method replaces an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
Only FSDirectory.unprotectedSetQuota(..) call this method so that the child is always an INodeDirectory.
> INodeDirectory.replaceChild(..) does not update parent
> ------------------------------------------------------
>
> Key: HDFS-4243
> URL: https://issues.apache.org/jira/browse/HDFS-4243
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Jing Zhao
>
> The method replaces an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
> Only FSDirectory.unprotectedSetQuota(..) calls this method so that the child is always an INodeDirectory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4243) INodeDirectory.replaceChild(..) does
not update parent
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509656#comment-13509656 ]
Hudson commented on HDFS-4243:
------------------------------
Integrated in Hadoop-Yarn-trunk #56 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/56/])
HDFS-4243. When replacing an INodeDirectory, the parent pointers of the children of the child have to be updated to the new child. Contributed by Jing Zhao (Revision 1416709)
Result = SUCCESS
szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1416709
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java
> INodeDirectory.replaceChild(..) does not update parent
> ------------------------------------------------------
>
> Key: HDFS-4243
> URL: https://issues.apache.org/jira/browse/HDFS-4243
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Jing Zhao
> Fix For: 3.0.0
>
> Attachments: HDFS-4243.001.patch, HDFS-4243.002.patch, HDFS-4243.003.patch
>
>
> The method replaces an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
> Only FSDirectory.unprotectedSetQuota(..) calls this method so that the child is always an INodeDirectory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4243) INodeDirectory.replaceChild(..) does
not update parent
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-4243:
-----------------------------------------
Resolution: Fixed
Fix Version/s: 3.0.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
+1 patch looks good.
I have committed this. Thanks, Jing!
> INodeDirectory.replaceChild(..) does not update parent
> ------------------------------------------------------
>
> Key: HDFS-4243
> URL: https://issues.apache.org/jira/browse/HDFS-4243
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Jing Zhao
> Fix For: 3.0.0
>
> Attachments: HDFS-4243.001.patch, HDFS-4243.002.patch, HDFS-4243.003.patch
>
>
> The method replaces an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
> Only FSDirectory.unprotectedSetQuota(..) calls this method so that the child is always an INodeDirectory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4243) INodeDirectory.replaceChild(..) does
not update parent
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509041#comment-13509041 ]
Hadoop QA commented on HDFS-4243:
---------------------------------
{color:green}+1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12555807/HDFS-4243.003.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3591//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3591//console
This message is automatically generated.
> INodeDirectory.replaceChild(..) does not update parent
> ------------------------------------------------------
>
> Key: HDFS-4243
> URL: https://issues.apache.org/jira/browse/HDFS-4243
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Jing Zhao
> Attachments: HDFS-4243.001.patch, HDFS-4243.002.patch, HDFS-4243.003.patch
>
>
> The method replaces an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
> Only FSDirectory.unprotectedSetQuota(..) calls this method so that the child is always an INodeDirectory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4243) INodeDirectory.replaceChild(..) does
not update parent
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508976#comment-13508976 ]
Tsz Wo (Nicholas), SZE commented on HDFS-4243:
----------------------------------------------
Jing, thanks for working on this. What the actual problem the bug would cause? Data loss or just some incorrect reports/logs?
> INodeDirectory.replaceChild(..) does not update parent
> ------------------------------------------------------
>
> Key: HDFS-4243
> URL: https://issues.apache.org/jira/browse/HDFS-4243
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Jing Zhao
> Attachments: HDFS-4243.001.patch, HDFS-4243.002.patch, HDFS-4243.003.patch
>
>
> The method replaces an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
> Only FSDirectory.unprotectedSetQuota(..) calls this method so that the child is always an INodeDirectory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4243) INodeDirectory.replaceChild(..) does
not update parent
Posted by "Jing Zhao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508992#comment-13508992 ]
Jing Zhao commented on HDFS-4243:
---------------------------------
The INode#parent is mainly used by INode#getFullPathName which is used in generating reports/logs and fsck (through FSNamesystem#listCorruptFileBlocks).
> INodeDirectory.replaceChild(..) does not update parent
> ------------------------------------------------------
>
> Key: HDFS-4243
> URL: https://issues.apache.org/jira/browse/HDFS-4243
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Jing Zhao
> Attachments: HDFS-4243.001.patch, HDFS-4243.002.patch, HDFS-4243.003.patch
>
>
> The method replaces an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
> Only FSDirectory.unprotectedSetQuota(..) calls this method so that the child is always an INodeDirectory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4243) INodeDirectory.replaceChild(..) does
not update parent
Posted by "Jing Zhao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jing Zhao updated HDFS-4243:
----------------------------
Attachment: HDFS-4243.001.patch
Patch uploaded. The patch also refactors FSDirectory#replaceNode() by removing unnecessary FSNamesystem locking and rooDir.addINode() call.
> INodeDirectory.replaceChild(..) does not update parent
> ------------------------------------------------------
>
> Key: HDFS-4243
> URL: https://issues.apache.org/jira/browse/HDFS-4243
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Jing Zhao
> Attachments: HDFS-4243.001.patch
>
>
> The method replaces an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
> Only FSDirectory.unprotectedSetQuota(..) calls this method so that the child is always an INodeDirectory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4243) INodeDirectory.replaceChild(..) does
not update parent
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-4243:
-----------------------------------------
Description:
The method replaces an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
Only FSDirectory.unprotectedSetQuota(..) call this method so that the child is always an INodeDirectory.
was:The method replace an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
> INodeDirectory.replaceChild(..) does not update parent
> ------------------------------------------------------
>
> Key: HDFS-4243
> URL: https://issues.apache.org/jira/browse/HDFS-4243
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Jing Zhao
>
> The method replaces an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
> Only FSDirectory.unprotectedSetQuota(..) call this method so that the child is always an INodeDirectory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4243) INodeDirectory.replaceChild(..) does
not update parent
Posted by "Jing Zhao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jing Zhao updated HDFS-4243:
----------------------------
Attachment: HDFS-4243.002.patch
> INodeDirectory.replaceChild(..) does not update parent
> ------------------------------------------------------
>
> Key: HDFS-4243
> URL: https://issues.apache.org/jira/browse/HDFS-4243
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Jing Zhao
> Attachments: HDFS-4243.001.patch, HDFS-4243.002.patch
>
>
> The method replaces an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
> Only FSDirectory.unprotectedSetQuota(..) calls this method so that the child is always an INodeDirectory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4243) INodeDirectory.replaceChild(..) does
not update parent
Posted by "Jing Zhao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jing Zhao updated HDFS-4243:
----------------------------
Attachment: HDFS-4243.003.patch
Add a new testcase in which we call setQuota (which calls
INodeDirectory#replaceChild) and rename for a directory. Without the fix to call the INode#getFullPathName for a file in the directory returns the wrong name.
> INodeDirectory.replaceChild(..) does not update parent
> ------------------------------------------------------
>
> Key: HDFS-4243
> URL: https://issues.apache.org/jira/browse/HDFS-4243
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Jing Zhao
> Attachments: HDFS-4243.001.patch, HDFS-4243.002.patch, HDFS-4243.003.patch
>
>
> The method replaces an existing child with a new child. However, it does not take care the case the child also an INodeDirectory. In such case, the parent pointers of the children of the child have to be updated to the new child.
> Only FSDirectory.unprotectedSetQuota(..) calls this method so that the child is always an INodeDirectory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira