You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Robert Chansler (JIRA)" <ji...@apache.org> on 2008/10/30 00:54:46 UTC
[jira] Created: (HADOOP-4542) Fault in TestDistributedUpgrade
Fault in TestDistributedUpgrade
-------------------------------
Key: HADOOP-4542
URL: https://issues.apache.org/jira/browse/HADOOP-4542
Project: Hadoop Core
Issue Type: Bug
Components: dfs, test
Affects Versions: 0.18.0
Reporter: Robert Chansler
Priority: Minor
A TestDistributedUpgrade subtest checks that the Name Node _does not_ start when a distributed upgrade is required. In 0.18, the subtest fails when the Name Node _does_ start. The fault is with the test, not HDFS. Not a problem in 0.19.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4542) Fault in TestDistributedUpgrade
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-4542:
------------------------------------
Konstantin writes:
This is the test problem. Directory names are messed up.
The name-node code is fine.
The test itself is fixed in 0.19 and works correctly.
I would not worry about this failure in 0.18
> Fault in TestDistributedUpgrade
> -------------------------------
>
> Key: HADOOP-4542
> URL: https://issues.apache.org/jira/browse/HADOOP-4542
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Robert Chansler
> Priority: Minor
>
> A TestDistributedUpgrade subtest checks that the Name Node _does not_ start when a distributed upgrade is required. In 0.18, the subtest fails when the Name Node _does_ start. The fault is with the test, not HDFS. Not a problem in 0.19.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4542) Fault in TestDistributedUpgrade
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650797#action_12650797 ]
Konstantin Shvachko commented on HADOOP-4542:
---------------------------------------------
Yes, and on Hudson the name-node does not fail (although it should) because some previous test does not cleanup directory {{build/test/dfs/name}}, which still contains a legal image.
> Fault in TestDistributedUpgrade
> -------------------------------
>
> Key: HADOOP-4542
> URL: https://issues.apache.org/jira/browse/HADOOP-4542
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Robert Chansler
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.18.3
>
> Attachments: HADOOP-4542.patch, HADOOP-4542.patch
>
>
> A TestDistributedUpgrade subtest checks that the Name Node _does not_ start when a distributed upgrade is required. In 0.18, the subtest fails when the Name Node _does_ start. The fault is with the test, not HDFS. Not a problem in 0.19.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4542) Fault in TestDistributedUpgrade
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650804#action_12650804 ]
Konstantin Shvachko commented on HADOOP-4542:
---------------------------------------------
+1.
This will re-point the name-node to a correct storage directory, will cause its failure with the correct exception, and let Hudson build succeed.
> Fault in TestDistributedUpgrade
> -------------------------------
>
> Key: HADOOP-4542
> URL: https://issues.apache.org/jira/browse/HADOOP-4542
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Robert Chansler
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.18.3
>
> Attachments: HADOOP-4542.patch, HADOOP-4542.patch
>
>
> A TestDistributedUpgrade subtest checks that the Name Node _does not_ start when a distributed upgrade is required. In 0.18, the subtest fails when the Name Node _does_ start. The fault is with the test, not HDFS. Not a problem in 0.19.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4542) Fault in TestDistributedUpgrade
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651104#action_12651104 ]
Raghu Angadi commented on HADOOP-4542:
--------------------------------------
Thanks Konstantin. I will commit this to 0.18. 'ant test-patch' : {noformat}
[exec] +1 overall.
[exec] +1 @author. The patch does not contain any @author tags.
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
{noformat}
> Fault in TestDistributedUpgrade
> -------------------------------
>
> Key: HADOOP-4542
> URL: https://issues.apache.org/jira/browse/HADOOP-4542
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Robert Chansler
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.18.3
>
> Attachments: HADOOP-4542.patch, HADOOP-4542.patch
>
>
> A TestDistributedUpgrade subtest checks that the Name Node _does not_ start when a distributed upgrade is required. In 0.18, the subtest fails when the Name Node _does_ start. The fault is with the test, not HDFS. Not a problem in 0.19.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-4542) Fault in TestDistributedUpgrade
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi resolved HADOOP-4542.
----------------------------------
Resolution: Fixed
Release Note: TestDistributedUpgrade used succeed for wrong reasons.
I just committed this.
> Fault in TestDistributedUpgrade
> -------------------------------
>
> Key: HADOOP-4542
> URL: https://issues.apache.org/jira/browse/HADOOP-4542
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Robert Chansler
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.18.3
>
> Attachments: HADOOP-4542.patch, HADOOP-4542.patch
>
>
> A TestDistributedUpgrade subtest checks that the Name Node _does not_ start when a distributed upgrade is required. In 0.18, the subtest fails when the Name Node _does_ start. The fault is with the test, not HDFS. Not a problem in 0.19.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4542) Fault in TestDistributedUpgrade
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650799#action_12650799 ]
Raghu Angadi commented on HADOOP-4542:
--------------------------------------
right. Thanks for looking into the root cause of this.
The patch could be smaller but I kept it same as 0.19 for consistency.
> Fault in TestDistributedUpgrade
> -------------------------------
>
> Key: HADOOP-4542
> URL: https://issues.apache.org/jira/browse/HADOOP-4542
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Robert Chansler
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.18.3
>
> Attachments: HADOOP-4542.patch, HADOOP-4542.patch
>
>
> A TestDistributedUpgrade subtest checks that the Name Node _does not_ start when a distributed upgrade is required. In 0.18, the subtest fails when the Name Node _does_ start. The fault is with the test, not HDFS. Not a problem in 0.19.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-4542) Fault in TestDistributedUpgrade
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi reassigned HADOOP-4542:
------------------------------------
Assignee: Raghu Angadi
> Fault in TestDistributedUpgrade
> -------------------------------
>
> Key: HADOOP-4542
> URL: https://issues.apache.org/jira/browse/HADOOP-4542
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Robert Chansler
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.18.3
>
>
> A TestDistributedUpgrade subtest checks that the Name Node _does not_ start when a distributed upgrade is required. In 0.18, the subtest fails when the Name Node _does_ start. The fault is with the test, not HDFS. Not a problem in 0.19.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4542) Fault in TestDistributedUpgrade
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4542:
---------------------------------
Attachment: HADOOP-4542.patch
Thanks Konstantin.
So on 0.18 test passes but always for wrong reason. The attached patch fixes that. This fix was part of HADOOP-2885.
> Fault in TestDistributedUpgrade
> -------------------------------
>
> Key: HADOOP-4542
> URL: https://issues.apache.org/jira/browse/HADOOP-4542
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Robert Chansler
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.18.3
>
> Attachments: HADOOP-4542.patch, HADOOP-4542.patch
>
>
> A TestDistributedUpgrade subtest checks that the Name Node _does not_ start when a distributed upgrade is required. In 0.18, the subtest fails when the Name Node _does_ start. The fault is with the test, not HDFS. Not a problem in 0.19.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4542) Fault in TestDistributedUpgrade
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650788#action_12650788 ]
Konstantin Shvachko commented on HADOOP-4542:
---------------------------------------------
Here is the exception thrown by 0.18 for TestDistributedUpgrade:
{code}
2008-11-25 22:50:18,060 ERROR fs.FSNamesystem (FSNamesystem.java:<init>(275)) - FSNamesystem initialization failed.
org.apache.hadoop.dfs.InconsistentFSStateException: Directory /home/shv/branch-0.18/build/test/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible.
at org.apache.hadoop.dfs.FSImage.recoverTransitionRead(FSImage.java:211)
at org.apache.hadoop.dfs.FSDirectory.loadFSImage(FSDirectory.java:80)
at org.apache.hadoop.dfs.FSNamesystem.initialize(FSNamesystem.java:294)
at org.apache.hadoop.dfs.FSNamesystem.<init>(FSNamesystem.java:273)
at org.apache.hadoop.dfs.NameNode.initialize(NameNode.java:148)
at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:193)
at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:179)
at org.apache.hadoop.dfs.NameNode.createNameNode(NameNode.java:830)
at org.apache.hadoop.dfs.MiniDFSCluster.<init>(MiniDFSCluster.java:264)
at org.apache.hadoop.dfs.MiniDFSCluster.<init>(MiniDFSCluster.java:93)
at org.apache.hadoop.dfs.TestDistributedUpgrade.startNameNodeShouldFail(TestDistributedUpgrade.java:54)
at org.apache.hadoop.dfs.TestDistributedUpgrade.testDistributedUpgrade(TestDistributedUpgrade.java:97)
{code}
And here is the correct exception that should be thrown in this case and is thrown in 0.20
{code}
2008-11-25 22:53:37,165 ERROR namenode.FSNamesystem (FSNamesystem.java:<init>(282)) - FSNamesystem initialization failed.
java.io.IOException:
File system image contains an old layout version -7.
An upgrade to version -18 is required.
Please restart NameNode with -upgrade option.
at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:312)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:87)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:299)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:280)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:169)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:247)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:907)
at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:275)
at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:168)
at org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade.startNameNodeShouldFail(TestDistributedUpgrade.java:63)
at org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade.testDistributedUpgrade(TestDistributedUpgrade.java:110)
{code}
The problem is that in 0.18 {{MiniDFSCluster}} is configured with the storage being in {{/build/test/dfs/name}}, while TestDFSUpgradeFromImage unpacks it into {{/build/test/data/dfs/name1}}. And this was fixed by HADOOP-3965 or HADOOP-3948. This should be controled by {{manageDfsDirs}} parameter.
> Fault in TestDistributedUpgrade
> -------------------------------
>
> Key: HADOOP-4542
> URL: https://issues.apache.org/jira/browse/HADOOP-4542
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Robert Chansler
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.18.3
>
> Attachments: HADOOP-4542.patch
>
>
> A TestDistributedUpgrade subtest checks that the Name Node _does not_ start when a distributed upgrade is required. In 0.18, the subtest fails when the Name Node _does_ start. The fault is with the test, not HDFS. Not a problem in 0.19.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4542) Fault in TestDistributedUpgrade
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley updated HADOOP-4542:
--------------------------------
Fix Version/s: 0.18.3
> Fault in TestDistributedUpgrade
> -------------------------------
>
> Key: HADOOP-4542
> URL: https://issues.apache.org/jira/browse/HADOOP-4542
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Robert Chansler
> Priority: Minor
> Fix For: 0.18.3
>
>
> A TestDistributedUpgrade subtest checks that the Name Node _does not_ start when a distributed upgrade is required. In 0.18, the subtest fails when the Name Node _does_ start. The fault is with the test, not HDFS. Not a problem in 0.19.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4542) Fault in TestDistributedUpgrade
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650680#action_12650680 ]
Raghu Angadi commented on HADOOP-4542:
--------------------------------------
'ant test-patch' on 0.18 : {noformat}
[exec] +1 overall.
[exec] +1 @author. The patch does not contain any @author tags.
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
{noformat}
> Fault in TestDistributedUpgrade
> -------------------------------
>
> Key: HADOOP-4542
> URL: https://issues.apache.org/jira/browse/HADOOP-4542
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Robert Chansler
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.18.3
>
> Attachments: HADOOP-4542.patch
>
>
> A TestDistributedUpgrade subtest checks that the Name Node _does not_ start when a distributed upgrade is required. In 0.18, the subtest fails when the Name Node _does_ start. The fault is with the test, not HDFS. Not a problem in 0.19.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4542) Fault in TestDistributedUpgrade
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12648687#action_12648687 ]
Raghu Angadi commented on HADOOP-4542:
--------------------------------------
I am not able to reproduce this in 0.18. Which subtest fails? Any info on the jira that fixed this for 0.19 would be useful.
> Fault in TestDistributedUpgrade
> -------------------------------
>
> Key: HADOOP-4542
> URL: https://issues.apache.org/jira/browse/HADOOP-4542
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Robert Chansler
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.18.3
>
>
> A TestDistributedUpgrade subtest checks that the Name Node _does not_ start when a distributed upgrade is required. In 0.18, the subtest fails when the Name Node _does_ start. The fault is with the test, not HDFS. Not a problem in 0.19.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4542) Fault in TestDistributedUpgrade
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-4542:
---------------------------------
Attachment: HADOOP-4542.patch
Attached simple patch makes {{testDistributedUpgrade()}} a no-op. This easy fix is preferred since :
- This is going only to 0.18
- There is no requirement for DistributedUpgrade in 0.18
- Already fixed in 0.19 and trunk (not exactly sure by which patch).
- Even if I fix it (from a patch from 0.19), it is hard for me to reproduce.
> Fault in TestDistributedUpgrade
> -------------------------------
>
> Key: HADOOP-4542
> URL: https://issues.apache.org/jira/browse/HADOOP-4542
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs, test
> Affects Versions: 0.18.0
> Reporter: Robert Chansler
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.18.3
>
> Attachments: HADOOP-4542.patch
>
>
> A TestDistributedUpgrade subtest checks that the Name Node _does not_ start when a distributed upgrade is required. In 0.18, the subtest fails when the Name Node _does_ start. The fault is with the test, not HDFS. Not a problem in 0.19.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.