You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Aaron T. Myers (Created) (JIRA)" <ji...@apache.org> on 2012/03/01 23:03:58 UTC
[jira] [Created] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
--------------------------------------------------------------------
Key: HDFS-3037
URL: https://issues.apache.org/jira/browse/HDFS-3037
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Affects Versions: 0.24.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
Priority: Minor
In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daryn Sharp updated HDFS-3037:
------------------------------
Fix Version/s: 2.0.1-alpha
0.23.3
> TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
> --------------------------------------------------------------------
>
> Key: HDFS-3037
> URL: https://issues.apache.org/jira/browse/HDFS-3037
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Priority: Minor
> Fix For: 0.23.3, 2.0.1-alpha, 3.0.0
>
> Attachments: HDFS-3037.patch
>
>
> In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220584#comment-13220584 ]
Hudson commented on HDFS-3037:
------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1893 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1893/])
HDFS-3037. TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy. Contributed by Aaron T. Myers. (Revision 1296016)
Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296016
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestMulitipleNNDataBlockScanner.java
> TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
> --------------------------------------------------------------------
>
> Key: HDFS-3037
> URL: https://issues.apache.org/jira/browse/HDFS-3037
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Priority: Minor
> Fix For: 0.24.0
>
> Attachments: HDFS-3037.patch
>
>
> In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410301#comment-13410301 ]
Hudson commented on HDFS-3037:
------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #309 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/309/])
svn merge -c 1296016 FIXES: HDFS-3037. TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy. Contributed by Aaron T. Myers. (Revision 1359247)
Result = SUCCESS
daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359247
Files :
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestMulitipleNNDataBlockScanner.java
> TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
> --------------------------------------------------------------------
>
> Key: HDFS-3037
> URL: https://issues.apache.org/jira/browse/HDFS-3037
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Priority: Minor
> Fix For: 0.23.3, 2.0.1-alpha, 3.0.0
>
> Attachments: HDFS-3037.patch
>
>
> In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220437#comment-13220437 ]
Todd Lipcon commented on HDFS-3037:
-----------------------------------
+1 pending Hudson
> TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
> --------------------------------------------------------------------
>
> Key: HDFS-3037
> URL: https://issues.apache.org/jira/browse/HDFS-3037
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Priority: Minor
> Attachments: HDFS-3037.patch
>
>
> In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220562#comment-13220562 ]
Hadoop QA commented on HDFS-3037:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12516742/HDFS-3037.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hdfs.TestHFlush
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1937//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1937//artifact/trunk/hadoop-hdfs-project/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1937//console
This message is automatically generated.
> TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
> --------------------------------------------------------------------
>
> Key: HDFS-3037
> URL: https://issues.apache.org/jira/browse/HDFS-3037
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Priority: Minor
> Attachments: HDFS-3037.patch
>
>
> In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220498#comment-13220498 ]
Hadoop QA commented on HDFS-3037:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12516742/HDFS-3037.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourceChecker
org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean
org.apache.hadoop.hdfs.server.namenode.TestEditLogFileOutputStream
org.apache.hadoop.hdfs.server.namenode.TestStreamFile
org.apache.hadoop.hdfs.server.namenode.TestSecondaryWebUi
org.apache.hadoop.hdfs.server.namenode.TestCorruptFilesJsp
org.apache.hadoop.hdfs.server.namenode.TestEditLogJournalFailures
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1936//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1936//artifact/trunk/hadoop-hdfs-project/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1936//console
This message is automatically generated.
> TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
> --------------------------------------------------------------------
>
> Key: HDFS-3037
> URL: https://issues.apache.org/jira/browse/HDFS-3037
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Priority: Minor
> Attachments: HDFS-3037.patch
>
>
> In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220955#comment-13220955 ]
Hudson commented on HDFS-3037:
------------------------------
Integrated in Hadoop-Mapreduce-trunk #1007 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1007/])
HDFS-3037. TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy. Contributed by Aaron T. Myers. (Revision 1296016)
Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296016
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestMulitipleNNDataBlockScanner.java
> TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
> --------------------------------------------------------------------
>
> Key: HDFS-3037
> URL: https://issues.apache.org/jira/browse/HDFS-3037
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Priority: Minor
> Fix For: 0.24.0
>
> Attachments: HDFS-3037.patch
>
>
> In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220588#comment-13220588 ]
Hudson commented on HDFS-3037:
------------------------------
Integrated in Hadoop-Common-trunk-Commit #1819 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1819/])
HDFS-3037. TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy. Contributed by Aaron T. Myers. (Revision 1296016)
Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296016
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestMulitipleNNDataBlockScanner.java
> TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
> --------------------------------------------------------------------
>
> Key: HDFS-3037
> URL: https://issues.apache.org/jira/browse/HDFS-3037
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Priority: Minor
> Fix For: 0.24.0
>
> Attachments: HDFS-3037.patch
>
>
> In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220613#comment-13220613 ]
Hudson commented on HDFS-3037:
------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1826 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1826/])
HDFS-3037. TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy. Contributed by Aaron T. Myers. (Revision 1296016)
Result = ABORTED
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296016
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestMulitipleNNDataBlockScanner.java
> TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
> --------------------------------------------------------------------
>
> Key: HDFS-3037
> URL: https://issues.apache.org/jira/browse/HDFS-3037
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Priority: Minor
> Fix For: 0.24.0
>
> Attachments: HDFS-3037.patch
>
>
> In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
Posted by "Aaron T. Myers (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220521#comment-13220521 ]
Aaron T. Myers commented on HDFS-3037:
--------------------------------------
The findbugs warning is unrelated and tracked by HDFS-3038. I strongly suspect that the test failures are spurious, but I just kicked Jenkins again for this patch just to be sure.
> TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
> --------------------------------------------------------------------
>
> Key: HDFS-3037
> URL: https://issues.apache.org/jira/browse/HDFS-3037
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Priority: Minor
> Attachments: HDFS-3037.patch
>
>
> In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409596#comment-13409596 ]
Daryn Sharp commented on HDFS-3037:
-----------------------------------
I've also committed to branch-2 & 23.
> TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
> --------------------------------------------------------------------
>
> Key: HDFS-3037
> URL: https://issues.apache.org/jira/browse/HDFS-3037
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Priority: Minor
> Fix For: 0.23.3, 2.0.1-alpha, 3.0.0
>
> Attachments: HDFS-3037.patch
>
>
> In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220892#comment-13220892 ]
Hudson commented on HDFS-3037:
------------------------------
Integrated in Hadoop-Hdfs-trunk #972 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/972/])
HDFS-3037. TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy. Contributed by Aaron T. Myers. (Revision 1296016)
Result = UNSTABLE
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1296016
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestMulitipleNNDataBlockScanner.java
> TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
> --------------------------------------------------------------------
>
> Key: HDFS-3037
> URL: https://issues.apache.org/jira/browse/HDFS-3037
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Priority: Minor
> Fix For: 0.24.0
>
> Attachments: HDFS-3037.patch
>
>
> In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
Posted by "Aaron T. Myers (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron T. Myers updated HDFS-3037:
---------------------------------
Attachment: HDFS-3037.patch
Here's a patch which addresses the issue. I tested this by putting a "Thread.sleep(5000)" at the top of BPOfferService#offerService. Without this patch, the test consistently fails. With the patch, it consistently passes.
> TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
> --------------------------------------------------------------------
>
> Key: HDFS-3037
> URL: https://issues.apache.org/jira/browse/HDFS-3037
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Priority: Minor
> Attachments: HDFS-3037.patch
>
>
> In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
Posted by "Todd Lipcon (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HDFS-3037:
------------------------------
Status: Patch Available (was: Open)
> TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
> --------------------------------------------------------------------
>
> Key: HDFS-3037
> URL: https://issues.apache.org/jira/browse/HDFS-3037
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Priority: Minor
> Attachments: HDFS-3037.patch
>
>
> In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
Posted by "Aaron T. Myers (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220575#comment-13220575 ]
Aaron T. Myers commented on HDFS-3037:
--------------------------------------
Pretty sure the TestHFlush failure is spurious and unrelated to this patch. I suspect it's due to HDFS-3041. I just ran TestHFlush locally several times with the patch applied and it passed just fine.
I'm going to commit this momentarily.
> TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
> --------------------------------------------------------------------
>
> Key: HDFS-3037
> URL: https://issues.apache.org/jira/browse/HDFS-3037
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Priority: Minor
> Attachments: HDFS-3037.patch
>
>
> In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy closed HDFS-3037.
-------------------------------
> TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
> --------------------------------------------------------------------
>
> Key: HDFS-3037
> URL: https://issues.apache.org/jira/browse/HDFS-3037
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Priority: Minor
> Fix For: 0.23.3, 2.0.2-alpha
>
> Attachments: HDFS-3037.patch
>
>
> In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3037)
TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
Posted by "Aaron T. Myers (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron T. Myers updated HDFS-3037:
---------------------------------
Resolution: Fixed
Fix Version/s: 0.24.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks a lot for the review, Todd. I've just committed this to trunk.
> TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
> --------------------------------------------------------------------
>
> Key: HDFS-3037
> URL: https://issues.apache.org/jira/browse/HDFS-3037
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Priority: Minor
> Fix For: 0.24.0
>
> Attachments: HDFS-3037.patch
>
>
> In this test, we restart a DN in a running cluster, call MiniDFS#waitActive, and then assert some things about the DN. Trouble is, MiniDFSCluster#waitActive won't wait any time at all, since the DN had previously registered with the NN and the NN never had time to realize the DN was dead.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira