You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Eli Collins (JIRA)" <ji...@apache.org> on 2012/09/10 17:24:07 UTC
[jira] [Created] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Eli Collins created HDFS-3910:
---------------------------------
Summary: DFSTestUtil#waitReplication should timeout
Key: HDFS-3910
URL: https://issues.apache.org/jira/browse/HDFS-3910
Project: Hadoop HDFS
Issue Type: Improvement
Components: test
Affects Versions: 2.0.0-alpha
Reporter: Eli Collins
Assignee: Eli Collins
DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453438#comment-13453438 ]
Hudson commented on HDFS-3910:
------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2783 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2783/])
HDFS-3910. DFSTestUtil#waitReplication should timeout. Contributed by Eli Collins (Revision 1383618)
Result = SUCCESS
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383618
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocal.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDatanodeBlockScanner.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileStatus.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancerWithMultipleNameNodes.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestOverReplicatedBlocks.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReplacement.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReport.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestDatanodeRestart.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProcessCorruptBlocks.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFileInputFormat.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMultipleLevelCaching.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/UtilsForTests.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestCombineFileInputFormat.java
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453963#comment-13453963 ]
Hudson commented on HDFS-3910:
------------------------------
Integrated in Hadoop-Hdfs-trunk #1163 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1163/])
HDFS-3910. DFSTestUtil#waitReplication should timeout. Contributed by Eli Collins (Revision 1383618)
Result = FAILURE
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383618
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocal.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDatanodeBlockScanner.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileStatus.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancerWithMultipleNameNodes.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestOverReplicatedBlocks.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReplacement.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReport.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestDatanodeRestart.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProcessCorruptBlocks.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFileInputFormat.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMultipleLevelCaching.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/UtilsForTests.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestCombineFileInputFormat.java
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453440#comment-13453440 ]
Hudson commented on HDFS-3910:
------------------------------
Integrated in Hadoop-Common-trunk-Commit #2720 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2720/])
HDFS-3910. DFSTestUtil#waitReplication should timeout. Contributed by Eli Collins (Revision 1383618)
Result = SUCCESS
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383618
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocal.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDatanodeBlockScanner.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileStatus.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancerWithMultipleNameNodes.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestOverReplicatedBlocks.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReplacement.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReport.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestDatanodeRestart.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProcessCorruptBlocks.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFileInputFormat.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMultipleLevelCaching.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/UtilsForTests.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestCombineFileInputFormat.java
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464036#comment-13464036 ]
Arun C Murthy commented on HDFS-3910:
-------------------------------------
I've reverted this from branch-2.0.2-alpha.
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HDFS-3910:
------------------------------
Priority: Minor (was: Major)
Hadoop Flags: Reviewed
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466772#comment-13466772 ]
Hudson commented on HDFS-3910:
------------------------------
Integrated in Hadoop-Hdfs-trunk #1182 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1182/])
MAPREDUCE-4681. Fix unit tests broken by HDFS-3910. Contributed by Arun C. Murthy. (Revision 1392075)
Result = SUCCESS
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1392075
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestJobQueueInformation.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSetupAndCleanupFailure.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/UtilsForTests.java
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466521#comment-13466521 ]
Hudson commented on HDFS-3910:
------------------------------
Integrated in Hadoop-Common-trunk-Commit #2796 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2796/])
MAPREDUCE-4681. Fix unit tests broken by HDFS-3910. Contributed by Arun C. Murthy. (Revision 1392075)
Result = SUCCESS
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1392075
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestJobQueueInformation.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSetupAndCleanupFailure.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/UtilsForTests.java
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453483#comment-13453483 ]
Hudson commented on HDFS-3910:
------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #2744 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2744/])
HDFS-3910. DFSTestUtil#waitReplication should timeout. Contributed by Eli Collins (Revision 1383618)
Result = FAILURE
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383618
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocal.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDatanodeBlockScanner.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileStatus.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancerWithMultipleNameNodes.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestOverReplicatedBlocks.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReplacement.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReport.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestDatanodeRestart.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProcessCorruptBlocks.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFileInputFormat.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMultipleLevelCaching.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/UtilsForTests.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestCombineFileInputFormat.java
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HDFS-3910:
------------------------------
Status: Patch Available (was: Open)
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13452173#comment-13452173 ]
Eli Collins commented on HDFS-3910:
-----------------------------------
With this patch reproducing HDFS-3902 now yields a test failure with an explicit error message:
{noformat}
Tests run: 5, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 92.861 sec <<< FAILURE!
testBlockCorruptionPolicy(org.apache.hadoop.hdfs.TestDatanodeBlockScanner) Time elapsed: 28.169 sec <<< ERROR!
java.util.concurrent.TimeoutException: Timed out waiting for /tmp/testBlockVerification/file1 to reach 3 replicas
at org.apache.hadoop.hdfs.DFSTestUtil.waitReplication(DFSTestUtil.java:534)
at org.apache.hadoop.hdfs.TestDatanodeBlockScanner.testBlockCorruptionPolicy(TestDatanodeBlockScanner.java:219)
{noformat}
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13452559#comment-13452559 ]
Andy Isaacson commented on HDFS-3910:
-------------------------------------
{code}
if (hostnames.length != replFactor) {
+ sufficientReplicas = false;
+ System.out.println("Block " + j + " of file " + fileName
+ " has replication factor " + hostnames.length + "; locations "
+ + Joiner.on(' ').join(hostnames));
{code}
(slightly reformatted for readability)
This println should also log the value of {{replFactor}}, something like
{code}
+ " has replication factor " + hostnames.length + " (desired " + replFactor + "); locations "
{code}
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HDFS-3910:
------------------------------
Attachment: hdfs-3910.txt
Patch attached.
- Updates waitReplication so that, like other DFSTestUtil wait methods, it throws TimeoutException after a fixed number of attempts and doesn't swallow InterruptedException
- Bumps the waitReplication sleep to 1s since we shouldn't have to sleep again after the first one
- Modified the various callers (MR uses DFSTestUtil)
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13452414#comment-13452414 ]
Eli Collins commented on HDFS-3910:
-----------------------------------
The test failure is HDFS-3902 (which now shows up as a test failure).
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HDFS-3910:
------------------------------
Attachment: hdfs-3910.txt
Thanks, addressed both points.
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463965#comment-13463965 ]
Arun C Murthy commented on HDFS-3910:
-------------------------------------
I'll revert this to unblock 2.0.2 release since this might be an incompatible change. Please discuss.
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13452366#comment-13452366 ]
Hadoop QA commented on HDFS-3910:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12544491/hdfs-3910.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 20 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
org.apache.hadoop.hdfs.TestDatanodeBlockScanner
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3166//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3166//console
This message is automatically generated.
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HDFS-3910:
------------------------------
Attachment: hdfs-3910.txt
Good points, updated patch addresses both.
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453432#comment-13453432 ]
Eli Collins commented on HDFS-3910:
-----------------------------------
Thanks for the reviews Andy and ATM.
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463860#comment-13463860 ]
Arun C Murthy commented on HDFS-3910:
-------------------------------------
Eli? Should we revert this change?
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466523#comment-13466523 ]
Hudson commented on HDFS-3910:
------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #2818 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2818/])
MAPREDUCE-4681. Fix unit tests broken by HDFS-3910. Contributed by Arun C. Murthy. (Revision 1392075)
Result = FAILURE
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1392075
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestJobQueueInformation.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSetupAndCleanupFailure.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/UtilsForTests.java
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463469#comment-13463469 ]
Arun C Murthy commented on HDFS-3910:
-------------------------------------
Eli - this broke MR tests' compilation: MAPREDUCE-4681.
Are we sure we are not breaking HBase et al by changing signatures for DFSTestUtil.* ?
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13452753#comment-13452753 ]
Hadoop QA commented on HDFS-3910:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12544571/hdfs-3910.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 20 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
org.apache.hadoop.hdfs.TestDatanodeBlockScanner
org.apache.hadoop.hdfs.TestPersistBlocks
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3169//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3169//console
This message is automatically generated.
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Andy Isaacson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13452524#comment-13452524 ]
Andy Isaacson commented on HDFS-3910:
-------------------------------------
LGTM. I might name the variable {{correctReplicas}} rather than {{sufficientReplicas}} since it is {{false}} when replFactor=2 and locs.length=3, but that is a minor point.
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HDFS-3910:
------------------------------
Resolution: Fixed
Fix Version/s: 2.0.2-alpha
Target Version/s: (was: 2.0.2-alpha)
Status: Resolved (was: Patch Available)
I've committed this and merged to branch-2, and branch-2.0.2-alpha.
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins resolved HDFS-3910.
-------------------------------
Resolution: Fixed
Fix Version/s: (was: 2.0.2-alpha)
2.0.3-alpha
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.3-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy reopened HDFS-3910:
---------------------------------
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13454010#comment-13454010 ]
Hudson commented on HDFS-3910:
------------------------------
Integrated in Hadoop-Mapreduce-trunk #1194 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1194/])
HDFS-3910. DFSTestUtil#waitReplication should timeout. Contributed by Eli Collins (Revision 1383618)
Result = SUCCESS
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1383618
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocal.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDatanodeBlockScanner.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileStatus.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancerWithMultipleNameNodes.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestOverReplicatedBlocks.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReplacement.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReport.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestDatanodeRestart.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProcessCorruptBlocks.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFileInputFormat.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMultipleLevelCaching.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/UtilsForTests.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestCombineFileInputFormat.java
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453224#comment-13453224 ]
Eli Collins commented on HDFS-3910:
-----------------------------------
TestPersistBlocks is unrelated, is HDFS-3811. Per above TestDatanodeBlockScanner is HDFS-3902.
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466787#comment-13466787 ]
Hudson commented on HDFS-3910:
------------------------------
Integrated in Hadoop-Mapreduce-trunk #1213 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1213/])
MAPREDUCE-4681. Fix unit tests broken by HDFS-3910. Contributed by Arun C. Murthy. (Revision 1392075)
Result = FAILURE
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1392075
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestJobQueueInformation.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSetupAndCleanupFailure.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/UtilsForTests.java
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13453339#comment-13453339 ]
Aaron T. Myers commented on HDFS-3910:
--------------------------------------
Patch looks pretty good to me. Two little nits:
# It would be good to include the exception stack trace here:
{code}
- } catch (IOException e) {
- LOG.warn("Shouldn't happen", e);
+ } catch (Exception e) {
+ Assert.fail("Failed to start BlockChecker");
{code}
# In a few places you indent the "{{throws}}" declaration of methods that continue onto the next line 2 spaces instead of 4, e.g. in TestFileInputFormat and1 UtilsForTests.
+1 once these are addressed.
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466525#comment-13466525 ]
Hudson commented on HDFS-3910:
------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2859 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2859/])
MAPREDUCE-4681. Fix unit tests broken by HDFS-3910. Contributed by Arun C. Murthy. (Revision 1392075)
Result = SUCCESS
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1392075
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestJobQueueInformation.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestSetupAndCleanupFailure.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/UtilsForTests.java
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3910) DFSTestUtil#waitReplication should
timeout
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464156#comment-13464156 ]
Eli Collins commented on HDFS-3910:
-----------------------------------
@Arun, I verified that HBase does not use DFSTestUtil. I fixed up the other sub-projects in this commit, modulo the mr1 code - thanks for catching that!
> DFSTestUtil#waitReplication should timeout
> ------------------------------------------
>
> Key: HDFS-3910
> URL: https://issues.apache.org/jira/browse/HDFS-3910
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: test
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Priority: Minor
> Fix For: 2.0.2-alpha
>
> Attachments: hdfs-3910.txt, hdfs-3910.txt, hdfs-3910.txt
>
>
> DFSTestUtil#waitReplication never times out so test execution fails only when the mvn test executor times out. This leaves a stray test process around, an example is HDFS-3902. Let's make waitReplication do something like bail after it has checked block locations 10 times for one file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira