You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Matt Foley (JIRA)" <ji...@apache.org> on 2011/04/05 23:41:06 UTC
[jira] [Created] (HDFS-1808) TestBalancer waits forever, errs
without giving information
TestBalancer waits forever, errs without giving information
-----------------------------------------------------------
Key: HDFS-1808
URL: https://issues.apache.org/jira/browse/HDFS-1808
Project: Hadoop HDFS
Issue Type: Bug
Components: data-node, name-node
Affects Versions: 0.22.0
Reporter: Matt Foley
Assignee: Matt Foley
In three locations in the code, TestBalancer waits forever on a condition. Failures result in Hudson/Jenkins "Timeout occurred" error message with no information about where or why. Need to replace with TimeoutExceptions that throw a stack trace and useful info about the failure mode.
In waitForHeartBeat(), it is waiting on an exact match for a value that may be coarsely quantized -- i.e., significant deviation from the exact "expected" result may occur. Replace with an allowed range of result.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1808) TestBalancer waits forever, errs
without giving information
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13022586#comment-13022586 ]
Hadoop QA commented on HDFS-1808:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12476951/1808_TestBalancer_v4.patch
against trunk revision 1095461.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 1 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these core unit tests:
org.apache.hadoop.hdfs.TestFileConcurrentReader
+1 contrib tests. The patch passed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/394//testReport/
Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/394//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/394//console
This message is automatically generated.
> TestBalancer waits forever, errs without giving information
> -----------------------------------------------------------
>
> Key: HDFS-1808
> URL: https://issues.apache.org/jira/browse/HDFS-1808
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node, name-node
> Affects Versions: 0.22.0
> Reporter: Matt Foley
> Assignee: Matt Foley
> Fix For: 0.23.0
>
> Attachments: 1808_TestBalancer_v3.patch, 1808_TestBalancer_v4.patch, TestBalancer.java.patch
>
>
> In three locations in the code, TestBalancer waits forever on a condition. Failures result in Hudson/Jenkins "Timeout occurred" error message with no information about where or why. Need to replace with TimeoutExceptions that throw a stack trace and useful info about the failure mode.
> In waitForHeartBeat(), it is waiting on an exact match for a value that may be coarsely quantized -- i.e., significant deviation from the exact "expected" result may occur. Replace with an allowed range of result.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1808) TestBalancer waits forever, errs
without giving information
Posted by "Matt Foley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt Foley updated HDFS-1808:
-----------------------------
Attachment: 1808_TestBalancer_v3.patch
Cleaned up a couple more things. Please review.
> TestBalancer waits forever, errs without giving information
> -----------------------------------------------------------
>
> Key: HDFS-1808
> URL: https://issues.apache.org/jira/browse/HDFS-1808
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node, name-node
> Affects Versions: 0.22.0
> Reporter: Matt Foley
> Assignee: Matt Foley
> Attachments: 1808_TestBalancer_v3.patch, TestBalancer.java.patch
>
>
> In three locations in the code, TestBalancer waits forever on a condition. Failures result in Hudson/Jenkins "Timeout occurred" error message with no information about where or why. Need to replace with TimeoutExceptions that throw a stack trace and useful info about the failure mode.
> In waitForHeartBeat(), it is waiting on an exact match for a value that may be coarsely quantized -- i.e., significant deviation from the exact "expected" result may occur. Replace with an allowed range of result.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1808) TestBalancer waits forever, errs
without giving information
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HDFS-1808:
------------------------------
Fix Version/s: 0.23.0
> TestBalancer waits forever, errs without giving information
> -----------------------------------------------------------
>
> Key: HDFS-1808
> URL: https://issues.apache.org/jira/browse/HDFS-1808
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node, name-node
> Affects Versions: 0.22.0
> Reporter: Matt Foley
> Assignee: Matt Foley
> Fix For: 0.23.0
>
> Attachments: 1808_TestBalancer_v3.patch, TestBalancer.java.patch
>
>
> In three locations in the code, TestBalancer waits forever on a condition. Failures result in Hudson/Jenkins "Timeout occurred" error message with no information about where or why. Need to replace with TimeoutExceptions that throw a stack trace and useful info about the failure mode.
> In waitForHeartBeat(), it is waiting on an exact match for a value that may be coarsely quantized -- i.e., significant deviation from the exact "expected" result may occur. Replace with an allowed range of result.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1808) TestBalancer waits forever, errs
without giving information
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026291#comment-13026291 ]
Hudson commented on HDFS-1808:
------------------------------
Integrated in Hadoop-Hdfs-trunk #650 (See [https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/650/])
> TestBalancer waits forever, errs without giving information
> -----------------------------------------------------------
>
> Key: HDFS-1808
> URL: https://issues.apache.org/jira/browse/HDFS-1808
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node, name-node
> Affects Versions: 0.22.0
> Reporter: Matt Foley
> Assignee: Matt Foley
> Fix For: 0.23.0
>
> Attachments: 1808_TestBalancer_v3.patch, 1808_TestBalancer_v4.patch, TestBalancer.java.patch
>
>
> In three locations in the code, TestBalancer waits forever on a condition. Failures result in Hudson/Jenkins "Timeout occurred" error message with no information about where or why. Need to replace with TimeoutExceptions that throw a stack trace and useful info about the failure mode.
> In waitForHeartBeat(), it is waiting on an exact match for a value that may be coarsely quantized -- i.e., significant deviation from the exact "expected" result may occur. Replace with an allowed range of result.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1808) TestBalancer waits forever, errs
without giving information
Posted by "Matt Foley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt Foley updated HDFS-1808:
-----------------------------
Status: Patch Available (was: Open)
> TestBalancer waits forever, errs without giving information
> -----------------------------------------------------------
>
> Key: HDFS-1808
> URL: https://issues.apache.org/jira/browse/HDFS-1808
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node, name-node
> Affects Versions: 0.22.0
> Reporter: Matt Foley
> Assignee: Matt Foley
> Attachments: 1808_TestBalancer_v3.patch, TestBalancer.java.patch
>
>
> In three locations in the code, TestBalancer waits forever on a condition. Failures result in Hudson/Jenkins "Timeout occurred" error message with no information about where or why. Need to replace with TimeoutExceptions that throw a stack trace and useful info about the failure mode.
> In waitForHeartBeat(), it is waiting on an exact match for a value that may be coarsely quantized -- i.e., significant deviation from the exact "expected" result may occur. Replace with an allowed range of result.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1808) TestBalancer waits forever, errs
without giving information
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025626#comment-13025626 ]
Hudson commented on HDFS-1808:
------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #609 (See [https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/609/])
HDFS-1808. TestBalancer waits forever, errs without giving information. Contributed by Matt Foley
> TestBalancer waits forever, errs without giving information
> -----------------------------------------------------------
>
> Key: HDFS-1808
> URL: https://issues.apache.org/jira/browse/HDFS-1808
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node, name-node
> Affects Versions: 0.22.0
> Reporter: Matt Foley
> Assignee: Matt Foley
> Fix For: 0.23.0
>
> Attachments: 1808_TestBalancer_v3.patch, 1808_TestBalancer_v4.patch, TestBalancer.java.patch
>
>
> In three locations in the code, TestBalancer waits forever on a condition. Failures result in Hudson/Jenkins "Timeout occurred" error message with no information about where or why. Need to replace with TimeoutExceptions that throw a stack trace and useful info about the failure mode.
> In waitForHeartBeat(), it is waiting on an exact match for a value that may be coarsely quantized -- i.e., significant deviation from the exact "expected" result may occur. Replace with an allowed range of result.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1808) TestBalancer waits forever, errs
without giving information
Posted by "Matt Foley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt Foley updated HDFS-1808:
-----------------------------
Attachment: 1808_TestBalancer_v4.patch
Incorporated Eli's suggestions.
I do plan to commit this separately, in fact I broke the link from HDFS-1295 for all the unit test modules except TestDatanodeBlockScanner which is directly related to it.
> TestBalancer waits forever, errs without giving information
> -----------------------------------------------------------
>
> Key: HDFS-1808
> URL: https://issues.apache.org/jira/browse/HDFS-1808
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node, name-node
> Affects Versions: 0.22.0
> Reporter: Matt Foley
> Assignee: Matt Foley
> Fix For: 0.23.0
>
> Attachments: 1808_TestBalancer_v3.patch, 1808_TestBalancer_v4.patch, TestBalancer.java.patch
>
>
> In three locations in the code, TestBalancer waits forever on a condition. Failures result in Hudson/Jenkins "Timeout occurred" error message with no information about where or why. Need to replace with TimeoutExceptions that throw a stack trace and useful info about the failure mode.
> In waitForHeartBeat(), it is waiting on an exact match for a value that may be coarsely quantized -- i.e., significant deviation from the exact "expected" result may occur. Replace with an allowed range of result.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1808) TestBalancer waits forever, errs
without giving information
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13021682#comment-13021682 ]
Eli Collins commented on HDFS-1808:
-----------------------------------
Change looks good. If you don't mind let's commit this patch independently so we track different changes in different jiras. Minor comments:
Nits:
* Rather than pass timeout and allowedRange to waitForHeartBeat I'd have them be constants in the test function, seems like the tests shouldn't require different values for these parameters.
* Line 199 "0-0.01" should be "0-0.1"
* Line 205 needs indenting
> TestBalancer waits forever, errs without giving information
> -----------------------------------------------------------
>
> Key: HDFS-1808
> URL: https://issues.apache.org/jira/browse/HDFS-1808
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node, name-node
> Affects Versions: 0.22.0
> Reporter: Matt Foley
> Assignee: Matt Foley
> Attachments: 1808_TestBalancer_v3.patch, TestBalancer.java.patch
>
>
> In three locations in the code, TestBalancer waits forever on a condition. Failures result in Hudson/Jenkins "Timeout occurred" error message with no information about where or why. Need to replace with TimeoutExceptions that throw a stack trace and useful info about the failure mode.
> In waitForHeartBeat(), it is waiting on an exact match for a value that may be coarsely quantized -- i.e., significant deviation from the exact "expected" result may occur. Replace with an allowed range of result.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1808) TestBalancer waits forever, errs
without giving information
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HDFS-1808:
------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I've committed this. Thanks Matt!
> TestBalancer waits forever, errs without giving information
> -----------------------------------------------------------
>
> Key: HDFS-1808
> URL: https://issues.apache.org/jira/browse/HDFS-1808
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node, name-node
> Affects Versions: 0.22.0
> Reporter: Matt Foley
> Assignee: Matt Foley
> Fix For: 0.23.0
>
> Attachments: 1808_TestBalancer_v3.patch, 1808_TestBalancer_v4.patch, TestBalancer.java.patch
>
>
> In three locations in the code, TestBalancer waits forever on a condition. Failures result in Hudson/Jenkins "Timeout occurred" error message with no information about where or why. Need to replace with TimeoutExceptions that throw a stack trace and useful info about the failure mode.
> In waitForHeartBeat(), it is waiting on an exact match for a value that may be coarsely quantized -- i.e., significant deviation from the exact "expected" result may occur. Replace with an allowed range of result.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1808) TestBalancer waits forever, errs
without giving information
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13022561#comment-13022561 ]
Eli Collins commented on HDFS-1808:
-----------------------------------
+1 lgtm
> TestBalancer waits forever, errs without giving information
> -----------------------------------------------------------
>
> Key: HDFS-1808
> URL: https://issues.apache.org/jira/browse/HDFS-1808
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node, name-node
> Affects Versions: 0.22.0
> Reporter: Matt Foley
> Assignee: Matt Foley
> Fix For: 0.23.0
>
> Attachments: 1808_TestBalancer_v3.patch, 1808_TestBalancer_v4.patch, TestBalancer.java.patch
>
>
> In three locations in the code, TestBalancer waits forever on a condition. Failures result in Hudson/Jenkins "Timeout occurred" error message with no information about where or why. Need to replace with TimeoutExceptions that throw a stack trace and useful info about the failure mode.
> In waitForHeartBeat(), it is waiting on an exact match for a value that may be coarsely quantized -- i.e., significant deviation from the exact "expected" result may occur. Replace with an allowed range of result.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1808) TestBalancer waits forever, errs
without giving information
Posted by "Matt Foley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt Foley updated HDFS-1808:
-----------------------------
Attachment: TestBalancer.java.patch
Replace the three infinite waits with 20-second timeouts. Throw TimeoutException WITH useful information as to current state. Now if it errs in the future, at least we'll be able to see why.
Replace the exact expected value with a bounded range (which can be set to "exact" if you really want that).
Corrected a large number of comments that were out of date and incorrect.
Test now passes Hudson automated testing in connection with HDFS-1295, where it previously failed.
Posted here for information, but I'm going to subordinate this bug to HDFS-1295 (which it was blocking) and submit a single patch to that Jira.
> TestBalancer waits forever, errs without giving information
> -----------------------------------------------------------
>
> Key: HDFS-1808
> URL: https://issues.apache.org/jira/browse/HDFS-1808
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node, name-node
> Affects Versions: 0.22.0
> Reporter: Matt Foley
> Assignee: Matt Foley
> Attachments: TestBalancer.java.patch
>
>
> In three locations in the code, TestBalancer waits forever on a condition. Failures result in Hudson/Jenkins "Timeout occurred" error message with no information about where or why. Need to replace with TimeoutExceptions that throw a stack trace and useful info about the failure mode.
> In waitForHeartBeat(), it is waiting on an exact match for a value that may be coarsely quantized -- i.e., significant deviation from the exact "expected" result may occur. Replace with an allowed range of result.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1808) TestBalancer waits forever, errs
without giving information
Posted by "Matt Foley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13022825#comment-13022825 ]
Matt Foley commented on HDFS-1808:
----------------------------------
The javadoc error is due to HDFS-1857, and unrelated to this patch.
The unit test failure on TestFileConcurrentReader is unrelated to this patch.
Ready to commit.
> TestBalancer waits forever, errs without giving information
> -----------------------------------------------------------
>
> Key: HDFS-1808
> URL: https://issues.apache.org/jira/browse/HDFS-1808
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node, name-node
> Affects Versions: 0.22.0
> Reporter: Matt Foley
> Assignee: Matt Foley
> Fix For: 0.23.0
>
> Attachments: 1808_TestBalancer_v3.patch, 1808_TestBalancer_v4.patch, TestBalancer.java.patch
>
>
> In three locations in the code, TestBalancer waits forever on a condition. Failures result in Hudson/Jenkins "Timeout occurred" error message with no information about where or why. Need to replace with TimeoutExceptions that throw a stack trace and useful info about the failure mode.
> In waitForHeartBeat(), it is waiting on an exact match for a value that may be coarsely quantized -- i.e., significant deviation from the exact "expected" result may occur. Replace with an allowed range of result.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira