You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org> on 2011/03/11 00:43:59 UTC
[jira] Created: (HDFS-1748) Balancer utilization classification is
not complete
Balancer utilization classification is not complete
---------------------------------------------------
Key: HDFS-1748
URL: https://issues.apache.org/jira/browse/HDFS-1748
Project: Hadoop HDFS
Issue Type: Bug
Components: balancer
Reporter: Tsz Wo (Nicholas), SZE
Attachments: h1748_20110310.patch
{code}
//Balancer.java
/* Return true if the given datanode is overUtilized */
private boolean isOverUtilized(BalancerDatanode datanode) {
return datanode.utilization > (avgUtilization+threshold);
}
/* Return true if the given datanode is above average utilized
* but not overUtilized */
private boolean isAboveAvgUtilized(BalancerDatanode datanode) {
return (datanode.utilization <= (avgUtilization+threshold))
&& (datanode.utilization > avgUtilization);
}
/* Return true if the given datanode is underUtilized */
private boolean isUnderUtilized(BalancerDatanode datanode) {
return datanode.utilization < (avgUtilization-threshold);
}
/* Return true if the given datanode is below average utilized
* but not underUtilized */
private boolean isBelowAvgUtilized(BalancerDatanode datanode) {
return (datanode.utilization >= (avgUtilization-threshold))
&& (datanode.utilization < avgUtilization);
}
{code}
Where is {{datanode.utilization == avgUtilization}}?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HDFS-1748) Balancer utilization classification
is incomplete
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13005948#comment-13005948 ]
Suresh Srinivas commented on HDFS-1748:
---------------------------------------
+1 for the patch. Patch looks good.
Minor comment: should belowAvgUtilizedDatanode be renamed belowOrEqualAvgUtilizedDatanode for consistency? However I am ok if you do not want to make this change.
> Balancer utilization classification is incomplete
> -------------------------------------------------
>
> Key: HDFS-1748
> URL: https://issues.apache.org/jira/browse/HDFS-1748
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h1748_20110310.patch
>
>
> {code}
> //Balancer.java
> /* Return true if the given datanode is overUtilized */
> private boolean isOverUtilized(BalancerDatanode datanode) {
> return datanode.utilization > (avgUtilization+threshold);
> }
>
> /* Return true if the given datanode is above average utilized
> * but not overUtilized */
> private boolean isAboveAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization <= (avgUtilization+threshold))
> && (datanode.utilization > avgUtilization);
> }
>
> /* Return true if the given datanode is underUtilized */
> private boolean isUnderUtilized(BalancerDatanode datanode) {
> return datanode.utilization < (avgUtilization-threshold);
> }
> /* Return true if the given datanode is below average utilized
> * but not underUtilized */
> private boolean isBelowAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization >= (avgUtilization-threshold))
> && (datanode.utilization < avgUtilization);
> }
> {code}
> Where is {{datanode.utilization == avgUtilization}}?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HDFS-1748) Balancer utilization classification
is incomplete
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13005476#comment-13005476 ]
Hadoop QA commented on HDFS-1748:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12473349/h1748_20110310.patch
against trunk revision 1080380.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these core unit tests:
org.apache.hadoop.hdfs.TestFileConcurrentReader
-1 contrib tests. The patch failed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/248//testReport/
Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/248//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/248//console
This message is automatically generated.
> Balancer utilization classification is incomplete
> -------------------------------------------------
>
> Key: HDFS-1748
> URL: https://issues.apache.org/jira/browse/HDFS-1748
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h1748_20110310.patch
>
>
> {code}
> //Balancer.java
> /* Return true if the given datanode is overUtilized */
> private boolean isOverUtilized(BalancerDatanode datanode) {
> return datanode.utilization > (avgUtilization+threshold);
> }
>
> /* Return true if the given datanode is above average utilized
> * but not overUtilized */
> private boolean isAboveAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization <= (avgUtilization+threshold))
> && (datanode.utilization > avgUtilization);
> }
>
> /* Return true if the given datanode is underUtilized */
> private boolean isUnderUtilized(BalancerDatanode datanode) {
> return datanode.utilization < (avgUtilization-threshold);
> }
> /* Return true if the given datanode is below average utilized
> * but not underUtilized */
> private boolean isBelowAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization >= (avgUtilization-threshold))
> && (datanode.utilization < avgUtilization);
> }
> {code}
> Where is {{datanode.utilization == avgUtilization}}?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HDFS-1748) Balancer utilization classification is
incomplete
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-1748:
-----------------------------------------
Resolution: Fixed
Fix Version/s: 0.23.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I have committed this. Thanks Suresh for the review.
> Balancer utilization classification is incomplete
> -------------------------------------------------
>
> Key: HDFS-1748
> URL: https://issues.apache.org/jira/browse/HDFS-1748
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.23.0
>
> Attachments: h1748_20110310.patch
>
>
> {code}
> //Balancer.java
> /* Return true if the given datanode is overUtilized */
> private boolean isOverUtilized(BalancerDatanode datanode) {
> return datanode.utilization > (avgUtilization+threshold);
> }
>
> /* Return true if the given datanode is above average utilized
> * but not overUtilized */
> private boolean isAboveAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization <= (avgUtilization+threshold))
> && (datanode.utilization > avgUtilization);
> }
>
> /* Return true if the given datanode is underUtilized */
> private boolean isUnderUtilized(BalancerDatanode datanode) {
> return datanode.utilization < (avgUtilization-threshold);
> }
> /* Return true if the given datanode is below average utilized
> * but not underUtilized */
> private boolean isBelowAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization >= (avgUtilization-threshold))
> && (datanode.utilization < avgUtilization);
> }
> {code}
> Where is {{datanode.utilization == avgUtilization}}?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HDFS-1748) Balancer utilization classification
is not complete
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13005416#comment-13005416 ]
Tsz Wo (Nicholas), SZE commented on HDFS-1748:
----------------------------------------------
The recent failures of {{TestBalancer}} seems related to this.
> Balancer utilization classification is not complete
> ---------------------------------------------------
>
> Key: HDFS-1748
> URL: https://issues.apache.org/jira/browse/HDFS-1748
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer
> Reporter: Tsz Wo (Nicholas), SZE
> Attachments: h1748_20110310.patch
>
>
> {code}
> //Balancer.java
> /* Return true if the given datanode is overUtilized */
> private boolean isOverUtilized(BalancerDatanode datanode) {
> return datanode.utilization > (avgUtilization+threshold);
> }
>
> /* Return true if the given datanode is above average utilized
> * but not overUtilized */
> private boolean isAboveAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization <= (avgUtilization+threshold))
> && (datanode.utilization > avgUtilization);
> }
>
> /* Return true if the given datanode is underUtilized */
> private boolean isUnderUtilized(BalancerDatanode datanode) {
> return datanode.utilization < (avgUtilization-threshold);
> }
> /* Return true if the given datanode is below average utilized
> * but not underUtilized */
> private boolean isBelowAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization >= (avgUtilization-threshold))
> && (datanode.utilization < avgUtilization);
> }
> {code}
> Where is {{datanode.utilization == avgUtilization}}?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HDFS-1748) Balancer utilization classification is
incomplete
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-1748:
-----------------------------------------
Fix Version/s: Federation Branch
Merged this (r1080836) from trunk to federation.
> Balancer utilization classification is incomplete
> -------------------------------------------------
>
> Key: HDFS-1748
> URL: https://issues.apache.org/jira/browse/HDFS-1748
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: Federation Branch, 0.23.0
>
> Attachments: h1748_20110310.patch
>
>
> {code}
> //Balancer.java
> /* Return true if the given datanode is overUtilized */
> private boolean isOverUtilized(BalancerDatanode datanode) {
> return datanode.utilization > (avgUtilization+threshold);
> }
>
> /* Return true if the given datanode is above average utilized
> * but not overUtilized */
> private boolean isAboveAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization <= (avgUtilization+threshold))
> && (datanode.utilization > avgUtilization);
> }
>
> /* Return true if the given datanode is underUtilized */
> private boolean isUnderUtilized(BalancerDatanode datanode) {
> return datanode.utilization < (avgUtilization-threshold);
> }
> /* Return true if the given datanode is below average utilized
> * but not underUtilized */
> private boolean isBelowAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization >= (avgUtilization-threshold))
> && (datanode.utilization < avgUtilization);
> }
> {code}
> Where is {{datanode.utilization == avgUtilization}}?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HDFS-1748) Balancer utilization classification is
incomplete
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-1748:
-----------------------------------------
Status: Patch Available (was: Open)
> Balancer utilization classification is incomplete
> -------------------------------------------------
>
> Key: HDFS-1748
> URL: https://issues.apache.org/jira/browse/HDFS-1748
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h1748_20110310.patch
>
>
> {code}
> //Balancer.java
> /* Return true if the given datanode is overUtilized */
> private boolean isOverUtilized(BalancerDatanode datanode) {
> return datanode.utilization > (avgUtilization+threshold);
> }
>
> /* Return true if the given datanode is above average utilized
> * but not overUtilized */
> private boolean isAboveAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization <= (avgUtilization+threshold))
> && (datanode.utilization > avgUtilization);
> }
>
> /* Return true if the given datanode is underUtilized */
> private boolean isUnderUtilized(BalancerDatanode datanode) {
> return datanode.utilization < (avgUtilization-threshold);
> }
> /* Return true if the given datanode is below average utilized
> * but not underUtilized */
> private boolean isBelowAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization >= (avgUtilization-threshold))
> && (datanode.utilization < avgUtilization);
> }
> {code}
> Where is {{datanode.utilization == avgUtilization}}?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HDFS-1748) Balancer utilization classification is
incomplete
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-1748:
-----------------------------------------
Assignee: Tsz Wo (Nicholas), SZE
Summary: Balancer utilization classification is incomplete (was: Balancer utilization classification is not complete)
> Balancer utilization classification is incomplete
> -------------------------------------------------
>
> Key: HDFS-1748
> URL: https://issues.apache.org/jira/browse/HDFS-1748
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h1748_20110310.patch
>
>
> {code}
> //Balancer.java
> /* Return true if the given datanode is overUtilized */
> private boolean isOverUtilized(BalancerDatanode datanode) {
> return datanode.utilization > (avgUtilization+threshold);
> }
>
> /* Return true if the given datanode is above average utilized
> * but not overUtilized */
> private boolean isAboveAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization <= (avgUtilization+threshold))
> && (datanode.utilization > avgUtilization);
> }
>
> /* Return true if the given datanode is underUtilized */
> private boolean isUnderUtilized(BalancerDatanode datanode) {
> return datanode.utilization < (avgUtilization-threshold);
> }
> /* Return true if the given datanode is below average utilized
> * but not underUtilized */
> private boolean isBelowAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization >= (avgUtilization-threshold))
> && (datanode.utilization < avgUtilization);
> }
> {code}
> Where is {{datanode.utilization == avgUtilization}}?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1748) Balancer utilization classification
is incomplete
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13022485#comment-13022485 ]
Hudson commented on HDFS-1748:
------------------------------
Integrated in Hadoop-Hdfs-trunk #643 (See [https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/643/])
> Balancer utilization classification is incomplete
> -------------------------------------------------
>
> Key: HDFS-1748
> URL: https://issues.apache.org/jira/browse/HDFS-1748
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: Federation Branch, 0.23.0
>
> Attachments: h1748_20110310.patch
>
>
> {code}
> //Balancer.java
> /* Return true if the given datanode is overUtilized */
> private boolean isOverUtilized(BalancerDatanode datanode) {
> return datanode.utilization > (avgUtilization+threshold);
> }
>
> /* Return true if the given datanode is above average utilized
> * but not overUtilized */
> private boolean isAboveAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization <= (avgUtilization+threshold))
> && (datanode.utilization > avgUtilization);
> }
>
> /* Return true if the given datanode is underUtilized */
> private boolean isUnderUtilized(BalancerDatanode datanode) {
> return datanode.utilization < (avgUtilization-threshold);
> }
> /* Return true if the given datanode is below average utilized
> * but not underUtilized */
> private boolean isBelowAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization >= (avgUtilization-threshold))
> && (datanode.utilization < avgUtilization);
> }
> {code}
> Where is {{datanode.utilization == avgUtilization}}?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HDFS-1748) Balancer utilization classification is
not complete
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-1748:
-----------------------------------------
Attachment: h1748_20110310.patch
h1748_20110310.patch: change isBelowAvgUtilized to isBelowOrEqualAvgUtilized
> Balancer utilization classification is not complete
> ---------------------------------------------------
>
> Key: HDFS-1748
> URL: https://issues.apache.org/jira/browse/HDFS-1748
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer
> Reporter: Tsz Wo (Nicholas), SZE
> Attachments: h1748_20110310.patch
>
>
> {code}
> //Balancer.java
> /* Return true if the given datanode is overUtilized */
> private boolean isOverUtilized(BalancerDatanode datanode) {
> return datanode.utilization > (avgUtilization+threshold);
> }
>
> /* Return true if the given datanode is above average utilized
> * but not overUtilized */
> private boolean isAboveAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization <= (avgUtilization+threshold))
> && (datanode.utilization > avgUtilization);
> }
>
> /* Return true if the given datanode is underUtilized */
> private boolean isUnderUtilized(BalancerDatanode datanode) {
> return datanode.utilization < (avgUtilization-threshold);
> }
> /* Return true if the given datanode is below average utilized
> * but not underUtilized */
> private boolean isBelowAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization >= (avgUtilization-threshold))
> && (datanode.utilization < avgUtilization);
> }
> {code}
> Where is {{datanode.utilization == avgUtilization}}?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HDFS-1748) Balancer utilization classification
is incomplete
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13005970#comment-13005970 ]
Hudson commented on HDFS-1748:
------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #558 (See [https://hudson.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/558/])
HDFS-1748. Balancer utilization classification is incomplete.
> Balancer utilization classification is incomplete
> -------------------------------------------------
>
> Key: HDFS-1748
> URL: https://issues.apache.org/jira/browse/HDFS-1748
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.23.0
>
> Attachments: h1748_20110310.patch
>
>
> {code}
> //Balancer.java
> /* Return true if the given datanode is overUtilized */
> private boolean isOverUtilized(BalancerDatanode datanode) {
> return datanode.utilization > (avgUtilization+threshold);
> }
>
> /* Return true if the given datanode is above average utilized
> * but not overUtilized */
> private boolean isAboveAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization <= (avgUtilization+threshold))
> && (datanode.utilization > avgUtilization);
> }
>
> /* Return true if the given datanode is underUtilized */
> private boolean isUnderUtilized(BalancerDatanode datanode) {
> return datanode.utilization < (avgUtilization-threshold);
> }
> /* Return true if the given datanode is below average utilized
> * but not underUtilized */
> private boolean isBelowAvgUtilized(BalancerDatanode datanode) {
> return (datanode.utilization >= (avgUtilization-threshold))
> && (datanode.utilization < avgUtilization);
> }
> {code}
> Where is {{datanode.utilization == avgUtilization}}?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira