You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Lei Xu (JIRA)" <ji...@apache.org> on 2009/04/14 22:51:15 UTC
[jira] Created: (HADOOP-5672) Use custom MAX_SIZE_TO_MOVE value
Balancer
Use custom MAX_SIZE_TO_MOVE value Balancer
-------------------------------------------
Key: HADOOP-5672
URL: https://issues.apache.org/jira/browse/HADOOP-5672
Project: Hadoop Core
Issue Type: Improvement
Components: dfs
Reporter: Lei Xu
Assignee: Lei Xu
Priority: Minor
Fix For: 0.21.0
Balancer will load "fs.balancer.max.move.size" for a custom value of MAX_SIZE_TO_MOVE.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5672) Use custom MAX_SIZE_TO_MOVE value
Balancer
Posted by "Lei Xu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lei Xu updated HADOOP-5672:
---------------------------
Status: Patch Available (was: Open)
This patch is used for custom max size to move for HDFS balancer.
> Use custom MAX_SIZE_TO_MOVE value Balancer
> -------------------------------------------
>
> Key: HADOOP-5672
> URL: https://issues.apache.org/jira/browse/HADOOP-5672
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Lei Xu
> Assignee: Lei Xu
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: balancer-1.patch, balancer.patch
>
>
> Balancer will load "fs.balancer.max.move.size" for a custom value of MAX_SIZE_TO_MOVE.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5672) Use custom MAX_SIZE_TO_MOVE value
Balancer
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699091#action_12699091 ]
Hairong Kuang commented on HADOOP-5672:
---------------------------------------
1. Better declare maxSize as a field of Balancer and set its value in setConf.
2. Need to add the property to hdfs-default.xml.
> Use custom MAX_SIZE_TO_MOVE value Balancer
> -------------------------------------------
>
> Key: HADOOP-5672
> URL: https://issues.apache.org/jira/browse/HADOOP-5672
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Lei Xu
> Assignee: Lei Xu
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: balancer.patch
>
>
> Balancer will load "fs.balancer.max.move.size" for a custom value of MAX_SIZE_TO_MOVE.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5672) Use custom MAX_SIZE_TO_MOVE value
Balancer
Posted by "Lei Xu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lei Xu updated HADOOP-5672:
---------------------------
Attachment: balancer-1.patch
1, Add a field : Balancer.maxSizeToMove and set it in setConf
2. Add a entry "dfs.balancer.maxSizeToMove" in src/hdfs/hdfs-default.xml
> Use custom MAX_SIZE_TO_MOVE value Balancer
> -------------------------------------------
>
> Key: HADOOP-5672
> URL: https://issues.apache.org/jira/browse/HADOOP-5672
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Lei Xu
> Assignee: Lei Xu
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: balancer-1.patch, balancer.patch
>
>
> Balancer will load "fs.balancer.max.move.size" for a custom value of MAX_SIZE_TO_MOVE.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5672) Use custom MAX_SIZE_TO_MOVE value
Balancer
Posted by "Lei Xu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lei Xu updated HADOOP-5672:
---------------------------
Attachment: balancer-2.patch
1, refine the description of the property dfs.balancer.maxSizeToMove
2, set the initial value of maxSizeToMove to be MAX_SIZE_TO_MOVE;
3. Change class BalancerDatanode from "private static class" to "private class", and access Balancer.maxMoveSize directly
> Use custom MAX_SIZE_TO_MOVE value Balancer
> -------------------------------------------
>
> Key: HADOOP-5672
> URL: https://issues.apache.org/jira/browse/HADOOP-5672
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Lei Xu
> Assignee: Lei Xu
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: balancer-1.patch, balancer-2.patch, balancer.patch
>
>
> Balancer will load "fs.balancer.max.move.size" for a custom value of MAX_SIZE_TO_MOVE.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5672) Use custom MAX_SIZE_TO_MOVE value
Balancer
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700221#action_12700221 ]
Hadoop QA commented on HADOOP-5672:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12405574/balancer-1.patch
against trunk revision 765959.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/210/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/210/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/210/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/210/console
This message is automatically generated.
> Use custom MAX_SIZE_TO_MOVE value Balancer
> -------------------------------------------
>
> Key: HADOOP-5672
> URL: https://issues.apache.org/jira/browse/HADOOP-5672
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Lei Xu
> Assignee: Lei Xu
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: balancer-1.patch, balancer.patch
>
>
> Balancer will load "fs.balancer.max.move.size" for a custom value of MAX_SIZE_TO_MOVE.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5672) Use custom MAX_SIZE_TO_MOVE value
Balancer
Posted by "Lei Xu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710521#action_12710521 ]
Lei Xu commented on HADOOP-5672:
--------------------------------
Hi, all
this patch need be reviewed. Thanks
> Use custom MAX_SIZE_TO_MOVE value Balancer
> -------------------------------------------
>
> Key: HADOOP-5672
> URL: https://issues.apache.org/jira/browse/HADOOP-5672
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Lei Xu
> Assignee: Lei Xu
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: balancer-1.patch, balancer-2.patch, balancer.patch
>
>
> Balancer will load "fs.balancer.max.move.size" for a custom value of MAX_SIZE_TO_MOVE.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5672) Use custom MAX_SIZE_TO_MOVE value
Balancer
Posted by "Lei Xu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700224#action_12700224 ]
Lei Xu commented on HADOOP-5672:
--------------------------------
For the method signature changes, maxSizeToMove is a field of Balancer but not a field of BalancerDatanode. How could it get maxSizeToMove from the constructor of BalancerDatanode?
> Use custom MAX_SIZE_TO_MOVE value Balancer
> -------------------------------------------
>
> Key: HADOOP-5672
> URL: https://issues.apache.org/jira/browse/HADOOP-5672
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Lei Xu
> Assignee: Lei Xu
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: balancer-1.patch, balancer.patch
>
>
> Balancer will load "fs.balancer.max.move.size" for a custom value of MAX_SIZE_TO_MOVE.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5672) Use custom MAX_SIZE_TO_MOVE value
Balancer
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699925#action_12699925 ]
Hairong Kuang commented on HADOOP-5672:
---------------------------------------
1. better set the initial value of maxSizeToMove to be MAX_SIZE_TO_MOVE;
2. since maxSizeToMove is a field of Balancer, there is no need to change any method signature.
3. description of the property dfs.balancer.maxSizeToMove shoud specify that this is the max number of bytes to move per iteration.
> Use custom MAX_SIZE_TO_MOVE value Balancer
> -------------------------------------------
>
> Key: HADOOP-5672
> URL: https://issues.apache.org/jira/browse/HADOOP-5672
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Lei Xu
> Assignee: Lei Xu
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: balancer-1.patch, balancer.patch
>
>
> Balancer will load "fs.balancer.max.move.size" for a custom value of MAX_SIZE_TO_MOVE.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5672) Use custom MAX_SIZE_TO_MOVE value
Balancer
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700498#action_12700498 ]
Hairong Kuang commented on HADOOP-5672:
---------------------------------------
You could make BalancerDatanode not a static class.
> Use custom MAX_SIZE_TO_MOVE value Balancer
> -------------------------------------------
>
> Key: HADOOP-5672
> URL: https://issues.apache.org/jira/browse/HADOOP-5672
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Lei Xu
> Assignee: Lei Xu
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: balancer-1.patch, balancer.patch
>
>
> Balancer will load "fs.balancer.max.move.size" for a custom value of MAX_SIZE_TO_MOVE.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5672) Use custom MAX_SIZE_TO_MOVE value
Balancer
Posted by "Lei Xu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lei Xu updated HADOOP-5672:
---------------------------
Attachment: balancer.patch
add custom value for MAX_SIZE_TO_MOVE
> Use custom MAX_SIZE_TO_MOVE value Balancer
> -------------------------------------------
>
> Key: HADOOP-5672
> URL: https://issues.apache.org/jira/browse/HADOOP-5672
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Lei Xu
> Assignee: Lei Xu
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: balancer.patch
>
>
> Balancer will load "fs.balancer.max.move.size" for a custom value of MAX_SIZE_TO_MOVE.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.