You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Ted Yu (JIRA)" <ji...@apache.org> on 2011/04/19 01:30:05 UTC
[jira] [Created] (HBASE-3799) Provide more accurate check for super
underloaded region server in load balancer
Provide more accurate check for super underloaded region server in load balancer
--------------------------------------------------------------------------------
Key: HBASE-3799
URL: https://issues.apache.org/jira/browse/HBASE-3799
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.90.2
Reporter: Ted Yu
Assignee: Ted Yu
HBASE-3609 used simple check for region server which recently joined the cluster so that both young and old regions from other region servers are assigned to it.
The check was too strict.
1 or more region may be assigned to this server before load balancer performs rebalancing.
The next time balancer runs, it wouldn't treat this server as seriously underloaded correctly and assign a lot of young regions to it.
We can use threshold over the number of regions to avoid such issue.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3799) Provide more accurate check for super
underloaded region server in load balancer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-3799:
--------------------------
Attachment: 3799.patch
> Provide more accurate check for super underloaded region server in load balancer
> --------------------------------------------------------------------------------
>
> Key: HBASE-3799
> URL: https://issues.apache.org/jira/browse/HBASE-3799
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.2
> Reporter: Ted Yu
> Assignee: Ted Yu
> Attachments: 3799.patch
>
>
> HBASE-3609 used simple check for region server which recently joined the cluster so that both young and old regions from other region servers are assigned to it.
> The check was too strict.
> 1 or more region may be assigned to this server before load balancer performs rebalancing.
> The next time balancer runs, it wouldn't treat this server as seriously underloaded correctly and assign a lot of young regions to it.
> We can use threshold over the number of regions to avoid such issue.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3799) Provide more accurate check for
super underloaded region server in load balancer
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455353#comment-13455353 ]
Lars Hofhansl commented on HBASE-3799:
--------------------------------------
@Ted: Do you still want this?
> Provide more accurate check for super underloaded region server in load balancer
> --------------------------------------------------------------------------------
>
> Key: HBASE-3799
> URL: https://issues.apache.org/jira/browse/HBASE-3799
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.2
> Reporter: Ted Yu
> Assignee: Ted Yu
> Attachments: 3799.patch
>
>
> HBASE-3609 used simple check for region server which recently joined the cluster so that both young and old regions from other region servers are assigned to it.
> The check was too strict.
> 1 or more region may be assigned to this server before load balancer performs rebalancing.
> The next time balancer runs, it wouldn't treat this server as seriously underloaded correctly and assign a lot of young regions to it.
> We can use threshold over the number of regions to avoid such issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-3799) Provide more accurate check for
super underloaded region server in load balancer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu resolved HBASE-3799.
---------------------------
Resolution: Later
> Provide more accurate check for super underloaded region server in load balancer
> --------------------------------------------------------------------------------
>
> Key: HBASE-3799
> URL: https://issues.apache.org/jira/browse/HBASE-3799
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.2
> Reporter: Ted Yu
> Assignee: Ted Yu
> Attachments: 3799.patch
>
>
> HBASE-3609 used simple check for region server which recently joined the cluster so that both young and old regions from other region servers are assigned to it.
> The check was too strict.
> 1 or more region may be assigned to this server before load balancer performs rebalancing.
> The next time balancer runs, it wouldn't treat this server as seriously underloaded correctly and assign a lot of young regions to it.
> We can use threshold over the number of regions to avoid such issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3799) Provide more accurate check for
super underloaded region server in load balancer
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13021661#comment-13021661 ]
stack commented on HBASE-3799:
------------------------------
A general comment on balancing (that probably fits better elsewhere than as a comment on this issue) is that we need 'smoothing' of region move.... Yesterday we brought a regionserver back online into a smallish cluster that was under load and the balance run unloaded a bunch of regions all in the one go which put a dent in the throughput; it'd be sweet if the balancer ran at an appropriate 'rate'. When under load, it should move regions 'gently' rather than all as a big bang (the decommission script will move a region at a time, verifying it deployed in its new location before moving another... this can take ages to complete but its proven minimally disruptive to loadings)
> Provide more accurate check for super underloaded region server in load balancer
> --------------------------------------------------------------------------------
>
> Key: HBASE-3799
> URL: https://issues.apache.org/jira/browse/HBASE-3799
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.2
> Reporter: Ted Yu
> Assignee: Ted Yu
>
> HBASE-3609 used simple check for region server which recently joined the cluster so that both young and old regions from other region servers are assigned to it.
> The check was too strict.
> 1 or more region may be assigned to this server before load balancer performs rebalancing.
> The next time balancer runs, it wouldn't treat this server as seriously underloaded correctly and assign a lot of young regions to it.
> We can use threshold over the number of regions to avoid such issue.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3799) Provide more accurate check for
super underloaded region server in load balancer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13021777#comment-13021777 ]
Ted Yu commented on HBASE-3799:
-------------------------------
We can limit the number (percentage) of regions that are moved off a single region server.
Should that be part of this JIRA ?
> Provide more accurate check for super underloaded region server in load balancer
> --------------------------------------------------------------------------------
>
> Key: HBASE-3799
> URL: https://issues.apache.org/jira/browse/HBASE-3799
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.2
> Reporter: Ted Yu
> Assignee: Ted Yu
> Attachments: 3799.patch
>
>
> HBASE-3609 used simple check for region server which recently joined the cluster so that both young and old regions from other region servers are assigned to it.
> The check was too strict.
> 1 or more region may be assigned to this server before load balancer performs rebalancing.
> The next time balancer runs, it wouldn't treat this server as seriously underloaded correctly and assign a lot of young regions to it.
> We can use threshold over the number of regions to avoid such issue.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HBASE-3799) Provide more accurate check for
super underloaded region server in load balancer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HBASE-3799 started by Ted Yu.
> Provide more accurate check for super underloaded region server in load balancer
> --------------------------------------------------------------------------------
>
> Key: HBASE-3799
> URL: https://issues.apache.org/jira/browse/HBASE-3799
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.2
> Reporter: Ted Yu
> Assignee: Ted Yu
> Attachments: 3799.patch
>
>
> HBASE-3609 used simple check for region server which recently joined the cluster so that both young and old regions from other region servers are assigned to it.
> The check was too strict.
> 1 or more region may be assigned to this server before load balancer performs rebalancing.
> The next time balancer runs, it wouldn't treat this server as seriously underloaded correctly and assign a lot of young regions to it.
> We can use threshold over the number of regions to avoid such issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3799) Provide more accurate check for
super underloaded region server in load balancer
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13022300#comment-13022300 ]
stack commented on HBASE-3799:
------------------------------
Shouldn't be part of this issue. I'll move my comment to a new issue.
> Provide more accurate check for super underloaded region server in load balancer
> --------------------------------------------------------------------------------
>
> Key: HBASE-3799
> URL: https://issues.apache.org/jira/browse/HBASE-3799
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.2
> Reporter: Ted Yu
> Assignee: Ted Yu
> Attachments: 3799.patch
>
>
> HBASE-3609 used simple check for region server which recently joined the cluster so that both young and old regions from other region servers are assigned to it.
> The check was too strict.
> 1 or more region may be assigned to this server before load balancer performs rebalancing.
> The next time balancer runs, it wouldn't treat this server as seriously underloaded correctly and assign a lot of young regions to it.
> We can use threshold over the number of regions to avoid such issue.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3799) Provide more accurate check for
super underloaded region server in load balancer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455363#comment-13455363 ]
Ted Yu commented on HBASE-3799:
-------------------------------
The notion of young vs. old regions can be more accurately described with various metrics for read / write on regions.
I will close this JIRA for now.
> Provide more accurate check for super underloaded region server in load balancer
> --------------------------------------------------------------------------------
>
> Key: HBASE-3799
> URL: https://issues.apache.org/jira/browse/HBASE-3799
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.2
> Reporter: Ted Yu
> Assignee: Ted Yu
> Attachments: 3799.patch
>
>
> HBASE-3609 used simple check for region server which recently joined the cluster so that both young and old regions from other region servers are assigned to it.
> The check was too strict.
> 1 or more region may be assigned to this server before load balancer performs rebalancing.
> The next time balancer runs, it wouldn't treat this server as seriously underloaded correctly and assign a lot of young regions to it.
> We can use threshold over the number of regions to avoid such issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira