You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by sunweiwei <su...@asiainfo-linkage.com> on 2014/06/17 05:19:03 UTC
regions not balance?
Hi
I have several regionservers. One regionserver has 434 regions and other
regionservers only have about 300+ regions.
I think it's not balance. Then I run balancer command in hbase shell
and return true, but It not works.
The regionserver always has 400+ regions. Is this normal?
Thanks
答复: regions not balance?
Posted by sunweiwei <su...@asiainfo-linkage.com>.
Thanks all
-----邮件原件-----
发件人: Jean-Marc Spaggiari [mailto:jean-marc@spaggiari.org]
发送时间: 2014年6月17日 19:25
收件人: user
主题: Re: regions not balance?
Also, you can set it to TRACE logs from the WebUI. That will help you to
see the logic displayed on the logs...
JM
2014-06-17 1:23 GMT-04:00 duanp <du...@neusoft.com>:
>
> I think that the balancer may not need to be executed. you can calculate
> the value with parameters you set.
>
> before, i have the same question as yours .
>
> In the 0.96 , org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer
> has a method BaseLoadBalancer.needsBalance determine it .
>
>
>
> protected boolean needsBalance(ClusterLoadState cs) {
> if (cs.getNumServers() < MIN_SERVER_BALANCE) {
> if (LOG.isDebugEnabled()) {
> LOG.debug("Not running balancer because only " + cs.getNumServers()
> + " active regionserver(s)");
> }
> return false;
> }
> // Check if we even need to do any load balancing
> // HBASE-3681 check sloppiness first
> float average = cs.getLoadAverage(); // for logging numRegions /
> numServers;
> int floor = (int) Math.floor(average * (1 - slop)); //
> hbase.regions.slop, default 0.2
> int ceiling = (int) Math.ceil(average * (1 + slop));
> if (!(cs.getMaxLoad() > ceiling || cs.getMinLoad() < floor)) { //
> NavigableMap<ServerAndLoad, List<HRegionInfo>> serversByLoad =
> cs.getServersByLoad();
> if (LOG.isTraceEnabled()) {
> // If nothing to balance, then don't say anything unless
> trace-level logging.
> LOG.trace("Skipping load balancing because balanced cluster; " +
> "servers=" + cs.getNumServers() + " " +
> "regions=" + cs.getNumRegions() + " average=" + average + " " +
> "mostloaded=" + serversByLoad.lastKey().getLoad() +
> " leastloaded=" + serversByLoad.firstKey().getLoad());
> }
> return false;
> }
> return true;
> }
>
>
>
> 发件人： sunweiwei
> 发送时间： 2014-06-17 12:55
> 收件人： user@hbase.apache.org
> 主题： 答复: regions not balance?
> I'm using hbase0.96.
> There is no balancer related logs in master log at that moment.
> I find Regions-in-transition is empty in web UI's debug dump.
>
> Thanks
>
>
> -----邮件原件-----
> 发件人: Ted Yu [mailto:yuzhihong@gmail.com]
> 发送时间: 2014年6月17日 11:41
> 收件人: user@hbase.apache.org
> 主题: Re: regions not balance?
>
> bq. and return true, but It not works
>
> Looking at master log, do you see balancer related logs around this time ?
> If there was region in transition at this moment, balancer wouldn't perform
> balancing.
>
> What version of HBase are you using ?
>
> Cheers
>
>
> On Mon, Jun 16, 2014 at 8:19 PM, sunweiwei <su...@asiainfo-linkage.com>
> wrote:
>
> > Hi
> >
> > I have several regionservers. One regionserver has 434 regions and
> > other
> > regionservers only have about 300+ regions.
> >
> > I think it's not balance. Then I run balancer command in hbase shell
> > and return true, but It not works.
> >
> > The regionserver always has 400+ regions. Is this normal?
> >
> >
> >
> > Thanks
> >
> >
>
> ---------------------------------------------------------------------------------------------------
> Confidentiality Notice: The information contained in this e-mail and any
> accompanying attachment(s)
> is intended only for the use of the intended recipient and may be
> confidential and/or privileged of
> Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader
> of this communication is
> not the intended recipient, unauthorized use, forwarding, printing,
> storing, disclosure or copying
> is strictly prohibited, and may be unlawful.If you have received this
> communication in error,please
> immediately notify the sender by return e-mail, and delete the original
> message and all copies from
> your system. Thank you.
>
> ---------------------------------------------------------------------------------------------------
>
Re: regions not balance?
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Also, you can set it to TRACE logs from the WebUI. That will help you to
see the logic displayed on the logs...
JM
2014-06-17 1:23 GMT-04:00 duanp <du...@neusoft.com>:
>
> I think that the balancer may not need to be executed. you can calculate
> the value with parameters you set.
>
> before, i have the same question as yours .
>
> In the 0.96 , org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer
> has a method BaseLoadBalancer.needsBalance determine it .
>
>
>
> protected boolean needsBalance(ClusterLoadState cs) {
> if (cs.getNumServers() < MIN_SERVER_BALANCE) {
> if (LOG.isDebugEnabled()) {
> LOG.debug("Not running balancer because only " + cs.getNumServers()
> + " active regionserver(s)");
> }
> return false;
> }
> // Check if we even need to do any load balancing
> // HBASE-3681 check sloppiness first
> float average = cs.getLoadAverage(); // for logging numRegions /
> numServers;
> int floor = (int) Math.floor(average * (1 - slop)); //
> hbase.regions.slop, default 0.2
> int ceiling = (int) Math.ceil(average * (1 + slop));
> if (!(cs.getMaxLoad() > ceiling || cs.getMinLoad() < floor)) { //
> NavigableMap<ServerAndLoad, List<HRegionInfo>> serversByLoad =
> cs.getServersByLoad();
> if (LOG.isTraceEnabled()) {
> // If nothing to balance, then don't say anything unless
> trace-level logging.
> LOG.trace("Skipping load balancing because balanced cluster; " +
> "servers=" + cs.getNumServers() + " " +
> "regions=" + cs.getNumRegions() + " average=" + average + " " +
> "mostloaded=" + serversByLoad.lastKey().getLoad() +
> " leastloaded=" + serversByLoad.firstKey().getLoad());
> }
> return false;
> }
> return true;
> }
>
>
>
> 发件人： sunweiwei
> 发送时间： 2014-06-17 12:55
> 收件人： user@hbase.apache.org
> 主题： 答复: regions not balance?
> I'm using hbase0.96.
> There is no balancer related logs in master log at that moment.
> I find Regions-in-transition is empty in web UI's debug dump.
>
> Thanks
>
>
> -----邮件原件-----
> 发件人: Ted Yu [mailto:yuzhihong@gmail.com]
> 发送时间: 2014年6月17日 11:41
> 收件人: user@hbase.apache.org
> 主题: Re: regions not balance?
>
> bq. and return true, but It not works
>
> Looking at master log, do you see balancer related logs around this time ?
> If there was region in transition at this moment, balancer wouldn't perform
> balancing.
>
> What version of HBase are you using ?
>
> Cheers
>
>
> On Mon, Jun 16, 2014 at 8:19 PM, sunweiwei <su...@asiainfo-linkage.com>
> wrote:
>
> > Hi
> >
> > I have several regionservers. One regionserver has 434 regions and
> > other
> > regionservers only have about 300+ regions.
> >
> > I think it's not balance. Then I run balancer command in hbase shell
> > and return true, but It not works.
> >
> > The regionserver always has 400+ regions. Is this normal?
> >
> >
> >
> > Thanks
> >
> >
>
> ---------------------------------------------------------------------------------------------------
> Confidentiality Notice: The information contained in this e-mail and any
> accompanying attachment(s)
> is intended only for the use of the intended recipient and may be
> confidential and/or privileged of
> Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader
> of this communication is
> not the intended recipient, unauthorized use, forwarding, printing,
> storing, disclosure or copying
> is strictly prohibited, and may be unlawful.If you have received this
> communication in error,please
> immediately notify the sender by return e-mail, and delete the original
> message and all copies from
> your system. Thank you.
>
> ---------------------------------------------------------------------------------------------------
>
Re: regions not balance?
Posted by duanp <du...@neusoft.com>.
I think that the balancer may not need to be executed. you can calculate the value with parameters you set.
before, i have the same question as yours .
In the 0.96 , org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer has a method BaseLoadBalancer.needsBalance determine it .
protected boolean needsBalance(ClusterLoadState cs) {
if (cs.getNumServers() < MIN_SERVER_BALANCE) {
if (LOG.isDebugEnabled()) {
LOG.debug("Not running balancer because only " + cs.getNumServers()
+ " active regionserver(s)");
}
return false;
}
// Check if we even need to do any load balancing
// HBASE-3681 check sloppiness first
float average = cs.getLoadAverage(); // for logging numRegions / numServers;
int floor = (int) Math.floor(average * (1 - slop)); // hbase.regions.slop, default 0.2
int ceiling = (int) Math.ceil(average * (1 + slop));
if (!(cs.getMaxLoad() > ceiling || cs.getMinLoad() < floor)) { //
NavigableMap<ServerAndLoad, List<HRegionInfo>> serversByLoad = cs.getServersByLoad();
if (LOG.isTraceEnabled()) {
// If nothing to balance, then don't say anything unless trace-level logging.
LOG.trace("Skipping load balancing because balanced cluster; " +
"servers=" + cs.getNumServers() + " " +
"regions=" + cs.getNumRegions() + " average=" + average + " " +
"mostloaded=" + serversByLoad.lastKey().getLoad() +
" leastloaded=" + serversByLoad.firstKey().getLoad());
}
return false;
}
return true;
}
发件人： sunweiwei
发送时间： 2014-06-17 12:55
收件人： user@hbase.apache.org
主题： 答复: regions not balance?
I'm using hbase0.96.
There is no balancer related logs in master log at that moment.
I find Regions-in-transition is empty in web UI's debug dump.
Thanks
-----邮件原件-----
发件人: Ted Yu [mailto:yuzhihong@gmail.com]
发送时间: 2014年6月17日 11:41
收件人: user@hbase.apache.org
主题: Re: regions not balance?
bq. and return true, but It not works
Looking at master log, do you see balancer related logs around this time ?
If there was region in transition at this moment, balancer wouldn't perform
balancing.
What version of HBase are you using ?
Cheers
On Mon, Jun 16, 2014 at 8:19 PM, sunweiwei <su...@asiainfo-linkage.com>
wrote:
> Hi
>
> I have several regionservers. One regionserver has 434 regions and
> other
> regionservers only have about 300+ regions.
>
> I think it's not balance. Then I run balancer command in hbase shell
> and return true, but It not works.
>
> The regionserver always has 400+ regions. Is this normal?
>
>
>
> Thanks
>
>
---------------------------------------------------------------------------------------------------
Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s)
is intended only for the use of the intended recipient and may be confidential and/or privileged of
Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication is
not the intended recipient, unauthorized use, forwarding, printing, storing, disclosure or copying
is strictly prohibited, and may be unlawful.If you have received this communication in error,please
immediately notify the sender by return e-mail, and delete the original message and all copies from
your system. Thank you.
---------------------------------------------------------------------------------------------------
答复: regions not balance?
Posted by sunweiwei <su...@asiainfo-linkage.com>.
I'm using hbase0.96.
There is no balancer related logs in master log at that moment.
I find Regions-in-transition is empty in web UI's debug dump.
Thanks
-----邮件原件-----
发件人: Ted Yu [mailto:yuzhihong@gmail.com]
发送时间: 2014年6月17日 11:41
收件人: user@hbase.apache.org
主题: Re: regions not balance?
bq. and return true, but It not works
Looking at master log, do you see balancer related logs around this time ?
If there was region in transition at this moment, balancer wouldn't perform
balancing.
What version of HBase are you using ?
Cheers
On Mon, Jun 16, 2014 at 8:19 PM, sunweiwei <su...@asiainfo-linkage.com>
wrote:
> Hi
>
> I have several regionservers. One regionserver has 434 regions and
> other
> regionservers only have about 300+ regions.
>
> I think it's not balance. Then I run balancer command in hbase shell
> and return true, but It not works.
>
> The regionserver always has 400+ regions. Is this normal?
>
>
>
> Thanks
>
>
Re: regions not balance?
Posted by Ted Yu <yu...@gmail.com>.
bq. and return true, but It not works
Looking at master log, do you see balancer related logs around this time ?
If there was region in transition at this moment, balancer wouldn't perform
balancing.
What version of HBase are you using ?
Cheers
On Mon, Jun 16, 2014 at 8:19 PM, sunweiwei <su...@asiainfo-linkage.com>
wrote:
> Hi
>
> I have several regionservers. One regionserver has 434 regions and
> other
> regionservers only have about 300+ regions.
>
> I think it's not balance. Then I run balancer command in hbase shell
> and return true, but It not works.
>
> The regionserver always has 400+ regions. Is this normal?
>
>
>
> Thanks
>
>