You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "ramkrishna.s.vasudevan (Created) (JIRA)" <ji...@apache.org> on 2012/02/27 09:39:48 UTC
[jira] [Created] (HBASE-5482) Balancer in 0.90 algo leading to same
region balanced twice and picking same region with Src and Destination as
same RS.
Balancer in 0.90 algo leading to same region balanced twice and picking same region with Src and Destination as same RS.
------------------------------------------------------------------------------------------------------------------------
Key: HBASE-5482
URL: https://issues.apache.org/jira/browse/HBASE-5482
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.90.5
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.90.7
There are possibility of 2 problems
-> When we populate regionsToMove while iterating the serverinfo in descending manner there is a chance that the same region can be added twice.
Because in the first loop we do a randomization of the regions.
Where as when we get we have neededRegions!= 0 we just get the region in the index and add it again . This may lead to have same region in the regionsToMove list.
-> Another problem is
when the problem in the first point happens then there is a chance that
the regionToMove can have the same src and destination and the same region can be picked every 5 mins.
{code}
for(Map.Entry<HServerInfo, List<HRegionInfo>> server :
serversByLoad.descendingMap().entrySet()) {
BalanceInfo balanceInfo = serverBalanceInfo.get(server.getKey());
int idx =
balanceInfo == null ? 0 : balanceInfo.getNextRegionForUnload();
if (idx >= server.getValue().size()) break;
HRegionInfo region = server.getValue().get(idx);
if (region.isMetaRegion()) continue; // Don't move meta regions.
regionsToMove.add(new RegionPlan(region, server.getKey(), null));
if(--neededRegions == 0) {
// No more regions needed, done shedding
break;
}
}
{code}
If i have meta and root in the top two loaded region server(totally 3 RS), we just skip the regions in those region server and populate the region from the least loaded RS.
Then in the next loop we iterate from the least loaded server and populate the destination as also the same server.
This is leading to a condition where every 5 min balancing happens and also the server is same for src and dest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5482) In 0.90, balancer algo leading to
same region balanced twice and picking same region with Src and Destination
as same RS.
Posted by "ramkrishna.s.vasudevan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan updated HBASE-5482:
------------------------------------------
Attachment: HBASE-5482_1.patch
Patch for 0.90.
> In 0.90, balancer algo leading to same region balanced twice and picking same region with Src and Destination as same RS.
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5482
> URL: https://issues.apache.org/jira/browse/HBASE-5482
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.90.5
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.90.7
>
> Attachments: HBASE-5482_1.patch
>
>
> There are possibility of 2 problems
> -> When we populate regionsToMove while iterating the serverinfo in descending manner there is a chance that the same region can be added twice.
> Because in the first loop we do a randomization of the regions.
> Where as when we get we have neededRegions!= 0 we just get the region in the index and add it again . This may lead to have same region in the regionsToMove list.
> -> Another problem is
> when the problem in the first point happens then there is a chance that
> the regionToMove can have the same src and destination and the same region can be picked every 5 mins.
> {code}
> for(Map.Entry<HServerInfo, List<HRegionInfo>> server :
> serversByLoad.descendingMap().entrySet()) {
> BalanceInfo balanceInfo = serverBalanceInfo.get(server.getKey());
> int idx =
> balanceInfo == null ? 0 : balanceInfo.getNextRegionForUnload();
> if (idx >= server.getValue().size()) break;
> HRegionInfo region = server.getValue().get(idx);
> if (region.isMetaRegion()) continue; // Don't move meta regions.
> regionsToMove.add(new RegionPlan(region, server.getKey(), null));
> if(--neededRegions == 0) {
> // No more regions needed, done shedding
> break;
> }
> }
> {code}
> If i have meta and root in the top two loaded region server(totally 3 RS), we just skip the regions in those region server and populate the region from the least loaded RS.
> Then in the next loop we iterate from the least loaded server and populate the destination as also the same server.
> This is leading to a condition where every 5 min balancing happens and also the server is same for src and dest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5482) In 0.90, balancer algo leading to
same region balanced twice and picking same region with Src and Destination
as same RS.
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13232757#comment-13232757 ]
ramkrishna.s.vasudevan commented on HBASE-5482:
-----------------------------------------------
Committed to 0.90. Thanks for the review Ted.
> In 0.90, balancer algo leading to same region balanced twice and picking same region with Src and Destination as same RS.
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5482
> URL: https://issues.apache.org/jira/browse/HBASE-5482
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.90.5
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.90.7
>
> Attachments: 5482-v2.txt, HBASE-5482_1.patch, HBASE-5482_2.patch
>
>
> There are possibility of 2 problems
> -> When we populate regionsToMove while iterating the serverinfo in descending manner there is a chance that the same region can be added twice.
> Because in the first loop we do a randomization of the regions.
> Where as when we get we have neededRegions!= 0 we just get the region in the index and add it again . This may lead to have same region in the regionsToMove list.
> -> Another problem is
> when the problem in the first point happens then there is a chance that
> the regionToMove can have the same src and destination and the same region can be picked every 5 mins.
> {code}
> for(Map.Entry<HServerInfo, List<HRegionInfo>> server :
> serversByLoad.descendingMap().entrySet()) {
> BalanceInfo balanceInfo = serverBalanceInfo.get(server.getKey());
> int idx =
> balanceInfo == null ? 0 : balanceInfo.getNextRegionForUnload();
> if (idx >= server.getValue().size()) break;
> HRegionInfo region = server.getValue().get(idx);
> if (region.isMetaRegion()) continue; // Don't move meta regions.
> regionsToMove.add(new RegionPlan(region, server.getKey(), null));
> if(--neededRegions == 0) {
> // No more regions needed, done shedding
> break;
> }
> }
> {code}
> If i have meta and root in the top two loaded region server(totally 3 RS), we just skip the regions in those region server and populate the region from the least loaded RS.
> Then in the next loop we iterate from the least loaded server and populate the destination as also the same server.
> This is leading to a condition where every 5 min balancing happens and also the server is same for src and dest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5482) In 0.90, balancer algo leading to
same region balanced twice and picking same region with Src and Destination
as same RS.
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219352#comment-13219352 ]
Zhihong Yu commented on HBASE-5482:
-----------------------------------
Patch looks good.
Please add curly braces around break statement:
{code}
+ if (idx >= server.getValue().size())
+ break;
{code}
In:
{code}
+ public void testBalanceClusterWithFirstRegionsRootMeta() throws Exception {
{code}
the following should end with 'META are':
{code}
+ // do not randomize in order to create a scenario where ROOT and META is
{code}
{code}
+ public void testPlanContainsDuplicateRegions() throws Exception {
{code}
Would testNoDuplicateRegionsInPlan() be a good name ?
> In 0.90, balancer algo leading to same region balanced twice and picking same region with Src and Destination as same RS.
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5482
> URL: https://issues.apache.org/jira/browse/HBASE-5482
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.90.5
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.90.7
>
> Attachments: 5482-v2.txt, HBASE-5482_1.patch
>
>
> There are possibility of 2 problems
> -> When we populate regionsToMove while iterating the serverinfo in descending manner there is a chance that the same region can be added twice.
> Because in the first loop we do a randomization of the regions.
> Where as when we get we have neededRegions!= 0 we just get the region in the index and add it again . This may lead to have same region in the regionsToMove list.
> -> Another problem is
> when the problem in the first point happens then there is a chance that
> the regionToMove can have the same src and destination and the same region can be picked every 5 mins.
> {code}
> for(Map.Entry<HServerInfo, List<HRegionInfo>> server :
> serversByLoad.descendingMap().entrySet()) {
> BalanceInfo balanceInfo = serverBalanceInfo.get(server.getKey());
> int idx =
> balanceInfo == null ? 0 : balanceInfo.getNextRegionForUnload();
> if (idx >= server.getValue().size()) break;
> HRegionInfo region = server.getValue().get(idx);
> if (region.isMetaRegion()) continue; // Don't move meta regions.
> regionsToMove.add(new RegionPlan(region, server.getKey(), null));
> if(--neededRegions == 0) {
> // No more regions needed, done shedding
> break;
> }
> }
> {code}
> If i have meta and root in the top two loaded region server(totally 3 RS), we just skip the regions in those region server and populate the region from the least loaded RS.
> Then in the next loop we iterate from the least loaded server and populate the destination as also the same server.
> This is leading to a condition where every 5 min balancing happens and also the server is same for src and dest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5482) In 0.90, balancer algo leading to
same region balanced twice and picking same region with Src and Destination
as same RS.
Posted by "ramkrishna.s.vasudevan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan updated HBASE-5482:
------------------------------------------
Attachment: HBASE-5482_2.patch
Patch addressing Ted's comments.
@Ted
If the patch is fine, we can commit it for 0.90.7?
> In 0.90, balancer algo leading to same region balanced twice and picking same region with Src and Destination as same RS.
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5482
> URL: https://issues.apache.org/jira/browse/HBASE-5482
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.90.5
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.90.7
>
> Attachments: 5482-v2.txt, HBASE-5482_1.patch, HBASE-5482_2.patch
>
>
> There are possibility of 2 problems
> -> When we populate regionsToMove while iterating the serverinfo in descending manner there is a chance that the same region can be added twice.
> Because in the first loop we do a randomization of the regions.
> Where as when we get we have neededRegions!= 0 we just get the region in the index and add it again . This may lead to have same region in the regionsToMove list.
> -> Another problem is
> when the problem in the first point happens then there is a chance that
> the regionToMove can have the same src and destination and the same region can be picked every 5 mins.
> {code}
> for(Map.Entry<HServerInfo, List<HRegionInfo>> server :
> serversByLoad.descendingMap().entrySet()) {
> BalanceInfo balanceInfo = serverBalanceInfo.get(server.getKey());
> int idx =
> balanceInfo == null ? 0 : balanceInfo.getNextRegionForUnload();
> if (idx >= server.getValue().size()) break;
> HRegionInfo region = server.getValue().get(idx);
> if (region.isMetaRegion()) continue; // Don't move meta regions.
> regionsToMove.add(new RegionPlan(region, server.getKey(), null));
> if(--neededRegions == 0) {
> // No more regions needed, done shedding
> break;
> }
> }
> {code}
> If i have meta and root in the top two loaded region server(totally 3 RS), we just skip the regions in those region server and populate the region from the least loaded RS.
> Then in the next loop we iterate from the least loaded server and populate the destination as also the same server.
> This is leading to a condition where every 5 min balancing happens and also the server is same for src and dest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5482) In 0.90, balancer algo leading to
same region balanced twice and picking same region with Src and Destination
as same RS.
Posted by "ramkrishna.s.vasudevan (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan resolved HBASE-5482.
-------------------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
> In 0.90, balancer algo leading to same region balanced twice and picking same region with Src and Destination as same RS.
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5482
> URL: https://issues.apache.org/jira/browse/HBASE-5482
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.90.5
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.90.7
>
> Attachments: 5482-v2.txt, HBASE-5482_1.patch, HBASE-5482_2.patch
>
>
> There are possibility of 2 problems
> -> When we populate regionsToMove while iterating the serverinfo in descending manner there is a chance that the same region can be added twice.
> Because in the first loop we do a randomization of the regions.
> Where as when we get we have neededRegions!= 0 we just get the region in the index and add it again . This may lead to have same region in the regionsToMove list.
> -> Another problem is
> when the problem in the first point happens then there is a chance that
> the regionToMove can have the same src and destination and the same region can be picked every 5 mins.
> {code}
> for(Map.Entry<HServerInfo, List<HRegionInfo>> server :
> serversByLoad.descendingMap().entrySet()) {
> BalanceInfo balanceInfo = serverBalanceInfo.get(server.getKey());
> int idx =
> balanceInfo == null ? 0 : balanceInfo.getNextRegionForUnload();
> if (idx >= server.getValue().size()) break;
> HRegionInfo region = server.getValue().get(idx);
> if (region.isMetaRegion()) continue; // Don't move meta regions.
> regionsToMove.add(new RegionPlan(region, server.getKey(), null));
> if(--neededRegions == 0) {
> // No more regions needed, done shedding
> break;
> }
> }
> {code}
> If i have meta and root in the top two loaded region server(totally 3 RS), we just skip the regions in those region server and populate the region from the least loaded RS.
> Then in the next loop we iterate from the least loaded server and populate the destination as also the same server.
> This is leading to a condition where every 5 min balancing happens and also the server is same for src and dest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5482) In 0.90, balancer algo leading to
same region balanced twice and picking same region with Src and Destination
as same RS.
Posted by "ramkrishna.s.vasudevan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan updated HBASE-5482:
------------------------------------------
Summary: In 0.90, balancer algo leading to same region balanced twice and picking same region with Src and Destination as same RS. (was: Balancer in 0.90 algo leading to same region balanced twice and picking same region with Src and Destination as same RS.)
> In 0.90, balancer algo leading to same region balanced twice and picking same region with Src and Destination as same RS.
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5482
> URL: https://issues.apache.org/jira/browse/HBASE-5482
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.90.5
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.90.7
>
>
> There are possibility of 2 problems
> -> When we populate regionsToMove while iterating the serverinfo in descending manner there is a chance that the same region can be added twice.
> Because in the first loop we do a randomization of the regions.
> Where as when we get we have neededRegions!= 0 we just get the region in the index and add it again . This may lead to have same region in the regionsToMove list.
> -> Another problem is
> when the problem in the first point happens then there is a chance that
> the regionToMove can have the same src and destination and the same region can be picked every 5 mins.
> {code}
> for(Map.Entry<HServerInfo, List<HRegionInfo>> server :
> serversByLoad.descendingMap().entrySet()) {
> BalanceInfo balanceInfo = serverBalanceInfo.get(server.getKey());
> int idx =
> balanceInfo == null ? 0 : balanceInfo.getNextRegionForUnload();
> if (idx >= server.getValue().size()) break;
> HRegionInfo region = server.getValue().get(idx);
> if (region.isMetaRegion()) continue; // Don't move meta regions.
> regionsToMove.add(new RegionPlan(region, server.getKey(), null));
> if(--neededRegions == 0) {
> // No more regions needed, done shedding
> break;
> }
> }
> {code}
> If i have meta and root in the top two loaded region server(totally 3 RS), we just skip the regions in those region server and populate the region from the least loaded RS.
> Then in the next loop we iterate from the least loaded server and populate the destination as also the same server.
> This is leading to a condition where every 5 min balancing happens and also the server is same for src and dest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5482) In 0.90, balancer algo leading to
same region balanced twice and picking same region with Src and Destination
as same RS.
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218246#comment-13218246 ]
ramkrishna.s.vasudevan commented on HBASE-5482:
-----------------------------------------------
Checking if the patch similar to this is needed in trunk. Once confirmed will prepare a patch for 0.92.
> In 0.90, balancer algo leading to same region balanced twice and picking same region with Src and Destination as same RS.
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5482
> URL: https://issues.apache.org/jira/browse/HBASE-5482
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.90.5
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.90.7
>
> Attachments: HBASE-5482_1.patch
>
>
> There are possibility of 2 problems
> -> When we populate regionsToMove while iterating the serverinfo in descending manner there is a chance that the same region can be added twice.
> Because in the first loop we do a randomization of the regions.
> Where as when we get we have neededRegions!= 0 we just get the region in the index and add it again . This may lead to have same region in the regionsToMove list.
> -> Another problem is
> when the problem in the first point happens then there is a chance that
> the regionToMove can have the same src and destination and the same region can be picked every 5 mins.
> {code}
> for(Map.Entry<HServerInfo, List<HRegionInfo>> server :
> serversByLoad.descendingMap().entrySet()) {
> BalanceInfo balanceInfo = serverBalanceInfo.get(server.getKey());
> int idx =
> balanceInfo == null ? 0 : balanceInfo.getNextRegionForUnload();
> if (idx >= server.getValue().size()) break;
> HRegionInfo region = server.getValue().get(idx);
> if (region.isMetaRegion()) continue; // Don't move meta regions.
> regionsToMove.add(new RegionPlan(region, server.getKey(), null));
> if(--neededRegions == 0) {
> // No more regions needed, done shedding
> break;
> }
> }
> {code}
> If i have meta and root in the top two loaded region server(totally 3 RS), we just skip the regions in those region server and populate the region from the least loaded RS.
> Then in the next loop we iterate from the least loaded server and populate the destination as also the same server.
> This is leading to a condition where every 5 min balancing happens and also the server is same for src and dest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5482) In 0.90, balancer algo leading to
same region balanced twice and picking same region with Src and Destination
as same RS.
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5482:
------------------------------
Attachment: 5482-v2.txt
Patch v2 corrected grammar in comments
> In 0.90, balancer algo leading to same region balanced twice and picking same region with Src and Destination as same RS.
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5482
> URL: https://issues.apache.org/jira/browse/HBASE-5482
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.90.5
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.90.7
>
> Attachments: 5482-v2.txt, HBASE-5482_1.patch
>
>
> There are possibility of 2 problems
> -> When we populate regionsToMove while iterating the serverinfo in descending manner there is a chance that the same region can be added twice.
> Because in the first loop we do a randomization of the regions.
> Where as when we get we have neededRegions!= 0 we just get the region in the index and add it again . This may lead to have same region in the regionsToMove list.
> -> Another problem is
> when the problem in the first point happens then there is a chance that
> the regionToMove can have the same src and destination and the same region can be picked every 5 mins.
> {code}
> for(Map.Entry<HServerInfo, List<HRegionInfo>> server :
> serversByLoad.descendingMap().entrySet()) {
> BalanceInfo balanceInfo = serverBalanceInfo.get(server.getKey());
> int idx =
> balanceInfo == null ? 0 : balanceInfo.getNextRegionForUnload();
> if (idx >= server.getValue().size()) break;
> HRegionInfo region = server.getValue().get(idx);
> if (region.isMetaRegion()) continue; // Don't move meta regions.
> regionsToMove.add(new RegionPlan(region, server.getKey(), null));
> if(--neededRegions == 0) {
> // No more regions needed, done shedding
> break;
> }
> }
> {code}
> If i have meta and root in the top two loaded region server(totally 3 RS), we just skip the regions in those region server and populate the region from the least loaded RS.
> Then in the next loop we iterate from the least loaded server and populate the destination as also the same server.
> This is leading to a condition where every 5 min balancing happens and also the server is same for src and dest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5482) In 0.90, balancer algo leading to
same region balanced twice and picking same region with Src and Destination
as same RS.
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220121#comment-13220121 ]
Zhihong Yu commented on HBASE-5482:
-----------------------------------
The following tests passed based on patch v2:
{code}
929 mt -Dtest=TestLoadBalancer
931 mt -Dtest=TestRegionRebalancing
{code}
+1 on patch v2.
> In 0.90, balancer algo leading to same region balanced twice and picking same region with Src and Destination as same RS.
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-5482
> URL: https://issues.apache.org/jira/browse/HBASE-5482
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.90.5
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.90.7
>
> Attachments: 5482-v2.txt, HBASE-5482_1.patch, HBASE-5482_2.patch
>
>
> There are possibility of 2 problems
> -> When we populate regionsToMove while iterating the serverinfo in descending manner there is a chance that the same region can be added twice.
> Because in the first loop we do a randomization of the regions.
> Where as when we get we have neededRegions!= 0 we just get the region in the index and add it again . This may lead to have same region in the regionsToMove list.
> -> Another problem is
> when the problem in the first point happens then there is a chance that
> the regionToMove can have the same src and destination and the same region can be picked every 5 mins.
> {code}
> for(Map.Entry<HServerInfo, List<HRegionInfo>> server :
> serversByLoad.descendingMap().entrySet()) {
> BalanceInfo balanceInfo = serverBalanceInfo.get(server.getKey());
> int idx =
> balanceInfo == null ? 0 : balanceInfo.getNextRegionForUnload();
> if (idx >= server.getValue().size()) break;
> HRegionInfo region = server.getValue().get(idx);
> if (region.isMetaRegion()) continue; // Don't move meta regions.
> regionsToMove.add(new RegionPlan(region, server.getKey(), null));
> if(--neededRegions == 0) {
> // No more regions needed, done shedding
> break;
> }
> }
> {code}
> If i have meta and root in the top two loaded region server(totally 3 RS), we just skip the regions in those region server and populate the region from the least loaded RS.
> Then in the next loop we iterate from the least loaded server and populate the destination as also the same server.
> This is leading to a condition where every 5 min balancing happens and also the server is same for src and dest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira