You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jimmy Xiang (Created) (JIRA)" <ji...@apache.org> on 2012/04/05 01:04:22 UTC
[jira] [Created] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Enhance hbck to sideline overlapped mega regions
------------------------------------------------
Key: HBASE-5719
URL: https://issues.apache.org/jira/browse/HBASE-5719
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.94.0, 0.96.0
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Fix For: 0.96.0
If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250092#comment-13250092 ]
jiraposter@reviews.apache.org commented on HBASE-5719:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4649/
-----------------------------------------------------------
(Updated 2012-04-09 19:27:53.361210)
Review request for hbase and jmhsieh.
Changes
-------
Addressed Jon's second review comments.
Summary
-------
Make it configurable to sideline some regions in big overlapped groups which hbck doesn't handle currently.
The regions chose to sideline are those which overlap with most other regions.
This addresses bug HBASE-5719.
https://issues.apache.org/jira/browse/HBASE-5719
Diffs (updated)
-----
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 54f9b21
src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java 17678dd
src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java ac3b225
Diff: https://reviews.apache.org/r/4649/diff
Testing
-------
mvn -PlocalTests -Dtest=TestHBaseFsck* clean test
Also tested in real system to fix inconsistencies.
Thanks,
Jimmy
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Jimmy Xiang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5719:
-------------------------------
Status: Patch Available (was: Open)
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Jimmy Xiang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5719:
-------------------------------
Status: Open (was: Patch Available)
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259521#comment-13259521 ]
Hudson commented on HBASE-5719:
-------------------------------
Integrated in HBase-0.92-security #105 (See [https://builds.apache.org/job/HBase-0.92-security/105/])
HBASE-5719 Enhance hbck to sideline overlapped mega regions (Jimmy Xiang) (Revision 1325405)
Result = FAILURE
jmhsieh :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: 5719.patch, 5719_0.90.patch, 5719_0.92.patch, 5719_0.94.patch, hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246895#comment-13246895 ]
Hadoop QA commented on HBASE-5719:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12521425/hbase-5719.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1399//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1399//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1399//console
This message is automatically generated.
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252184#comment-13252184 ]
Hadoop QA commented on HBASE-5719:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522372/5719_0.94.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestFromClientSide
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1492//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1492//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1492//console
This message is automatically generated.
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: 5719.patch, 5719_0.90.patch, 5719_0.92.patch, 5719_0.94.patch, hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Jimmy Xiang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5719:
-------------------------------
Status: Open (was: Patch Available)
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: 5719.patch, hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251768#comment-13251768 ]
jiraposter@reviews.apache.org commented on HBASE-5719:
------------------------------------------------------
bq. On 2012-04-11 15:44:08, jmhsieh wrote:
bq. > src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java, lines 384-392
bq. > <https://reviews.apache.org/r/4649/diff/2-3/?file=100674#file100674line384>
bq. >
bq. > nit: would be easier to read if
bq. > assertTrue((r1.equals(ac) && r2.equals(ae)) || (r1.equals(ae) && r2.equals(ac)));
bq.
bq. Jimmy Xiang wrote:
bq. That's the original version actually. However, the equals method somehow doesn't work. I didn't try to implement the equals method in the SimpleRange class.
I see. Either is fine by me -- as is or you can make the change. :)
- jmhsieh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4649/#review6851
-----------------------------------------------------------
On 2012-04-09 19:27:53, Jimmy Xiang wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4649/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-09 19:27:53)
bq.
bq.
bq. Review request for hbase and jmhsieh.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Make it configurable to sideline some regions in big overlapped groups which hbck doesn't handle currently.
bq.
bq. The regions chose to sideline are those which overlap with most other regions.
bq.
bq.
bq. This addresses bug HBASE-5719.
bq. https://issues.apache.org/jira/browse/HBASE-5719
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 54f9b21
bq. src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java 17678dd
bq. src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java ac3b225
bq.
bq. Diff: https://reviews.apache.org/r/4649/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. mvn -PlocalTests -Dtest=TestHBaseFsck* clean test
bq.
bq. Also tested in real system to fix inconsistencies.
bq.
bq.
bq. Thanks,
bq.
bq. Jimmy
bq.
bq.
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Jonathan Hsieh (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247865#comment-13247865 ]
Jonathan Hsieh commented on HBASE-5719:
---------------------------------------
More context:
We ran into a corrupted cluster that had encountered HBASE-4238 and had several generations of "grandparent" and regions lingering in HDFS. If you looked at a region map, we had overlapping regions that looked like this:
[A-I], [A-E], [E-H], [A-C], [A-B], [B-C] ...
The HBASE-5128 version of hbck would see that all these regions fit inside of A-I and then attempt to merge the all into one mega region. This is technically correct but could result merging all the regions in an overlap group into one region that was significantly larger than all others (worst case all regions of a table could get combined into one region). HBASE-5128 includes some safeguards to prevent these "mega merges". In order to fix these situations, we sidelined (close, offline, move to different dir) the grandparent regions with the largest overlapped with the most other regions. This leaves us with many small groups of overlapping regions instead of a single large set of overlapping regions. These smaller regions could be safely repaired automatically via merges, and any data from the sidelined grandparent regions could be restored via a bulk load later on.
So in the example above, the [A-I], [A-E], [E-H] grandparent regions would get sidelined, and leaving us with [A-C], [A-B],[B-C]. These smaller regions could get safely merged automatically into a single [A-C]' region. We'd then bulk load [A-I], [A-E], and [E-H] regions back in afterwards to restore data.
The goal of this patch is to automatically id and sideline overlapping grandparent regions.
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247976#comment-13247976 ]
jiraposter@reviews.apache.org commented on HBASE-5719:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4649/#review6713
-----------------------------------------------------------
Nice feature. Can we call it something other than "breakXxx"? Maybe "-sidelineLargeOverlap"
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14511>
why commented?
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14512>
why commented?
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14513>
break? maybe a different word?
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14514>
instead of nesting, maybe set hfiles to an empty array?
if (hfiles==null) {
hfiles = new FileStatus[0];
}
for (FileStatus hfile : hfiles)
...
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14565>
Instead of "None/Multiple" just say how many htds?
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14516>
We lost info here.
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14608>
Pretty important to note that we only "break" groups that exceed the max number of regions to merge!
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14569>
Comment about what you are trying to accomplish with this method?
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14607>
Is this really a failure?
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14570>
I think we should bail out in this case.
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14571>
LOG.debug/trace?
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14572>
LOG.debug/trace?
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14562>
wrong comment for this method
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14563>
Please update usage with new options.
src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java
<https://reviews.apache.org/r/4649/#comment14568>
Why this change?
src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java
<https://reviews.apache.org/r/4649/#comment14567>
This is a really important function.
Please java doc (maybe an example?) and at least add a unit tests that cover edge cases here for this.
What is the definition of a "megaOverlap"?
What are the properties of the argument and the return values?
What is the intuition behind how the regions selected are selected and returned?
- jmhsieh
On 2012-04-04 23:52:50, Jimmy Xiang wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4649/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-04 23:52:50)
bq.
bq.
bq. Review request for hbase and jmhsieh.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Make it configurable to sideline some regions in big overlapped groups which hbck doesn't handle currently.
bq.
bq. The regions chose to sideline are those which overlap with most other regions.
bq.
bq.
bq. This addresses bug HBASE-5719.
bq. https://issues.apache.org/jira/browse/HBASE-5719
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 54f9b21
bq. src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java 17678dd
bq.
bq. Diff: https://reviews.apache.org/r/4649/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. mvn -PlocalTests -Dtest=TestHBaseFsck* clean test
bq.
bq. Also tested in real system to fix inconsistencies.
bq.
bq.
bq. Thanks,
bq.
bq. Jimmy
bq.
bq.
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252890#comment-13252890 ]
Hudson commented on HBASE-5719:
-------------------------------
Integrated in HBase-0.92 #369 (See [https://builds.apache.org/job/HBase-0.92/369/])
HBASE-5719 Enhance hbck to sideline overlapped mega regions (Jimmy Xiang) (Revision 1325405)
Result = FAILURE
jmhsieh :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: 5719.patch, 5719_0.90.patch, 5719_0.92.patch, 5719_0.94.patch, hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251791#comment-13251791 ]
Hadoop QA commented on HBASE-5719:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522285/hbase-5719_v3-new.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1481//console
This message is automatically generated.
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251774#comment-13251774 ]
Hadoop QA commented on HBASE-5719:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522283/hbase-5719_v3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1479//console
This message is automatically generated.
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251743#comment-13251743 ]
jiraposter@reviews.apache.org commented on HBASE-5719:
------------------------------------------------------
bq. On 2012-04-11 15:44:08, jmhsieh wrote:
bq. > src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java, lines 384-392
bq. > <https://reviews.apache.org/r/4649/diff/2-3/?file=100674#file100674line384>
bq. >
bq. > nit: would be easier to read if
bq. > assertTrue((r1.equals(ac) && r2.equals(ae)) || (r1.equals(ae) && r2.equals(ac)));
That's the original version actually. However, the equals method somehow doesn't work. I didn't try to implement the equals method in the SimpleRange class.
- Jimmy
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4649/#review6851
-----------------------------------------------------------
On 2012-04-09 19:27:53, Jimmy Xiang wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4649/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-09 19:27:53)
bq.
bq.
bq. Review request for hbase and jmhsieh.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Make it configurable to sideline some regions in big overlapped groups which hbck doesn't handle currently.
bq.
bq. The regions chose to sideline are those which overlap with most other regions.
bq.
bq.
bq. This addresses bug HBASE-5719.
bq. https://issues.apache.org/jira/browse/HBASE-5719
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 54f9b21
bq. src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java 17678dd
bq. src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java ac3b225
bq.
bq. Diff: https://reviews.apache.org/r/4649/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. mvn -PlocalTests -Dtest=TestHBaseFsck* clean test
bq.
bq. Also tested in real system to fix inconsistencies.
bq.
bq.
bq. Thanks,
bq.
bq. Jimmy
bq.
bq.
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252155#comment-13252155 ]
Hadoop QA commented on HBASE-5719:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522366/5719.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.TestRegionRebalancing
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1491//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1491//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1491//console
This message is automatically generated.
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: 5719.patch, hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252718#comment-13252718 ]
Hudson commented on HBASE-5719:
-------------------------------
Integrated in HBase-0.94 #106 (See [https://builds.apache.org/job/HBase-0.94/106/])
HBASE-5719 Enhance hbck to sideline overlapped mega regions (Jimmy Xiang) (Revision 1325403)
Result = SUCCESS
jmhsieh :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: 5719.patch, 5719_0.90.patch, 5719_0.92.patch, 5719_0.94.patch, hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Jimmy Xiang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5719:
-------------------------------
Attachment: hbase-5719_v3.patch
hbase-5719_0.94.patch
hbase-5719_0.92.patch
hbase-5719_0.90.patch
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Jimmy Xiang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5719:
-------------------------------
Status: Open (was: Patch Available)
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Jimmy Xiang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5719:
-------------------------------
Attachment: 5719.patch
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: 5719.patch, hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253633#comment-13253633 ]
Hudson commented on HBASE-5719:
-------------------------------
Integrated in HBase-0.94-security #9 (See [https://builds.apache.org/job/HBase-0.94-security/9/])
HBASE-5719 Enhance hbck to sideline overlapped mega regions (Jimmy Xiang) (Revision 1325403)
Result = SUCCESS
jmhsieh :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: 5719.patch, 5719_0.90.patch, 5719_0.92.patch, 5719_0.94.patch, hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248574#comment-13248574 ]
jiraposter@reviews.apache.org commented on HBASE-5719:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4649/
-----------------------------------------------------------
(Updated 2012-04-06 18:27:34.615205)
Review request for hbase and jmhsieh.
Changes
-------
Addressed Jon's comments.
Summary
-------
Make it configurable to sideline some regions in big overlapped groups which hbck doesn't handle currently.
The regions chose to sideline are those which overlap with most other regions.
This addresses bug HBASE-5719.
https://issues.apache.org/jira/browse/HBASE-5719
Diffs (updated)
-----
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 54f9b21
src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java 17678dd
src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java ac3b225
Diff: https://reviews.apache.org/r/4649/diff
Testing
-------
mvn -PlocalTests -Dtest=TestHBaseFsck* clean test
Also tested in real system to fix inconsistencies.
Thanks,
Jimmy
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247994#comment-13247994 ]
jiraposter@reviews.apache.org commented on HBASE-5719:
------------------------------------------------------
bq. On 2012-04-06 02:26:10, jmhsieh wrote:
bq. > Nice feature. Can we call it something other than "breakXxx"? Maybe "-sidelineLargeOverlap"
Sure. Will do.
bq. On 2012-04-06 02:26:10, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 142
bq. > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line142>
bq. >
bq. > why commented?
bq. >
It is not used so I commented it out.
bq. On 2012-04-06 02:26:10, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 157
bq. > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line157>
bq. >
bq. > why commented?
It is not used so I commented it out.
bq. On 2012-04-06 02:26:10, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 174
bq. > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line174>
bq. >
bq. > break? maybe a different word?
sideline.
bq. On 2012-04-06 02:26:10, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 880
bq. > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line880>
bq. >
bq. > instead of nesting, maybe set hfiles to an empty array?
bq. >
bq. > if (hfiles==null) {
bq. > hfiles = new FileStatus[0];
bq. > }
bq. > for (FileStatus hfile : hfiles)
bq. > ...
That means an extra object. I think the current checking is fine.
bq. On 2012-04-06 02:26:10, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, lines 1572-1574
bq. > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line1572>
bq. >
bq. > Instead of "None/Multiple" just say how many htds?
bq. >
bq. >
All the htds are printed out, so it should be clear enough.
bq. On 2012-04-06 02:26:10, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 1834
bq. > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line1834>
bq. >
bq. > Comment about what you are trying to accomplish with this method?
Will do.
bq. On 2012-04-06 02:26:10, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 1876
bq. > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line1876>
bq. >
bq. > LOG.debug/trace?
Will change to LOG.debug
bq. On 2012-04-06 02:26:10, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 1872
bq. > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line1872>
bq. >
bq. > LOG.debug/trace?
bq. >
Will change to LOG.debug, good catch.
bq. On 2012-04-06 02:26:10, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java, lines 167-215
bq. > <https://reviews.apache.org/r/4649/diff/1/?file=100074#file100074line167>
bq. >
bq. > This is a really important function.
bq. >
bq. > Please java doc (maybe an example?) and at least add a unit tests that cover edge cases here for this.
bq. >
bq. > What is the definition of a "megaOverlap"?
bq. >
bq. > What are the properties of the argument and the return values?
bq. >
bq. > What is the intuition behind how the regions selected are selected and returned?
Sure, will do.
bq. On 2012-04-06 02:26:10, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, lines 2826-2831
bq. > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line2826>
bq. >
bq. > wrong comment for this method
Will fix it.
bq. On 2012-04-06 02:26:10, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, lines 2896-2904
bq. > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line2896>
bq. >
bq. > Please update usage with new options.
Will do.
bq. On 2012-04-06 02:26:10, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 1843
bq. > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line1843>
bq. >
bq. > Is this really a failure?
This should not happen. Let me remove this checking.
bq. On 2012-04-06 02:26:10, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 1755
bq. > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line1755>
bq. >
bq. > We lost info here.
Will add it back.
bq. On 2012-04-06 02:26:10, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, lines 1756-1757
bq. > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line1756>
bq. >
bq. > Pretty important to note that we only "break" groups that exceed the max number of regions to merge!
Right, I will add some comment.
bq. On 2012-04-06 02:26:10, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 1858
bq. > <https://reviews.apache.org/r/4649/diff/1/?file=100073#file100073line1858>
bq. >
bq. > I think we should bail out in this case.
Agree, will fix it.
- Jimmy
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4649/#review6713
-----------------------------------------------------------
On 2012-04-04 23:52:50, Jimmy Xiang wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4649/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-04 23:52:50)
bq.
bq.
bq. Review request for hbase and jmhsieh.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Make it configurable to sideline some regions in big overlapped groups which hbck doesn't handle currently.
bq.
bq. The regions chose to sideline are those which overlap with most other regions.
bq.
bq.
bq. This addresses bug HBASE-5719.
bq. https://issues.apache.org/jira/browse/HBASE-5719
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 54f9b21
bq. src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java 17678dd
bq.
bq. Diff: https://reviews.apache.org/r/4649/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. mvn -PlocalTests -Dtest=TestHBaseFsck* clean test
bq.
bq. Also tested in real system to fix inconsistencies.
bq.
bq.
bq. Thanks,
bq.
bq. Jimmy
bq.
bq.
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Jimmy Xiang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5719:
-------------------------------
Hadoop Flags: Reviewed
Status: Patch Available (was: Open)
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13251683#comment-13251683 ]
jiraposter@reviews.apache.org commented on HBASE-5719:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4649/#review6851
-----------------------------------------------------------
Ship it!
Looks good to me Jimmy. Mind checking 0.90/0.92/0.94 and doing ports if necessary? Should be trivial.
I have one nit that you can address or ignore. :)
src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java
<https://reviews.apache.org/r/4649/#comment15241>
nit: would be easier to read if
assertTrue((r1.equals(ac) && r2.equals(ae)) || (r1.equals(ae) && r2.equals(ac)));
- jmhsieh
On 2012-04-09 19:27:53, Jimmy Xiang wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4649/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-09 19:27:53)
bq.
bq.
bq. Review request for hbase and jmhsieh.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Make it configurable to sideline some regions in big overlapped groups which hbck doesn't handle currently.
bq.
bq. The regions chose to sideline are those which overlap with most other regions.
bq.
bq.
bq. This addresses bug HBASE-5719.
bq. https://issues.apache.org/jira/browse/HBASE-5719
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 54f9b21
bq. src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java 17678dd
bq. src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java ac3b225
bq.
bq. Diff: https://reviews.apache.org/r/4649/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. mvn -PlocalTests -Dtest=TestHBaseFsck* clean test
bq.
bq. Also tested in real system to fix inconsistencies.
bq.
bq.
bq. Thanks,
bq.
bq. Jimmy
bq.
bq.
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250087#comment-13250087 ]
jiraposter@reviews.apache.org commented on HBASE-5719:
------------------------------------------------------
bq. On 2012-04-09 18:20:11, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 1869
bq. > <https://reviews.apache.org/r/4649/diff/2/?file=100672#file100672line1869>
bq. >
bq. > Maybe this one is at info level?
Will change to info level.
bq. On 2012-04-09 18:20:11, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 2828
bq. > <https://reviews.apache.org/r/4649/diff/2/?file=100672#file100672line2828>
bq. >
bq. > nit: typo (old was correct)
Changed it back. It must be because the mouse moved when I was typing something. I didn't remember I changed it. :)
bq. On 2012-04-09 18:20:11, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 2895
bq. > <https://reviews.apache.org/r/4649/diff/2/?file=100672#file100672line2895>
bq. >
bq. > Is this <n> per group or globally? please make it clear in usage
It is per group. Will fix it.
bq. On 2012-04-09 18:20:11, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java, line 2981
bq. > <https://reviews.apache.org/r/4649/diff/2/?file=100672#file100672line2981>
bq. >
bq. > nit: should be like previous -maxMerge's comments.
Will fix it.
bq. On 2012-04-09 18:20:11, jmhsieh wrote:
bq. > src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java, line 360
bq. > <https://reviews.apache.org/r/4649/diff/2/?file=100674#file100674line360>
bq. >
bq. > nit: it would easer to read if this was called ai, the next ae, and the one after that ac.
bq. >
bq. > Also would help if you put the expected overlap count in comments!
Sure.
bq. On 2012-04-09 18:20:11, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java, line 212
bq. > <https://reviews.apache.org/r/4649/diff/2/?file=100673#file100673line212>
bq. >
bq. > what is the intuition here?
Added comment
bq. On 2012-04-09 18:20:11, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java, line 189
bq. > <https://reviews.apache.org/r/4649/diff/2/?file=100673#file100673line189>
bq. >
bq. > add comment?
bq. >
bq. > We always overlap with ourselves -- which is why > 1 is the condition below..
Will do.
bq. On 2012-04-09 18:20:11, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java, line 178
bq. > <https://reviews.apache.org/r/4649/diff/2/?file=100673#file100673line178>
bq. >
bq. > add comment: calculates the # of overlaps for each region and populates rangeOverlapMap
Sure
bq. On 2012-04-09 18:20:11, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java, line 177
bq. > <https://reviews.apache.org/r/4649/diff/2/?file=100673#file100673line177>
bq. >
bq. > Add comment about structure contents. something like: Integer key is overlap count and List is regions that have that many overlaps.
bq. > maybe rename to overlapRangeMap?
Sure.
bq. On 2012-04-09 18:20:11, jmhsieh wrote:
bq. > src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java, line 171
bq. > <https://reviews.apache.org/r/4649/diff/2/?file=100673#file100673line171>
bq. >
bq. > To find? Would returning a sort make sense? if there are no overlaps will this return the specified number (or less?)
bq. >
bq. > Could this return several regions that now leave holes?
bq. >
bq. > nit: It might make testing clearer to return the overlapCount -> regions map and check that.
Added comment.
- Jimmy
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4649/#review6764
-----------------------------------------------------------
On 2012-04-06 18:27:34, Jimmy Xiang wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4649/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-06 18:27:34)
bq.
bq.
bq. Review request for hbase and jmhsieh.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Make it configurable to sideline some regions in big overlapped groups which hbck doesn't handle currently.
bq.
bq. The regions chose to sideline are those which overlap with most other regions.
bq.
bq.
bq. This addresses bug HBASE-5719.
bq. https://issues.apache.org/jira/browse/HBASE-5719
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 54f9b21
bq. src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java 17678dd
bq. src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java ac3b225
bq.
bq. Diff: https://reviews.apache.org/r/4649/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. mvn -PlocalTests -Dtest=TestHBaseFsck* clean test
bq.
bq. Also tested in real system to fix inconsistencies.
bq.
bq.
bq. Thanks,
bq.
bq. Jimmy
bq.
bq.
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Jimmy Xiang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5719:
-------------------------------
Status: Patch Available (was: Open)
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: 5719.patch, hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252713#comment-13252713 ]
Hudson commented on HBASE-5719:
-------------------------------
Integrated in HBase-TRUNK #2746 (See [https://builds.apache.org/job/HBase-TRUNK/2746/])
HBASE-5719 Enhance hbck to sideline overlapped mega regions (Jimmy Xiang) (Revision 1325404)
Result = SUCCESS
jmhsieh :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: 5719.patch, 5719_0.90.patch, 5719_0.92.patch, 5719_0.94.patch, hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Jimmy Xiang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5719:
-------------------------------
Status: Patch Available (was: Open)
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Jimmy Xiang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5719:
-------------------------------
Attachment: 5719_0.94.patch
5719_0.92.patch
5719_0.90.patch
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: 5719.patch, 5719_0.90.patch, 5719_0.92.patch, 5719_0.94.patch, hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250024#comment-13250024 ]
jiraposter@reviews.apache.org commented on HBASE-5719:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4649/#review6764
-----------------------------------------------------------
Jimmy,
Looks good to me. Mostly some questions and a little bit of of follow up to make it easier to read and to provide user more info.
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14916>
nice catch.
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment15130>
Maybe this one is at info level?
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14917>
nit: typo (old was correct)
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14918>
Is this <n> per group or globally? please make it clear in usage
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
<https://reviews.apache.org/r/4649/#comment14919>
nit: should be like previous -maxMerge's comments.
src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java
<https://reviews.apache.org/r/4649/#comment15129>
To find? Would returning a sort make sense? if there are no overlaps will this return the specified number (or less?)
Could this return several regions that now leave holes?
nit: It might make testing clearer to return the overlapCount -> regions map and check that.
src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java
<https://reviews.apache.org/r/4649/#comment15132>
Add comment about structure contents. something like: Integer key is overlap count and List is regions that have that many overlaps.
maybe rename to overlapRangeMap?
src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java
<https://reviews.apache.org/r/4649/#comment15135>
add comment: calculates the # of overlaps for each region and populates rangeOverlapMap
src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java
<https://reviews.apache.org/r/4649/#comment15133>
add comment?
We always overlap with ourselves -- which is why > 1 is the condition below..
src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java
<https://reviews.apache.org/r/4649/#comment15136>
what is the intuition here?
src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java
<https://reviews.apache.org/r/4649/#comment15140>
nit: it would easer to read if this was called ai, the next ae, and the one after that ac.
Also would help if you put the expected overlap count in comments!
- jmhsieh
On 2012-04-06 18:27:34, Jimmy Xiang wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4649/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-06 18:27:34)
bq.
bq.
bq. Review request for hbase and jmhsieh.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Make it configurable to sideline some regions in big overlapped groups which hbck doesn't handle currently.
bq.
bq. The regions chose to sideline are those which overlap with most other regions.
bq.
bq.
bq. This addresses bug HBASE-5719.
bq. https://issues.apache.org/jira/browse/HBASE-5719
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 54f9b21
bq. src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java 17678dd
bq. src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java ac3b225
bq.
bq. Diff: https://reviews.apache.org/r/4649/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. mvn -PlocalTests -Dtest=TestHBaseFsck* clean test
bq.
bq. Also tested in real system to fix inconsistencies.
bq.
bq.
bq. Thanks,
bq.
bq. Jimmy
bq.
bq.
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Jimmy Xiang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5719:
-------------------------------
Status: Patch Available (was: Open)
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: 5719.patch, hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246861#comment-13246861 ]
jiraposter@reviews.apache.org commented on HBASE-5719:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4649/
-----------------------------------------------------------
Review request for hbase.
Summary
-------
Make it configurable to sideline some regions in big overlapped groups which hbck doesn't handle currently.
The regions chose to sideline are those which overlap with most other regions.
This addresses bug HBASE-5719.
https://issues.apache.org/jira/browse/HBASE-5719
Diffs
-----
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 54f9b21
src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java 17678dd
Diff: https://reviews.apache.org/r/4649/diff
Testing
-------
mvn -PlocalTests -Dtest=TestHBaseFsck* clean test
Also tested in real system to fix inconsistencies.
Thanks,
Jimmy
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Jonathan Hsieh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Hsieh updated HBASE-5719:
----------------------------------
Resolution: Fixed
Fix Version/s: 0.94.0
0.92.2
0.90.7
Status: Resolved (was: Patch Available)
Thanks for the patch and the backports Jimmy!
Commited to 0.96/0.94/0.92/0.90.
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: 5719.patch, 5719_0.90.patch, 5719_0.92.patch, 5719_0.94.patch, hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Jimmy Xiang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5719:
-------------------------------
Attachment: hbase-5719_v3-new.patch
Try it again. The patch applies fine for me.
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246864#comment-13246864 ]
jiraposter@reviews.apache.org commented on HBASE-5719:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4649/
-----------------------------------------------------------
(Updated 2012-04-04 23:52:50.930344)
Review request for hbase and jmhsieh.
Summary
-------
Make it configurable to sideline some regions in big overlapped groups which hbck doesn't handle currently.
The regions chose to sideline are those which overlap with most other regions.
This addresses bug HBASE-5719.
https://issues.apache.org/jira/browse/HBASE-5719
Diffs
-----
src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 54f9b21
src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java 17678dd
Diff: https://reviews.apache.org/r/4649/diff
Testing
-------
mvn -PlocalTests -Dtest=TestHBaseFsck* clean test
Also tested in real system to fix inconsistencies.
Thanks,
Jimmy
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Jimmy Xiang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5719:
-------------------------------
Status: Open (was: Patch Available)
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253146#comment-13253146 ]
Hudson commented on HBASE-5719:
-------------------------------
Integrated in HBase-TRUNK-security #169 (See [https://builds.apache.org/job/HBase-TRUNK-security/169/])
HBASE-5719 Enhance hbck to sideline overlapped mega regions (Jimmy Xiang) (Revision 1325404)
Result = FAILURE
jmhsieh :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/RegionSplitCalculator.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitCalculator.java
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: 5719.patch, 5719_0.90.patch, 5719_0.92.patch, 5719_0.94.patch, hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped
mega regions
Posted by "Jimmy Xiang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-5719:
-------------------------------
Attachment: hbase-5719.patch
> Enhance hbck to sideline overlapped mega regions
> ------------------------------------------------
>
> Key: HBASE-5719
> URL: https://issues.apache.org/jira/browse/HBASE-5719
> Project: HBase
> Issue Type: New Feature
> Components: hbck
> Affects Versions: 0.94.0, 0.96.0
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5719.patch
>
>
> If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time.
> In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira