You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Marc Limotte (JIRA)" <ji...@apache.org> on 2011/03/23 23:33:05 UTC
[jira] [Created] (HBASE-3695) Some improvements to Hbck to test the
entire region chain in Meta and provide better error reporting
Some improvements to Hbck to test the entire region chain in Meta and provide better error reporting
----------------------------------------------------------------------------------------------------
Key: HBASE-3695
URL: https://issues.apache.org/jira/browse/HBASE-3695
Project: HBase
Issue Type: Improvement
Components: util
Affects Versions: 0.90.1
Reporter: Marc Limotte
Assignee: Marc Limotte
The current Hbck tool will miss some inconsistencies in Meta, and in other cases will detect an issue, but does not provide much in the way of useful feedback.
* Incorporate the full region chain tests (similar to check_meta.rb). I.e. look for overlaps, holes and cycles. I believe check_meta.rb will be redundant after this change.
* More unit tests, and better tests that will test the actual error discovered, instead of just errors true/false.
* In the case of overlaps and holes, output both ends of the broken chain.
* Previous implementation runs check() twice. This is inefficient and, more importantly, reports redundant errors which could be confusing to the user.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3695) Some improvements to Hbck to test
the entire region chain in Meta and provide better error reporting
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13011385#comment-13011385 ]
stack commented on HBASE-3695:
------------------------------
Resolved the dupe. There is already an online merge too. And yes, need to make it so region hacking in .META. is a bit easier, perhaps some facility in shell (Saw someone trying to do "put '.META.', 'STRING_VERSION_OF_HREGIONINFO', etc" which plain won't work, but, if you think a second, should work).
> Some improvements to Hbck to test the entire region chain in Meta and provide better error reporting
> ----------------------------------------------------------------------------------------------------
>
> Key: HBASE-3695
> URL: https://issues.apache.org/jira/browse/HBASE-3695
> Project: HBase
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.90.1
> Reporter: Marc Limotte
> Assignee: Marc Limotte
> Fix For: 0.90.3
>
>
> The current Hbck tool will miss some inconsistencies in Meta, and in other cases will detect an issue, but does not provide much in the way of useful feedback.
> * Incorporate the full region chain tests (similar to check_meta.rb). I.e. look for overlaps, holes and cycles. I believe check_meta.rb will be redundant after this change.
> * More unit tests, and better tests that will test the actual error discovered, instead of just errors true/false.
> * In the case of overlaps and holes, output both ends of the broken chain.
> * Previous implementation runs check() twice. This is inefficient and, more importantly, reports redundant errors which could be confusing to the user.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3695) Some improvements to Hbck to test
the entire region chain in Meta and provide better error reporting
Posted by "Wayne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13059082#comment-13059082 ]
Wayne commented on HBASE-3695:
------------------------------
It would be very useful to have the -fix flag actually be able to resolve most issues. Once the region keys get messed up it is very hard to manually fix these issues. This tool should be able to find and fix the major problems that can occur. These issues include no region with an empty start key, no region with an empty end key, a hole in the region chain, overlapping regions, regions with the same start key, etc. etc.
If this tool can find the issues it should be able to fix them as well.
> Some improvements to Hbck to test the entire region chain in Meta and provide better error reporting
> ----------------------------------------------------------------------------------------------------
>
> Key: HBASE-3695
> URL: https://issues.apache.org/jira/browse/HBASE-3695
> Project: HBase
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.90.1
> Reporter: Marc Limotte
> Assignee: Marc Limotte
> Fix For: 0.90.3
>
>
> The current Hbck tool will miss some inconsistencies in Meta, and in other cases will detect an issue, but does not provide much in the way of useful feedback.
> * Incorporate the full region chain tests (similar to check_meta.rb). I.e. look for overlaps, holes and cycles. I believe check_meta.rb will be redundant after this change.
> * More unit tests, and better tests that will test the actual error discovered, instead of just errors true/false.
> * In the case of overlaps and holes, output both ends of the broken chain.
> * Previous implementation runs check() twice. This is inefficient and, more importantly, reports redundant errors which could be confusing to the user.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3695) Some improvements to Hbck to test
the entire region chain in Meta and provide better error reporting
Posted by "Marc Limotte (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13011262#comment-13011262 ]
Marc Limotte commented on HBASE-3695:
-------------------------------------
@Stack Yea. That one is a dupe. It was earlier, but it is only a subset of what's here.
Separately, I think there should be a distinct JIRA to cover the merge tool issue (it doesn't work with the low, overlapping sequence numbers generated by the bulk load tool).
Also (possibly another JIRA), I wonder if the ability to manually create a region should be exposed (through the shell or other tool). We had to do this a couple of times to resolve the issues that had come up; but, on the other hand, if the merge tool worked in those scenarios, maybe this wouldn't have been necessary.
> Some improvements to Hbck to test the entire region chain in Meta and provide better error reporting
> ----------------------------------------------------------------------------------------------------
>
> Key: HBASE-3695
> URL: https://issues.apache.org/jira/browse/HBASE-3695
> Project: HBase
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.90.1
> Reporter: Marc Limotte
> Assignee: Marc Limotte
> Fix For: 0.90.3
>
>
> The current Hbck tool will miss some inconsistencies in Meta, and in other cases will detect an issue, but does not provide much in the way of useful feedback.
> * Incorporate the full region chain tests (similar to check_meta.rb). I.e. look for overlaps, holes and cycles. I believe check_meta.rb will be redundant after this change.
> * More unit tests, and better tests that will test the actual error discovered, instead of just errors true/false.
> * In the case of overlaps and holes, output both ends of the broken chain.
> * Previous implementation runs check() twice. This is inefficient and, more importantly, reports redundant errors which could be confusing to the user.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3695) Some improvements to Hbck to test
the entire region chain in Meta and provide better error reporting
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030208#comment-13030208 ]
Hudson commented on HBASE-3695:
-------------------------------
Integrated in HBase-TRUNK #1909 (See [https://builds.apache.org/hudson/job/HBase-TRUNK/1909/])
> Some improvements to Hbck to test the entire region chain in Meta and provide better error reporting
> ----------------------------------------------------------------------------------------------------
>
> Key: HBASE-3695
> URL: https://issues.apache.org/jira/browse/HBASE-3695
> Project: HBase
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.90.1
> Reporter: Marc Limotte
> Assignee: Marc Limotte
> Fix For: 0.90.3
>
>
> The current Hbck tool will miss some inconsistencies in Meta, and in other cases will detect an issue, but does not provide much in the way of useful feedback.
> * Incorporate the full region chain tests (similar to check_meta.rb). I.e. look for overlaps, holes and cycles. I believe check_meta.rb will be redundant after this change.
> * More unit tests, and better tests that will test the actual error discovered, instead of just errors true/false.
> * In the case of overlaps and holes, output both ends of the broken chain.
> * Previous implementation runs check() twice. This is inefficient and, more importantly, reports redundant errors which could be confusing to the user.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3695) Some improvements to Hbck to test the
entire region chain in Meta and provide better error reporting
Posted by "Marc Limotte (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marc Limotte updated HBASE-3695:
--------------------------------
Fix Version/s: 0.90.3
> Some improvements to Hbck to test the entire region chain in Meta and provide better error reporting
> ----------------------------------------------------------------------------------------------------
>
> Key: HBASE-3695
> URL: https://issues.apache.org/jira/browse/HBASE-3695
> Project: HBase
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.90.1
> Reporter: Marc Limotte
> Assignee: Marc Limotte
> Fix For: 0.90.3
>
>
> The current Hbck tool will miss some inconsistencies in Meta, and in other cases will detect an issue, but does not provide much in the way of useful feedback.
> * Incorporate the full region chain tests (similar to check_meta.rb). I.e. look for overlaps, holes and cycles. I believe check_meta.rb will be redundant after this change.
> * More unit tests, and better tests that will test the actual error discovered, instead of just errors true/false.
> * In the case of overlaps and holes, output both ends of the broken chain.
> * Previous implementation runs check() twice. This is inefficient and, more importantly, reports redundant errors which could be confusing to the user.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3695) Some improvements to Hbck to test
the entire region chain in Meta and provide better error reporting
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13011105#comment-13011105 ]
stack commented on HBASE-3695:
------------------------------
@Marc Should I resolve 'HBASE-3602) [hbck] Print out better info if detects overlapping regions in .META.' as dup of this? Thanks boss.
> Some improvements to Hbck to test the entire region chain in Meta and provide better error reporting
> ----------------------------------------------------------------------------------------------------
>
> Key: HBASE-3695
> URL: https://issues.apache.org/jira/browse/HBASE-3695
> Project: HBase
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.90.1
> Reporter: Marc Limotte
> Assignee: Marc Limotte
> Fix For: 0.90.3
>
>
> The current Hbck tool will miss some inconsistencies in Meta, and in other cases will detect an issue, but does not provide much in the way of useful feedback.
> * Incorporate the full region chain tests (similar to check_meta.rb). I.e. look for overlaps, holes and cycles. I believe check_meta.rb will be redundant after this change.
> * More unit tests, and better tests that will test the actual error discovered, instead of just errors true/false.
> * In the case of overlaps and holes, output both ends of the broken chain.
> * Previous implementation runs check() twice. This is inefficient and, more importantly, reports redundant errors which could be confusing to the user.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-3695) Some improvements to Hbck to test
the entire region chain in Meta and provide better error reporting
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-3695.
--------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Applied to trunk and branch after trying it up on cluster to make sure its not 'broke'. Thanks for the patch Marc.
> Some improvements to Hbck to test the entire region chain in Meta and provide better error reporting
> ----------------------------------------------------------------------------------------------------
>
> Key: HBASE-3695
> URL: https://issues.apache.org/jira/browse/HBASE-3695
> Project: HBase
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.90.1
> Reporter: Marc Limotte
> Assignee: Marc Limotte
> Fix For: 0.90.3
>
>
> The current Hbck tool will miss some inconsistencies in Meta, and in other cases will detect an issue, but does not provide much in the way of useful feedback.
> * Incorporate the full region chain tests (similar to check_meta.rb). I.e. look for overlaps, holes and cycles. I believe check_meta.rb will be redundant after this change.
> * More unit tests, and better tests that will test the actual error discovered, instead of just errors true/false.
> * In the case of overlaps and holes, output both ends of the broken chain.
> * Previous implementation runs check() twice. This is inefficient and, more importantly, reports redundant errors which could be confusing to the user.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira