You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Anoop Sam John (Jira)" <ji...@apache.org> on 2021/06/01 11:59:00 UTC
[jira] [Commented] (HBASE-24734) Wrong comparator opening Region
when 'split-to-WAL' enabled.
[ https://issues.apache.org/jira/browse/HBASE-24734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17355058#comment-17355058 ]
Anoop Sam John commented on HBASE-24734:
----------------------------------------
When WAL split happens for META region, we will hit this issue right?
bq. we should at least fail bulk load if trying to bulk load meta
So we should not use split to HFile for META table's WALs? [~stack]
> Wrong comparator opening Region when 'split-to-WAL' enabled.
> ------------------------------------------------------------
>
> Key: HBASE-24734
> URL: https://issues.apache.org/jira/browse/HBASE-24734
> Project: HBase
> Issue Type: Sub-task
> Components: HFile, MTTR
> Reporter: Michael Stack
> Priority: Major
>
> Came across this when we were testing the 'split-to-hfile' feature running ITBLL:
>
> {code:java}
> 2020-07-10 10:16:49,983 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closing region hbase:meta,,1.15882307402020-07-10 10:16:49,997 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed hbase:meta,,1.15882307402020-07-10 10:16:49,998 WARN org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler: Fatal error occurred while opening region hbase:meta,,1.1588230740, aborting...java.lang.IllegalArgumentException: Invalid range: IntegrationTestBigLinkedList,,1594350463222.8f89e01a5245e79946e22d8a8ab4698b. > IntegrationTestBigLinkedList,\x10\x02J\xA1,1594349535271.be24dc276f686e6dcc7fb9d3f91c8387. at org.apache.hadoop.hbase.client.RegionInfoBuilder$MutableRegionInfo.containsRange(RegionInfoBuilder.java:300) at org.apache.hadoop.hbase.regionserver.HStore.tryCommitRecoveredHFile(HStore.java:1111) at org.apache.hadoop.hbase.regionserver.HRegion.loadRecoveredHFilesIfAny(HRegion.java:5442) at org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:1010) at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:950) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7490) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegionFromTableDir(HRegion.java:7448) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7424) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7382) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7333) at org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:135) at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834)2020-07-10 10:16:50,005 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: ***** ABORTING region server hbasedn149.example.org,16020,1594375563853: Failed to open region hbase:meta,,1.1588230740 and can not recover *****java.lang.IllegalArgumentException: Invalid range: IntegrationTestBigLinkedList,,1594350463222.8f89e01a5245e79946e22d8a8ab4698b. > IntegrationTestBigLinkedList,\x10\x02J\xA1,1594349535271.be24dc276f686e6dcc7fb9d3f91c8387.
> {code}
> Seems basic case of wrong comparator. Below passes if I use the meta comparator
> {code:java}
> @Test
> public void testBinaryKeys() throws Exception {
> Set<Cell> set = new TreeSet<>(CellComparatorImpl.COMPARATOR);
> final byte [] fam = Bytes.toBytes("col");
> final byte [] qf = Bytes.toBytes("umn");
> final byte [] nb = new byte[0];
> Cell [] keys = {
> createByteBufferKeyValueFromKeyValue(
> new KeyValue(Bytes.toBytes("aaaaa,\u0000\u0000,2"), fam, qf, 2, nb)),
> createByteBufferKeyValueFromKeyValue(
> new KeyValue(Bytes.toBytes("aaaaa,\u0001,3"), fam, qf, 3, nb)),
> createByteBufferKeyValueFromKeyValue(
> new KeyValue(Bytes.toBytes("aaaaa,,1"), fam, qf, 1, nb)),
> createByteBufferKeyValueFromKeyValue(
> new KeyValue(Bytes.toBytes("aaaaa,\u1000,5"), fam, qf, 5, nb)),
> createByteBufferKeyValueFromKeyValue(
> new KeyValue(Bytes.toBytes("aaaaa,a,4"), fam, qf, 4, nb)),
> createByteBufferKeyValueFromKeyValue(
> new KeyValue(Bytes.toBytes("a,a,0"), fam, qf, 0, nb)),
> };
> // Add to set with bad comparator
> Collections.addAll(set, keys);
> // This will output the keys incorrectly.
> boolean assertion = false;
> int count = 0;
> try {
> for (Cell k: set) {
> assertTrue("count=" + count + ", " + k.toString(), count++ == k.getTimestamp());
> }
> } catch (AssertionError e) {
> // Expected
> assertion = true;
> }
> assertTrue(assertion);
> // Make set with good comparator
> set = new TreeSet<>(CellComparatorImpl.META_COMPARATOR);
> Collections.addAll(set, keys);
> count = 0;
> for (Cell k: set) {
> assertTrue("count=" + count + ", " + k.toString(), count++ == k.getTimestamp());
> }
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)