You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "James Taylor (Created) (JIRA)" <ji...@apache.org> on 2012/04/04 03:04:24 UTC
[jira] [Created] (HBASE-5710) NPE in MiniCluster during metadata
scan for a pre-split table with multiple column families
NPE in MiniCluster during metadata scan for a pre-split table with multiple column families
-------------------------------------------------------------------------------------------
Key: HBASE-5710
URL: https://issues.apache.org/jira/browse/HBASE-5710
Project: HBase
Issue Type: Bug
Components: test, util
Affects Versions: 0.94.0
Environment: MiniCluster
Reporter: James Taylor
Priority: Minor
Fix For: 0.94.0
In the MiniCluster test environment, an NPE occurs while scanning regions
of a pre-split table with multiple column families. Without this working
in the test environment, you cannot write unit tests for these types of
scenarios.
Add the following to TestMetaScanner to repro:
@Test
public void testMultiFamilyMultiRegionMetaScanner() throws Exception {
LOG.info("Starting testMetaScanner");
final byte[] TABLENAME = Bytes.toBytes("testMetaScanner");
final byte[] FAMILY1 = Bytes.toBytes("family1");
final byte[] FAMILY2 = Bytes.toBytes("family2");
TEST_UTIL.createTable(TABLENAME, new byte[][] {FAMILY1,FAMILY2});
Configuration conf = TEST_UTIL.getConfiguration();
HTable table = new HTable(conf, TABLENAME);
TEST_UTIL.createMultiRegions(conf, table, FAMILY1,
new byte[][]{
HConstants.EMPTY_START_ROW,
Bytes.toBytes("region_a"),
Bytes.toBytes("region_b")});
TEST_UTIL.createMultiRegions(conf, table, FAMILY2,
new byte[][]{
HConstants.EMPTY_START_ROW,
Bytes.toBytes("region_a"),
Bytes.toBytes("region_b")});
// Make sure all the regions are deployed
TEST_UTIL.countRows(table);
// This fails with an NPE currently
MetaScanner.allTableRegions(conf, TABLENAME, false).keySet();
table.close();
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5710) NPE in MiniCluster during metadata
scan for a pre-split table with multiple column families
Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-5710:
---------------------------------
Fix Version/s: (was: 0.94.0)
> NPE in MiniCluster during metadata scan for a pre-split table with multiple column families
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-5710
> URL: https://issues.apache.org/jira/browse/HBASE-5710
> Project: HBase
> Issue Type: Bug
> Components: test, util
> Affects Versions: 0.94.0
> Environment: MiniCluster
> Reporter: James Taylor
> Priority: Minor
>
> In the MiniCluster test environment, an NPE occurs while scanning regions
> of a pre-split table with multiple column families. Without this working
> in the test environment, you cannot write unit tests for these types of
> scenarios.
> Add the following to TestMetaScanner to repro:
> @Test
> public void testMultiFamilyMultiRegionMetaScanner() throws Exception {
> LOG.info("Starting testMetaScanner");
> final byte[] TABLENAME = Bytes.toBytes("testMetaScanner");
> final byte[] FAMILY1 = Bytes.toBytes("family1");
> final byte[] FAMILY2 = Bytes.toBytes("family2");
> TEST_UTIL.createTable(TABLENAME, new byte[][] {FAMILY1,FAMILY2});
> Configuration conf = TEST_UTIL.getConfiguration();
> HTable table = new HTable(conf, TABLENAME);
> TEST_UTIL.createMultiRegions(conf, table, FAMILY1,
> new byte[][]{
> HConstants.EMPTY_START_ROW,
> Bytes.toBytes("region_a"),
> Bytes.toBytes("region_b")});
> TEST_UTIL.createMultiRegions(conf, table, FAMILY2,
> new byte[][]{
> HConstants.EMPTY_START_ROW,
> Bytes.toBytes("region_a"),
> Bytes.toBytes("region_b")});
> // Make sure all the regions are deployed
> TEST_UTIL.countRows(table);
> // This fails with an NPE currently
> MetaScanner.allTableRegions(conf, TABLENAME, false).keySet();
> table.close();
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5710) NPE in MiniCluster during metadata
scan for a pre-split table with multiple column families
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247641#comment-13247641 ]
Lars Hofhansl commented on HBASE-5710:
--------------------------------------
I took a look. The problem is this:
The problem occurs because createMultiRegions tries to clean up the empty default region for an existing table. It does that by listing the existing regions and then deleting them *after* the new regions were added.
Since regions are per table, not per column family createMultiRegions should only be called *once*, regardless of how many column families there are.
So createMultiRegions really should not have columnFamily as parameter, this is pure convenience to add the column family if the table did not have it. IMHO it is more confusing than useful. And it should document that it should only be called once.
I don't think this necessarily needs to be fixed, though. In your case you create the table with the two column families and then just pass the first column family to a single call to createMultiRegions, which will set up the table's regions correctly:
{code}
@Test
public void testMultiFamilyMultiRegionMetaScanner() throws Exception {
LOG.info("Starting testMetaScanner");
final byte[] TABLENAME = Bytes.toBytes("testMetaScanner");
final byte[] FAMILY1 = Bytes.toBytes("family1");
final byte[] FAMILY2 = Bytes.toBytes("family2");
TEST_UTIL.createTable(TABLENAME, new byte[][] { FAMILY1, FAMILY2 });
Configuration conf = TEST_UTIL.getConfiguration();
HTable table = new HTable(conf, TABLENAME);
TEST_UTIL.createMultiRegions(conf, table, FAMILY1, new byte[][] { HConstants.EMPTY_START_ROW,
Bytes.toBytes("region_a"), Bytes.toBytes("region_b") });
TEST_UTIL.countRows(table);
// This fails with an NPE currently
MetaScanner.allTableRegions(conf, TABLENAME, false).keySet();
table.close();
}
{code}
> NPE in MiniCluster during metadata scan for a pre-split table with multiple column families
> -------------------------------------------------------------------------------------------
>
> Key: HBASE-5710
> URL: https://issues.apache.org/jira/browse/HBASE-5710
> Project: HBase
> Issue Type: Bug
> Components: test, util
> Affects Versions: 0.94.0
> Environment: MiniCluster
> Reporter: James Taylor
> Priority: Minor
> Fix For: 0.94.0
>
>
> In the MiniCluster test environment, an NPE occurs while scanning regions
> of a pre-split table with multiple column families. Without this working
> in the test environment, you cannot write unit tests for these types of
> scenarios.
> Add the following to TestMetaScanner to repro:
> @Test
> public void testMultiFamilyMultiRegionMetaScanner() throws Exception {
> LOG.info("Starting testMetaScanner");
> final byte[] TABLENAME = Bytes.toBytes("testMetaScanner");
> final byte[] FAMILY1 = Bytes.toBytes("family1");
> final byte[] FAMILY2 = Bytes.toBytes("family2");
> TEST_UTIL.createTable(TABLENAME, new byte[][] {FAMILY1,FAMILY2});
> Configuration conf = TEST_UTIL.getConfiguration();
> HTable table = new HTable(conf, TABLENAME);
> TEST_UTIL.createMultiRegions(conf, table, FAMILY1,
> new byte[][]{
> HConstants.EMPTY_START_ROW,
> Bytes.toBytes("region_a"),
> Bytes.toBytes("region_b")});
> TEST_UTIL.createMultiRegions(conf, table, FAMILY2,
> new byte[][]{
> HConstants.EMPTY_START_ROW,
> Bytes.toBytes("region_a"),
> Bytes.toBytes("region_b")});
> // Make sure all the regions are deployed
> TEST_UTIL.countRows(table);
> // This fails with an NPE currently
> MetaScanner.allTableRegions(conf, TABLENAME, false).keySet();
> table.close();
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira