[jira] [Created] (DRILL-4304) Select from hive hbase table fails with NPE when hbase.table.name is not used

["Krystal (JIRA)" <ji...@apache.org>]

Krystal created DRILL-4304:
------------------------------

             Summary: Select from hive hbase table fails with NPE when hbase.table.name is not used
                 Key: DRILL-4304
                 URL: https://issues.apache.org/jira/browse/DRILL-4304
             Project: Apache Drill
          Issue Type: Bug
          Components: Storage - Hive
    Affects Versions: 1.5.0
            Reporter: Krystal


git.commit.id.abbrev=3d0b4b0

I can query against the following hive/hbase table fine:

create table hbase_voter(key int,name varchar(30),age int,registration string,contributions decimal(6,2),voternum smallint,create_date timestamp)STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ':key,onecf:name,twocf:age,twocf:registration,threecf:contributions,threecf:voterzone,fourcf:create_date') TBLPROPERTIES("hbase.table.name" = "hbase_voter");

However, if I omit the "hbase.table.name" (which is optional in hive), it fails when I query against this table from sqlline:

create table hbase_voter(key int,name varchar(30),age int,registration string,contributions decimal(6,2),voternum smallint,create_date timestamp)STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ':key,onecf:name,twocf:age,twocf:registration,threecf:contributions,threecf:voterzone,fourcf:create_date');

Here is the stack trace:
{code}
SYSTEM ERROR: NullPointerException



  (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception during fragment initialization: Internal error: Error while applying rule DrillPushProjIntoScan, args [rel#8:LogicalProject.NONE.ANY([]).[](input=rel#7:Subset#0.ENUMERABLE.ANY([]).[],key=$0,name=$1,age=$2,registration=$3,contributions=$4,voternum=$5,create_date=$6), rel#2:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[hive, hbase_voter2])]
    org.apache.drill.exec.work.foreman.Foreman.run():261
    java.util.concurrent.ThreadPoolExecutor.runWorker():1145
    java.util.concurrent.ThreadPoolExecutor$Worker.run():615
    java.lang.Thread.run():745
  Caused By (java.lang.AssertionError) Internal error: Error while applying rule DrillPushProjIntoScan, args [rel#8:LogicalProject.NONE.ANY([]).[](input=rel#7:Subset#0.ENUMERABLE.ANY([]).[],key=$0,name=$1,age=$2,registration=$3,contributions=$4,voternum=$5,create_date=$6), rel#2:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[hive, hbase_voter2])]
    org.apache.calcite.util.Util.newInternal():792
    org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():251
    org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():808
    org.apache.calcite.tools.Programs$RuleSetProgram.run():303
    org.apache.calcite.prepare.PlannerImpl.transform():313
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.doLogicalPlanning():542
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():218
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():252
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():172
    org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():199
    org.apache.drill.exec.work.foreman.Foreman.runSQL():924
    org.apache.drill.exec.work.foreman.Foreman.run():250
    java.util.concurrent.ThreadPoolExecutor.runWorker():1145
    java.util.concurrent.ThreadPoolExecutor$Worker.run():615
    java.lang.Thread.run():745
  Caused By (java.lang.AssertionError) Internal error: Error occurred while applying rule DrillPushProjIntoScan
    org.apache.calcite.util.Util.newInternal():792
    org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo():150
    org.apache.calcite.plan.RelOptRuleCall.transformTo():213
    org.apache.drill.exec.planner.logical.DrillPushProjIntoScan.onMatch():86
    org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():228
    org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():808
    org.apache.calcite.tools.Programs$RuleSetProgram.run():303
    org.apache.calcite.prepare.PlannerImpl.transform():313
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.doLogicalPlanning():542
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():218
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():252
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():172
    org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():199
    org.apache.drill.exec.work.foreman.Foreman.runSQL():924
    org.apache.drill.exec.work.foreman.Foreman.run():250
    java.util.concurrent.ThreadPoolExecutor.runWorker():1145
    java.util.concurrent.ThreadPoolExecutor$Worker.run():615
    java.lang.Thread.run():745
  Caused By (java.lang.reflect.UndeclaredThrowableException) null
    com.sun.proxy.$Proxy65.getNonCumulativeCost():-1
    org.apache.calcite.rel.metadata.RelMetadataQuery.getNonCumulativeCost():115
    org.apache.calcite.plan.volcano.VolcanoPlanner.getCost():1112
    org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0():363
    org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements():344
    org.apache.calcite.plan.volcano.VolcanoPlanner.addRelToSet():1827
    org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl():1760
    org.apache.calcite.plan.volcano.VolcanoPlanner.register():1017
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():1037
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():117
    org.apache.calcite.rel.AbstractRelNode.onRegister():305
    org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl():1658
    org.apache.calcite.plan.volcano.VolcanoPlanner.register():1017
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():1037
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():1940
    org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo():138
    org.apache.calcite.plan.RelOptRuleCall.transformTo():213
    org.apache.drill.exec.planner.logical.DrillPushProjIntoScan.onMatch():86
    org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():228
    org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():808
    org.apache.calcite.tools.Programs$RuleSetProgram.run():303
    org.apache.calcite.prepare.PlannerImpl.transform():313
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.doLogicalPlanning():542
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():218
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():252
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():172
    org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():199
    org.apache.drill.exec.work.foreman.Foreman.runSQL():924
    org.apache.drill.exec.work.foreman.Foreman.run():250
    java.util.concurrent.ThreadPoolExecutor.runWorker():1145
    java.util.concurrent.ThreadPoolExecutor$Worker.run():615
    java.lang.Thread.run():745
  Caused By (java.lang.reflect.InvocationTargetException) null
    sun.reflect.NativeMethodAccessorImpl.invoke0():-2
    sun.reflect.NativeMethodAccessorImpl.invoke():57
    sun.reflect.DelegatingMethodAccessorImpl.invoke():43
    java.lang.reflect.Method.invoke():606
    org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$1$1.invoke():182
    com.sun.proxy.$Proxy65.getNonCumulativeCost():-1
    org.apache.calcite.rel.metadata.RelMetadataQuery.getNonCumulativeCost():115
    org.apache.calcite.plan.volcano.VolcanoPlanner.getCost():1112
    org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0():363
    org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements():344
    org.apache.calcite.plan.volcano.VolcanoPlanner.addRelToSet():1827
    org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl():1760
    org.apache.calcite.plan.volcano.VolcanoPlanner.register():1017
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():1037
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():117
    org.apache.calcite.rel.AbstractRelNode.onRegister():305
    org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl():1658
    org.apache.calcite.plan.volcano.VolcanoPlanner.register():1017
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():1037
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():1940
    org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo():138
    org.apache.calcite.plan.RelOptRuleCall.transformTo():213
    org.apache.drill.exec.planner.logical.DrillPushProjIntoScan.onMatch():86
    org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():228
    org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():808
    org.apache.calcite.tools.Programs$RuleSetProgram.run():303
    org.apache.calcite.prepare.PlannerImpl.transform():313
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.doLogicalPlanning():542
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():218
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():252
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():172
    org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():199
    org.apache.drill.exec.work.foreman.Foreman.runSQL():924
    org.apache.drill.exec.work.foreman.Foreman.run():250
    java.util.concurrent.ThreadPoolExecutor.runWorker():1145
    java.util.concurrent.ThreadPoolExecutor$Worker.run():615
    java.lang.Thread.run():745
  Caused By (org.apache.drill.common.exceptions.DrillRuntimeException) java.io.IOException: Failed to get numRows from HiveTable
    org.apache.drill.exec.store.hive.HiveScan.getScanStats():233
    org.apache.drill.exec.physical.base.AbstractGroupScan.getScanStats():76
    org.apache.drill.exec.planner.logical.DrillScanRel.computeSelfCost():164
    org.apache.calcite.rel.metadata.RelMdPercentageOriginalRows.getNonCumulativeCost():165
    sun.reflect.NativeMethodAccessorImpl.invoke0():-2
    sun.reflect.NativeMethodAccessorImpl.invoke():57
    sun.reflect.DelegatingMethodAccessorImpl.invoke():43
    java.lang.reflect.Method.invoke():606
    org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$1$1.invoke():182
    com.sun.proxy.$Proxy65.getNonCumulativeCost():-1
    org.apache.calcite.rel.metadata.RelMetadataQuery.getNonCumulativeCost():115
    org.apache.calcite.plan.volcano.VolcanoPlanner.getCost():1112
    org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0():363
    org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements():344
    org.apache.calcite.plan.volcano.VolcanoPlanner.addRelToSet():1827
    org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl():1760
    org.apache.calcite.plan.volcano.VolcanoPlanner.register():1017
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():1037
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():117
    org.apache.calcite.rel.AbstractRelNode.onRegister():305
    org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl():1658
    org.apache.calcite.plan.volcano.VolcanoPlanner.register():1017
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():1037
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():1940
    org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo():138
    org.apache.calcite.plan.RelOptRuleCall.transformTo():213
    org.apache.drill.exec.planner.logical.DrillPushProjIntoScan.onMatch():86
    org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():228
    org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():808
    org.apache.calcite.tools.Programs$RuleSetProgram.run():303
    org.apache.calcite.prepare.PlannerImpl.transform():313
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.doLogicalPlanning():542
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():218
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():252
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():172
    org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():199
    org.apache.drill.exec.work.foreman.Foreman.runSQL():924
    org.apache.drill.exec.work.foreman.Foreman.run():250
    java.util.concurrent.ThreadPoolExecutor.runWorker():1145
    java.util.concurrent.ThreadPoolExecutor$Worker.run():615
    java.lang.Thread.run():745
  Caused By (java.io.IOException) Failed to get numRows from HiveTable
    org.apache.drill.exec.store.hive.HiveMetadataProvider.getStats():110
    org.apache.drill.exec.store.hive.HiveScan.getScanStats():224
    org.apache.drill.exec.physical.base.AbstractGroupScan.getScanStats():76
    org.apache.drill.exec.planner.logical.DrillScanRel.computeSelfCost():164
    org.apache.calcite.rel.metadata.RelMdPercentageOriginalRows.getNonCumulativeCost():165
    sun.reflect.NativeMethodAccessorImpl.invoke0():-2
    sun.reflect.NativeMethodAccessorImpl.invoke():57
    sun.reflect.DelegatingMethodAccessorImpl.invoke():43
    java.lang.reflect.Method.invoke():606
    org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$1$1.invoke():182
    com.sun.proxy.$Proxy65.getNonCumulativeCost():-1
    org.apache.calcite.rel.metadata.RelMetadataQuery.getNonCumulativeCost():115
    org.apache.calcite.plan.volcano.VolcanoPlanner.getCost():1112
    org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0():363
    org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements():344
    org.apache.calcite.plan.volcano.VolcanoPlanner.addRelToSet():1827
    org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl():1760
    org.apache.calcite.plan.volcano.VolcanoPlanner.register():1017
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():1037
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():117
    org.apache.calcite.rel.AbstractRelNode.onRegister():305
    org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl():1658
    org.apache.calcite.plan.volcano.VolcanoPlanner.register():1017
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():1037
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():1940
    org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo():138
    org.apache.calcite.plan.RelOptRuleCall.transformTo():213
    org.apache.drill.exec.planner.logical.DrillPushProjIntoScan.onMatch():86
    org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():228
    org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():808
    org.apache.calcite.tools.Programs$RuleSetProgram.run():303
    org.apache.calcite.prepare.PlannerImpl.transform():313
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.doLogicalPlanning():542
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():218
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():252
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():172
    org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():199
    org.apache.drill.exec.work.foreman.Foreman.runSQL():924
    org.apache.drill.exec.work.foreman.Foreman.run():250
    java.util.concurrent.ThreadPoolExecutor.runWorker():1145
    java.util.concurrent.ThreadPoolExecutor$Worker.run():615
    java.lang.Thread.run():745
  Caused By (java.lang.NullPointerException) null
    org.apache.hadoop.hbase.util.Bytes.toBytes():503
    org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat.getSplitsInternal():444
    org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat.getSplits():432
    org.apache.drill.exec.store.hive.HiveMetadataProvider$1.run():250
    org.apache.drill.exec.store.hive.HiveMetadataProvider$1.run():238
    java.security.AccessController.doPrivileged():-2
    javax.security.auth.Subject.doAs():415
    org.apache.hadoop.security.UserGroupInformation.doAs():1595
    org.apache.drill.exec.store.hive.HiveMetadataProvider.splitInputWithUGI():238
    org.apache.drill.exec.store.hive.HiveMetadataProvider.getTableInputSplits():125
    org.apache.drill.exec.store.hive.HiveMetadataProvider.getStats():93
    org.apache.drill.exec.store.hive.HiveScan.getScanStats():224
    org.apache.drill.exec.physical.base.AbstractGroupScan.getScanStats():76
    org.apache.drill.exec.planner.logical.DrillScanRel.computeSelfCost():164
    org.apache.calcite.rel.metadata.RelMdPercentageOriginalRows.getNonCumulativeCost():165
    sun.reflect.NativeMethodAccessorImpl.invoke0():-2
    sun.reflect.NativeMethodAccessorImpl.invoke():57
    sun.reflect.DelegatingMethodAccessorImpl.invoke():43
    java.lang.reflect.Method.invoke():606
    org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$1$1.invoke():182
    com.sun.proxy.$Proxy65.getNonCumulativeCost():-1
    org.apache.calcite.rel.metadata.RelMetadataQuery.getNonCumulativeCost():115
    org.apache.calcite.plan.volcano.VolcanoPlanner.getCost():1112
    org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0():363
    org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements():344
    org.apache.calcite.plan.volcano.VolcanoPlanner.addRelToSet():1827
    org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl():1760
    org.apache.calcite.plan.volcano.VolcanoPlanner.register():1017
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():1037
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():117
    org.apache.calcite.rel.AbstractRelNode.onRegister():305
    org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl():1658
    org.apache.calcite.plan.volcano.VolcanoPlanner.register():1017
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():1037
    org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered():1940
    org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo():138
    org.apache.calcite.plan.RelOptRuleCall.transformTo():213
    org.apache.drill.exec.planner.logical.DrillPushProjIntoScan.onMatch():86
    org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch():228
    org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp():808
    org.apache.calcite.tools.Programs$RuleSetProgram.run():303
    org.apache.calcite.prepare.PlannerImpl.transform():313
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.doLogicalPlanning():542
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():218
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel():252
    org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():172
    org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():199
    org.apache.drill.exec.work.foreman.Foreman.runSQL():924
    org.apache.drill.exec.work.foreman.Foreman.run():250
    java.util.concurrent.ThreadPoolExecutor.runWorker():1145
    java.util.concurrent.ThreadPoolExecutor$Worker.run():615
    java.lang.Thread.run():745
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)