You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Kristam Subba Swathi (JIRA)" <ji...@apache.org> on 2012/08/06 12:51:03 UTC
[jira] [Created] (HIVE-3336) [Hive-Hbase Integration]:Data is not
inserted into the hbase table from the hive table if the value is null even
if the row key is not null
Kristam Subba Swathi created HIVE-3336:
------------------------------------------
Summary: [Hive-Hbase Integration]:Data is not inserted into the hbase table from the hive table if the value is null even if the row key is not null
Key: HIVE-3336
URL: https://issues.apache.org/jira/browse/HIVE-3336
Project: Hive
Issue Type: Bug
Components: HBase Handler
Affects Versions: 0.9.0
Reporter: Kristam Subba Swathi
Scenario
------------
1.create a new HBase table which is to be managed by Hive, use the STORED BY clause on CREATE TABLE as follows
Query:
{noformat}
CREATE TABLE hbase_table_join(key int, value string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
TBLPROPERTIES ("hbase.table.name" = "xyz_join");
{noformat}
2.Create two tables in hive and load data into the table as follows
table1:
{noformat}
describe hive_hbase_1;
OK
empno int
ename string
deptno int
hive> select * from hive_hbase_1;
OK
1 a 1
2 q 10
3 w 12
4 w 12
4 w 12
5 r 13
{noformat}
table2:
{noformat}
hive> describe hive_hbase_2;
OK
deptno int
dname string
count int
Time taken: 0.214 seconds
hive> select * from hive_hbase_2;
OK
11 a 11
12 q 101
13 w 121
14 w 121
2 we13 NULL
4 w 12
5 r 13
{noformat}
3.Insert the join result of these tables into hbase_table_join as follows
{noformat}
hive> insert into table hbase_table_join select /*+mapjoin(hive_hbase_1)*/ hive_hbase_2.deptno,hive_hbase_1.ename from hive_hbase_1 right outer join hive_hbase_2 on hive_hbase_1.deptno=hive_hbase_2.deptno;
{noformat}
Actual result of the select query is
---------------
{noformat}
11 NULL
12 w
12 w
12 w
13 r
14 NULL
2 NULL
4 NULL
5 NULL
{noformat}
But the above insert query is failing because of the following exception
---------------------
{noformat}
2012-07-17 18:41:45,003 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"deptno":11,"dname":"a","count":11}
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:400)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:335)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"deptno":11,"dname":"a","count":11}
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
... 8 more
Caused by: java.lang.IllegalArgumentException: No columns to insert
at org.apache.hadoop.hbase.client.HTable.validatePut(HTable.java:969)
at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:762)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:748)
at org.apache.hadoop.hive.hbase.HiveHBaseTableOutputFormat$1.write(HiveHBaseTableOutputFormat.java:98)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:588)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-3336) [Hive-Hbase Integration]:Data is not
inserted into the hbase table from the hive table if the value is null even
if the row key is not null
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam reassigned HIVE-3336:
--------------------------------------
Assignee: Chinna Rao Lalam
> [Hive-Hbase Integration]:Data is not inserted into the hbase table from the hive table if the value is null even if the row key is not null
> -------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-3336
> URL: https://issues.apache.org/jira/browse/HIVE-3336
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
> Affects Versions: 0.9.0
> Reporter: Kristam Subba Swathi
> Assignee: Chinna Rao Lalam
>
> Scenario
> ------------
> 1.create a new HBase table which is to be managed by Hive, use the STORED BY clause on CREATE TABLE as follows
> Query:
> {noformat}
> CREATE TABLE hbase_table_join(key int, value string)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "xyz_join");
> {noformat}
> 2.Create two tables in hive and load data into the table as follows
> table1:
> {noformat}
> describe hive_hbase_1;
> OK
> empno int
> ename string
> deptno int
> hive> select * from hive_hbase_1;
> OK
> 1 a 1
> 2 q 10
> 3 w 12
> 4 w 12
> 4 w 12
> 5 r 13
> {noformat}
> table2:
> {noformat}
> hive> describe hive_hbase_2;
> OK
> deptno int
> dname string
> count int
> Time taken: 0.214 seconds
> hive> select * from hive_hbase_2;
> OK
> 11 a 11
> 12 q 101
> 13 w 121
> 14 w 121
> 2 we13 NULL
> 4 w 12
> 5 r 13
> {noformat}
> 3.Insert the join result of these tables into hbase_table_join as follows
> {noformat}
> hive> insert into table hbase_table_join select /*+mapjoin(hive_hbase_1)*/ hive_hbase_2.deptno,hive_hbase_1.ename from hive_hbase_1 right outer join hive_hbase_2 on hive_hbase_1.deptno=hive_hbase_2.deptno;
> {noformat}
> Actual result of the select query is
> ---------------
> {noformat}
> 11 NULL
> 12 w
> 12 w
> 12 w
> 13 r
> 14 NULL
> 2 NULL
> 4 NULL
> 5 NULL
> {noformat}
> But the above insert query is failing because of the following exception
> ---------------------
> {noformat}
> 2012-07-17 18:41:45,003 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"deptno":11,"dname":"a","count":11}
> at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:400)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:335)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"deptno":11,"dname":"a","count":11}
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
> ... 8 more
> Caused by: java.lang.IllegalArgumentException: No columns to insert
> at org.apache.hadoop.hbase.client.HTable.validatePut(HTable.java:969)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:762)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:748)
> at org.apache.hadoop.hive.hbase.HiveHBaseTableOutputFormat$1.write(HiveHBaseTableOutputFormat.java:98)
> at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:588)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira