You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by "Omer, Farah" <fo...@microstrategy.com> on 2012/08/14 17:05:38 UTC
Issue with creating table in hbase
Hi all,
I was testing hbase integrated with hive, and running into an issue. Would anyone has an idea what it means?
hbase(main):001:0> CREATE TABLE hbase_mstr_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name" = "xyz")
SyntaxError: (hbase):1: syntax error, unexpected tIDENTIFIER
CREATE TABLE hbase_mstr_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name" = "xyz")
^
hbase(main):002:0>
Hive version is 0.7, hbase version is 0.90
Thanks.
Farah Omer
Sr. DB Engineer | MicroStrategy, Inc.
Tel 703.270.2230 | fomer@microstrategy.com
1850 Towers Crescent Plaza | Tysons Corner, VA 22182
www.microstrategy.com
RE: Issue with creating table in hbase
Posted by "Omer, Farah" <fo...@microstrategy.com>.
Here is the complete stacktrace.
Killed Tasks
Task
Complete
Status
Start Time
Finish Time
Errors
Counters
task_201207251201_0678_m_000000<http://hadoop001:50030/taskdetails.jsp?tipid=task_201207251201_0678_m_000000>
0.00%
14-Aug-2012 12:30:20
14-Aug-2012 12:30:44 (24sec)
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:387)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.jav
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:387)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.jav
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:387)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.jav
0<http://hadoop001:50030/taskstats.jsp?tipid=task_201207251201_0678_m_000000>
Farah Omer
Sr. DB Engineer | MicroStrategy, Inc.
Tel 703.270.2230 | fomer@microstrategy.com
1850 Towers Crescent Plaza | Tysons Corner, VA 22182
www.microstrategy.com
From: kulkarni.swarnim@gmail.com [mailto:kulkarni.swarnim@gmail.com]
Sent: Tuesday, August 14, 2012 1:06 PM
To: user@hive.apache.org
Subject: Re: Issue with creating table in hbase
Is that the complete stacktrace?
On Tue, Aug 14, 2012 at 12:01 PM, Omer, Farah <fo...@microstrategy.com>> wrote:
Unfortunately the job's log also doesn't tell me anything very meaningful. Have you or anyone might have seen this before?
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:387)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.jav
Thanks.
Farah Omer
Sr. DB Engineer | MicroStrategy, Inc.
Tel 703.270.2230<tel:703.270.2230> | fomer@microstrategy.com<ht...@microstrategy.com>
1850 Towers Crescent Plaza | Tysons Corner, VA 22182
www.microstrategy.com<http://www.microstrategy.com>
From: kulkarni.swarnim@gmail.com<ma...@gmail.com> [mailto:kulkarni.swarnim@gmail.com<ma...@gmail.com>]
Sent: Tuesday, August 14, 2012 12:49 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: Issue with creating table in hbase
It seems like your Map reduce job is failing. Refer to the logs in the tracking URL "http://hadoop001:50030/jobdetails.jsp?jobid=job_201207251201_0678" to see why exactly it is failing.
On Tue, Aug 14, 2012 at 11:35 AM, Omer, Farah <fo...@microstrategy.com>> wrote:
Thanks. That helped.
Another related question:
I created this table on HIVE:
hive> CREATE TABLE hbase_mstr_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name<http://hbase.table.name>" = "hbase_mstr_1_xyz");
My intention is to load it and then access it via HIVE. But I am not able to insert values into this table:
hive> insert overwrite table hbase_mstr_1 select * from eatwh1_ft1;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201207251201_0678, Tracking URL = http://hadoop001:50030/jobdetails.jsp?jobid=job_201207251201_0678
Kill Command = /usr/lib/hadoop/bin/hadoop job -Dmapred.job.tracker=hadoop001:6932 -kill job_201207251201_0678
2012-08-14 12:30:20,977 Stage-0 map = 0%, reduce = 0%
2012-08-14 12:30:47,110 Stage-0 map = 100%, reduce = 100%
Ended Job = job_201207251201_0678 with errors
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
The hive.log doesn't tell a lot of useful information.
2012-08-14 11:55:51,893 WARN hbase.HBaseConfiguration (HBaseConfiguration.java:<init>(45)) - instantiating HBaseConfiguration() is deprecated. Please use HBaseConfiguration#create() to construct a plain Configuration
2012-08-14 11:55:52,302 WARN mapred.JobClient (JobClient.java:copyAndConfigureFiles(649)) - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
2012-08-14 11:56:21,132 ERROR exec.MapRedTask (SessionState.java:printError(365)) - Ended Job = job_201207251201_0677 with errors
2012-08-14 11:56:21,151 ERROR ql.Driver (SessionState.java:printError(365)) - FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
Any ideas?
Thank you.
Farah Omer
Sr. DB Engineer | MicroStrategy, Inc.
Tel 703.270.2230<tel:703.270.2230> | fomer@microstrategy.com<ht...@microstrategy.com>
1850 Towers Crescent Plaza | Tysons Corner, VA 22182
www.microstrategy.com<http://www.microstrategy.com>
From: kulkarni.swarnim@gmail.com<ma...@gmail.com> [mailto:kulkarni.swarnim@gmail.com<ma...@gmail.com>]
Sent: Tuesday, August 14, 2012 11:11 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: Issue with creating table in hbase
> hbase(main):001:0> CREATE TABLE hbase_mstr_1.....
Are you running the CREATE TABLE from the hbase shell? You should run it from the hive shell. You can start it from "$HIVE_HOME/bin/hive"
On Tue, Aug 14, 2012 at 10:05 AM, Omer, Farah <fo...@microstrategy.com>> wrote:
Hi all,
I was testing hbase integrated with hive, and running into an issue. Would anyone has an idea what it means?
hbase(main):001:0> CREATE TABLE hbase_mstr_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name<http://hbase.table.name>" = "xyz")
SyntaxError: (hbase):1: syntax error, unexpected tIDENTIFIER
CREATE TABLE hbase_mstr_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name<http://hbase.table.name>" = "xyz")
^
hbase(main):002:0>
Hive version is 0.7, hbase version is 0.90
Thanks.
Farah Omer
Sr. DB Engineer | MicroStrategy, Inc.
Tel 703.270.2230<tel:703.270.2230> | fomer@microstrategy.com<ht...@microstrategy.com>
1850 Towers Crescent Plaza | Tysons Corner, VA 22182
www.microstrategy.com<http://www.microstrategy.com>
--
Swarnim
--
Swarnim
--
Swarnim
Re: Issue with creating table in hbase
Posted by "kulkarni.swarnim@gmail.com" <ku...@gmail.com>.
Is that the complete stacktrace?
On Tue, Aug 14, 2012 at 12:01 PM, Omer, Farah <fo...@microstrategy.com>wrote:
> Unfortunately the job’s log also doesn’t tell me anything very
> meaningful. Have you or anyone might have seen this before?****
>
> ****
>
> java.lang.RuntimeException: Error in configuring object****
>
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> ****
>
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)***
> *
>
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> ****
>
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:387)
> ****
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)****
>
> at org.apache.hadoop.mapred.Child$4.run(Child.java:270)****
>
> at java.security.AccessController.doPrivileged(Native Method)****
>
> at javax.security.auth.Subject.doAs(Subject.java:396)****
>
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
> ****
>
> at org.apache.hadoop.mapred.Child.main(Child.java:264)****
>
> Caused by: java.lang.reflect.InvocationTargetException****
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)****
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> ****
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.jav
> ****
>
> ** **
>
> ** **
>
> Thanks.****
>
> ** **
>
> ** **
>
> Farah Omer****
>
> Sr. DB Engineer | MicroStrategy, Inc.****
>
> Tel 703.270.2230 | fomer@microstrategy.com****
>
> 1850 Towers Crescent Plaza | Tysons Corner, VA 22182****
>
> www.microstrategy.com****
>
> ** **
>
> *From:* kulkarni.swarnim@gmail.com [mailto:kulkarni.swarnim@gmail.com]
> *Sent:* Tuesday, August 14, 2012 12:49 PM
>
> *To:* user@hive.apache.org
> *Subject:* Re: Issue with creating table in hbase****
>
> ** **
>
> It seems like your Map reduce job is failing. Refer to the logs in the
> tracking URL "
> http://hadoop001:50030/jobdetails.jsp?jobid=job_201207251201_0678" to see
> why exactly it is failing.****
>
> ** **
>
> On Tue, Aug 14, 2012 at 11:35 AM, Omer, Farah <fo...@microstrategy.com>
> wrote:****
>
> Thanks. That helped.****
>
> ****
>
> Another related question:****
>
> ****
>
> I created this table on HIVE:****
>
> hive> CREATE TABLE hbase_mstr_1(key int, value string) STORED BY
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES
> ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("
> hbase.table.name" = "hbase_mstr_1_xyz");****
>
> ****
>
> My intention is to load it and then access it via HIVE. But I am not able
> to insert values into this table:****
>
> ****
>
> hive> insert overwrite table hbase_mstr_1 select * from eatwh1_ft1;****
>
> Total MapReduce jobs = 1****
>
> Launching Job 1 out of 1****
>
> Number of reduce tasks is set to 0 since there's no reduce operator****
>
> Starting Job = job_201207251201_0678, Tracking URL =
> http://hadoop001:50030/jobdetails.jsp?jobid=job_201207251201_0678****
>
> Kill Command = /usr/lib/hadoop/bin/hadoop job
> -Dmapred.job.tracker=hadoop001:6932 -kill job_201207251201_0678****
>
> 2012-08-14 12:30:20,977 Stage-0 map = 0%, reduce = 0%****
>
> 2012-08-14 12:30:47,110 Stage-0 map = 100%, reduce = 100%****
>
> Ended Job = job_201207251201_0678 with errors****
>
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.MapRedTask****
>
> ****
>
> The hive.log doesn’t tell a lot of useful information.****
>
> ****
>
> 2012-08-14 11:55:51,893 WARN hbase.HBaseConfiguration
> (HBaseConfiguration.java:<init>(45)) - instantiating HBaseConfiguration()
> is deprecated. Please use HBaseConfiguration#create() to construct a plain
> Configuration****
>
> 2012-08-14 11:55:52,302 WARN mapred.JobClient
> (JobClient.java:copyAndConfigureFiles(649)) - Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.***
> *
>
> 2012-08-14 11:56:21,132 ERROR exec.MapRedTask
> (SessionState.java:printError(365)) - Ended Job = job_201207251201_0677
> with errors****
>
> 2012-08-14 11:56:21,151 ERROR ql.Driver
> (SessionState.java:printError(365)) - FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.MapRedTask****
>
> ****
>
> Any ideas?****
>
> ****
>
> Thank you.****
>
> ****
>
> Farah Omer****
>
> Sr. DB Engineer | MicroStrategy, Inc.****
>
> Tel 703.270.2230 | fomer@microstrategy.com****
>
> 1850 Towers Crescent Plaza | Tysons Corner, VA 22182****
>
> www.microstrategy.com****
>
> ****
>
> *From:* kulkarni.swarnim@gmail.com [mailto:kulkarni.swarnim@gmail.com]
> *Sent:* Tuesday, August 14, 2012 11:11 AM
> *To:* user@hive.apache.org
> *Subject:* Re: Issue with creating table in hbase****
>
> ****
>
> > *hbase(main):001:0*> CREATE TABLE hbase_mstr_1.....****
>
> ****
>
> Are you running the CREATE TABLE from the hbase shell? You should run it
> from the hive shell. You can start it from "$HIVE_HOME/bin/hive"****
>
> On Tue, Aug 14, 2012 at 10:05 AM, Omer, Farah <fo...@microstrategy.com>
> wrote:****
>
> Hi all,****
>
> I was testing hbase integrated with hive, and running into an issue. Would
> anyone has an idea what it means?****
>
> ****
>
> hbase(main):001:0> CREATE TABLE hbase_mstr_1(key int, value string) STORED
> BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES
> ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("
> hbase.table.name" = "xyz")****
>
> SyntaxError: (hbase):1: syntax error, unexpected tIDENTIFIER****
>
> ****
>
> CREATE TABLE hbase_mstr_1(key int, value string) STORED BY
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES
> ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("
> hbase.table.name" = "xyz")****
>
> ^****
>
> ****
>
> hbase(main):002:0>****
>
> ****
>
> Hive version is 0.7, hbase version is 0.90****
>
> ****
>
> Thanks.****
>
> ****
>
> Farah Omer****
>
> Sr. DB Engineer | MicroStrategy, Inc.****
>
> Tel 703.270.2230 | fomer@microstrategy.com****
>
> 1850 Towers Crescent Plaza | Tysons Corner, VA 22182****
>
> www.microstrategy.com****
>
> ****
>
> ****
>
>
>
> ****
>
> ****
>
> --
> Swarnim****
>
>
>
> ****
>
> ** **
>
> --
> Swarnim****
>
--
Swarnim
RE: Issue with creating table in hbase
Posted by "Omer, Farah" <fo...@microstrategy.com>.
Unfortunately the job's log also doesn't tell me anything very meaningful. Have you or anyone might have seen this before?
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:387)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.jav
Thanks.
Farah Omer
Sr. DB Engineer | MicroStrategy, Inc.
Tel 703.270.2230 | fomer@microstrategy.com
1850 Towers Crescent Plaza | Tysons Corner, VA 22182
www.microstrategy.com
From: kulkarni.swarnim@gmail.com [mailto:kulkarni.swarnim@gmail.com]
Sent: Tuesday, August 14, 2012 12:49 PM
To: user@hive.apache.org
Subject: Re: Issue with creating table in hbase
It seems like your Map reduce job is failing. Refer to the logs in the tracking URL "http://hadoop001:50030/jobdetails.jsp?jobid=job_201207251201_0678" to see why exactly it is failing.
On Tue, Aug 14, 2012 at 11:35 AM, Omer, Farah <fo...@microstrategy.com>> wrote:
Thanks. That helped.
Another related question:
I created this table on HIVE:
hive> CREATE TABLE hbase_mstr_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name<http://hbase.table.name>" = "hbase_mstr_1_xyz");
My intention is to load it and then access it via HIVE. But I am not able to insert values into this table:
hive> insert overwrite table hbase_mstr_1 select * from eatwh1_ft1;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201207251201_0678, Tracking URL = http://hadoop001:50030/jobdetails.jsp?jobid=job_201207251201_0678
Kill Command = /usr/lib/hadoop/bin/hadoop job -Dmapred.job.tracker=hadoop001:6932 -kill job_201207251201_0678
2012-08-14 12:30:20,977 Stage-0 map = 0%, reduce = 0%
2012-08-14 12:30:47,110 Stage-0 map = 100%, reduce = 100%
Ended Job = job_201207251201_0678 with errors
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
The hive.log doesn't tell a lot of useful information.
2012-08-14 11:55:51,893 WARN hbase.HBaseConfiguration (HBaseConfiguration.java:<init>(45)) - instantiating HBaseConfiguration() is deprecated. Please use HBaseConfiguration#create() to construct a plain Configuration
2012-08-14 11:55:52,302 WARN mapred.JobClient (JobClient.java:copyAndConfigureFiles(649)) - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
2012-08-14 11:56:21,132 ERROR exec.MapRedTask (SessionState.java:printError(365)) - Ended Job = job_201207251201_0677 with errors
2012-08-14 11:56:21,151 ERROR ql.Driver (SessionState.java:printError(365)) - FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
Any ideas?
Thank you.
Farah Omer
Sr. DB Engineer | MicroStrategy, Inc.
Tel 703.270.2230<tel:703.270.2230> | fomer@microstrategy.com<ht...@microstrategy.com>
1850 Towers Crescent Plaza | Tysons Corner, VA 22182
www.microstrategy.com<http://www.microstrategy.com>
From: kulkarni.swarnim@gmail.com<ma...@gmail.com> [mailto:kulkarni.swarnim@gmail.com<ma...@gmail.com>]
Sent: Tuesday, August 14, 2012 11:11 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: Issue with creating table in hbase
> hbase(main):001:0> CREATE TABLE hbase_mstr_1.....
Are you running the CREATE TABLE from the hbase shell? You should run it from the hive shell. You can start it from "$HIVE_HOME/bin/hive"
On Tue, Aug 14, 2012 at 10:05 AM, Omer, Farah <fo...@microstrategy.com>> wrote:
Hi all,
I was testing hbase integrated with hive, and running into an issue. Would anyone has an idea what it means?
hbase(main):001:0> CREATE TABLE hbase_mstr_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name<http://hbase.table.name>" = "xyz")
SyntaxError: (hbase):1: syntax error, unexpected tIDENTIFIER
CREATE TABLE hbase_mstr_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name<http://hbase.table.name>" = "xyz")
^
hbase(main):002:0>
Hive version is 0.7, hbase version is 0.90
Thanks.
Farah Omer
Sr. DB Engineer | MicroStrategy, Inc.
Tel 703.270.2230<tel:703.270.2230> | fomer@microstrategy.com<ht...@microstrategy.com>
1850 Towers Crescent Plaza | Tysons Corner, VA 22182
www.microstrategy.com<http://www.microstrategy.com>
--
Swarnim
--
Swarnim
Re: Issue with creating table in hbase
Posted by "kulkarni.swarnim@gmail.com" <ku...@gmail.com>.
It seems like your Map reduce job is failing. Refer to the logs in the
tracking URL "
http://hadoop001:50030/jobdetails.jsp?jobid=job_201207251201_0678" to see
why exactly it is failing.
On Tue, Aug 14, 2012 at 11:35 AM, Omer, Farah <fo...@microstrategy.com>wrote:
> Thanks. That helped.****
>
> ** **
>
> Another related question:****
>
> ** **
>
> I created this table on HIVE:****
>
> hive> CREATE TABLE hbase_mstr_1(key int, value string) STORED BY
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES
> ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("
> hbase.table.name" = "hbase_mstr_1_xyz");****
>
> ** **
>
> My intention is to load it and then access it via HIVE. But I am not able
> to insert values into this table:****
>
> ** **
>
> hive> insert overwrite table hbase_mstr_1 select * from eatwh1_ft1;****
>
> Total MapReduce jobs = 1****
>
> Launching Job 1 out of 1****
>
> Number of reduce tasks is set to 0 since there's no reduce operator****
>
> Starting Job = job_201207251201_0678, Tracking URL =
> http://hadoop001:50030/jobdetails.jsp?jobid=job_201207251201_0678****
>
> Kill Command = /usr/lib/hadoop/bin/hadoop job
> -Dmapred.job.tracker=hadoop001:6932 -kill job_201207251201_0678****
>
> 2012-08-14 12:30:20,977 Stage-0 map = 0%, reduce = 0%****
>
> 2012-08-14 12:30:47,110 Stage-0 map = 100%, reduce = 100%****
>
> Ended Job = job_201207251201_0678 with errors****
>
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.MapRedTask****
>
> ** **
>
> The hive.log doesn’t tell a lot of useful information.****
>
> ** **
>
> 2012-08-14 11:55:51,893 WARN hbase.HBaseConfiguration
> (HBaseConfiguration.java:<init>(45)) - instantiating HBaseConfiguration()
> is deprecated. Please use HBaseConfiguration#create() to construct a plain
> Configuration****
>
> 2012-08-14 11:55:52,302 WARN mapred.JobClient
> (JobClient.java:copyAndConfigureFiles(649)) - Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.***
> *
>
> 2012-08-14 11:56:21,132 ERROR exec.MapRedTask
> (SessionState.java:printError(365)) - Ended Job = job_201207251201_0677
> with errors****
>
> 2012-08-14 11:56:21,151 ERROR ql.Driver
> (SessionState.java:printError(365)) - FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.MapRedTask****
>
> ** **
>
> Any ideas?****
>
> ** **
>
> Thank you.****
>
> ** **
>
> Farah Omer****
>
> Sr. DB Engineer | MicroStrategy, Inc.****
>
> Tel 703.270.2230 | fomer@microstrategy.com****
>
> 1850 Towers Crescent Plaza | Tysons Corner, VA 22182****
>
> www.microstrategy.com****
>
> ** **
>
> *From:* kulkarni.swarnim@gmail.com [mailto:kulkarni.swarnim@gmail.com]
> *Sent:* Tuesday, August 14, 2012 11:11 AM
> *To:* user@hive.apache.org
> *Subject:* Re: Issue with creating table in hbase****
>
> ** **
>
> > *hbase(main):001:0*> CREATE TABLE hbase_mstr_1.....****
>
> ** **
>
> Are you running the CREATE TABLE from the hbase shell? You should run it
> from the hive shell. You can start it from "$HIVE_HOME/bin/hive"****
>
> On Tue, Aug 14, 2012 at 10:05 AM, Omer, Farah <fo...@microstrategy.com>
> wrote:****
>
> Hi all,****
>
> I was testing hbase integrated with hive, and running into an issue. Would
> anyone has an idea what it means?****
>
> ****
>
> hbase(main):001:0> CREATE TABLE hbase_mstr_1(key int, value string) STORED
> BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES
> ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("
> hbase.table.name" = "xyz")****
>
> SyntaxError: (hbase):1: syntax error, unexpected tIDENTIFIER****
>
> ****
>
> CREATE TABLE hbase_mstr_1(key int, value string) STORED BY
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES
> ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("
> hbase.table.name" = "xyz")****
>
> ^****
>
> ****
>
> hbase(main):002:0>****
>
> ****
>
> Hive version is 0.7, hbase version is 0.90****
>
> ****
>
> Thanks.****
>
> ****
>
> Farah Omer****
>
> Sr. DB Engineer | MicroStrategy, Inc.****
>
> Tel 703.270.2230 | fomer@microstrategy.com****
>
> 1850 Towers Crescent Plaza | Tysons Corner, VA 22182****
>
> www.microstrategy.com****
>
> ****
>
> ****
>
>
>
> ****
>
> ** **
>
> --
> Swarnim****
>
--
Swarnim
RE: Issue with creating table in hbase
Posted by "Omer, Farah" <fo...@microstrategy.com>.
Thanks. That helped.
Another related question:
I created this table on HIVE:
hive> CREATE TABLE hbase_mstr_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name" = "hbase_mstr_1_xyz");
My intention is to load it and then access it via HIVE. But I am not able to insert values into this table:
hive> insert overwrite table hbase_mstr_1 select * from eatwh1_ft1;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201207251201_0678, Tracking URL = http://hadoop001:50030/jobdetails.jsp?jobid=job_201207251201_0678
Kill Command = /usr/lib/hadoop/bin/hadoop job -Dmapred.job.tracker=hadoop001:6932 -kill job_201207251201_0678
2012-08-14 12:30:20,977 Stage-0 map = 0%, reduce = 0%
2012-08-14 12:30:47,110 Stage-0 map = 100%, reduce = 100%
Ended Job = job_201207251201_0678 with errors
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
The hive.log doesn't tell a lot of useful information.
2012-08-14 11:55:51,893 WARN hbase.HBaseConfiguration (HBaseConfiguration.java:<init>(45)) - instantiating HBaseConfiguration() is deprecated. Please use HBaseConfiguration#create() to construct a plain Configuration
2012-08-14 11:55:52,302 WARN mapred.JobClient (JobClient.java:copyAndConfigureFiles(649)) - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
2012-08-14 11:56:21,132 ERROR exec.MapRedTask (SessionState.java:printError(365)) - Ended Job = job_201207251201_0677 with errors
2012-08-14 11:56:21,151 ERROR ql.Driver (SessionState.java:printError(365)) - FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
Any ideas?
Thank you.
Farah Omer
Sr. DB Engineer | MicroStrategy, Inc.
Tel 703.270.2230 | fomer@microstrategy.com
1850 Towers Crescent Plaza | Tysons Corner, VA 22182
www.microstrategy.com
From: kulkarni.swarnim@gmail.com [mailto:kulkarni.swarnim@gmail.com]
Sent: Tuesday, August 14, 2012 11:11 AM
To: user@hive.apache.org
Subject: Re: Issue with creating table in hbase
> hbase(main):001:0> CREATE TABLE hbase_mstr_1.....
Are you running the CREATE TABLE from the hbase shell? You should run it from the hive shell. You can start it from "$HIVE_HOME/bin/hive"
On Tue, Aug 14, 2012 at 10:05 AM, Omer, Farah <fo...@microstrategy.com>> wrote:
Hi all,
I was testing hbase integrated with hive, and running into an issue. Would anyone has an idea what it means?
hbase(main):001:0> CREATE TABLE hbase_mstr_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name<http://hbase.table.name>" = "xyz")
SyntaxError: (hbase):1: syntax error, unexpected tIDENTIFIER
CREATE TABLE hbase_mstr_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name<http://hbase.table.name>" = "xyz")
^
hbase(main):002:0>
Hive version is 0.7, hbase version is 0.90
Thanks.
Farah Omer
Sr. DB Engineer | MicroStrategy, Inc.
Tel 703.270.2230<tel:703.270.2230> | fomer@microstrategy.com<ht...@microstrategy.com>
1850 Towers Crescent Plaza | Tysons Corner, VA 22182
www.microstrategy.com<http://www.microstrategy.com>
--
Swarnim
Re: Issue with creating table in hbase
Posted by "kulkarni.swarnim@gmail.com" <ku...@gmail.com>.
> *hbase(main):001:0*> CREATE TABLE hbase_mstr_1.....
Are you running the CREATE TABLE from the hbase shell? You should run it
from the hive shell. You can start it from "$HIVE_HOME/bin/hive"
On Tue, Aug 14, 2012 at 10:05 AM, Omer, Farah <fo...@microstrategy.com>wrote:
> Hi all,****
>
> I was testing hbase integrated with hive, and running into an issue. Would
> anyone has an idea what it means?****
>
> ** **
>
> hbase(main):001:0> CREATE TABLE hbase_mstr_1(key int, value string) STORED
> BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES
> ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("
> hbase.table.name" = "xyz")****
>
> SyntaxError: (hbase):1: syntax error, unexpected tIDENTIFIER****
>
> ** **
>
> CREATE TABLE hbase_mstr_1(key int, value string) STORED BY
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES
> ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("
> hbase.table.name" = "xyz")****
>
> ^****
>
> ** **
>
> hbase(main):002:0>****
>
> ** **
>
> Hive version is 0.7, hbase version is 0.90****
>
> ** **
>
> Thanks.****
>
> ** **
>
> Farah Omer****
>
> Sr. DB Engineer | MicroStrategy, Inc.****
>
> Tel 703.270.2230 | fomer@microstrategy.com****
>
> 1850 Towers Crescent Plaza | Tysons Corner, VA 22182****
>
> www.microstrategy.com****
>
> ** **
>
> ** **
>
--
Swarnim