You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Jimmy Hu (JIRA)" <ji...@apache.org> on 2010/08/05 02:07:16 UTC
[jira] Created: (HIVE-1512) Need to get hive_hbase-handler to work
with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
Need to get hive_hbase-handler to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
---------------------------------------------------------------------------------------------------
Key: HIVE-1512
URL: https://issues.apache.org/jira/browse/HIVE-1512
Project: Hadoop Hive
Issue Type: Improvement
Components: HBase Handler
Affects Versions: 0.7.0
Reporter: Jimmy Hu
Fix For: 0.7.0
the current trunk hive_hbase-handler only works with hbase 0.20.3, we need to get it to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: retirement of org.apache.hadoop.security.UnixUserGroupInformation
in new version of hadoop
Posted by Vinithra Varadharajan <vi...@cloudera.com>.
Jimmy,
The patch that you need to make hive compile with Hadoop 0.20S is HIVE-1264,
which has been committed to Hive trunk. Is your Hive repo up to date?
If you'd like to apply patches of cdh3b3 Hive to Hive trunk, you can find
them in cdh3b3's hive tarball in hive-0.5.0+32/src/cloudera/patches.
Hope this helps.
-Vinithra
On Mon, Nov 8, 2010 at 3:14 PM, Jinsong Hu <ji...@hotmail.com> wrote:
> I am very interested in hive_hbase-handler in 0.6 version of hive. I have
> compiled the hive trunk's hive_hbase-handler and got it working with
> cdh3b2's hbase after some minor code change. However, in my testing, I found
> that cdh3b2's hbase is not stable enough. I tested cdh3b3's hbase and I
> found it was much better. However,
> when I tried to get hive to work with it, I found that neither hive 0.6 nor
> the hive trunk are compatible with cdh3b3 hbase, as they use
> UnixUserGroupInformation .
>
> the hive released with cdh3b3 is still 0.5 version. it doesn't have the
> hive_hbase-handler . and according to my reading, hive itself needs a patch
> HIVE-705 in order to work with hive_hbase-handler, and hive-0.5 version
> doesn't have that patch. This means I can't really use the hive shipped with
> cdh3b3. and compiling hive-0.6 so it work with cdh3b3 is my best hope.
>
> Which goes back to the question: can se use UserGroupInformation to
> replace UnixUserGroupInformation
> in hive-0.6 ?
>
>
> Jimmy
>
>
>
>
> --------------------------------------------------
> From: "Vinithra Varadharajan" <vi...@cloudera.com>
> Sent: Monday, November 08, 2010 2:52 PM
> To: <de...@hive.apache.org>; <ji...@hotmail.com>
> Cc: <hi...@hadoop.apache.org>
> Subject: Re: retirement of
> org.apache.hadoop.security.UnixUserGroupInformation in new version of hadoop
>
>
> Hi Jimmy,
>>
>> cdh3b3 Hadoop includes the patches written by Yahoo! to introduce
>> authentication based on Kerberos. These patches include removal of
>> UnixUserGroupInformation. cdh3b3's Hive release includes patches to
>> Driver.java to not use UnixUserGroupInformation and instead make calls to
>> a
>> shim layer that works with the authentication system in Hadoop.
>>
>> Are there features in hive trunk/0.6 that you are interested in using but
>> are not available in cdh3b3 Hive?
>>
>> -Vinithra
>>
>> On Mon, Nov 8, 2010 at 12:28 PM, Jinsong Hu <ji...@hotmail.com>
>> wrote:
>>
>> Hi, There:
>>> I was trying to compile trunk version of hive with cloudera cdh3b3
>>> distribution and noticed that in
>>> org.apache.hadoop.hive.ql Driver.java was using:
>>>
>>> org.apache.hadoop.security.UnixUserGroupInformation;
>>>
>>> However, in the cdh3b3 version of hadoop, this class is gone, so
>>> Driver.java doesn't compile.
>>>
>>> I checked 0.6 version of hive, the same issue exists.
>>>
>>> I noticed that class org.apache.hadoop.security.UserGroupInformation is
>>> still there, both
>>> in cdh3b2 and cdh3b3 version of hadoop. Can we change the Driver.java to
>>> use
>>>
>>> org.apache.hadoop.security.UserGroupInformation;
>>>
>>> instead ? This class does sound like a more generic class that in the
>>> future, we may
>>> even get it to work under windows or other os.
>>>
>>> Jimmy.
>>>
>>>
>>>
>>>
>>>
>>
Re: retirement of org.apache.hadoop.security.UnixUserGroupInformation in new version of hadoop
Posted by Jinsong Hu <ji...@hotmail.com>.
I am very interested in hive_hbase-handler in 0.6 version of hive. I have
compiled the hive trunk's hive_hbase-handler and got it working with
cdh3b2's hbase after some minor code change. However, in my testing, I found
that cdh3b2's hbase is not stable enough. I tested cdh3b3's hbase and I
found it was much better. However,
when I tried to get hive to work with it, I found that neither hive 0.6 nor
the hive trunk are compatible with cdh3b3 hbase, as they use
UnixUserGroupInformation .
the hive released with cdh3b3 is still 0.5 version. it doesn't have the
hive_hbase-handler . and according to my reading, hive itself needs a patch
HIVE-705 in order to work with hive_hbase-handler, and hive-0.5 version
doesn't have that patch. This means I can't really use the hive shipped with
cdh3b3. and compiling hive-0.6 so it work with cdh3b3 is my best hope.
Which goes back to the question: can se use UserGroupInformation to replace
UnixUserGroupInformation
in hive-0.6 ?
Jimmy
--------------------------------------------------
From: "Vinithra Varadharajan" <vi...@cloudera.com>
Sent: Monday, November 08, 2010 2:52 PM
To: <de...@hive.apache.org>; <ji...@hotmail.com>
Cc: <hi...@hadoop.apache.org>
Subject: Re: retirement of
org.apache.hadoop.security.UnixUserGroupInformation in new version of hadoop
> Hi Jimmy,
>
> cdh3b3 Hadoop includes the patches written by Yahoo! to introduce
> authentication based on Kerberos. These patches include removal of
> UnixUserGroupInformation. cdh3b3's Hive release includes patches to
> Driver.java to not use UnixUserGroupInformation and instead make calls to
> a
> shim layer that works with the authentication system in Hadoop.
>
> Are there features in hive trunk/0.6 that you are interested in using but
> are not available in cdh3b3 Hive?
>
> -Vinithra
>
> On Mon, Nov 8, 2010 at 12:28 PM, Jinsong Hu <ji...@hotmail.com>
> wrote:
>
>> Hi, There:
>> I was trying to compile trunk version of hive with cloudera cdh3b3
>> distribution and noticed that in
>> org.apache.hadoop.hive.ql Driver.java was using:
>>
>> org.apache.hadoop.security.UnixUserGroupInformation;
>>
>> However, in the cdh3b3 version of hadoop, this class is gone, so
>> Driver.java doesn't compile.
>>
>> I checked 0.6 version of hive, the same issue exists.
>>
>> I noticed that class org.apache.hadoop.security.UserGroupInformation is
>> still there, both
>> in cdh3b2 and cdh3b3 version of hadoop. Can we change the Driver.java to
>> use
>>
>> org.apache.hadoop.security.UserGroupInformation;
>>
>> instead ? This class does sound like a more generic class that in the
>> future, we may
>> even get it to work under windows or other os.
>>
>> Jimmy.
>>
>>
>>
>>
>
Re: retirement of org.apache.hadoop.security.UnixUserGroupInformation
in new version of hadoop
Posted by Vinithra Varadharajan <vi...@cloudera.com>.
Hi Jimmy,
cdh3b3 Hadoop includes the patches written by Yahoo! to introduce
authentication based on Kerberos. These patches include removal of
UnixUserGroupInformation. cdh3b3's Hive release includes patches to
Driver.java to not use UnixUserGroupInformation and instead make calls to a
shim layer that works with the authentication system in Hadoop.
Are there features in hive trunk/0.6 that you are interested in using but
are not available in cdh3b3 Hive?
-Vinithra
On Mon, Nov 8, 2010 at 12:28 PM, Jinsong Hu <ji...@hotmail.com> wrote:
> Hi, There:
> I was trying to compile trunk version of hive with cloudera cdh3b3
> distribution and noticed that in
> org.apache.hadoop.hive.ql Driver.java was using:
>
> org.apache.hadoop.security.UnixUserGroupInformation;
>
> However, in the cdh3b3 version of hadoop, this class is gone, so
> Driver.java doesn't compile.
>
> I checked 0.6 version of hive, the same issue exists.
>
> I noticed that class org.apache.hadoop.security.UserGroupInformation is
> still there, both
> in cdh3b2 and cdh3b3 version of hadoop. Can we change the Driver.java to
> use
>
> org.apache.hadoop.security.UserGroupInformation;
>
> instead ? This class does sound like a more generic class that in the
> future, we may
> even get it to work under windows or other os.
>
> Jimmy.
>
>
>
>
retirement of org.apache.hadoop.security.UnixUserGroupInformation in new version of hadoop
Posted by Jinsong Hu <ji...@hotmail.com>.
Hi, There:
I was trying to compile trunk version of hive with cloudera cdh3b3
distribution and noticed that in
org.apache.hadoop.hive.ql Driver.java was using:
org.apache.hadoop.security.UnixUserGroupInformation;
However, in the cdh3b3 version of hadoop, this class is gone, so Driver.java
doesn't compile.
I checked 0.6 version of hive, the same issue exists.
I noticed that class org.apache.hadoop.security.UserGroupInformation is
still there, both
in cdh3b2 and cdh3b3 version of hadoop. Can we change the Driver.java to use
org.apache.hadoop.security.UserGroupInformation;
instead ? This class does sound like a more generic class that in the
future, we may
even get it to work under windows or other os.
Jimmy.
Re: [jira] Resolved: (HIVE-1512) Need to get hive_hbase-handler to
work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
Posted by Ted Yu <yu...@gmail.com>.
Right now I see when accessing https://issues.apache.org/jira/browse/HIVE-1512:
com.atlassian.jira.exception.DataAccessException:
org.ofbiz.core.entity.GenericDataSourceException: Unable to esablish a
connection with the database. (FATAL: database is not accepting
commands to avoid wraparound data loss in database "postgres")
Maybe Apache should switch to HBase as data source :-)
On Wed, Aug 25, 2010 at 12:10 PM, Basab Maulik <bk...@gmail.com> wrote:
> I have created HIVE 1597 to track this issue. The fix for HIVE-1512 does
> not
> resolve the issue below.
>
> Basab
>
> On Mon, Aug 23, 2010 at 3:09 PM, Rajappa Iyer <rs...@mayin.org> wrote:
>
> > Perhaps I am missing something, but I cannot get this to work:
> >
> > /Users/riyer/src/ext/hive-trunk/build/dist/bin/hive --auxpath
> >
> >
> /Users/riyer/src/ext/hive-trunk/build/hbase-handler/hive_hbase-handler.jar,/Users/riyer/.jars/hbase-0.89.20100621+17.jar,/Users/riyer/src/ext/hive-trunk/hbase-handler/lib/zookeeper-3.2.2.jar
> > -hiveconf hbase.zookeeper.quorum=10.2.128.92,10.2.130.82,10.2.131.57
> > Hive history
> file=/tmp/riyer/hive_job_log_riyer_201008231505_440471748.txt
> > hive> CREATE TABLE hbase_table_1(key int, value string)
> > > STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> > > WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> > > TBLPROPERTIES ("hbase.table.name" = "xyz");
> > FAILED: Error in metadata:
> > MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException:
> > 10.2.128.92:60000
> > at
> >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:376)
> > at
> > org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:72)
> > at
> >
> >
> org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:62)
> > at
> >
> >
> org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:141)
> > at
> >
> >
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:285)
> > at
> > org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:326)
> > at
> > org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2173)
> > at
> org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:148)
> > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:108)
> > at
> >
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
> > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:610)
> > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:479)
> > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:357)
> > at
> > org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:140)
> > at
> > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:199)
> > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:351)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> > )
> > FAILED: Execution Error, return code 1 from
> > org.apache.hadoop.hive.ql.exec.DDLTask
> > hive>
> >
> > I get the same results with zookeeper-3.3.1+7.jar from CDH3
> >
> >
> > On Mon, Aug 23, 2010 at 10:12 AM, John Sichi (JIRA) <ji...@apache.org>
> > wrote:
> >
> > >
> > > [
> > >
> >
> https://issues.apache.org/jira/browse/HIVE-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> > ]
> > >
> > > John Sichi resolved HIVE-1512.
> > > ------------------------------
> > >
> > > Hadoop Flags: [Reviewed]
> > > Resolution: Fixed
> > >
> > > Committed. Thanks Basab!
> > >
> > >
> > > > Need to get hive_hbase-handler to work with hbase versions 0.20.4
> > 0.20.5
> > > and cloudera CDH3 version
> > > >
> > >
> >
> ---------------------------------------------------------------------------------------------------
> > > >
> > > > Key: HIVE-1512
> > > > URL: https://issues.apache.org/jira/browse/HIVE-1512
> > > > Project: Hadoop Hive
> > > > Issue Type: Improvement
> > > > Components: HBase Handler
> > > > Affects Versions: 0.7.0
> > > > Reporter: Jimmy Hu
> > > > Assignee: Basab Maulik
> > > > Fix For: 0.7.0
> > > >
> > > > Attachments: HIVE-1512.2.patch, HIVE-1512.3.patch,
> > > HIVE-1512.patch
> > > >
> > > > Original Estimate: 48h
> > > > Remaining Estimate: 48h
> > > >
> > > > the current trunk hive_hbase-handler only works with hbase 0.20.3,
> we
> > > need to get it to work with hbase versions 0.20.4 0.20.5 and cloudera
> > CDH3
> > > version
> > >
> > > --
> > > This message is automatically generated by JIRA.
> > > -
> > > You can reply to this email to add a comment to the issue online.
> > >
> > >
> >
>
Re: [jira] Resolved: (HIVE-1512) Need to get hive_hbase-handler to
work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
Posted by Basab Maulik <bk...@gmail.com>.
I have created HIVE 1597 to track this issue. The fix for HIVE-1512 does not
resolve the issue below.
Basab
On Mon, Aug 23, 2010 at 3:09 PM, Rajappa Iyer <rs...@mayin.org> wrote:
> Perhaps I am missing something, but I cannot get this to work:
>
> /Users/riyer/src/ext/hive-trunk/build/dist/bin/hive --auxpath
>
> /Users/riyer/src/ext/hive-trunk/build/hbase-handler/hive_hbase-handler.jar,/Users/riyer/.jars/hbase-0.89.20100621+17.jar,/Users/riyer/src/ext/hive-trunk/hbase-handler/lib/zookeeper-3.2.2.jar
> -hiveconf hbase.zookeeper.quorum=10.2.128.92,10.2.130.82,10.2.131.57
> Hive history file=/tmp/riyer/hive_job_log_riyer_201008231505_440471748.txt
> hive> CREATE TABLE hbase_table_1(key int, value string)
> > STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> > WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> > TBLPROPERTIES ("hbase.table.name" = "xyz");
> FAILED: Error in metadata:
> MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException:
> 10.2.128.92:60000
> at
>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:376)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:72)
> at
>
> org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:62)
> at
>
> org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:141)
> at
>
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:285)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:326)
> at
> org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2173)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:148)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:108)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:610)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:479)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:357)
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:140)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:199)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:351)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> )
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask
> hive>
>
> I get the same results with zookeeper-3.3.1+7.jar from CDH3
>
>
> On Mon, Aug 23, 2010 at 10:12 AM, John Sichi (JIRA) <ji...@apache.org>
> wrote:
>
> >
> > [
> >
> https://issues.apache.org/jira/browse/HIVE-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> ]
> >
> > John Sichi resolved HIVE-1512.
> > ------------------------------
> >
> > Hadoop Flags: [Reviewed]
> > Resolution: Fixed
> >
> > Committed. Thanks Basab!
> >
> >
> > > Need to get hive_hbase-handler to work with hbase versions 0.20.4
> 0.20.5
> > and cloudera CDH3 version
> > >
> >
> ---------------------------------------------------------------------------------------------------
> > >
> > > Key: HIVE-1512
> > > URL: https://issues.apache.org/jira/browse/HIVE-1512
> > > Project: Hadoop Hive
> > > Issue Type: Improvement
> > > Components: HBase Handler
> > > Affects Versions: 0.7.0
> > > Reporter: Jimmy Hu
> > > Assignee: Basab Maulik
> > > Fix For: 0.7.0
> > >
> > > Attachments: HIVE-1512.2.patch, HIVE-1512.3.patch,
> > HIVE-1512.patch
> > >
> > > Original Estimate: 48h
> > > Remaining Estimate: 48h
> > >
> > > the current trunk hive_hbase-handler only works with hbase 0.20.3, we
> > need to get it to work with hbase versions 0.20.4 0.20.5 and cloudera
> CDH3
> > version
> >
> > --
> > This message is automatically generated by JIRA.
> > -
> > You can reply to this email to add a comment to the issue online.
> >
> >
>
Re: [jira] Resolved: (HIVE-1512) Need to get hive_hbase-handler to
work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
Posted by Rajappa Iyer <rs...@mayin.org>.
Perhaps I am missing something, but I cannot get this to work:
/Users/riyer/src/ext/hive-trunk/build/dist/bin/hive --auxpath
/Users/riyer/src/ext/hive-trunk/build/hbase-handler/hive_hbase-handler.jar,/Users/riyer/.jars/hbase-0.89.20100621+17.jar,/Users/riyer/src/ext/hive-trunk/hbase-handler/lib/zookeeper-3.2.2.jar
-hiveconf hbase.zookeeper.quorum=10.2.128.92,10.2.130.82,10.2.131.57
Hive history file=/tmp/riyer/hive_job_log_riyer_201008231505_440471748.txt
hive> CREATE TABLE hbase_table_1(key int, value string)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "xyz");
FAILED: Error in metadata:
MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException:
10.2.128.92:60000
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:376)
at
org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:72)
at
org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:62)
at
org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:141)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:285)
at
org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:326)
at
org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:2173)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:148)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:108)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:610)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:479)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:357)
at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:140)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:199)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:351)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
)
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask
hive>
I get the same results with zookeeper-3.3.1+7.jar from CDH3
On Mon, Aug 23, 2010 at 10:12 AM, John Sichi (JIRA) <ji...@apache.org> wrote:
>
> [
> https://issues.apache.org/jira/browse/HIVE-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel]
>
> John Sichi resolved HIVE-1512.
> ------------------------------
>
> Hadoop Flags: [Reviewed]
> Resolution: Fixed
>
> Committed. Thanks Basab!
>
>
> > Need to get hive_hbase-handler to work with hbase versions 0.20.4 0.20.5
> and cloudera CDH3 version
> >
> ---------------------------------------------------------------------------------------------------
> >
> > Key: HIVE-1512
> > URL: https://issues.apache.org/jira/browse/HIVE-1512
> > Project: Hadoop Hive
> > Issue Type: Improvement
> > Components: HBase Handler
> > Affects Versions: 0.7.0
> > Reporter: Jimmy Hu
> > Assignee: Basab Maulik
> > Fix For: 0.7.0
> >
> > Attachments: HIVE-1512.2.patch, HIVE-1512.3.patch,
> HIVE-1512.patch
> >
> > Original Estimate: 48h
> > Remaining Estimate: 48h
> >
> > the current trunk hive_hbase-handler only works with hbase 0.20.3, we
> need to get it to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3
> version
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>
[jira] Assigned: (HIVE-1512) Need to get hive_hbase-handler to work
with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi reassigned HIVE-1512:
--------------------------------
Assignee: Basab Maulik (was: John Sichi)
> Need to get hive_hbase-handler to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
> ---------------------------------------------------------------------------------------------------
>
> Key: HIVE-1512
> URL: https://issues.apache.org/jira/browse/HIVE-1512
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: HBase Handler
> Affects Versions: 0.7.0
> Reporter: Jimmy Hu
> Assignee: Basab Maulik
> Fix For: 0.7.0
>
> Attachments: HIVE-1512.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> the current trunk hive_hbase-handler only works with hbase 0.20.3, we need to get it to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1512) Need to get hive_hbase-handler to work
with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
Posted by "Basab Maulik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Basab Maulik updated HIVE-1512:
-------------------------------
Attachment: HIVE-1512.3.patch
Thanks John. This is a small change to the patch, fixes a potential NPE. Also, HBase 0.89.x introduces an additional runtime dependency for the tests, guava-r05.jar, I think the Google collections library jar.
> Need to get hive_hbase-handler to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
> ---------------------------------------------------------------------------------------------------
>
> Key: HIVE-1512
> URL: https://issues.apache.org/jira/browse/HIVE-1512
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: HBase Handler
> Affects Versions: 0.7.0
> Reporter: Jimmy Hu
> Assignee: Basab Maulik
> Fix For: 0.7.0
>
> Attachments: HIVE-1512.2.patch, HIVE-1512.3.patch, HIVE-1512.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> the current trunk hive_hbase-handler only works with hbase 0.20.3, we need to get it to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1512) Need to get hive_hbase-handler to
work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12900495#action_12900495 ]
John Sichi commented on HIVE-1512:
----------------------------------
+1. Will commit when tests pass.
> Need to get hive_hbase-handler to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
> ---------------------------------------------------------------------------------------------------
>
> Key: HIVE-1512
> URL: https://issues.apache.org/jira/browse/HIVE-1512
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: HBase Handler
> Affects Versions: 0.7.0
> Reporter: Jimmy Hu
> Assignee: Basab Maulik
> Fix For: 0.7.0
>
> Attachments: HIVE-1512.2.patch, HIVE-1512.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> the current trunk hive_hbase-handler only works with hbase 0.20.3, we need to get it to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1512) Need to get hive_hbase-handler to work
with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
Posted by "Jimmy Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Hu updated HIVE-1512:
---------------------------
Attachment: HIVE-1512.patch
patch for 3 files to make the hive-hbase-handler to be compatible with 0.20.4 and 0.20.5 and cloudera CDH3 hbase.
> Need to get hive_hbase-handler to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
> ---------------------------------------------------------------------------------------------------
>
> Key: HIVE-1512
> URL: https://issues.apache.org/jira/browse/HIVE-1512
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: HBase Handler
> Affects Versions: 0.7.0
> Reporter: Jimmy Hu
> Fix For: 0.7.0
>
> Attachments: HIVE-1512.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> the current trunk hive_hbase-handler only works with hbase 0.20.3, we need to get it to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1512) Need to get hive_hbase-handler to
work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12895829#action_12895829 ]
John Sichi commented on HIVE-1512:
----------------------------------
This patch can't be applied until we actually upgrade the Hbase libs, since it is incompatible with 0.20.3. I'll link it to HIVE-1235.
Also, when supplying patches, please base them off of hive trunk (not off of a subdirectory).
Thanks!
> Need to get hive_hbase-handler to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
> ---------------------------------------------------------------------------------------------------
>
> Key: HIVE-1512
> URL: https://issues.apache.org/jira/browse/HIVE-1512
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: HBase Handler
> Affects Versions: 0.7.0
> Reporter: Jimmy Hu
> Fix For: 0.7.0
>
> Attachments: HIVE-1512.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> the current trunk hive_hbase-handler only works with hbase 0.20.3, we need to get it to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-1512) Need to get hive_hbase-handler to work
with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi reassigned HIVE-1512:
--------------------------------
Assignee: John Sichi
> Need to get hive_hbase-handler to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
> ---------------------------------------------------------------------------------------------------
>
> Key: HIVE-1512
> URL: https://issues.apache.org/jira/browse/HIVE-1512
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: HBase Handler
> Affects Versions: 0.7.0
> Reporter: Jimmy Hu
> Assignee: John Sichi
> Fix For: 0.7.0
>
> Attachments: HIVE-1512.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> the current trunk hive_hbase-handler only works with hbase 0.20.3, we need to get it to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1512) Need to get hive_hbase-handler to
work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
Posted by "Basab Maulik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12897948#action_12897948 ]
Basab Maulik commented on HIVE-1512:
------------------------------------
Quick clarification, the current code works fine against HBase 0.20.5 which I tested (and presumably against 0.20.4 as well). Be sure to use the correct version of the ZooKeeper libs, 3.2.2.
A patch is needed to get it to build with hbase 0.89.0 snapshots.
> Need to get hive_hbase-handler to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
> ---------------------------------------------------------------------------------------------------
>
> Key: HIVE-1512
> URL: https://issues.apache.org/jira/browse/HIVE-1512
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: HBase Handler
> Affects Versions: 0.7.0
> Reporter: Jimmy Hu
> Assignee: John Sichi
> Fix For: 0.7.0
>
> Attachments: HIVE-1512.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> the current trunk hive_hbase-handler only works with hbase 0.20.3, we need to get it to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1512) Need to get hive_hbase-handler to
work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
Posted by "HBase Review Board (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898761#action_12898761 ]
HBase Review Board commented on HIVE-1512:
------------------------------------------
Message from: bkm.hadoop@gmail.com
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/660/
-----------------------------------------------------------
Review request for Hive Developers and John Sichi.
Summary
-------
Fixes Hive HBase storage handler to work with both HBase 0.89.0 SNAPSHOT releases and the 0.20.x releases.
This addresses bug HIVE-1512.
http://issues.apache.org/jira/browse/HIVE-1512
Diffs
-----
trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java 985559
trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java 985559
trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java 985559
trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java 985559
trunk/hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestHBaseSerDe.java 985559
trunk/hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestLazyHBaseObject.java 985559
Diff: http://review.cloudera.org/r/660/diff
Testing
-------
TestHBaseSerDe, TestLazyHBaseObject, TestHBaseCliDriver, TestHBaseMinimrCliDriver tests pass.
Thanks,
bkm
> Need to get hive_hbase-handler to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
> ---------------------------------------------------------------------------------------------------
>
> Key: HIVE-1512
> URL: https://issues.apache.org/jira/browse/HIVE-1512
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: HBase Handler
> Affects Versions: 0.7.0
> Reporter: Jimmy Hu
> Assignee: Basab Maulik
> Fix For: 0.7.0
>
> Attachments: HIVE-1512.2.patch, HIVE-1512.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> the current trunk hive_hbase-handler only works with hbase 0.20.3, we need to get it to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1512) Need to get hive_hbase-handler to work
with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi resolved HIVE-1512.
------------------------------
Hadoop Flags: [Reviewed]
Resolution: Fixed
Committed. Thanks Basab!
> Need to get hive_hbase-handler to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
> ---------------------------------------------------------------------------------------------------
>
> Key: HIVE-1512
> URL: https://issues.apache.org/jira/browse/HIVE-1512
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: HBase Handler
> Affects Versions: 0.7.0
> Reporter: Jimmy Hu
> Assignee: Basab Maulik
> Fix For: 0.7.0
>
> Attachments: HIVE-1512.2.patch, HIVE-1512.3.patch, HIVE-1512.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> the current trunk hive_hbase-handler only works with hbase 0.20.3, we need to get it to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1512) Need to get hive_hbase-handler to work
with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
Posted by "Basab Maulik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Basab Maulik updated HIVE-1512:
-------------------------------
Attachment: HIVE-1512.2.patch
This patch allows the HBase handler to work with both the hbase 0.20.x/ zookeeper 3.2.2 libraries and the hbase 0.89.0 snapshot / zk 3.3.1 libraries.
The patch changes code to use HBase client API's which are not deprecated or removed in 0.89.0. Primarily we consistently use a column family / column qualifier pair for HBase client api's instead of "<column-family>:<column-qualifier>" colon-separated form allowed in the 0.20.x apis.
Tests pass with 0.20.3, 0.20.5 / zk 3.2.2 and 0.89.0 snapshot / zk 3.3.1.
> Need to get hive_hbase-handler to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
> ---------------------------------------------------------------------------------------------------
>
> Key: HIVE-1512
> URL: https://issues.apache.org/jira/browse/HIVE-1512
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: HBase Handler
> Affects Versions: 0.7.0
> Reporter: Jimmy Hu
> Assignee: Basab Maulik
> Fix For: 0.7.0
>
> Attachments: HIVE-1512.2.patch, HIVE-1512.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> the current trunk hive_hbase-handler only works with hbase 0.20.3, we need to get it to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.