You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Tianqiang Li <pe...@gmail.com> on 2010/10/02 01:09:30 UTC
Re: Hive table export?
Hi, Carl,
I got sometime today to install sqoop-1.0.0+3 pointed out by you,
installation is successful, thanks, however, when I tried to use it and
type:
sqoop import --connect jdbc::mysql://mysql_host/my_db --table mytable
--username xxx --password yyy
*10/10/01 16:23:32 ERROR sqoop.Sqoop: Got exception running Sqoop:
java.lang.RuntimeException: Could not find appropriate Hadoop shim for
0.20.1+169.113*
java.lang.RuntimeException: Could not find appropriate Hadoop shim for
0.20.1+169.113
at com.cloudera.sqoop.shims.ShimLoader.loadShim(ShimLoader.java:190)
at
com.cloudera.sqoop.shims.ShimLoader.getHadoopShim(ShimLoader.java:109)
at
com.cloudera.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:141)
at com.cloudera.sqoop.tool.ImportTool.init(ImportTool.java:66)
at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:142)
at com.cloudera.sqoop.Sqoop.run(Sqoop.java:134)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:170)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:196)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:205)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
I looked error msg up in internet, get some very related hit.
https://issues.cloudera.org/browse/SQOOP-82
Our environment is cdh2 hadoop-0.20.1, my question is that is it true there
is no way to let sqoop-1.0.0 work on my current hadoop version, other than
upgrading to chd3 beta-2 ? Currently I am not very tending to upgrade our
system since some important jobs are running on top of it on a daily basis.
Thanks.
-Peter
On Mon, Sep 20, 2010 at 6:03 PM, Carl Steinbach <ca...@cloudera.com> wrote:
> Hi Peter,
>
> Please try the version of Sqoop that ships with CDH3. This version does not
> have dependencies on ZK or HBase. You can get copy of the tarball here:
> http://archive.cloudera.com/cdh/3/sqoop-1.0.0+3.tar.gz
>
> Thanks.
>
> Carl
>
>
> On Mon, Sep 20, 2010 at 5:47 PM, Tianqiang Li <pe...@gmail.com> wrote:
>
>> Hi, Carl,
>> I tried to compile the package you mentioned, but it said Hbase and
>> zookeeper are required dependencies, our environment doesn't have these two,
>> and our hadoop version is from CDH2, which we probably are not convenient to
>> upgrade to CDH3 in short term, any chance to let the sqoop v1.1.0 run with
>> CDH2 without hbase and zookeeper installed ?
>>
>> -Peter
>>
>>
>> On Mon, Sep 20, 2010 at 2:38 PM, Tianqiang Li <pe...@gmail.com> wrote:
>>
>>> Thanks Carl for you hints, let me play with this version of sqoop.
>>>
>>> -Peter
>>>
>>>
>>> On Mon, Sep 20, 2010 at 2:03 PM, Carl Steinbach <ca...@cloudera.com>wrote:
>>>
>>>> Hi Peter,
>>>>
>>>> Sqoop supports both import (RDBMS --> HDFS) and export (HDFS --> RDBMS)
>>>> modes of operation. Please consult the user guide in the latest version of
>>>> Sqoop for more information:
>>>> http://github.com/downloads/cloudera/sqoop/sqoop-1.1.0.tar.gz
>>>>
>>>> Thanks.
>>>>
>>>> Carl
>>>>
>>>>
>>>> On Mon, Sep 20, 2010 at 1:15 PM, Tianqiang Li <pe...@gmail.com>wrote:
>>>>
>>>>> Hi,
>>>>> I have quick questioin, are there any existing tools that are able to
>>>>> export hive table rows into table database like mysql ? I checked sqoop, but
>>>>> it seems it's for importing data to hadoop/hive, but not the other
>>>>> way(export). Any of light you'd like to shed are welcomed, thanks!
>>>>>
>>>>> Regards,
>>>>> Peter
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Hive table export?
Posted by Arvind Prabhakar <ar...@cloudera.com>.
Sqoop 1.0.0+3 is supported against CDH3 B3 vesion of Hadoop
(0.20.2+320). Please upgrade your Hadoop installation and try again.
Arvind
On 10/01/2010 04:09 PM, Tianqiang Li wrote:
> Hi, Carl,
> I got sometime today to install sqoop-1.0.0+3 pointed out by you,
> installation is successful, thanks, however, when I tried to use it
> and type:
>
> sqoop import --connect jdbc::mysql://mysql_host/my_db --table mytable
> --username xxx --password yyy
>
> *10/10/01 16:23:32 ERROR sqoop.Sqoop: Got exception running Sqoop:
> java.lang.RuntimeException: Could not find appropriate Hadoop shim for
> 0.20.1+169.113*
> java.lang.RuntimeException: Could not find appropriate Hadoop shim for
> 0.20.1+169.113
> at
> com.cloudera.sqoop.shims.ShimLoader.loadShim(ShimLoader.java:190)
> at
> com.cloudera.sqoop.shims.ShimLoader.getHadoopShim(ShimLoader.java:109)
> at
> com.cloudera.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:141)
> at com.cloudera.sqoop.tool.ImportTool.init(ImportTool.java:66)
> at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:142)
> at com.cloudera.sqoop.Sqoop.run(Sqoop.java:134)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:170)
> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:196)
> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:205)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
> I looked error msg up in internet, get some very related hit.
> https://issues.cloudera.org/browse/SQOOP-82
>
> Our environment is cdh2 hadoop-0.20.1, my question is that is it true
> there is no way to let sqoop-1.0.0 work on my current hadoop version,
> other than upgrading to chd3 beta-2 ? Currently I am not very tending
> to upgrade our system since some important jobs are running on top of
> it on a daily basis. Thanks.
>
> -Peter
>
> On Mon, Sep 20, 2010 at 6:03 PM, Carl Steinbach <carl@cloudera.com
> <ma...@cloudera.com>> wrote:
>
> Hi Peter,
>
> Please try the version of Sqoop that ships with CDH3. This version
> does not have dependencies on ZK or HBase. You can get copy of the
> tarball here: http://archive.cloudera.com/cdh/3/sqoop-1.0.0+3.tar.gz
>
> Thanks.
>
> Carl
>
>
> On Mon, Sep 20, 2010 at 5:47 PM, Tianqiang Li <peterltq@gmail.com
> <ma...@gmail.com>> wrote:
>
> Hi, Carl,
> I tried to compile the package you mentioned, but it said
> Hbase and zookeeper are required dependencies, our environment
> doesn't have these two, and our hadoop version is from CDH2,
> which we probably are not convenient to upgrade to CDH3 in
> short term, any chance to let the sqoop v1.1.0 run with CDH2
> without hbase and zookeeper installed ?
>
> -Peter
>
>
> On Mon, Sep 20, 2010 at 2:38 PM, Tianqiang Li
> <peterltq@gmail.com <ma...@gmail.com>> wrote:
>
> Thanks Carl for you hints, let me play with this version
> of sqoop.
>
> -Peter
>
>
> On Mon, Sep 20, 2010 at 2:03 PM, Carl Steinbach
> <carl@cloudera.com <ma...@cloudera.com>> wrote:
>
> Hi Peter,
>
> Sqoop supports both import (RDBMS --> HDFS) and export
> (HDFS --> RDBMS) modes of operation. Please consult
> the user guide in the latest version of Sqoop for more
> information:
> http://github.com/downloads/cloudera/sqoop/sqoop-1.1.0.tar.gz
>
> Thanks.
>
> Carl
>
>
> On Mon, Sep 20, 2010 at 1:15 PM, Tianqiang Li
> <peterltq@gmail.com <ma...@gmail.com>> wrote:
>
> Hi,
> I have quick questioin, are there any existing
> tools that are able to export hive table rows into
> table database like mysql ? I checked sqoop, but
> it seems it's for importing data to hadoop/hive,
> but not the other way(export). Any of light you'd
> like to shed are welcomed, thanks!
>
> Regards,
> Peter
>
>
>
>
>
>