You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@sqoop.apache.org by Otto Rossbach <ot...@googlemail.com> on 2011/12/20 19:26:26 UTC

Sqoop Installation Problem

Hi,
I do have some trouble setting up Sqoop. I downloaded the latest
version of Sqoop from http://www.apache.org/dyn/closer.cgi/incubator/sqoop/
.
I figured out, that I don't have to build sqoop using ant, because the
binaries and the jar's are already in the tar.

I have set the environment variables:
HADOOP_HOME
HIVE_HOME
SQOOP_HOME
and have their bin subfolders added to the PATH variable.

Running 'sqoop' or 'sqoop version' leads to the following error:

hadoop@dev1:/usr/local/sqoop> sqoop
Warning: /usr/lib/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Exception in thread "main" java.lang.NoClassDefFoundError: com/
cloudera/sqoop/Sqoop
Caused by: java.lang.ClassNotFoundException: com.cloudera.sqoop.Sqoop
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
Could not find the main class: com.cloudera.sqoop.Sqoop.  Program will
exit.

That's why I tried to add the missing jars in the HADOOP_CLASSPATH in
sqoop/bin/configure-sqoop. I added the following line at the end of
the file: export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/local/sqoop/
sqoop-1.4.0-incubating.jar
So that the com/cloudera/sqoop/Sqoop class was found. But
unfortunately this does not solve the problem. Now I do get the
following error when executing "sqoop version". (I know this is a very
dirty hack, but I really don't know what variable I need to set, so
that sqoop finds its own jar-files?! Very frustrating!)

hadoop@dev1:/usr/local/sqoop> sqoop version
Warning: /usr/lib/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.hadoop.conf.Configuration.getInstances(Ljava/lang/
String;Ljava/lang/Class;)Ljava/util/List;
at org.apache.sqoop.tool.SqoopTool.loadPlugins(SqoopTool.java:139)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:210)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)

I could do the same trick for the
org.apache.hadoop.conf.Configuration.getInstances method, but I
haven't found the correct jar-file yet. And this is definitely not the
correct way to do it.

Can anyone help me setting up Sqoop correctly?

Thanks for your help in advance.

Re: Sqoop Installation Problem

Posted by Jarek Jarcec Cecho <ja...@apache.org>.

Hi Otto,
Hadoop version 0.20.x is not yet officially supported. Reason why you can't find jar with method org.apache.hadoop.conf.Configuration.getInstances is that your hadoop version do not contain it at all.

We've recently started working on SQOOP-384 that will add support for 0.20 branch. There are already patches available and Arvind has reported that he manage to get working simple MySQL import on 0.20.205. So I would suggest to apply patches from both subtasks (SQOOP-412 and SQOOP-413) on top of your sqoop version and try them out.

Jarcec

SQOOP-384: https://issues.apache.org/jira/browse/SQOOP-384
SQOOP-412: https://issues.apache.org/jira/browse/SQOOP-412
SQOOP-413: https://issues.apache.org/jira/browse/SQOOP-413

On Wed, Dec 21, 2011 at 08:49:42AM +0100, Otto Rossbach wrote:
> Hi,
> I tried running sqoop without the environment variable. But unfortunately
> it doesn't solve the problem. The classes cannot be found. So I am very
> sure, that I need to set a classpath somewhere. But I don't know where.
> 
> I am working with version: Hadoop 0.20.203.0
> 
> 
> 2011/12/21 Kate Ting <ka...@cloudera.com>
> 
> > Hi Otto - since you don't need to set SQOOP_HOME, can you go ahead and
> > unset that environment variable and then try running sqoop?
> >
> > Also, which Hadoop version are you on?
> >
> > Regards, Kate
> >
> > On Tue, Dec 20, 2011 at 10:26 AM, Otto Rossbach
> > <ot...@googlemail.com> wrote:
> > > Hi,
> > > I do have some trouble setting up Sqoop. I downloaded the latest
> > > version of Sqoop
> > > from http://www.apache.org/dyn/closer.cgi/incubator/sqoop/.
> > > I figured out, that I don't have to build sqoop using ant, because the
> > > binaries and the jar's are already in the tar.
> > >
> > > I have set the environment variables:
> > > HADOOP_HOME
> > > HIVE_HOME
> > > SQOOP_HOME
> > > and have their bin subfolders added to the PATH variable.
> > >
> > > Running 'sqoop' or 'sqoop version' leads to the following error:
> > >
> > > hadoop@dev1:/usr/local/sqoop> sqoop
> > > Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> > > Please set $HBASE_HOME to the root of your HBase installation.
> > > Exception in thread "main" java.lang.NoClassDefFoundError: com/
> > > cloudera/sqoop/Sqoop
> > > Caused by: java.lang.ClassNotFoundException: com.cloudera.sqoop.Sqoop
> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
> > > at java.security.AccessController.doPrivileged(Native Method)
> > > at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> > > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
> > > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
> > > Could not find the main class: com.cloudera.sqoop.Sqoop.  Program will
> > > exit.
> > >
> > > That's why I tried to add the missing jars in the HADOOP_CLASSPATH in
> > > sqoop/bin/configure-sqoop. I added the following line at the end of
> > > the file: export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/local/sqoop/
> > > sqoop-1.4.0-incubating.jar
> > > So that the com/cloudera/sqoop/Sqoop class was found. But
> > > unfortunately this does not solve the problem. Now I do get the
> > > following error when executing "sqoop version". (I know this is a very
> > > dirty hack, but I really don't know what variable I need to set, so
> > > that sqoop finds its own jar-files?! Very frustrating!)
> > >
> > > hadoop@dev1:/usr/local/sqoop> sqoop version
> > > Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> > > Please set $HBASE_HOME to the root of your HBase installation.
> > > Exception in thread "main" java.lang.NoSuchMethodError:
> > > org.apache.hadoop.conf.Configuration.getInstances(Ljava/lang/
> > > String;Ljava/lang/Class;)Ljava/util/List;
> > > at org.apache.sqoop.tool.SqoopTool.loadPlugins(SqoopTool.java:139)
> > > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:210)
> > > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
> > > at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> > > at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
> > >
> > > I could do the same trick for the
> > > org.apache.hadoop.conf.Configuration.getInstances method, but I
> > > haven't found the correct jar-file yet. And this is definitely not the
> > > correct way to do it.
> > >
> > > Can anyone help me setting up Sqoop correctly?
> > >
> > > Thanks for your help in advance.
> >

Re: Sqoop Installation Problem

Posted by Otto Rossbach <ot...@googlemail.com>.

Hi,
I tried running sqoop without the environment variable. But unfortunately
it doesn't solve the problem. The classes cannot be found. So I am very
sure, that I need to set a classpath somewhere. But I don't know where.

I am working with version: Hadoop 0.20.203.0


2011/12/21 Kate Ting <ka...@cloudera.com>

> Hi Otto - since you don't need to set SQOOP_HOME, can you go ahead and
> unset that environment variable and then try running sqoop?
>
> Also, which Hadoop version are you on?
>
> Regards, Kate
>
> On Tue, Dec 20, 2011 at 10:26 AM, Otto Rossbach
> <ot...@googlemail.com> wrote:
> > Hi,
> > I do have some trouble setting up Sqoop. I downloaded the latest
> > version of Sqoop
> > from http://www.apache.org/dyn/closer.cgi/incubator/sqoop/.
> > I figured out, that I don't have to build sqoop using ant, because the
> > binaries and the jar's are already in the tar.
> >
> > I have set the environment variables:
> > HADOOP_HOME
> > HIVE_HOME
> > SQOOP_HOME
> > and have their bin subfolders added to the PATH variable.
> >
> > Running 'sqoop' or 'sqoop version' leads to the following error:
> >
> > hadoop@dev1:/usr/local/sqoop> sqoop
> > Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> > Please set $HBASE_HOME to the root of your HBase installation.
> > Exception in thread "main" java.lang.NoClassDefFoundError: com/
> > cloudera/sqoop/Sqoop
> > Caused by: java.lang.ClassNotFoundException: com.cloudera.sqoop.Sqoop
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
> > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
> > Could not find the main class: com.cloudera.sqoop.Sqoop.  Program will
> > exit.
> >
> > That's why I tried to add the missing jars in the HADOOP_CLASSPATH in
> > sqoop/bin/configure-sqoop. I added the following line at the end of
> > the file: export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/local/sqoop/
> > sqoop-1.4.0-incubating.jar
> > So that the com/cloudera/sqoop/Sqoop class was found. But
> > unfortunately this does not solve the problem. Now I do get the
> > following error when executing "sqoop version". (I know this is a very
> > dirty hack, but I really don't know what variable I need to set, so
> > that sqoop finds its own jar-files?! Very frustrating!)
> >
> > hadoop@dev1:/usr/local/sqoop> sqoop version
> > Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> > Please set $HBASE_HOME to the root of your HBase installation.
> > Exception in thread "main" java.lang.NoSuchMethodError:
> > org.apache.hadoop.conf.Configuration.getInstances(Ljava/lang/
> > String;Ljava/lang/Class;)Ljava/util/List;
> > at org.apache.sqoop.tool.SqoopTool.loadPlugins(SqoopTool.java:139)
> > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:210)
> > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
> > at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> > at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
> >
> > I could do the same trick for the
> > org.apache.hadoop.conf.Configuration.getInstances method, but I
> > haven't found the correct jar-file yet. And this is definitely not the
> > correct way to do it.
> >
> > Can anyone help me setting up Sqoop correctly?
> >
> > Thanks for your help in advance.
>

Re: Sqoop Installation Problem

Posted by Kate Ting <ka...@cloudera.com>.

Hi Otto - since you don't need to set SQOOP_HOME, can you go ahead and
unset that environment variable and then try running sqoop?

Also, which Hadoop version are you on?

Regards, Kate

On Tue, Dec 20, 2011 at 10:26 AM, Otto Rossbach
<ot...@googlemail.com> wrote:
> Hi,
> I do have some trouble setting up Sqoop. I downloaded the latest
> version of Sqoop
> from http://www.apache.org/dyn/closer.cgi/incubator/sqoop/.
> I figured out, that I don't have to build sqoop using ant, because the
> binaries and the jar's are already in the tar.
>
> I have set the environment variables:
> HADOOP_HOME
> HIVE_HOME
> SQOOP_HOME
> and have their bin subfolders added to the PATH variable.
>
> Running 'sqoop' or 'sqoop version' leads to the following error:
>
> hadoop@dev1:/usr/local/sqoop> sqoop
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Exception in thread "main" java.lang.NoClassDefFoundError: com/
> cloudera/sqoop/Sqoop
> Caused by: java.lang.ClassNotFoundException: com.cloudera.sqoop.Sqoop
> at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
> Could not find the main class: com.cloudera.sqoop.Sqoop.  Program will
> exit.
>
> That's why I tried to add the missing jars in the HADOOP_CLASSPATH in
> sqoop/bin/configure-sqoop. I added the following line at the end of
> the file: export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/local/sqoop/
> sqoop-1.4.0-incubating.jar
> So that the com/cloudera/sqoop/Sqoop class was found. But
> unfortunately this does not solve the problem. Now I do get the
> following error when executing "sqoop version". (I know this is a very
> dirty hack, but I really don't know what variable I need to set, so
> that sqoop finds its own jar-files?! Very frustrating!)
>
> hadoop@dev1:/usr/local/sqoop> sqoop version
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.apache.hadoop.conf.Configuration.getInstances(Ljava/lang/
> String;Ljava/lang/Class;)Ljava/util/List;
> at org.apache.sqoop.tool.SqoopTool.loadPlugins(SqoopTool.java:139)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:210)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
>
> I could do the same trick for the
> org.apache.hadoop.conf.Configuration.getInstances method, but I
> haven't found the correct jar-file yet. And this is definitely not the
> correct way to do it.
>
> Can anyone help me setting up Sqoop correctly?
>
> Thanks for your help in advance.