You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Usman Waheed <us...@opera.com> on 2011/02/20 22:27:28 UTC

Hbase Bulk Loader

Hi,

I am getting the following error when trying to run an import job using  
hadoop with the importtsv tool. My HADOOP_CLASSPATH is set to the  
following in hadoop-env.sh:

export  
HADOOP_CLASSPATH=/usr/lib/hbase-0.90.0/lib/hbase-0.90.0.jar:/usr/lib/hbase-0.90.0/lib/zookeeper-3.3.2.jar:/usr/lib/hbase-0.90.0/lib/google-collections-1.0-rc1-sources.jar

The error message i am getting is:

hadoop jar /usr/lib/hbase-0.90.0/hbase-0.90.0.jar importtsv  
-Dimporttsv.columns=HBASE_ROW_KEY,metrics:a,metrics:b,metrics:c,metrics:d  
test /var/data/input -Dimporttsv.bulk.output=/var/data/output
Exception in thread "main" java.lang.NoClassDefFoundError:  
com/google/common/base/Function
         at  
org.apache.hadoop.hbase.mapreduce.ImportTsv.createSubmittableJob(ImportTsv.java:299)
         at  
org.apache.hadoop.hbase.mapreduce.ImportTsv.main(ImportTsv.java:370)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at  
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at  
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at  
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
         at  
org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
         at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:49)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at  
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at  
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
Caused by: java.lang.ClassNotFoundException:  
com.google.common.base.Function
         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)

I am running CDH3b3 and Hbase 0.90.0

Any clues?

Regards,
Usman

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

Re: Hbase Bulk Loader

Posted by Usman Waheed <us...@opera.com>.
The below can be fixed using guava-rc06.jar in the -libjars as follows:

hadoop jar /usr/lib/hbase-0.90.0/hbase-0.90.0.jar importtsv -libjars  
/usr/lib/hbase-0.90.0/lib/guava-r06.jar  
-Dimporttsv.columns=HBASE_ROW_KEY,metrics:a,metrics:b,metrics:c,metrics:d  
-Dimporttsv.bulk.output=/var/data/output test /var/data/input

I had to remove google-collections-1.0-rc1-sources.jar from the  
HADOOP_CLASSPATH and replace it with guava-rc06.jar.


> Hi,
>
> I am getting the following error when trying to run an import job using  
> hadoop with the importtsv tool. My HADOOP_CLASSPATH is set to the  
> following in hadoop-env.sh:
>
> export  
> HADOOP_CLASSPATH=/usr/lib/hbase-0.90.0/lib/hbase-0.90.0.jar:/usr/lib/hbase-0.90.0/lib/zookeeper-3.3.2.jar:/usr/lib/hbase-0.90.0/lib/google-collections-1.0-rc1-sources.jar
>
> The error message i am getting is:
>
> hadoop jar /usr/lib/hbase-0.90.0/hbase-0.90.0.jar importtsv  
> -Dimporttsv.columns=HBASE_ROW_KEY,metrics:a,metrics:b,metrics:c,metrics:d  
> test /var/data/input -Dimporttsv.bulk.output=/var/data/output
> Exception in thread "main" java.lang.NoClassDefFoundError:  
> com/google/common/base/Function
>          at  
> org.apache.hadoop.hbase.mapreduce.ImportTsv.createSubmittableJob(ImportTsv.java:299)
>          at  
> org.apache.hadoop.hbase.mapreduce.ImportTsv.main(ImportTsv.java:370)
>          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>          at  
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>          at  
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>          at java.lang.reflect.Method.invoke(Method.java:597)
>          at  
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
>          at  
> org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
>          at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:49)
>          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>          at  
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>          at  
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>          at java.lang.reflect.Method.invoke(Method.java:597)
>          at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> Caused by: java.lang.ClassNotFoundException:  
> com.google.common.base.Function
>          at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>          at java.security.AccessController.doPrivileged(Native Method)
>          at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>          at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>          at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>          at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>
> I am running CDH3b3 and Hbase 0.90.0
>
> Any clues?
>
> Regards,
> Usman
>


-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/