You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Hari Sreekumar <hs...@clickable.com> on 2011/03/29 14:25:24 UTC

Confusion regarding version of hadoop to use in hbase 0.90.1

Hi,

I went through this thread (
http://www.apacheserver.net/Using-Hadoop-bundled-in-lib-directory-HBase-at1134614.htm)
which mentions that the hadoop jars ( hadoop-0.20.2/hadoop-0.20.2-core.jar
and hadoop-0.20.2/hadoop-0.20.2-core.jar) can simply be replaced by
hbase-0.90.1/lib/hadoop-*.jar files. Now I have the following jars in
hbase-0.90.1/lib:
hadoop-core-0.20-append-r1056497.jar

In my hadoop-0.20.2/*.jar, I have these jars :
hadoop-0.20.2-ant.jar
hadoop-0.20.2-examples.jar
hadoop-0.20.2-tools.jar
hadoop-0.20.2-core.jar
hadoop-0.20.2-test.jar

If anyone else has tried this way, which jar did you replace?
I tried replacing hadoop-0.20.2-core.jar
with hadoop-core-0.20-append-r1056497.jar. I get this error when I tried to
start dfs:

Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/util/PlatformName
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.util.PlatformName
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
Could not find the main class: org.apache.hadoop.util.PlatformName.  Program
will exit.
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/hdfs/server/namenode/NameNode
hadoop1: starting datanode, logging to
/opt/hadoop/bin/../logs/hadoop-hadoop-datanode-hadoop1.out
hadoop2: starting datanode, logging to
/opt/hadoop/bin/../logs/hadoop-hadoop-datanode-hadoop2.out
hadoop3: starting datanode, logging to
/opt/hadoop/bin/../logs/hadoop-hadoop-datanode-hadoop3.out
ejabber: starting datanode, logging to
/opt/hadoop/bin/../logs/hadoop-hadoop-datanode-ejabber.out
hadoop1: Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/util/PlatformName
hadoop1: Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.util.PlatformName
hadoop1:        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
hadoop1:        at java.security.AccessController.doPrivileged(Native
Method)
hadoop1:        at
java.net.URLClassLoader.findClass(URLClassLoader.java:190)
hadoop1:        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
hadoop1:        at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
hadoop1:        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
hadoop1: Could not find the main class: org.apache.hadoop.util.PlatformName.
 Program will exit.
hadoop1: Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/hdfs/server/datanode/DataNode
hadoop2: Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/util/PlatformName
hadoop2: Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.util.PlatformName
hadoop2:        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
hadoop2:        at java.security.AccessController.doPrivileged(Native
Method)
hadoop2:        at
java.net.URLClassLoader.findClass(URLClassLoader.java:190)
hadoop2:        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
hadoop2:        at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
hadoop2:        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)

Am I instead supposed to keep both jars and not replace one with another ?
Won't that lead to conflict?

Thanks,
Hari

Re: Confusion regarding version of hadoop to use in hbase 0.90.1

Posted by Hari Sreekumar <hs...@clickable.com>.
It worked when I added the append jar file to HADOOP_CLASSPATH in
hadoop-env.sh. I think this line in bin/hadoop is the culprit:

for f in $HADOOP_HOME/hadoop-*-core.jar; do
  CLASSPATH=${CLASSPATH}:$f;

The append jar file doesn't match the hadoop-*-core.jar pattern, so it did
not get added to the classpath.

Thanks,
Hari



On Tue, Mar 29, 2011 at 6:30 PM, Harsh J <qw...@gmail.com> wrote:

> Hello,
>
> On Tue, Mar 29, 2011 at 5:55 PM, Hari Sreekumar
> <hs...@clickable.com> wrote:
> > Hi,
> >
> > I went through this thread (
> >
> http://www.apacheserver.net/Using-Hadoop-bundled-in-lib-directory-HBase-at1134614.htm
> )
> > which mentions that the hadoop jars (
> hadoop-0.20.2/hadoop-0.20.2-core.jar
> > and hadoop-0.20.2/hadoop-0.20.2-core.jar) can simply be replaced by
> > hbase-0.90.1/lib/hadoop-*.jar files. Now I have the following jars in
> > hbase-0.90.1/lib:
> > hadoop-core-0.20-append-r1056497.jar
> > If anyone else has tried this way, which jar did you replace?
> > I tried replacing hadoop-0.20.2-core.jar
> > with hadoop-core-0.20-append-r1056497.jar. I get this error when I tried
> to
> > start dfs:
> >
> > Exception in thread "main" java.lang.NoClassDefFoundError:
> > org/apache/hadoop/util/PlatformName
> > Caused by: java.lang.ClassNotFoundException:
> > org.apache.hadoop.util.PlatformName
>
> It must be replaced, yes (not placed along-side, which would indeed
> lead to conflicts).
>
> Could you verify if the supplied jar indeed contains the required classes?
> Something like `jar tvf <jar> | grep ClassName` should throw up a result.
>
> An opinion though, given a chance, I'd actually pick up CDH3B4 packs
> from Cloudera to setup a HBase cluster (and not really have to worry
> too much about these things).
>
> --
> Harsh J
> http://harshj.com
>

Re: Confusion regarding version of hadoop to use in hbase 0.90.1

Posted by Harsh J <qw...@gmail.com>.
Hello,

On Tue, Mar 29, 2011 at 5:55 PM, Hari Sreekumar
<hs...@clickable.com> wrote:
> Hi,
>
> I went through this thread (
> http://www.apacheserver.net/Using-Hadoop-bundled-in-lib-directory-HBase-at1134614.htm)
> which mentions that the hadoop jars ( hadoop-0.20.2/hadoop-0.20.2-core.jar
> and hadoop-0.20.2/hadoop-0.20.2-core.jar) can simply be replaced by
> hbase-0.90.1/lib/hadoop-*.jar files. Now I have the following jars in
> hbase-0.90.1/lib:
> hadoop-core-0.20-append-r1056497.jar
> If anyone else has tried this way, which jar did you replace?
> I tried replacing hadoop-0.20.2-core.jar
> with hadoop-core-0.20-append-r1056497.jar. I get this error when I tried to
> start dfs:
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/util/PlatformName
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.util.PlatformName

It must be replaced, yes (not placed along-side, which would indeed
lead to conflicts).

Could you verify if the supplied jar indeed contains the required classes?
Something like `jar tvf <jar> | grep ClassName` should throw up a result.

An opinion though, given a chance, I'd actually pick up CDH3B4 packs
from Cloudera to setup a HBase cluster (and not really have to worry
too much about these things).

-- 
Harsh J
http://harshj.com