You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Andy Li <an...@gmail.com> on 2008/02/24 13:24:38 UTC
Exception when running HBase from Hadoop 0.16 package - java.lang.NoClassDefFoundError org/apache/hadoop/hbase/HBaseConfiguration
Hi folks,
Have anyone encountered the same problem when running HBase on Hadoop 0.16?
I installed Hadoop 0.16 from the website, everything comes with default.
Here is the following config file for Hbase. I observed that Hbase
directory structure is
quite different than the one came from Hadoop 0.15.3.
The config file I use is under this directory.
/opt/hadoop/contrib/hbase/conf/
Other config file hadoop-env.sh and hbase-env.sh are both from the default,
nothing has been changed.
Here is my hbase-site.xml config file.
========== COPY/PASTE STARTS HERE ==========
<configuration>
<property>
<name>hbase.master</name>
<value>10.1.1.2:60000</value>
<final>true</final>
</property>
<property>
<name>hbase.client.retries.number</name>
<value>10</value>
</property>
<property>
<name>hbase.master.lease.period</name>
<value>60000</value>
</property>
<property>
<name>hbase.regionserver.lease.period</name>
<value>120000</value>
</property>
<property>
<name>hbase.regionserver.optionalcacheflushinterval</name>
<value>600000</value>
</property>
</configuration>
========== COPY/PASTE ENDS HERE ==========
Does anyone know what is the cause?
My guess is the hbase classpath is not setup properly on the system base on
the exception message,
but not sure why it happened and how to solve it. Any inputs are welcome.
Thanks,
-Andy
Re: Exception when running HBase from Hadoop 0.16 package - java.lang.NoClassDefFoundError
org/apache/hadoop/hbase/HBaseConfiguration
Posted by stack <st...@duboce.net>.
Andy:
This page, http://wiki.apache.org/hadoop/Hbase/MapReduce, is at attempt
at an hadoop+hbase HOWTO. It suggests that the easiest means of getting
hbase resources into the mix is to add them to the HADOOP_CLASSPATH.
See what you think.
St.Ack
Andy Li wrote:
> A work around is to create a symbolic link in the /opt/hadoop/lib/ that
> points to the hbase library JAR file.
>
> This is definitely not the formal or correct way to config HBase library
> with Hadoop,
> but it works for me so far. However, I would like to know the formal way to
> configure the
> HBase classpath that is used by Hadoop when I run a MapReduce job that needs
> to access HBase.
>
> Thanks,
> Andy
>
> On Sun, Feb 24, 2008 at 10:22 AM, Andy Li <an...@gmail.com> wrote:
>
>
>> Sorry, I forgot to copy/paste the stack trace.
>>
>> Here is what I get by running ./bin/hadoop jar XXXXXXX
>> On the region server, I get the same following exception in the syslog
>> form the userlog.
>> ========== COPY/PASTE STARTS HERE ===========
>> java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:247)
>> at org.apache.hadoop.conf.Configuration.getClassByName(
>> Configuration.java:587)
>> at org.apache.hadoop.conf.Configuration.getClass(
>> Configuration.java:605)
>> at org.apache.hadoop.conf.Configuration.getClass(
>> Configuration.java:631)
>> at org.apache.hadoop.mapred.JobConf.getReducerClass(JobConf.java
>> :788)
>> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:250)
>> at org.apache.hadoop.mapred.TaskTracker$Child.main(
>> TaskTracker.java:2071)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.hadoop.hbase.HBaseConfiguration
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
>> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
>> ... 8 more
>> ========== COPY/PASTE ENDS HERE ===========
>>
>>
>>
>> On Sun, Feb 24, 2008 at 8:13 AM, Jim Kellerman <ji...@powerset.com> wrote:
>>
>>
>>> It would be helpful if you could paste the entire exception and
>>> stack trace.
>>>
>>> ---
>>> Jim Kellerman, Senior Engineer; Powerset
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Andy Li [mailto:annndy.lee@gmail.com]
>>>> Sent: Sunday, February 24, 2008 4:25 AM
>>>> To: hbase-user@hadoop.apache.org
>>>> Subject: Exception when running HBase from Hadoop 0.16
>>>> package - java.lang.NoClassDefFoundError
>>>> org/apache/hadoop/hbase/HBaseConfiguration
>>>>
>>>> Hi folks,
>>>>
>>>> Have anyone encountered the same problem when running HBase
>>>> on Hadoop 0.16?
>>>>
>>>> I installed Hadoop 0.16 from the website, everything comes
>>>> with default.
>>>>
>>>> Here is the following config file for Hbase. I observed that
>>>> Hbase directory structure is quite different than the one
>>>> came from Hadoop 0.15.3.
>>>>
>>>> The config file I use is under this directory.
>>>> /opt/hadoop/contrib/hbase/conf/
>>>> Other config file hadoop-env.sh and hbase-env.sh are both
>>>> from the default, nothing has been changed.
>>>>
>>>> Here is my hbase-site.xml config file.
>>>> ========== COPY/PASTE STARTS HERE ========== <configuration>
>>>> <property>
>>>> <name>hbase.master</name>
>>>> <value>10.1.1.2:60000</value>
>>>> <final>true</final>
>>>> </property>
>>>> <property>
>>>> <name>hbase.client.retries.number</name>
>>>> <value>10</value>
>>>> </property>
>>>> <property>
>>>> <name>hbase.master.lease.period</name>
>>>> <value>60000</value>
>>>> </property>
>>>> <property>
>>>> <name>hbase.regionserver.lease.period</name>
>>>> <value>120000</value>
>>>> </property>
>>>> <property>
>>>> <name>hbase.regionserver.optionalcacheflushinterval</name>
>>>> <value>600000</value>
>>>> </property>
>>>> </configuration>
>>>> ========== COPY/PASTE ENDS HERE ==========
>>>>
>>>> Does anyone know what is the cause?
>>>> My guess is the hbase classpath is not setup properly on the
>>>> system base on the exception message, but not sure why it
>>>> happened and how to solve it. Any inputs are welcome.
>>>>
>>>> Thanks,
>>>> -Andy
>>>>
>>>> No virus found in this incoming message.
>>>> Checked by AVG Free Edition.
>>>> Version: 7.5.516 / Virus Database: 269.20.9/1294 - Release
>>>> Date: 2/22/2008 6:39 PM
>>>>
>>>>
>>>>
>>> No virus found in this outgoing message.
>>> Checked by AVG Free Edition.
>>> Version: 7.5.516 / Virus Database: 269.20.9/1294 - Release Date:
>>> 2/22/2008 6:39 PM
>>>
>>>
>>>
>
>
Re: Exception when running HBase from Hadoop 0.16 package - java.lang.NoClassDefFoundError org/apache/hadoop/hbase/HBaseConfiguration
Posted by Andy Li <an...@gmail.com>.
A work around is to create a symbolic link in the /opt/hadoop/lib/ that
points to the hbase library JAR file.
This is definitely not the formal or correct way to config HBase library
with Hadoop,
but it works for me so far. However, I would like to know the formal way to
configure the
HBase classpath that is used by Hadoop when I run a MapReduce job that needs
to access HBase.
Thanks,
Andy
On Sun, Feb 24, 2008 at 10:22 AM, Andy Li <an...@gmail.com> wrote:
> Sorry, I forgot to copy/paste the stack trace.
>
> Here is what I get by running ./bin/hadoop jar XXXXXXX
> On the region server, I get the same following exception in the syslog
> form the userlog.
> ========== COPY/PASTE STARTS HERE ===========
> java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at org.apache.hadoop.conf.Configuration.getClassByName(
> Configuration.java:587)
> at org.apache.hadoop.conf.Configuration.getClass(
> Configuration.java:605)
> at org.apache.hadoop.conf.Configuration.getClass(
> Configuration.java:631)
> at org.apache.hadoop.mapred.JobConf.getReducerClass(JobConf.java
> :788)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:250)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(
> TaskTracker.java:2071)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.HBaseConfiguration
> at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
> ... 8 more
> ========== COPY/PASTE ENDS HERE ===========
>
>
>
> On Sun, Feb 24, 2008 at 8:13 AM, Jim Kellerman <ji...@powerset.com> wrote:
>
> > It would be helpful if you could paste the entire exception and
> > stack trace.
> >
> > ---
> > Jim Kellerman, Senior Engineer; Powerset
> >
> >
> > > -----Original Message-----
> > > From: Andy Li [mailto:annndy.lee@gmail.com]
> > > Sent: Sunday, February 24, 2008 4:25 AM
> > > To: hbase-user@hadoop.apache.org
> > > Subject: Exception when running HBase from Hadoop 0.16
> > > package - java.lang.NoClassDefFoundError
> > > org/apache/hadoop/hbase/HBaseConfiguration
> > >
> > > Hi folks,
> > >
> > > Have anyone encountered the same problem when running HBase
> > > on Hadoop 0.16?
> > >
> > > I installed Hadoop 0.16 from the website, everything comes
> > > with default.
> > >
> > > Here is the following config file for Hbase. I observed that
> > > Hbase directory structure is quite different than the one
> > > came from Hadoop 0.15.3.
> > >
> > > The config file I use is under this directory.
> > > /opt/hadoop/contrib/hbase/conf/
> > > Other config file hadoop-env.sh and hbase-env.sh are both
> > > from the default, nothing has been changed.
> > >
> > > Here is my hbase-site.xml config file.
> > > ========== COPY/PASTE STARTS HERE ========== <configuration>
> > > <property>
> > > <name>hbase.master</name>
> > > <value>10.1.1.2:60000</value>
> > > <final>true</final>
> > > </property>
> > > <property>
> > > <name>hbase.client.retries.number</name>
> > > <value>10</value>
> > > </property>
> > > <property>
> > > <name>hbase.master.lease.period</name>
> > > <value>60000</value>
> > > </property>
> > > <property>
> > > <name>hbase.regionserver.lease.period</name>
> > > <value>120000</value>
> > > </property>
> > > <property>
> > > <name>hbase.regionserver.optionalcacheflushinterval</name>
> > > <value>600000</value>
> > > </property>
> > > </configuration>
> > > ========== COPY/PASTE ENDS HERE ==========
> > >
> > > Does anyone know what is the cause?
> > > My guess is the hbase classpath is not setup properly on the
> > > system base on the exception message, but not sure why it
> > > happened and how to solve it. Any inputs are welcome.
> > >
> > > Thanks,
> > > -Andy
> > >
> > > No virus found in this incoming message.
> > > Checked by AVG Free Edition.
> > > Version: 7.5.516 / Virus Database: 269.20.9/1294 - Release
> > > Date: 2/22/2008 6:39 PM
> > >
> > >
> >
> > No virus found in this outgoing message.
> > Checked by AVG Free Edition.
> > Version: 7.5.516 / Virus Database: 269.20.9/1294 - Release Date:
> > 2/22/2008 6:39 PM
> >
> >
>
Re: Exception when running HBase from Hadoop 0.16 package - java.lang.NoClassDefFoundError org/apache/hadoop/hbase/HBaseConfiguration
Posted by Andy Li <an...@gmail.com>.
Sorry, I forgot to copy/paste the stack trace.
Here is what I get by running ./bin/hadoop jar XXXXXXX
On the region server, I get the same following exception in the syslog form
the userlog.
========== COPY/PASTE STARTS HERE ===========
java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.conf.Configuration.getClassByName(
Configuration.java:587)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java
:605)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java
:631)
at org.apache.hadoop.mapred.JobConf.getReducerClass(JobConf.java
:788)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:250)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java
:2071)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.hbase.HBaseConfiguration
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
... 8 more
========== COPY/PASTE ENDS HERE ===========
On Sun, Feb 24, 2008 at 8:13 AM, Jim Kellerman <ji...@powerset.com> wrote:
> It would be helpful if you could paste the entire exception and
> stack trace.
>
> ---
> Jim Kellerman, Senior Engineer; Powerset
>
>
> > -----Original Message-----
> > From: Andy Li [mailto:annndy.lee@gmail.com]
> > Sent: Sunday, February 24, 2008 4:25 AM
> > To: hbase-user@hadoop.apache.org
> > Subject: Exception when running HBase from Hadoop 0.16
> > package - java.lang.NoClassDefFoundError
> > org/apache/hadoop/hbase/HBaseConfiguration
> >
> > Hi folks,
> >
> > Have anyone encountered the same problem when running HBase
> > on Hadoop 0.16?
> >
> > I installed Hadoop 0.16 from the website, everything comes
> > with default.
> >
> > Here is the following config file for Hbase. I observed that
> > Hbase directory structure is quite different than the one
> > came from Hadoop 0.15.3.
> >
> > The config file I use is under this directory.
> > /opt/hadoop/contrib/hbase/conf/
> > Other config file hadoop-env.sh and hbase-env.sh are both
> > from the default, nothing has been changed.
> >
> > Here is my hbase-site.xml config file.
> > ========== COPY/PASTE STARTS HERE ========== <configuration>
> > <property>
> > <name>hbase.master</name>
> > <value>10.1.1.2:60000</value>
> > <final>true</final>
> > </property>
> > <property>
> > <name>hbase.client.retries.number</name>
> > <value>10</value>
> > </property>
> > <property>
> > <name>hbase.master.lease.period</name>
> > <value>60000</value>
> > </property>
> > <property>
> > <name>hbase.regionserver.lease.period</name>
> > <value>120000</value>
> > </property>
> > <property>
> > <name>hbase.regionserver.optionalcacheflushinterval</name>
> > <value>600000</value>
> > </property>
> > </configuration>
> > ========== COPY/PASTE ENDS HERE ==========
> >
> > Does anyone know what is the cause?
> > My guess is the hbase classpath is not setup properly on the
> > system base on the exception message, but not sure why it
> > happened and how to solve it. Any inputs are welcome.
> >
> > Thanks,
> > -Andy
> >
> > No virus found in this incoming message.
> > Checked by AVG Free Edition.
> > Version: 7.5.516 / Virus Database: 269.20.9/1294 - Release
> > Date: 2/22/2008 6:39 PM
> >
> >
>
> No virus found in this outgoing message.
> Checked by AVG Free Edition.
> Version: 7.5.516 / Virus Database: 269.20.9/1294 - Release Date: 2/22/2008
> 6:39 PM
>
>
RE: Exception when running HBase from Hadoop 0.16 package -
java.lang.NoClassDefFoundError org/apache/hadoop/hbase/HBaseConfiguration
Posted by Jim Kellerman <ji...@powerset.com>.
It would be helpful if you could paste the entire exception and
stack trace.
---
Jim Kellerman, Senior Engineer; Powerset
> -----Original Message-----
> From: Andy Li [mailto:annndy.lee@gmail.com]
> Sent: Sunday, February 24, 2008 4:25 AM
> To: hbase-user@hadoop.apache.org
> Subject: Exception when running HBase from Hadoop 0.16
> package - java.lang.NoClassDefFoundError
> org/apache/hadoop/hbase/HBaseConfiguration
>
> Hi folks,
>
> Have anyone encountered the same problem when running HBase
> on Hadoop 0.16?
>
> I installed Hadoop 0.16 from the website, everything comes
> with default.
>
> Here is the following config file for Hbase. I observed that
> Hbase directory structure is quite different than the one
> came from Hadoop 0.15.3.
>
> The config file I use is under this directory.
> /opt/hadoop/contrib/hbase/conf/
> Other config file hadoop-env.sh and hbase-env.sh are both
> from the default, nothing has been changed.
>
> Here is my hbase-site.xml config file.
> ========== COPY/PASTE STARTS HERE ========== <configuration>
> <property>
> <name>hbase.master</name>
> <value>10.1.1.2:60000</value>
> <final>true</final>
> </property>
> <property>
> <name>hbase.client.retries.number</name>
> <value>10</value>
> </property>
> <property>
> <name>hbase.master.lease.period</name>
> <value>60000</value>
> </property>
> <property>
> <name>hbase.regionserver.lease.period</name>
> <value>120000</value>
> </property>
> <property>
> <name>hbase.regionserver.optionalcacheflushinterval</name>
> <value>600000</value>
> </property>
> </configuration>
> ========== COPY/PASTE ENDS HERE ==========
>
> Does anyone know what is the cause?
> My guess is the hbase classpath is not setup properly on the
> system base on the exception message, but not sure why it
> happened and how to solve it. Any inputs are welcome.
>
> Thanks,
> -Andy
>
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.5.516 / Virus Database: 269.20.9/1294 - Release
> Date: 2/22/2008 6:39 PM
>
>
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.5.516 / Virus Database: 269.20.9/1294 - Release Date: 2/22/2008 6:39 PM