[jira] [Created] (OOZIE-3598) Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration

["TomQeeMoer (Jira)" <ji...@apache.org>]

TomQeeMoer created OOZIE-3598:
---------------------------------

             Summary: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
                 Key: OOZIE-3598
                 URL: https://issues.apache.org/jira/browse/OOZIE-3598
             Project: Oozie
          Issue Type: Bug
          Components: scripts
    Affects Versions: 5.1.0
            Reporter: TomQeeMoer
         Attachments: oozie-setup.sh

This is my first time to try oozie,I followed the steps in quick start:

prepare step: copy the libs into libext

 
{code:java}
mkdir libext
cp /opt/hadoop/share/hadoop/*/*.jar libext/
cp /opt/hadoop/share/hadoop/*/lib/*.jar libext/
cp /tmp/ext-2.2.zip libext/{code}
 

step1:  upload the sharelib
{code:java}
./bin/oozie-setup.sh sharelib create -fs hdfs://hadoop01/ -locallib oozie-sharelib-5.1.0.tar.gz
{code}
step2: create database

 

 
{code:java}
./bin/ooziedb.sh create -sqlfile oozie.sql -run
{code}
step3:  run oozied

 

 
{code:java}
./bin/oozied.sh run
{code}
Error occured:

 

 
{code:java}
Error: A JNI error has occurred, please check your installation and try againError: A JNI error has occurred, please check your installation and try againException in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) at java.lang.Class.privateGetMethodRecursive(Class.java:3048) at java.lang.Class.getMethod0(Class.java:3018) at java.lang.Class.getMethod(Class.java:1784) at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544) at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:419) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:352) ... 7 more
{code}
 

 

I checked the jar files for many times, and I confirmed that the class org.apache.hadoop.conf.Configuration was included within *hadoop-common-2.7.7.jar*

now I find out why:

when I run step1, the *oozie-setup.sh* scipt , code in lines 136~140,create the 'lib' directory, and copied the self contained jars  in ${JETTY_LIB_DIR} to lib

 
{code:java}
#Create lib directory from war if lib doesn't exist    
if [ ! -d "${BASEDIR}/lib" ]; then
      mkdir ${BASEDIR}/lib      
      cp ${JETTY_LIB_DIR}/*  ${BASEDIR}/lib
fi
{code}
 

 

when I run step3, the function *prepare_jetty()*  in ** *oozie-setup.sh* scipt copied the jars in libext to  ${JETTY_LIB_DIR},there is some mistakes in lines 236-237:

 
{code:java}
found=`ls ${JETTY_LIB_DIR}/${jarPath} 2> /dev/null | wc -l`
checkExec "looking for JAR ${jarPath} in ${JETTY_LIB_DIR}"
{code}
1. when the ${JETTY_LIB_DIR} is */opt/oozie-5.1.0/embedded-oozie-server/webapp/WEB-INF/lib and* the ${jarPath} is */opt/oozie-5.1.0/libext/somejar.jar,*

 

the Combination ${JETTY_LIB_DIR}/${jarPath} is mistaken

I thought it should be like this:
{code:java}
found=`ls ${JETTY_LIB_DIR}/${jarPath##*/} 2> /dev/null | wc -l`
{code}
2. Since the return status of statement "ls ${JETTY_LIB_DIR}/${jarPath} 2> /dev/null | wc -l" is always 0, the error message "looking for JAR ${jarPath} in ${JETTY_LIB_DIR}" will never be printed.

 

In any case, the jars in libext were copied into ${JETTY_LIB_DIR} successfully. But ${JETTY_LIB_DIR}  in *NOT* included in "-Djava.library.path -cp"

code in oozie-jetty-server.sh,lines 81-83

 
{code:java}
jetty_opts="${jetty_opts} -Djava.library.path=${JAVA_LIBRARY_PATH}";
jetty_opts="${jetty_opts} -cp ${JETTY_DIR}/*:${JETTY_DIR}/dependency/*:${BASEDIR}/lib/*:${BASEDIR}/libtools/*:${JETTY_DIR}"
{code}
 

and the function *symlink_lib()* do NOT work , since the 'lib' directory was created in step1,which is not including the hadoop jars.
{code:java}
symlink_lib() {  test -e ${BASEDIR}/lib || ln -s ${JETTY_DIR}/webapp/WEB-INF/lib ${BASEDIR}/lib}
{code}
 

I modified the *oozie-setup.sh*, ** and it worked for me . see it attachement

Forgive my poor english

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)