You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bigtop.apache.org by "Masatake Iwasaki (Jira)" <ji...@apache.org> on 2020/11/04 12:28:00 UTC
[jira] [Commented] (BIGTOP-3439) Oozie's smoke test fails on CentOS
and Fedora
[ https://issues.apache.org/jira/browse/BIGTOP-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17226021#comment-17226021 ]
Masatake Iwasaki commented on BIGTOP-3439:
------------------------------------------
The cause is addition to bigtop-packages/src/common/oozie/install_oozie.sh by BIGTOP-3330 which only cares .deb package.
{noformat}
b37aafaa2 (Luca Toscano 2020-03-22 18:25:40 +0100 235)# Remove jars provided by 'oozie-client' from 'oozie' to avoid run-time issues
b37aafaa2 (Luca Toscano 2020-03-22 18:25:40 +0100 236)# while installing the package.
b37aafaa2 (Luca Toscano 2020-03-22 18:25:40 +0100 237)for oozie_client_jar_file in $(ls $CLIENT_LIB_DIR/lib); do
b37aafaa2 (Luca Toscano 2020-03-22 18:25:40 +0100 238) rm -f $SERVER_LIB_DIR/lib/$oozie_client_jar_file
b37aafaa2 (Luca Toscano 2020-03-22 18:25:40 +0100 239)done
{noformat}
For deb, --server-dir and ---client-dir are independent.
{noformat}
sh -x debian/install_oozie.sh --extra-dir=debian/ --build-dir=$(PWD) --server-dir=./debian/oozie --client-dir=./debian/oozie-client --docs-dir=./debian/oozie-client/usr/share/doc/oozie\|getent passwd oozie >/dev/null || /usr/sbin/useradd --comment "Oozie User" --shell /bin/false -M -r -g oozie --home %{data_oozie} oozie >/dev/null
--initd-dir=./debian/oozie/etc/init.d --conf-dir=./debian/oozie/etc/oozie/conf.dist
{noformat}
For rpm, --server-dir and ---client-dir have the same value.
{noformat}
sh %{SOURCE2} --extra-dir=$RPM_SOURCE_DIR --build-dir=$PWD --server-dir=$RPM_BUILD_ROOT --client-dir=$RPM_BUILD_ROOT --docs-dir=$RPM_BUILD_ROOT%{doc_oozie} --initd-dir=$RPM_BUILD_ROOT%{initd_dir} --conf-dir=$RPM_BUILD_ROOT%{conf_oozie_dist}
{noformat}
> Oozie's smoke test fails on CentOS and Fedora
> ---------------------------------------------
>
> Key: BIGTOP-3439
> URL: https://issues.apache.org/jira/browse/BIGTOP-3439
> Project: Bigtop
> Issue Type: Bug
> Components: tests
> Reporter: Kengo Seki
> Assignee: Masatake Iwasaki
> Priority: Major
> Fix For: 1.5.0
>
>
> It's a follow-up issue of BIGTOP-3406. With that change, Oozie's smoke test got to pass on Debian and Ubuntu, but it still fails on CentOS and Fedora with the following error.
> {code}
> ===== START TO RUN SMOKE TESTS: oozie =====
> + su -s /bin/bash hdfs -c 'JAVA_LIBRARY_PATH=/usr/lib/qfs hadoop fs -chmod 777 /user/vagrant'
> + su -s /bin/bash hdfs -c 'JAVA_LIBRARY_PATH=/usr/lib/qfs hadoop fs -chmod 777 /user/root'
> + su -s /bin/bash hdfs -c 'JAVA_LIBRARY_PATH=/usr/lib/qfs hadoop fs -chown yarn:yarn /user/yarn'
> + [[ oozie == *\q\f\s* ]]
> + [[ oozie == *\a\l\l\u\x\i\o* ]]
> + [[ oozie == *\o\o\z\i\e* ]]
> + su -s /bin/bash hdfs -c 'hadoop fs -mkdir -p /user/oozie/share/lib'
> + su -s /bin/bash hdfs -c 'hadoop fs -chown -R oozie:oozie /user/oozie'
> ++ hostname -f
> + oozie-setup sharelib create -fs hdfs://247b00365170.bigtop.apache.org:8020/
> setting OOZIE_DATA=/var/lib/oozie
> setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
> setting CATALINA_TMPDIR=/var/lib/oozie
> setting CATALINA_PID=/var/run/oozie/oozie.pid
> setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
> setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
> setting OOZIE_CONFIG=/etc/oozie/conf
> setting OOZIE_LOG=/var/log/oozie
> setting OOZIE_DATA=/var/lib/oozie
> setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
> setting CATALINA_TMPDIR=/var/lib/oozie
> setting CATALINA_PID=/var/run/oozie/oozie.pid
> setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
> setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
> setting OOZIE_CONFIG=/etc/oozie/conf
> setting OOZIE_LOG=/var/log/oozie
> Error: A JNI error has occurred, please check your installation and try again
> Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/cli/ParseException
> at java.lang.Class.getDeclaredMethods0(Native Method)
> at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
> at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
> at java.lang.Class.getMethod0(Class.java:3018)
> at java.lang.Class.getMethod(Class.java:1784)
> at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
> at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
> Caused by: java.lang.ClassNotFoundException: org.apache.commons.cli.ParseException
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 7 more
> + oozie admin -sharelibupdate
> Error: Could not find or load main class org.apache.oozie.cli.OozieCLI
> (snip)
> {code}
> As [~elukey] [pointed out|https://github.com/apache/bigtop/pull/676#issuecomment-715899945], the required jar files don't seem to be contained in the generated rpm files, but I've not digged into its reason yet.
> {code}
> $ rpm -ql output/oozie/noarch/oozie-4.3.0-1.el8.noarch.rpm | grep /usr/lib/oozie/lib
> /usr/lib/oozie/lib
> /usr/lib/oozie/lib/hadoop-annotations.jar
> /usr/lib/oozie/lib/hadoop-auth.jar
> /usr/lib/oozie/lib/hadoop-common.jar
> /usr/lib/oozie/lib/hadoop-hdfs-client.jar
> /usr/lib/oozie/lib/hadoop-mapreduce-client-app.jar
> /usr/lib/oozie/lib/hadoop-mapreduce-client-common.jar
> /usr/lib/oozie/lib/hadoop-mapreduce-client-core.jar
> /usr/lib/oozie/lib/hadoop-mapreduce-client-jobclient.jar
> /usr/lib/oozie/lib/hadoop-mapreduce-client-shuffle.jar
> /usr/lib/oozie/lib/hadoop-yarn-api.jar
> /usr/lib/oozie/lib/hadoop-yarn-client.jar
> /usr/lib/oozie/lib/hadoop-yarn-common.jar
> /usr/lib/oozie/lib/hadoop-yarn-server-common.jar
> /usr/lib/oozie/libext
> /usr/lib/oozie/libtools
> /usr/lib/oozie/libtools/gson-2.7.jar
> /usr/lib/oozie/libtools/oozie-tools-4.3.0.jar
> $ rpm -ql output/oozie/noarch/oozie-client-4.3.0-1.el8.noarch.rpm | grep /usr/lib/oozie/lib
> /usr/lib/oozie/lib
> /usr/lib/oozie/lib/hadoop-annotations.jar
> /usr/lib/oozie/lib/hadoop-auth.jar
> /usr/lib/oozie/lib/hadoop-common.jar
> /usr/lib/oozie/lib/hadoop-hdfs-client.jar
> /usr/lib/oozie/lib/hadoop-mapreduce-client-app.jar
> /usr/lib/oozie/lib/hadoop-mapreduce-client-common.jar
> /usr/lib/oozie/lib/hadoop-mapreduce-client-core.jar
> /usr/lib/oozie/lib/hadoop-mapreduce-client-jobclient.jar
> /usr/lib/oozie/lib/hadoop-mapreduce-client-shuffle.jar
> /usr/lib/oozie/lib/hadoop-yarn-api.jar
> /usr/lib/oozie/lib/hadoop-yarn-client.jar
> /usr/lib/oozie/lib/hadoop-yarn-common.jar
> /usr/lib/oozie/lib/hadoop-yarn-server-common.jar
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)