You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bigtop.apache.org by "Masatake Iwasaki (Jira)" <ji...@apache.org> on 2021/07/14 11:31:00 UTC
[jira] [Commented] (BIGTOP-3572) Oozie smoke test fails on the ARM
platform
[ https://issues.apache.org/jira/browse/BIGTOP-3572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17380509#comment-17380509 ]
Masatake Iwasaki commented on BIGTOP-3572:
------------------------------------------
Hmm. I could not reproduce the issue too.
> Oozie smoke test fails on the ARM platform
> ------------------------------------------
>
> Key: BIGTOP-3572
> URL: https://issues.apache.org/jira/browse/BIGTOP-3572
> Project: Bigtop
> Issue Type: Bug
> Components: tests
> Reporter: Kengo Seki
> Priority: Major
> Fix For: 3.0.0
>
>
> https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/247/COMPONENTS=hdfs.yarn.mapreduce.hive.oozie@oozie,OS=debian-10-arm64-deploy/console
> https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/247/COMPONENTS=hdfs.yarn.mapreduce.hive.oozie@oozie,OS=debian-9-arm64-deploy/console
> https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/247/COMPONENTS=hdfs.yarn.mapreduce.hive.oozie@oozie,OS=ubuntu-18.04-arm64-deploy/console
> https://ci.bigtop.apache.org/view/Test/job/Bigtop-trunk-smoke-tests/247/COMPONENTS=hdfs.yarn.mapreduce.hive.oozie@oozie,OS=ubuntu-20.04-arm64-deploy/console
> It looks like to be caused by the version mismatch of Guava, but I'm not sure why it happens only on arm64 (and maybe ppc64le?) platform.
> {code}
> + oozie-setup sharelib create -fs hdfs://28c610858918.bigtop.apache.org:8020/
> setting OOZIE_DATA=/var/lib/oozie
> setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
> setting CATALINA_TMPDIR=/var/lib/oozie
> setting CATALINA_PID=/var/run/oozie/oozie.pid
> setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
> setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
> setting OOZIE_CONFIG=/etc/oozie/conf
> setting OOZIE_LOG=/var/log/oozie
> setting JETTY_PID_FILE=/var/run/oozie/oozie.pid
> setting OOZIE_DATA=/var/lib/oozie
> setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
> setting CATALINA_TMPDIR=/var/lib/oozie
> setting CATALINA_PID=/var/run/oozie/oozie.pid
> setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
> setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
> setting OOZIE_CONFIG=/etc/oozie/conf
> setting OOZIE_LOG=/var/log/oozie
> setting JETTY_PID_FILE=/var/run/oozie/oozie.pid
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/usr/lib/oozie/embedded-oozie-server/webapp/WEB-INF/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/usr/lib/oozie/lib-client/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
> Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
> at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
> at org.apache.oozie.service.XLogService.extractInfoForLogWebService(XLogService.java:218)
> at org.apache.oozie.service.XLogService.init(XLogService.java:203)
> at org.apache.oozie.service.Services.setServiceInternal(Services.java:387)
> at org.apache.oozie.service.Services.<init>(Services.java:111)
> at org.apache.oozie.tools.OozieSharelibCLI.run(OozieSharelibCLI.java:198)
> at org.apache.oozie.tools.OozieSharelibCLI.main(OozieSharelibCLI.java:103)
> {code}
> I also tried to reproduce it, but I couldn't on the m6g EC2 instance.
> {code}
> ubuntu@ip-172-30-1-254:~/bigtop/provisioner/docker$ cat /etc/issue
> Ubuntu 18.04.5 LTS \n \l
> ubuntu@ip-172-30-1-254:~/bigtop/provisioner/docker$ arch
> aarch64
> ubuntu@ip-172-30-1-254:~/bigtop/provisioner/docker$ ./docker-hadoop.sh -d -C config_ubuntu-18.04.yaml -G -L -k hdfs,yarn,mapreduce,hive,oozie -s oozie -c 1
> ...
> + oozie-setup sharelib create -fs hdfs://3349cb325787.bigtop.apache.org:8020/
> setting OOZIE_DATA=/var/lib/oozie
> setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
> setting CATALINA_TMPDIR=/var/lib/oozie
> setting CATALINA_PID=/var/run/oozie/oozie.pid
> setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
> setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
> setting OOZIE_CONFIG=/etc/oozie/conf
> setting OOZIE_LOG=/var/log/oozie
> setting JETTY_PID_FILE=/var/run/oozie/oozie.pid
> setting OOZIE_DATA=/var/lib/oozie
> setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
> setting CATALINA_TMPDIR=/var/lib/oozie
> setting CATALINA_PID=/var/run/oozie/oozie.pid
> setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
> setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
> setting OOZIE_CONFIG=/etc/oozie/conf
> setting OOZIE_LOG=/var/log/oozie
> setting JETTY_PID_FILE=/var/run/oozie/oozie.pid
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/usr/lib/oozie/lib-client/slf4j-simple-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/usr/lib/oozie/embedded-oozie-server/webapp/WEB-INF/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
> 8324 [main] INFO org.apache.oozie.service.XLogService -
> *******************************************************************************
> STARTUP MSG: Oozie BUILD_VERSION [5.2.1] compiled by [ubuntu] on [2021.07.12-00:33:17GMT -Dvc.revision=unavailable -Dvc.url=unavailable -DgenerateDocs]
> STARTUP MSG: revision [branch-5.2@8f0e5ee09]@[git@github.com:apache/oozie.git]
> *******************************************************************************
> 8325 [main] INFO oozieops -
> *******************************************************************************
> STARTUP MSG: Oozie BUILD_VERSION [5.2.1] compiled by [ubuntu] on [2021.07.12-00:33:17GMT -Dvc.revision=unavailable -Dvc.url=unavailable -DgenerateDocs]
> STARTUP MSG: revision [branch-5.2@8f0e5ee09]@[git@github.com:apache/oozie.git]
> *******************************************************************************
> 8325 [main] INFO org.apache.oozie.service.XLogService - Log4j configuration file [oozie-log4j.properties]
> 8325 [main] INFO org.apache.oozie.service.XLogService - Log4j configuration file loaded from [/etc/oozie/conf]
> 8326 [main] INFO org.apache.oozie.service.XLogService - Log4j reload interval [10 sec]
> 8333 [main] INFO org.apache.oozie.service.ConfigurationService - Oozie home dir [/usr/lib/oozie]
> 8334 [main] INFO org.apache.oozie.service.ConfigurationService - Oozie conf dir [/etc/oozie/conf]
> 8334 [main] INFO org.apache.oozie.service.ConfigurationService - Oozie conf file [oozie-site.xml]
> 8613 [main] WARN org.apache.oozie.service.ConfigurationService - Invalid configuration defined, [use.system.libpath.for.mapreduce.and.pig.jobs]
> 8625 [main] WARN org.apache.oozie.service.ConfigurationService - Invalid configuration defined, [oozie.authentication.signature.secret]
> 8625 [main] WARN org.apache.oozie.service.ConfigurationService - Invalid configuration defined, [oozie.service.AuthorizationService.security.enabled]
> 8626 [main] WARN org.apache.oozie.util.DateUtils - GMT, UTC or Region/City Timezone formats are preferred instead of UTC
> 8627 [main] WARN org.apache.oozie.service.Services - System ID [oozie-oozi] exceeds maximum length [10], trimming
> 8629 [main] INFO org.apache.oozie.service.Services - Exiting null Entering NORMAL
> 8629 [main] INFO oozieops - Exiting null Entering NORMAL
> 8630 [main] INFO org.apache.oozie.service.Services - Initialized runtime directory [/tmp/oozie-oozi2045196064457886659.dir]
> 8630 [main] INFO org.apache.oozie.service.ConfigurationService - Programmatic configuration change, property[oozie.services]=[org.apache.oozie.service.LiteWorkflowAppService, org.apache.oozie.service.HadoopAccessorService]
> 8630 [main] INFO oozieops - Programmatic configuration change, property[oozie.services]=[org.apache.oozie.service.LiteWorkflowAppService, org.apache.oozie.service.HadoopAccessorService]
> 8631 [main] INFO org.apache.oozie.service.ConfigurationService - Programmatic configuration change, property[oozie.services.ext]=[]
> 8631 [main] INFO oozieops - Programmatic configuration change, property[oozie.services.ext]=[]
> 8636 [main] INFO org.apache.oozie.service.HadoopAccessorService - JOB_TRACKER_WHITELIST :[], Total entries :0
> 8636 [main] INFO org.apache.oozie.service.HadoopAccessorService - NAME_NODE_WHITELIST :[], Total entries :0
> 8636 [main] INFO org.apache.oozie.service.HadoopAccessorService - Oozie Kerberos Authentication [disabled]
> 8872 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> 9041 [main] INFO org.apache.hadoop.security.SecurityUtil - Updating Configuration
> 9041 [main] INFO org.apache.oozie.service.Services - Initialized
> 9044 [main] INFO org.apache.oozie.service.Services - Running with JARs for Hadoop version [3.2.2]
> 9048 [main] INFO org.apache.oozie.service.Services - Oozie System ID [oozie-oozi] started!
> Found Hadoop that supports Erasure Coding. Trying to disable Erasure Coding for path: /user/oozie/share/lib
> Done
> the destination path for sharelib is: /user/oozie/share/lib/lib_20210713230159
> 44357 [main] INFO org.apache.oozie.service.Services - Shutdown
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)