You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bigtop.apache.org by "Kengo Seki (Jira)" <ji...@apache.org> on 2020/11/03 07:11:00 UTC
[jira] [Updated] (BIGTOP-3334) Fix Oozie build after upgrading
Spark to 2.4.5
[ https://issues.apache.org/jira/browse/BIGTOP-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kengo Seki updated BIGTOP-3334:
-------------------------------
Fix Version/s: 1.5.0
> Fix Oozie build after upgrading Spark to 2.4.5
> ----------------------------------------------
>
> Key: BIGTOP-3334
> URL: https://issues.apache.org/jira/browse/BIGTOP-3334
> Project: Bigtop
> Issue Type: Bug
> Components: build
> Reporter: Kengo Seki
> Assignee: Kengo Seki
> Priority: Major
> Fix For: 1.5.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> After BIGTOP-3165, building Oozie came to fail as follows, because [Scala 2.10 support was removed from Spark 2.3|https://spark.apache.org/docs/2.4.5/#downloading].
> {code}
> $ ./gradlew oozie-pkg
> (snip)
> [INFO] ------------------< org.apache.oozie:oozie-examples >-------------------
> [INFO] Building Apache Oozie Examples 4.3.0 [15/23]
> [INFO] --------------------------------[ jar ]---------------------------------
> Downloading from central: https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.10/2.4.5/spark-core_2.10-2.4.5.pom
> Downloading from apache.snapshots.repo: https://repository.apache.org/content/groups/snapshots/org/apache/spark/spark-core_2.10/2.4.5/spark-core_2.10-2.4.5.pom
> Downloading from redhat: https://maven.repository.redhat.com/ga/org/apache/spark/spark-core_2.10/2.4.5/spark-core_2.10-2.4.5.pom
> [WARNING] The POM for org.apache.spark:spark-core_2.10:jar:2.4.5 is missing, no dependency information available
> Downloading from central: https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.10/2.4.5/spark-core_2.10-2.4.5.jar
> Downloading from apache.snapshots.repo: https://repository.apache.org/content/groups/snapshots/org/apache/spark/spark-core_2.10/2.4.5/spark-core_2.10-2.4.5.jar
> Downloading from redhat: https://maven.repository.redhat.com/ga/org/apache/spark/spark-core_2.10/2.4.5/spark-core_2.10-2.4.5.jar
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Oozie Main 4.3.0 ............................ SUCCESS [ 1.526 s]
> [INFO] Apache Oozie Hadoop Utils hadoop-2-4.3.0 hadoop-2-4.3.0 SUCCESS [ 2.526 s]
> [INFO] Apache Oozie Hadoop Distcp hadoop-2-4.3.0 hadoop-2-4.3.0 SUCCESS [ 0.121 s]
> [INFO] Apache Oozie Hadoop Auth hadoop-2-4.3.0 Test hadoop-2-4.3.0 SUCCESS [ 0.275 s]
> [INFO] Apache Oozie Hadoop Libs 4.3.0 ..................... SUCCESS [ 0.020 s]
> [INFO] Apache Oozie Client 4.3.0 .......................... SUCCESS [ 9.910 s]
> [INFO] Apache Oozie Share Lib Oozie 4.3.0 ................. SUCCESS [ 2.319 s]
> [INFO] Apache Oozie Share Lib HCatalog 4.3.0 .............. SUCCESS [ 7.790 s]
> [INFO] Apache Oozie Share Lib Distcp 4.3.0 ................ SUCCESS [ 0.645 s]
> [INFO] Apache Oozie Core 4.3.0 ............................ SUCCESS [ 39.494 s]
> [INFO] Apache Oozie Share Lib Streaming 4.3.0 ............. SUCCESS [ 4.785 s]
> [INFO] Apache Oozie Share Lib Hive 4.3.0 .................. SUCCESS [ 8.152 s]
> [INFO] Apache Oozie Share Lib Hive 2 4.3.0 ................ SUCCESS [ 6.853 s]
> [INFO] Apache Oozie Share Lib Sqoop 4.3.0 ................. SUCCESS [ 3.060 s]
> [INFO] Apache Oozie Examples 4.3.0 ........................ FAILURE [ 5.389 s]
> [INFO] Apache Oozie Share Lib Spark 4.3.0 ................. SKIPPED
> [INFO] Apache Oozie Share Lib 4.3.0 ....................... SKIPPED
> [INFO] Apache Oozie Docs 4.3.0 ............................ SKIPPED
> [INFO] Apache Oozie WebApp 4.3.0 .......................... SKIPPED
> [INFO] Apache Oozie Tools 4.3.0 ........................... SKIPPED
> [INFO] Apache Oozie MiniOozie 4.3.0 ....................... SKIPPED
> [INFO] Apache Oozie Distro 4.3.0 .......................... SKIPPED
> [INFO] Apache Oozie ZooKeeper Security Tests 4.3.0 ........ SKIPPED
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 01:33 min
> [INFO] Finished at: 2020-04-03T11:48:09+09:00
> [INFO] ------------------------------------------------------------------------
> [ERROR] Failed to execute goal on project oozie-examples: Could not resolve dependencies for project org.apache.oozie:oozie-examples:jar:4.3.0: Could not find artifact org.apache.spark:spark-core_2.10:jar:2.4.5 in central (https://repo1.maven.org/maven2) -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please read the following articles:
> [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the command
> [ERROR] mvn <args> -rf :oozie-examples
> ERROR, Oozie distro creation failed
> + MKDISTRO_RESULT=255
> + '[' 255 '!=' 0 ']'
> + echo 'ERROR: mkdistro.sh failed with error 255'
> ERROR: mkdistro.sh failed with error 255
> + echo 'Build files from failed build: /home/sekikn/repos/bigtop/output/oozie/oozie-4.3.0'
> Build files from failed build: /home/sekikn/repos/bigtop/output/oozie/oozie-4.3.0
> + exit 255
> debian/rules:33: recipe for target 'override_dh_auto_build' failed
> make[1]: *** [override_dh_auto_build] Error 255
> make[1]: Leaving directory '/home/sekikn/repos/bigtop/output/oozie/oozie-4.3.0'
> debian/rules:27: recipe for target 'build' failed
> make: *** [build] Error 2
> dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
> debuild: fatal error at line 1152:
> dpkg-buildpackage -rfakeroot -us -uc -ui -b
> failed
> > Task :oozie-deb FAILED
> FAILURE: Build failed with an exception.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)