You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@bigtop.apache.org by "William (Jira)" <ji...@apache.org> on 2022/09/13 15:09:00 UTC

[jira] [Updated] (BIGTOP-3813) Incompatible guava & commons-lang3 Versions in Oozie Sharelib

     [ https://issues.apache.org/jira/browse/BIGTOP-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

William updated BIGTOP-3813:
----------------------------
    Attachment:     (was: Oozie-Sharelib-Dependency-Update.patch)

> Incompatible guava & commons-lang3 Versions in Oozie Sharelib
> -------------------------------------------------------------
>
>                 Key: BIGTOP-3813
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-3813
>             Project: Bigtop
>          Issue Type: Bug
>          Components: hadoop
>    Affects Versions: 3.0.1
>            Reporter: William
>            Priority: Major
>
> Within the Oozie do-component-build script, the version of dependent Hadoop components (Hadoop, Spark, Hive, etc) are updated to the target Bigtop versions. These target versions are included in the Oozie sharelib. However, the build does not update the version of dependencies for these components, namely Spark in the case we ran into. We found that when Spark was running in yarn-cluster mode, and the oozie sharelib was uploaded to HDFS, any Oozie workflows with Spark tasks would fail due to guava and commons-lang3 versions included in the oozie sharelib that were still targetting the old spark version.
> Updating these dependencies in the oozie sharelib resolved the workflow failures:
> guava: 14.0.1 -> 27.0-jre
> commons-lang3: 3.3.9 -> 3.9



--
This message was sent by Atlassian Jira
(v8.20.10#820010)