You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Artem Ervits (JIRA)" <ji...@apache.org> on 2017/05/16 18:13:04 UTC
[jira] [Comment Edited] (OOZIE-2886) Ensure consistent versioning
of hadoop jars in server and webapp
[ https://issues.apache.org/jira/browse/OOZIE-2886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16012835#comment-16012835 ]
Artem Ervits edited comment on OOZIE-2886 at 5/16/17 6:12 PM:
--------------------------------------------------------------
[~rkanter] the issue is not with Hive sharelibs. I explicitly pull the version specified by {code}hadoop.version{code} and it fixes the extraneous hadoop jars. My patch fixes mix-up.
{code}
mvn clean package -Dhadoop.version=2.7.0 -DskipTests=true
{code}
{noformat}
find . -name hadoop*.jar | grep 2.7.0
./server/target/dependency/hadoop-annotations-2.7.0.jar
./server/target/dependency/hadoop-auth-2.7.0.jar
./server/target/dependency/hadoop-common-2.7.0.jar
./server/target/dependency/hadoop-yarn-api-2.7.0.jar
./server/target/dependency/hadoop-yarn-common-2.7.0.jar
./server/target/dependency/hadoop-yarn-server-applicationhistoryservice-2.7.0.jar
./server/target/dependency/hadoop-yarn-server-common-2.7.0.jar
./server/target/dependency/hadoop-yarn-server-resourcemanager-2.7.0.jar
./server/target/dependency/hadoop-yarn-server-web-proxy-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-annotations-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-api-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-common-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-server-applicationhistoryservice-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-server-common-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-server-resourcemanager-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-server-web-proxy-2.7.0.jar
{noformat}
now wondering, should I remove the exclusion declaration from each sharelib for the {code}hadoop-yarn-api{code}, {code}hadoop-yarn-common{code}, etc.?
was (Author: dbist13):
[~rkanter] the issue is not with Hive sharelibs. I explicitly pull the version specified by {code}hadoop.version{code} and it fixes the extraneous hadoop jars. My patch fixes mix-up.
{code}
mvn clean package -Dhadoop.version=2.7.0 -DskipTests=true
{code}
{noformat}
find . -name hadoop*.jar | grep 2.7.0
./server/target/dependency/hadoop-annotations-2.7.0.jar
./server/target/dependency/hadoop-auth-2.7.0.jar
./server/target/dependency/hadoop-common-2.7.0.jar
./server/target/dependency/hadoop-yarn-api-2.7.0.jar
./server/target/dependency/hadoop-yarn-common-2.7.0.jar
./server/target/dependency/hadoop-yarn-server-applicationhistoryservice-2.7.0.jar
./server/target/dependency/hadoop-yarn-server-common-2.7.0.jar
./server/target/dependency/hadoop-yarn-server-resourcemanager-2.7.0.jar
./server/target/dependency/hadoop-yarn-server-web-proxy-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-annotations-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-api-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-common-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-server-applicationhistoryservice-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-server-common-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-server-resourcemanager-2.7.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-server-web-proxy-2.7.0.jar
{noformat}
> Ensure consistent versioning of hadoop jars in server and webapp
> ----------------------------------------------------------------
>
> Key: OOZIE-2886
> URL: https://issues.apache.org/jira/browse/OOZIE-2886
> Project: Oozie
> Issue Type: Bug
> Components: build
> Affects Versions: 5.0.0
> Reporter: Robert Kanter
> Assignee: Artem Ervits
> Attachments: OOZIE-2886-0.patch
>
>
> The hive and hive2 sharelibs are pulling in Hadoop jars - they shouldn't be.
> Also, we should ensure that if you specify {{-Dhadoop.version=foo}} (e.g. 2.7.0), that all Hadoop jars end up as that version.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)