[jira] [Comment Edited] (BIGTOP-3348) Fix Hive test failures caused by NoSuchMethodError of ApplicationId.fromString

["Masatake Iwasaki (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BIGTOP-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17104058#comment-17104058 ] 

Masatake Iwasaki edited comment on BIGTOP-3348 at 5/11/20, 4:42 AM:
--------------------------------------------------------------------

{quote}Maybe there's a bug on Tez side for specifying correct Hadoop version. As a result change on Tez side does not work. The PR you're posting is the way to fix it.
{quote}
{{mvn install -Dhadoop.version=2.10.0}} did not changed the version of Hadoop as transitive dependency of Tez. While modifying {{<hadoop.version>2.7.2</hadoop.version>}} in pom.xml of Tez worked, we can not fix already publicly available Tez artifact anyway. The [PR|https://github.com/apache/bigtop/pull/636] is adding explicit dependency on Hadoop to Hive itself and excluding Hadoop from transitive dependency of Tez. Since Hive itself uses YARN API, I think it is the right way to resolve [dependency convergence|http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#transitive-dependencies].


was (Author: iwasakims):
{quote}Maybe there's a bug on Tez side for specifying correct Hadoop version. As a result change on Tez side does not work. The PR you're posting is the way to fix it.
{quote}
{{mvn install -Dhadoop.version=2.10.0}} did not changed the version of Hadoop as transitive dependency of Tez. While modifying {{<hadoop.version>2.7.2</hadoop.version>}} in pom.xml of Tez worked, we can not publicly available Tez artifact anyway. The [PR|https://github.com/apache/bigtop/pull/636] is adding explicit dependency on Hadoop to Hive itself and excluding Hadoop from transitive dependency of Tez. Since Hive itself uses YARN API, I think it is the right way to resolve [dependency convergence|http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#transitive-dependencies].

> Fix Hive test failures caused by NoSuchMethodError of ApplicationId.fromString
> ------------------------------------------------------------------------------
>
>                 Key: BIGTOP-3348
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-3348
>             Project: Bigtop
>          Issue Type: Bug
>            Reporter: Masatake Iwasaki
>            Assignee: Masatake Iwasaki
>            Priority: Major
>         Attachments: hive-deptree.txt
>
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Some tests fails due to change of YARN API.
> {noformat}
> Tests run: 8, Failures: 0, Errors: 8, Skipped: 0, Time elapsed: 0.981 sec <<< FAILURE! - in org.apache.hadoop.hive.llap.daemon.impl.TestTaskExecutorService
> testPreemptionStateOnTaskMoveToNonFinishableState(org.apache.hadoop.hive.llap.daemon.impl.TestTaskExecutorService)  Time elapsed: 0.93 sec  <<< ERROR!
> java.lang.NoSuchMethodError: org.apache.hadoop.yarn.api.records.ApplicationId.fromString(Ljava/lang/String;)Lorg/apache/hadoop/yarn/api/records/ApplicationId;
>         at org.apache.hadoop.yarn.util.ConverterUtils.toApplicationId(ConverterUtils.java:162)
>         at org.apache.hadoop.hive.llap.tez.Converters.createTaskAttemptId(Converters.java:99)
>         at org.apache.hadoop.hive.llap.tez.Converters.getTaskSpecfromProto(Converters.java:53)
>         at org.apache.hadoop.hive.llap.daemon.impl.TaskRunnerCallable.<init>(TaskRunnerCallable.java:147)
>         at org.apache.hadoop.hive.llap.daemon.impl.TaskExecutorTestHelpers$MockRequest.<init>(TaskExecutorTestHelpers.java:192)
>         at org.apache.hadoop.hive.llap.daemon.impl.TaskExecutorTestHelpers.createMockRequest(TaskExecutorTestHelpers.java:76)
>         at org.apache.hadoop.hive.llap.daemon.impl.TaskExecutorTestHelpers.createMockRequest(TaskExecutorTestHelpers.java:57)
>         at org.apache.hadoop.hive.llap.daemon.impl.TestTaskExecutorService.testPreemptionStateOnTaskMoveToNonFinishableState(TestTaskExecutorService.java:150)
>         ...
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)