You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Yuanhao Lu (Jira)" <ji...@apache.org> on 2020/06/03 20:11:00 UTC
[jira] [Updated] (HIVE-23604) LLAP does not have correct version of
guava after HIVE-22126
[ https://issues.apache.org/jira/browse/HIVE-23604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yuanhao Lu updated HIVE-23604:
------------------------------
Description:
This JIRA https://issues.apache.org/jira/browse/HIVE-22126 shaded guava in hive-exec. There is a issue that LLAP is also using this guava, so after shading, it cannot use hive-exec guava and may turn to `./tez/guava-11.0.2.jar` which will cause following error
{code:java}
Status: Running (Executing on YARN cluster with App id application_1591081923777_0005)
Map 1: -/- Reducer 2: 0/1
Map 1: 0/11 Reducer 2: 0/1
Map 1: 0(+11,-11)/11 Reducer 2: 0/1
Map 1: 0(+0,-32)/11 Reducer 2: 0/1
Status: Failed
Vertex failed, vertexName=Map 1, vertexId=vertex_1591081923777_0005_1_00, diagnostics=[Task failed, taskId=task_1591081923777_0005_1_00_000002, diagnostics=[TaskAttempt 0 failed, info=[org.apache.hadoop.ipc.RemoteException(java.lang.NoSuchMethodError): com.google.common.base.Stopwatch.createUnstarted()Lcom/google/common/base/Stopwatch;
at org.apache.hadoop.hive.llap.daemon.impl.TaskRunnerCallable.<init>(TaskRunnerCallable.java:122)
at org.apache.hadoop.hive.llap.daemon.impl.ContainerRunnerImpl.submitWork(ContainerRunnerImpl.java:274)
at org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.submitWork(LlapDaemon.java:558)
at org.apache.hadoop.hive.llap.daemon.impl.LlapProtocolServerImpl.submitWork(LlapProtocolServerImpl.java:104)
at org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$LlapDaemonProtocol$2.callBlockingMethod(LlapDaemonProtocolProtos.java:19020)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:528)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:999)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:927)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2915)
{code}
This can be solved by adding `com.google.common.base.Stopwatch.class` to this section in hive 3.1.2 release [https://github.com/apache/hive/blob/release-3.1.2-rc0/llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapServiceDriver.java#L385-L409] and probably to this file for master branch https://github.com/apache/hive/blob/2385361ef33ff81760115cfee4de31ec18afe2f3/llap-server/src/java/org/apache/hadoop/hive/llap/cli/service/AsyncTaskCopyLocalJars.java
was:
This JIRA https://issues.apache.org/jira/browse/HIVE-22126 shaded guava in hive-exec. There is a issue that LLAP is also using this guava, so after shading, it cannot use hive-exec guava and may turn to `./tez/guava-11.0.2.jar` which will cause following error
{code:java}
Status: Running (Executing on YARN cluster with App id application_1591081923777_0005)
Map 1: -/- Reducer 2: 0/1
Map 1: 0/11 Reducer 2: 0/1
Map 1: 0(+11,-11)/11 Reducer 2: 0/1
Map 1: 0(+0,-32)/11 Reducer 2: 0/1
Status: Failed
Vertex failed, vertexName=Map 1, vertexId=vertex_1591081923777_0005_1_00, diagnostics=[Task failed, taskId=task_1591081923777_0005_1_00_000002, diagnostics=[TaskAttempt 0 failed, info=[org.apache.hadoop.ipc.RemoteException(java.lang.NoSuchMethodError): com.google.common.base.Stopwatch.createUnstarted()Lcom/google/common/base/Stopwatch;
at org.apache.hadoop.hive.llap.daemon.impl.TaskRunnerCallable.<init>(TaskRunnerCallable.java:122)
at org.apache.hadoop.hive.llap.daemon.impl.ContainerRunnerImpl.submitWork(ContainerRunnerImpl.java:274)
at org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.submitWork(LlapDaemon.java:558)
at org.apache.hadoop.hive.llap.daemon.impl.LlapProtocolServerImpl.submitWork(LlapProtocolServerImpl.java:104)
at org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$LlapDaemonProtocol$2.callBlockingMethod(LlapDaemonProtocolProtos.java:19020)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:528)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:999)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:927)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2915)
{code}
This can be solved by adding `com.google.common.base.Stopwatch.class` to this section [https://github.com/apache/hive/blob/release-3.1.2-rc0/llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapServiceDriver.java#L385-L409]
> LLAP does not have correct version of guava after HIVE-22126
> ------------------------------------------------------------
>
> Key: HIVE-23604
> URL: https://issues.apache.org/jira/browse/HIVE-23604
> Project: Hive
> Issue Type: Bug
> Components: llap
> Reporter: Yuanhao Lu
> Priority: Major
>
> This JIRA https://issues.apache.org/jira/browse/HIVE-22126 shaded guava in hive-exec. There is a issue that LLAP is also using this guava, so after shading, it cannot use hive-exec guava and may turn to `./tez/guava-11.0.2.jar` which will cause following error
> {code:java}
> Status: Running (Executing on YARN cluster with App id application_1591081923777_0005)
>
> Map 1: -/- Reducer 2: 0/1
> Map 1: 0/11 Reducer 2: 0/1
> Map 1: 0(+11,-11)/11 Reducer 2: 0/1
> Map 1: 0(+0,-32)/11 Reducer 2: 0/1
> Status: Failed
> Vertex failed, vertexName=Map 1, vertexId=vertex_1591081923777_0005_1_00, diagnostics=[Task failed, taskId=task_1591081923777_0005_1_00_000002, diagnostics=[TaskAttempt 0 failed, info=[org.apache.hadoop.ipc.RemoteException(java.lang.NoSuchMethodError): com.google.common.base.Stopwatch.createUnstarted()Lcom/google/common/base/Stopwatch;
> at org.apache.hadoop.hive.llap.daemon.impl.TaskRunnerCallable.<init>(TaskRunnerCallable.java:122)
> at org.apache.hadoop.hive.llap.daemon.impl.ContainerRunnerImpl.submitWork(ContainerRunnerImpl.java:274)
> at org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.submitWork(LlapDaemon.java:558)
> at org.apache.hadoop.hive.llap.daemon.impl.LlapProtocolServerImpl.submitWork(LlapProtocolServerImpl.java:104)
> at org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$LlapDaemonProtocol$2.callBlockingMethod(LlapDaemonProtocolProtos.java:19020)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:528)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:999)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:927)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2915)
> {code}
> This can be solved by adding `com.google.common.base.Stopwatch.class` to this section in hive 3.1.2 release [https://github.com/apache/hive/blob/release-3.1.2-rc0/llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapServiceDriver.java#L385-L409] and probably to this file for master branch https://github.com/apache/hive/blob/2385361ef33ff81760115cfee4de31ec18afe2f3/llap-server/src/java/org/apache/hadoop/hive/llap/cli/service/AsyncTaskCopyLocalJars.java
--
This message was sent by Atlassian Jira
(v8.3.4#803005)