[jira] [Commented] (TEZ-787) Guava conflicts with Hadoop 2.2.0

["Siddharth Seth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888310#comment-13888310 ] 

Siddharth Seth commented on TEZ-787:
------------------------------------

[~cheolsoo], is it possible for you to access the classpath for the task which failed. This will be within the configured yarn local dirs. Ideally, tez should have localized the updated version of guava - and that should show up in the classpath before the one provided by Hadoop.

Is your app (assuming PIG) also pulling in a version of Guava ?

> Guava conflicts with Hadoop 2.2.0
> ---------------------------------
>
>                 Key: TEZ-787
>                 URL: https://issues.apache.org/jira/browse/TEZ-787
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.3.0
>            Reporter: Cheolsoo Park
>
> After upgrading tez to the current HEAD, I found my jobs fail in my Hadoop 2.2 cluster with the following error-
> {code}
> 2014-01-31 20:59:31,123 FATAL [AsyncDispatcher event handler] org.apache.hadoop.yarn.event.AsyncDispatcher: Error in dispatcher thread
> java.lang.NoSuchMethodError: com.google.common.collect.Range.closedOpen(Ljava/lang/Comparable;Ljava/lang/Comparable;)Lcom/google/common/collect/Range;
> 	at org.apache.tez.runtime.api.events.CompositeDataMovementEvent.getEvents(CompositeDataMovementEvent.java:87)
> 	at org.apache.tez.dag.app.dag.impl.Edge.handleCompositeDataMovementEvent(Edge.java:236)
> 	at org.apache.tez.dag.app.dag.impl.Edge.sendTezEventToDestinationTasks(Edge.java:248)
> 	at org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:2022)
> 	at org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:1976)
> 	at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
> 	at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> 	at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> 	at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> 	at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:955)
> 	at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:143)
> 	at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1354)
> 	at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1340)
> 	at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:134)
> 	at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:81)
> 	at java.lang.Thread.run(Thread.java:744)
> {code}
> I believe that TEZ-646 bumped guava version to 0.15, and that causes conflicts with the guava version that Hadoop 2.2 depends on. This is troublesome to me since I am planning to deploy tez on Hadoop 2.2 cluster, so is it possible to avoid this conflict?



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)