[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

[GitBox <gi...@apache.org>]

viirya commented on pull request #29326:
URL: https://github.com/apache/spark/pull/29326#issuecomment-667801138


   It is a trouble that hive-exec uses a method that became package-private since Guava version 20. So there is incompatibility with Guava versions > 19.0.
   
   ```
   sbt.ForkMain$ForkError: sbt.ForkMain$ForkError: java.lang.IllegalAccessError: tried to access method com.google.common.collect.Iterators.emptyIterator()Lcom/google/common/collect/UnmodifiableIterator; from class org.apache.hadoop.hive.ql.exec.FetchOperator
   	at org.apache.hadoop.hive.ql.exec.FetchOperator.<init>(FetchOperator.java:108)
   	at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:87)
   	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:541)
   	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
   	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
   	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
   	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
   ```
   
   hive-exec doesn't shade Guava until https://issues.apache.org/jira/browse/HIVE-22126 that targets 4.0.0.
   
   This seems a dead end for upgrading Guava in Spark for now.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org