[GitHub] [spark] sunchao commented on a change in pull request #33989: [SPARK-36676][SQL][BUILD] Create shaded Hive module and upgrade Guava version to 30.1.1-jre

[GitBox <gi...@apache.org>]

sunchao commented on a change in pull request #33989:
URL: https://github.com/apache/spark/pull/33989#discussion_r708669970



##########
File path: dev/deps/spark-deps-hadoop-2.7-hive-2.3
##########
@@ -50,22 +46,20 @@ commons-lang3/3.12.0//commons-lang3-3.12.0.jar
 commons-logging/1.1.3//commons-logging-1.1.3.jar
 commons-math3/3.4.1//commons-math3-3.4.1.jar
 commons-net/3.1//commons-net-3.1.jar
-commons-pool/1.5.4//commons-pool-1.5.4.jar
 commons-text/1.6//commons-text-1.6.jar
 compress-lzf/1.0.3//compress-lzf-1.0.3.jar
 core/1.1.2//core-1.1.2.jar
 curator-client/2.7.1//curator-client-2.7.1.jar
 curator-framework/2.7.1//curator-framework-2.7.1.jar
 curator-recipes/2.7.1//curator-recipes-2.7.1.jar
-datanucleus-api-jdo/4.2.4//datanucleus-api-jdo-4.2.4.jar
 datanucleus-core/4.1.17//datanucleus-core-4.1.17.jar
-datanucleus-rdbms/4.1.19//datanucleus-rdbms-4.1.19.jar
 derby/10.14.2.0//derby-10.14.2.0.jar
-dropwizard-metrics-hadoop-metrics2-reporter/0.1.2//dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar
+error_prone_annotations/2.5.1//error_prone_annotations-2.5.1.jar
+failureaccess/1.0.1//failureaccess-1.0.1.jar
 flatbuffers-java/1.9.0//flatbuffers-java-1.9.0.jar
 generex/1.0.2//generex-1.0.2.jar
 gson/2.2.4//gson-2.2.4.jar
-guava/14.0.1//guava-14.0.1.jar
+guava/30.1.1-jre//guava-30.1.1-jre.jar

Review comment:
       It actually might be possible: seems Spark now shade Guava usage everywhere within itself now. I remember seeing a few modules which are not using shaded version but now can't find them anymore. Perhaps some mistake from my side.
   
   It's a bit strange that Spark packs the shaded Guava classes in its `spark-network-common` jar. This is working since it is dependent by `spark-core` and thus transitively dependent by other modules.
   
   I need to fix the PR since the "inherited" tag will prevent Spark from shading jetty & guava in other modules, which is bad. Thinking about how to do proper plugin inheritance in Maven...
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org