[jira] [Created] (SPARK-54049) spark-network-common no longer shades all of Guava

["Yosef Fertel (Jira)" <ji...@apache.org>]

Yosef Fertel created SPARK-54049:
------------------------------------

             Summary: spark-network-common no longer shades all of Guava
                 Key: SPARK-54049
                 URL: https://issues.apache.org/jira/browse/SPARK-54049
             Project: Spark
          Issue Type: Bug
          Components: Build
    Affects Versions: 4.0.0
            Reporter: Yosef Fertel


Shading of Guava in `spark-network-common` has changed from 3.x to 4.x such that conflicting classes prevent downstream code from using Guava. 

In spark jars prior to 4.x, `com/google/thirdparty/publicsuffix/PublicSuffixPatterns.class` was not a class included in the jar. Post 4.x, Guava is still shaded to `sparkproject` but `com/google/thirdparty/publicsuffix/PublicSuffixPatterns.class`, used in Guava's `InternetDomainName` and elsewhere, is now present in the jar. 
{code:java}
$ jar tf spark-network-common_2.13-4.0.0.jar | rg PublicSuffixPatterns 
com/google/thirdparty/publicsuffix/PublicSuffixPatterns.class {code}
If a library depends on spark but also depends on Guava, calls to `InternetDomainName` that call `PublicSuffixPatterns` will fail with `Exception in thread "main" java.lang.NoSuchFieldError: EXACT`. 

Inspecting the code locations in such a library via `classOf[InternetDomainName].getProtectionDomain.getCodeSource.getLocation` and `classOf[PublicSuffixPatterns].getProtectionDomain.getCodeSource.getLocation` reveals that `InternetDomainName` is sourced from Guava, `target/bg-jobs/sbt_2062a9c3/target/5fcb43b5/1685140132000/guava-32.0.0-jre.jar`, while `PublicSuffixPatterns` is sourced instead from spark jar, `target/bg-jobs/sbt_2062a9c3/target/db746978/1747651686000/spark-network-common_2.13-4.0.0.jar`.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org