You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2025/11/25 00:16:15 UTC
(spark) branch master updated: [SPARK-54488][BUILD] Fix Connect JVM client leaks `protobuf-java-util` in shading rules
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 21b61660bcd0 [SPARK-54488][BUILD] Fix Connect JVM client leaks `protobuf-java-util` in shading rules
21b61660bcd0 is described below
commit 21b61660bcd00240f4c250f4a64fc544bc4c6117
Author: Cheng Pan <ch...@apache.org>
AuthorDate: Mon Nov 24 16:16:04 2025 -0800
[SPARK-54488][BUILD] Fix Connect JVM client leaks `protobuf-java-util` in shading rules
### What changes were proposed in this pull request?
Fix Connect JVM client leaks `protobuf-java-util` in shading rules by setting `protobuf-java-util` to `compile` scope (was `provided`)
### Why are the changes needed?
A bug fix, `spark-connect-client-jvm` jar should correctly shade all protobuf classes, same as the `spark-connect` server jar does.
### Does this PR introduce _any_ user-facing change?
Yes, users who pull `spark-connect-client-jvm` as dependencies, will not see transitive `protobuf-java-util` dependency.
### How was this patch tested?
```
build/mvn clean install -DskipTests -pl sql/connect/client/jvm -am
```
```patch
[INFO] --- shade:3.6.0:shade (default) spark-connect-client-jvm_2.13 ---
[INFO] Including org.apache.spark:spark-connect-common_2.13:jar:4.2.0-SNAPSHOT in the shaded jar.
[INFO] Including io.grpc:grpc-netty:jar:1.76.0 in the shaded jar.
[INFO] Including io.grpc:grpc-api:jar:1.76.0 in the shaded jar.
[INFO] Including org.codehaus.mojo:animal-sniffer-annotations:jar:1.24 in the shaded jar.
[INFO] Including io.grpc:grpc-core:jar:1.76.0 in the shaded jar.
[INFO] Including com.google.android:annotations:jar:4.1.1.4 in the shaded jar.
[INFO] Including io.grpc:grpc-context:jar:1.76.0 in the shaded jar.
[INFO] Including io.perfmark:perfmark-api:jar:0.27.0 in the shaded jar.
[INFO] Including io.grpc:grpc-util:jar:1.76.0 in the shaded jar.
[INFO] Including io.grpc:grpc-protobuf:jar:1.76.0 in the shaded jar.
[INFO] Including com.google.api.grpc:proto-google-common-protos:jar:2.59.2 in the shaded jar.
[INFO] Including io.grpc:grpc-protobuf-lite:jar:1.76.0 in the shaded jar.
[INFO] Including io.grpc:grpc-services:jar:1.76.0 in the shaded jar.
[INFO] Including io.grpc:grpc-stub:jar:1.76.0 in the shaded jar.
[INFO] Including io.grpc:grpc-inprocess:jar:1.76.0 in the shaded jar.
[INFO] Including io.netty:netty-codec-http2:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-common:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-buffer:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-transport:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-resolver:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-codec-base:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-handler:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-codec-http:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-codec-compression:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-handler-proxy:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-codec-socks:jar:4.2.7.Final in the shaded jar.
[INFO] Including io.netty:netty-transport-native-unix-common:jar:4.2.7.Final in the shaded jar.
[INFO] Including org.apache.spark:spark-sql-api_2.13:jar:4.2.0-SNAPSHOT in the shaded jar.
[INFO] Including org.apache.arrow:arrow-vector:jar:18.3.0 in the shaded jar.
[INFO] Including org.apache.arrow:arrow-format:jar:18.3.0 in the shaded jar.
[INFO] Including org.apache.arrow:arrow-memory-core:jar:18.3.0 in the shaded jar.
[INFO] Including com.google.flatbuffers:flatbuffers-java:jar:25.2.10 in the shaded jar.
[INFO] Including org.apache.arrow:arrow-memory-netty:jar:18.3.0 in the shaded jar.
[INFO] Including org.apache.arrow:arrow-memory-netty-buffer-patch:jar:18.3.0 in the shaded jar.
[INFO] Including com.google.protobuf:protobuf-java:jar:4.33.0 in the shaded jar.
+ [INFO] Including com.google.protobuf:protobuf-java-util:jar:4.33.0 in the shaded jar.
[INFO] Including com.google.code.gson:gson:jar:2.11.0 in the shaded jar.
[INFO] Including com.google.guava:guava:jar:33.4.8-jre in the shaded jar.
[INFO] Including com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava in the shaded jar.
[INFO] Including com.google.guava:failureaccess:jar:1.0.3 in the shaded jar.
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #53191 from pan3793/SPARK-54488.
Authored-by: Cheng Pan <ch...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
sql/connect/client/jvm/pom.xml | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/sql/connect/client/jvm/pom.xml b/sql/connect/client/jvm/pom.xml
index 6c0c0864f083..2767d7fdfcec 100644
--- a/sql/connect/client/jvm/pom.xml
+++ b/sql/connect/client/jvm/pom.xml
@@ -74,6 +74,11 @@
<artifactId>protobuf-java</artifactId>
<scope>compile</scope>
</dependency>
+ <dependency>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java-util</artifactId>
+ <scope>compile</scope>
+ </dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org