You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2026/03/30 07:47:12 UTC
(spark-kubernetes-operator) branch main updated: [SPARK-56285] Add `minimize` to `shadowJar` task
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new 204da24 [SPARK-56285] Add `minimize` to `shadowJar` task
204da24 is described below
commit 204da243ebf4d55449fa7b8dd8bda68d30efd0d2
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Mon Mar 30 00:47:01 2026 -0700
[SPARK-56285] Add `minimize` to `shadowJar` task
### What changes were proposed in this pull request?
This PR adds `minimize` to the `shadowJar` task in `spark-operator` to reduce the fat jar size by **~45% (136MB to 75MB)** by removing unused classes.
```diff
$ ls -al spark-operator/build/libs/spark-kubernetes-operator-0.9.0-SNAPSHOT-all.jar
--rw-r--r-- 1 dongjoon staff 136446076 Mar 30 00:03 spark-operator/build/libs/spark-kubernetes-operator-0.9.0-SNAPSHOT-all.jar
+-rw-r--r-- 1 dongjoon staff 75410071 Mar 30 00:17 spark-operator/build/libs/spark-kubernetes-operator-0.9.0-SNAPSHOT-all.jar
```
The following dependencies are excluded from minimization to preserve reflection-based and runtime-loaded classes:
- `com.fasterxml.jackson` (Jackson serialization/deserialization)
- `io.dropwizard.metrics` (metrics system)
- `io.fabric8` (Kubernetes client model binding)
- `io.javaoperatorsdk` (operator framework lifecycle)
- `io.vertx` (HTTP client implementation)
- `org.apache.logging.log4j` (logging plugin discovery)
- `org.slf4j` (logging facade)
### Why are the changes needed?
To reduce the size of the operator fat jar by removing unused classes from dependencies.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the existing CIs.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (claude-opus-4-6)
Closes #603 from dongjoon-hyun/SPARK-56285.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
spark-operator/build.gradle | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/spark-operator/build.gradle b/spark-operator/build.gradle
index f7b4672..db2e3f1 100644
--- a/spark-operator/build.gradle
+++ b/spark-operator/build.gradle
@@ -116,6 +116,15 @@ shadowJar {
// Netty io_uring native libraries - not needed for K8s API server communication
exclude("META-INF/native/libnetty_io_uring*")
exclude("META-INF/native/netty_io_uring*")
+ minimize {
+ exclude(dependency('com.fasterxml.jackson:.*'))
+ exclude(dependency('io.dropwizard.metrics:.*'))
+ exclude(dependency('io.fabric8:.*'))
+ exclude(dependency('io.javaoperatorsdk:.*'))
+ exclude(dependency('io.vertx:.*'))
+ exclude(dependency('org.apache.logging.log4j:.*'))
+ exclude(dependency('org.slf4j:.*'))
+ }
manifest {
attributes(
'Implementation-Title': project.description,
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org