You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2024/07/23 00:42:41 UTC
(spark-kubernetes-operator) branch main updated: [SPARK-48971] Ban `org.apache.commons.collections` in favor of `collections4`
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new f3d31d2 [SPARK-48971] Ban `org.apache.commons.collections` in favor of `collections4`
f3d31d2 is described below
commit f3d31d2ab0b9930bb783cb877362a2874de4ebe9
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Mon Jul 22 17:42:35 2024 -0700
[SPARK-48971] Ban `org.apache.commons.collections` in favor of `collections4`
### What changes were proposed in this pull request?
This PR aims to ban `org.apache.commons.collections` in favor of `org.apache.commons.collections4`.
### Why are the changes needed?
Since Apache Spark 3.3.0 (SPARK-37968), Apache Spark uses `collections4` while keeping `org.apache.commons.collections` for Hadoop 3 compatibility (SPARK-38696).
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs and manual review.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #22 from dongjoon-hyun/SPARK-48971.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
config/checkstyle/checkstyle.xml | 1 +
spark-submission-worker/build.gradle | 4 +++-
.../java/org/apache/spark/k8s/operator/SparkAppSubmissionWorker.java | 2 +-
3 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/config/checkstyle/checkstyle.xml b/config/checkstyle/checkstyle.xml
index 90161fe..ee4f9bd 100644
--- a/config/checkstyle/checkstyle.xml
+++ b/config/checkstyle/checkstyle.xml
@@ -203,6 +203,7 @@
<module name="IllegalImport">
<property name="illegalPkgs" value="org.apache.log4j"/>
<property name="illegalPkgs" value="org.apache.commons.lang"/>
+ <property name="illegalPkgs" value="org.apache.commons.collections"/>
</module>
</module>
</module>
diff --git a/spark-submission-worker/build.gradle b/spark-submission-worker/build.gradle
index da78026..5cef16a 100644
--- a/spark-submission-worker/build.gradle
+++ b/spark-submission-worker/build.gradle
@@ -1,7 +1,9 @@
dependencies {
implementation project(":spark-operator-api")
- implementation("org.apache.spark:spark-kubernetes_$scalaVersion:$sparkVersion")
+ implementation("org.apache.spark:spark-kubernetes_$scalaVersion:$sparkVersion") {
+ exclude group: 'commons-collections', module: 'commons-collections'
+ }
compileOnly("org.projectlombok:lombok:$lombokVersion")
annotationProcessor("org.projectlombok:lombok:$lombokVersion")
diff --git a/spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppSubmissionWorker.java b/spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppSubmissionWorker.java
index 57013cb..31d243b 100644
--- a/spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppSubmissionWorker.java
+++ b/spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppSubmissionWorker.java
@@ -26,7 +26,7 @@ import scala.Option;
import io.fabric8.kubernetes.client.KubernetesClient;
import org.apache.commons.codec.digest.DigestUtils;
-import org.apache.commons.collections.MapUtils;
+import org.apache.commons.collections4.MapUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.spark.SparkConf;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org