You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by ch...@apache.org on 2026/08/06 12:51:41 UTC
(kyuubi) branch branch-1.11 updated: [KYUUBI #7608][AUTHZ] Include commons-collections into authz
This is an automated email from the ASF dual-hosted git repository.
pan3793 pushed a commit to branch branch-1.11
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/branch-1.11 by this push:
new f0450572b7 [KYUUBI #7608][AUTHZ] Include commons-collections into authz
f0450572b7 is described below
commit f0450572b77e0732aa7c8f78b70e25215104080d
Author: lifumao <li...@tencent.com>
AuthorDate: Thu Aug 6 20:51:01 2026 +0800
[KYUUBI #7608][AUTHZ] Include commons-collections into authz
### Why are the changes needed?
Fix https://github.com/apache/kyuubi/issues/7608. kyuubi-spark-authz fails during plugin initialization on Spark 4.1+
```
java.lang.NoClassDefFoundError: org/apache/commons/collections/CollectionUtils
at org.apache.ranger.authorization.hadoop.config.RangerPluginConfig.setSuperUsersGroups(RangerPluginConfig.java:239)
at org.apache.ranger.plugin.service.RangerBasePlugin.setSuperUsersAndGroups(RangerBasePlugin.java:263)
at org.apache.ranger.plugin.service.RangerBasePlugin.<init>(RangerBasePlugin.java:112)
at org.apache.kyuubi.plugin.spark.authz.ranger.SparkRangerAdminPlugin$.<init>(SparkRangerAdminPlugin.scala:30)
at org.apache.kyuubi.plugin.spark.authz.ranger.RangerSparkExtension.<init>(RangerSparkExtension.scala:44)
```
Ranger's `RangerPluginConfig` uses `org.apache.commons.collections.CollectionUtils` (commons-collections 3.x) but declares that dependency with provided scope. Spark 3.x ships commons-collections-3.2.2.jar in $SPARK_HOME/jars/, but Spark 4.1+ only ships commons-collections4, and Hadoop 3.4.2+ removed the 3.x transitive dependency via [HADOOP-15760](https://issues.apache.org/jira/browse/HADOOP-15760).
### How was this patch tested?
```
$ build/mvn dependency:tree -pl :kyuubi-spark-authz_2.13 -am -Pspark-4.1 -Pscala-2.13 -Dincludes=commons-collections:commons-collections
[INFO] org.apache.kyuubi:kyuubi-spark-authz_2.13:jar:1.13.0-SNAPSHOT
[INFO] \- commons-collections:commons-collections:jar:3.2.2:compile
$ build/mvn clean package -DskipTests -pl :kyuubi-spark-authz-shaded_2.13 -am -Pspark-4.1 -Pscala-2.13
$ jar tf extensions/spark/kyuubi-spark-authz-shaded/target/kyuubi-spark-authz-shaded_2.13-1.13.0-SNAPSHOT.jar | grep org/apache/commons/collections/CollectionUtils
org/apache/kyuubi/shade/org/apache/commons/collections/CollectionUtils.class
```
### Was this patch authored or co-authored using generative AI tooling?
Assisted-by: Claude:claude-opus-4-7
Closes #7610 from maomaodev/kyuubi-7608.
Closes #7608
476db54a4 [lifumao] relocation commons-collections
c85165428 [lifumao] [KYUUBI-7608][AUTHZ] Include commons-collections into authz
Authored-by: lifumao <li...@tencent.com>
Signed-off-by: Cheng Pan <ch...@apache.org>
(cherry picked from commit 0e77fd4a5fcd2cd39af00e74fda52de53cf4e6e0)
Signed-off-by: Cheng Pan <ch...@apache.org>
---
extensions/spark/kyuubi-spark-authz-shaded/pom.xml | 5 +++++
extensions/spark/kyuubi-spark-authz/pom.xml | 1 -
2 files changed, 5 insertions(+), 1 deletion(-)
diff --git a/extensions/spark/kyuubi-spark-authz-shaded/pom.xml b/extensions/spark/kyuubi-spark-authz-shaded/pom.xml
index e6d7bf24a3..e86ee5cc21 100644
--- a/extensions/spark/kyuubi-spark-authz-shaded/pom.xml
+++ b/extensions/spark/kyuubi-spark-authz-shaded/pom.xml
@@ -56,6 +56,7 @@
<include>com.fasterxml.jackson.jaxrs:*</include>
<include>com.sun.jersey:*</include>
<include>javax.ws.rs:jsr311-api</include>
+ <include>commons-collections:commons-collections</include>
</includes>
</artifactSet>
<filters>
@@ -105,6 +106,10 @@
<pattern>com.kstruct.gethostname4j</pattern>
<shadedPattern>${kyuubi.shade.packageName}.com.kstruct.gethostname4j</shadedPattern>
</relocation>
+ <relocation>
+ <pattern>org.apache.commons.collections</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.org.apache.commons.collections</shadedPattern>
+ </relocation>
</relocations>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"></transformer>
diff --git a/extensions/spark/kyuubi-spark-authz/pom.xml b/extensions/spark/kyuubi-spark-authz/pom.xml
index 02277cce34..ccb1d22d5b 100644
--- a/extensions/spark/kyuubi-spark-authz/pom.xml
+++ b/extensions/spark/kyuubi-spark-authz/pom.xml
@@ -283,7 +283,6 @@
<dependency>
<groupId>commons-collections</groupId>
<artifactId>commons-collections</artifactId>
- <scope>provided</scope>
</dependency>
<dependency>