You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@submarine.apache.org by "Gustavo Martin (Jira)" <ji...@apache.org> on 2020/06/20 12:33:00 UTC
[jira] [Updated] (SUBMARINE-537) submarine-spark-security: missing
JacksonJsonProvider in uber jar
[ https://issues.apache.org/jira/browse/SUBMARINE-537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gustavo Martin updated SUBMARINE-537:
-------------------------------------
Description:
Missing org.codehaus.jackson.jaxrs.JacksonJsonProvider in uber jar which is required by RangerRESTClient in versions 1.1, 1.2 and 2.0
submarine-spark-security throws the following exception:
{noformat}
java.lang.NoClassDefFoundError: submarine_spark_ranger_project/org/codehaus/jackson/jaxrs/JacksonJsonProvider
at org.apache.ranger.plugin.util.RangerRESTClient.buildClient(RangerRESTClient.java:209)
at org.apache.ranger.plugin.util.RangerRESTClient.getClient(RangerRESTClient.java:176)
at org.apache.ranger.plugin.util.RangerRESTClient.getResource(RangerRESTClient.java:156)
at org.apache.ranger.admin.client.RangerAdminRESTClient.createWebResource(RangerAdminRESTClient.java:635)
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:142)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:251)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:191)
at org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:137)
at org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:229)
at org.apache.submarine.spark.security.RangerSparkPlugin$.init(RangerSparkPlugin.scala:43)
at org.apache.submarine.spark.security.RangerSparkPlugin$.<init>(RangerSparkPlugin.scala:64)
at org.apache.submarine.spark.security.RangerSparkPlugin$.<clinit>(RangerSparkPlugin.scala)
at org.apache.submarine.spark.security.RangerSparkAuthorizer$.org$apache$submarine$spark$security$RangerSparkAuthorizer$$getSparkResource(RangerSparkAuthorizer.scala:257)
at org.apache.submarine.spark.security.RangerSparkAuthorizer$$anonfun$addAccessRequest$1$1.apply(RangerSparkAuthorizer.scala:75)
at org.apache.submarine.spark.security.RangerSparkAuthorizer$$anonfun$addAccessRequest$1$1.apply(RangerSparkAuthorizer.scala:74)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at org.apache.submarine.spark.security.RangerSparkAuthorizer$.addAccessRequest$1(RangerSparkAuthorizer.scala:74)
at org.apache.submarine.spark.security.RangerSparkAuthorizer$.checkPrivileges(RangerSparkAuthorizer.scala:98)
at org.apache.spark.sql.catalyst.optimizer.SubmarineSparkRangerAuthorizationExtension.apply(SubmarineSparkRangerAuthorizationExtension.scala:64)
at org.apache.spark.sql.catalyst.optimizer.SubmarineSparkRangerAuthorizationExtension.apply(SubmarineSparkRangerAuthorizationExtension.scala:39)
at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1$$anonfun$apply$1.apply(RuleExecutor.scala:87)
at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1$$anonfun$apply$1.apply(RuleExecutor.scala:84)
at scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:124)
at scala.collection.immutable.List.foldLeft(List.scala:84)
at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1.apply(RuleExecutor.scala:84)
at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1.apply(RuleExecutor.scala:76)
at scala.collection.immutable.List.foreach(List.scala:392)
at org.apache.spark.sql.catalyst.rules.RuleExecutor.execute(RuleExecutor.scala:76)
at org.apache.spark.sql.execution.QueryExecution.optimizedPlan$lzycompute(QueryExecution.scala:67)
at org.apache.spark.sql.execution.QueryExecution.optimizedPlan(QueryExecution.scala:67)
at org.apache.spark.sql.execution.QueryExecution.sparkPlan$lzycompute(QueryExecution.scala:73)
at org.apache.spark.sql.execution.QueryExecution.sparkPlan(QueryExecution.scala:69)
at org.apache.spark.sql.execution.QueryExecution.executedPlan$lzycompute(QueryExecution.scala:78)
at org.apache.spark.sql.execution.QueryExecution.executedPlan(QueryExecution.scala:78)
at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3365)
at org.apache.spark.sql.Dataset.<init>(Dataset.scala:194)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:79)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:642)
at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:694)
Caused by: java.lang.ClassNotFoundException: submarine_spark_ranger_project.org.codehaus.jackson.jaxrs.JacksonJsonProvider
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
{noformat}
RangerRESTClient is using JacksonJsonProvider in versions 1.1, 1.2 and 2.0:
* ranger-1.1: https://github.com/apache/ranger/blob/ranger-1.1/agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
* ranger-1.2: https://github.com/apache/ranger/blob/ranger-1.2/agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
* ranger-2.0: https://github.com/apache/ranger/blob/ranger-2.0/agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
All of them in the same version, 1.9.13:
* ranger-1.1: https://github.com/apache/ranger/blob/ranger-1.1/pom.xml#L93
* ranger-1.2: https://github.com/apache/ranger/blob/ranger-1.2/pom.xml#L93
* ranger-2.0: https://github.com/apache/ranger/blob/ranger-2.0/pom.xml#L93
This version is the same as the one used by submarine:
* https://github.com/apache/submarine/blob/master/pom.xml#L119
was:
Missing org.codehaus.jackson.jaxrs.JacksonJsonProvider in uber jar which is required by RangerRESTClient in versions 1.1, 1.2 and 2.0
submarine-spark-security throws the following exception:
{noformat}
java.lang.NoClassDefFoundError: submarine_spark_ranger_project/org/codehaus/jackson/jaxrs/JacksonJsonProvider
at org.apache.ranger.plugin.util.RangerRESTClient.buildClient(RangerRESTClient.java:209)
at org.apache.ranger.plugin.util.RangerRESTClient.getClient(RangerRESTClient.java:176)
at org.apache.ranger.plugin.util.RangerRESTClient.getResource(RangerRESTClient.java:156)
at org.apache.ranger.admin.client.RangerAdminRESTClient.createWebResource(RangerAdminRESTClient.java:635)
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:142)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:251)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:191)
at org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:137)
at org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:229)
at org.apache.submarine.spark.security.RangerSparkPlugin$.init(RangerSparkPlugin.scala:43)
at org.apache.submarine.spark.security.RangerSparkPlugin$.<init>(RangerSparkPlugin.scala:64)
at org.apache.submarine.spark.security.RangerSparkPlugin$.<clinit>(RangerSparkPlugin.scala)
at org.apache.submarine.spark.security.RangerSparkAuthorizer$.org$apache$submarine$spark$security$RangerSparkAuthorizer$$getSparkResource(RangerSparkAuthorizer.scala:257)
at org.apache.submarine.spark.security.RangerSparkAuthorizer$$anonfun$addAccessRequest$1$1.apply(RangerSparkAuthorizer.scala:75)
at org.apache.submarine.spark.security.RangerSparkAuthorizer$$anonfun$addAccessRequest$1$1.apply(RangerSparkAuthorizer.scala:74)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at org.apache.submarine.spark.security.RangerSparkAuthorizer$.addAccessRequest$1(RangerSparkAuthorizer.scala:74)
at org.apache.submarine.spark.security.RangerSparkAuthorizer$.checkPrivileges(RangerSparkAuthorizer.scala:98)
at org.apache.spark.sql.catalyst.optimizer.SubmarineSparkRangerAuthorizationExtension.apply(SubmarineSparkRangerAuthorizationExtension.scala:64)
at org.apache.spark.sql.catalyst.optimizer.SubmarineSparkRangerAuthorizationExtension.apply(SubmarineSparkRangerAuthorizationExtension.scala:39)
at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1$$anonfun$apply$1.apply(RuleExecutor.scala:87)
at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1$$anonfun$apply$1.apply(RuleExecutor.scala:84)
at scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:124)
at scala.collection.immutable.List.foldLeft(List.scala:84)
at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1.apply(RuleExecutor.scala:84)
at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1.apply(RuleExecutor.scala:76)
at scala.collection.immutable.List.foreach(List.scala:392)
at org.apache.spark.sql.catalyst.rules.RuleExecutor.execute(RuleExecutor.scala:76)
at org.apache.spark.sql.execution.QueryExecution.optimizedPlan$lzycompute(QueryExecution.scala:67)
at org.apache.spark.sql.execution.QueryExecution.optimizedPlan(QueryExecution.scala:67)
at org.apache.spark.sql.execution.QueryExecution.sparkPlan$lzycompute(QueryExecution.scala:73)
at org.apache.spark.sql.execution.QueryExecution.sparkPlan(QueryExecution.scala:69)
at org.apache.spark.sql.execution.QueryExecution.executedPlan$lzycompute(QueryExecution.scala:78)
at org.apache.spark.sql.execution.QueryExecution.executedPlan(QueryExecution.scala:78)
at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3365)
at org.apache.spark.sql.Dataset.<init>(Dataset.scala:194)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:79)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:642)
at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:694)
Caused by: java.lang.ClassNotFoundException: submarine_spark_ranger_project.org.codehaus.jackson.jaxrs.JacksonJsonProvider
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
{noformat}
RangerRESTClient is using JacksonJsonProvider in versions 1.1, 1.2 and 2.0:
* ranger-1.1: https://github.com/apache/ranger/blob/ranger-1.1/agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
* ranger-1.2: https://github.com/apache/ranger/blob/ranger-1.2/agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
* ranger-2.0: https://github.com/apache/ranger/blob/ranger-2.0/agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
All of them in the same version: 1.9.13:
* ranger-1.1: https://github.com/apache/ranger/blob/ranger-1.1/pom.xml#L93
* ranger-1.2: https://github.com/apache/ranger/blob/ranger-1.2/pom.xml#L93
* ranger-2.0: https://github.com/apache/ranger/blob/ranger-2.0/pom.xml#L93
This version is the same as the one used by submarine:
* https://github.com/apache/submarine/blob/master/pom.xml#L119
> submarine-spark-security: missing JacksonJsonProvider in uber jar
> -----------------------------------------------------------------
>
> Key: SUBMARINE-537
> URL: https://issues.apache.org/jira/browse/SUBMARINE-537
> Project: Apache Submarine
> Issue Type: Bug
> Components: Security
> Reporter: Gustavo Martin
> Priority: Major
>
> Missing org.codehaus.jackson.jaxrs.JacksonJsonProvider in uber jar which is required by RangerRESTClient in versions 1.1, 1.2 and 2.0
> submarine-spark-security throws the following exception:
> {noformat}
> java.lang.NoClassDefFoundError: submarine_spark_ranger_project/org/codehaus/jackson/jaxrs/JacksonJsonProvider
> at org.apache.ranger.plugin.util.RangerRESTClient.buildClient(RangerRESTClient.java:209)
> at org.apache.ranger.plugin.util.RangerRESTClient.getClient(RangerRESTClient.java:176)
> at org.apache.ranger.plugin.util.RangerRESTClient.getResource(RangerRESTClient.java:156)
> at org.apache.ranger.admin.client.RangerAdminRESTClient.createWebResource(RangerAdminRESTClient.java:635)
> at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:142)
> at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:251)
> at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:191)
> at org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:137)
> at org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:229)
> at org.apache.submarine.spark.security.RangerSparkPlugin$.init(RangerSparkPlugin.scala:43)
> at org.apache.submarine.spark.security.RangerSparkPlugin$.<init>(RangerSparkPlugin.scala:64)
> at org.apache.submarine.spark.security.RangerSparkPlugin$.<clinit>(RangerSparkPlugin.scala)
> at org.apache.submarine.spark.security.RangerSparkAuthorizer$.org$apache$submarine$spark$security$RangerSparkAuthorizer$$getSparkResource(RangerSparkAuthorizer.scala:257)
> at org.apache.submarine.spark.security.RangerSparkAuthorizer$$anonfun$addAccessRequest$1$1.apply(RangerSparkAuthorizer.scala:75)
> at org.apache.submarine.spark.security.RangerSparkAuthorizer$$anonfun$addAccessRequest$1$1.apply(RangerSparkAuthorizer.scala:74)
> at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
> at org.apache.submarine.spark.security.RangerSparkAuthorizer$.addAccessRequest$1(RangerSparkAuthorizer.scala:74)
> at org.apache.submarine.spark.security.RangerSparkAuthorizer$.checkPrivileges(RangerSparkAuthorizer.scala:98)
> at org.apache.spark.sql.catalyst.optimizer.SubmarineSparkRangerAuthorizationExtension.apply(SubmarineSparkRangerAuthorizationExtension.scala:64)
> at org.apache.spark.sql.catalyst.optimizer.SubmarineSparkRangerAuthorizationExtension.apply(SubmarineSparkRangerAuthorizationExtension.scala:39)
> at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1$$anonfun$apply$1.apply(RuleExecutor.scala:87)
> at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1$$anonfun$apply$1.apply(RuleExecutor.scala:84)
> at scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:124)
> at scala.collection.immutable.List.foldLeft(List.scala:84)
> at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1.apply(RuleExecutor.scala:84)
> at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1.apply(RuleExecutor.scala:76)
> at scala.collection.immutable.List.foreach(List.scala:392)
> at org.apache.spark.sql.catalyst.rules.RuleExecutor.execute(RuleExecutor.scala:76)
> at org.apache.spark.sql.execution.QueryExecution.optimizedPlan$lzycompute(QueryExecution.scala:67)
> at org.apache.spark.sql.execution.QueryExecution.optimizedPlan(QueryExecution.scala:67)
> at org.apache.spark.sql.execution.QueryExecution.sparkPlan$lzycompute(QueryExecution.scala:73)
> at org.apache.spark.sql.execution.QueryExecution.sparkPlan(QueryExecution.scala:69)
> at org.apache.spark.sql.execution.QueryExecution.executedPlan$lzycompute(QueryExecution.scala:78)
> at org.apache.spark.sql.execution.QueryExecution.executedPlan(QueryExecution.scala:78)
> at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3365)
> at org.apache.spark.sql.Dataset.<init>(Dataset.scala:194)
> at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:79)
> at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:642)
> at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:694)
> Caused by: java.lang.ClassNotFoundException: submarine_spark_ranger_project.org.codehaus.jackson.jaxrs.JacksonJsonProvider
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> {noformat}
> RangerRESTClient is using JacksonJsonProvider in versions 1.1, 1.2 and 2.0:
> * ranger-1.1: https://github.com/apache/ranger/blob/ranger-1.1/agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
> * ranger-1.2: https://github.com/apache/ranger/blob/ranger-1.2/agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
> * ranger-2.0: https://github.com/apache/ranger/blob/ranger-2.0/agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
> All of them in the same version, 1.9.13:
> * ranger-1.1: https://github.com/apache/ranger/blob/ranger-1.1/pom.xml#L93
> * ranger-1.2: https://github.com/apache/ranger/blob/ranger-1.2/pom.xml#L93
> * ranger-2.0: https://github.com/apache/ranger/blob/ranger-2.0/pom.xml#L93
> This version is the same as the one used by submarine:
> * https://github.com/apache/submarine/blob/master/pom.xml#L119
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@submarine.apache.org
For additional commands, e-mail: dev-help@submarine.apache.org