You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "wForget (via GitHub)" <gi...@apache.org> on 2025/06/05 05:57:55 UTC
[PR] [KYUUBI #7079] Improve performance of authz rules [kyuubi]
wForget opened a new pull request, #7081:
URL: https://github.com/apache/kyuubi/pull/7081
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/master/contributing/code/index.html
2. If the PR is related to an issue in https://github.com/apache/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug, and what versions are affected.
-->
closes #7079
### How was this patch tested?
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7079] Improve performance of authz rules [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #7081:
URL: https://github.com/apache/kyuubi/pull/7081#discussion_r2128050027
##########
kyuubi-util-scala/src/main/scala/org/apache/kyuubi/util/reflect/ReflectUtils.scala:
##########
@@ -64,6 +66,27 @@ object ReflectUtils {
}
}
+ /**
+ * Get a method with the given name and argument classes from the given class.
+ * @param clz the class to get the method from
+ * @param methodName the method name from declared field names
+ * @param argClasses the classes of the arguments
+ * @return an unbound method that can be invoked later
+ */
+ def getMethod(clz: Class[_], methodName: String, argClasses: Class[_]*): UnboundMethod = {
Review Comment:
this does not simplify much code, can we inline it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7079] Improve performance of AccessRequest initialization [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #7081:
URL: https://github.com/apache/kyuubi/pull/7081#discussion_r2193938046
##########
extensions/spark/kyuubi-spark-authz/src/main/scala/org/apache/kyuubi/plugin/spark/authz/ranger/AccessRequest.scala:
##########
@@ -68,17 +70,43 @@ object AccessRequest {
req
}
+ private val getRolesFromUserAndGroupsMethod: Option[UnboundMethod] =
+ getMethod(
+ SparkRangerAdminPlugin.getClass,
+ "getRolesFromUserAndGroups",
+ classOf[String],
+ classOf[JSet[String]])
+
+ private val setUserRolesMethod: Option[UnboundMethod] =
+ getMethod(classOf[AccessRequest], "setUserRoles", classOf[JSet[String]])
+
private def getUserGroupsFromUgi(user: UserGroupInformation): JSet[String] = {
user.getGroupNames.toSet.asJava
}
+ private lazy val getUserStoreEnricherMethod: Option[UnboundMethod] =
+ getMethod(SparkRangerAdminPlugin.getClass, "getUserStoreEnricher")
+
+ private lazy val getRangerUserStoreMethod: Option[UnboundMethod] =
+ getMethod(
+ Class.forName("org.apache.ranger.plugin.contextenricher.RangerUserStoreEnricher"),
+ "getRangerUserStore")
+
+ private lazy val getUserGroupMappingMethod: Option[UnboundMethod] =
+ getMethod(
+ Class.forName("org.apache.ranger.plugin.util.RangerUserStore"),
Review Comment:
I see, it's a new feature of Ranger 2.1, we must use reflection as long as we support lower Ranger versions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7079] Improve performance of authz rules [kyuubi]
Posted by "wForget (via GitHub)" <gi...@apache.org>.
wForget commented on code in PR #7081:
URL: https://github.com/apache/kyuubi/pull/7081#discussion_r2128097971
##########
kyuubi-util-scala/src/main/scala/org/apache/kyuubi/util/reflect/ReflectUtils.scala:
##########
@@ -64,6 +66,27 @@ object ReflectUtils {
}
}
+ /**
+ * Get a method with the given name and argument classes from the given class.
+ * @param clz the class to get the method from
+ * @param methodName the method name from declared field names
+ * @param argClasses the classes of the arguments
+ * @return an unbound method that can be invoked later
+ */
+ def getMethod(clz: Class[_], methodName: String, argClasses: Class[_]*): UnboundMethod = {
Review Comment:
> this does not simplify much code, can we inline it?
Makes sense, changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7079] Improve performance of AccessRequest initialization [kyuubi]
Posted by "wForget (via GitHub)" <gi...@apache.org>.
wForget commented on code in PR #7081:
URL: https://github.com/apache/kyuubi/pull/7081#discussion_r2191981608
##########
extensions/spark/kyuubi-spark-authz/src/main/scala/org/apache/kyuubi/plugin/spark/authz/ranger/AccessRequest.scala:
##########
@@ -68,17 +70,43 @@ object AccessRequest {
req
}
+ private val getRolesFromUserAndGroupsMethod: Option[UnboundMethod] =
+ getMethod(
+ SparkRangerAdminPlugin.getClass,
Review Comment:
`SparkRangerAdminPlugin` is an object and does not seem to work with `classOf`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7079] Improve performance of AccessRequest initialization [kyuubi]
Posted by "wForget (via GitHub)" <gi...@apache.org>.
wForget commented on code in PR #7081:
URL: https://github.com/apache/kyuubi/pull/7081#discussion_r2135343043
##########
extensions/spark/kyuubi-spark-authz/src/main/scala/org/apache/kyuubi/plugin/spark/authz/ranger/AccessRequest.scala:
##########
@@ -68,17 +69,39 @@ object AccessRequest {
req
}
+ private val getRolesFromUserAndGroupsMethod: Option[UnboundMethod] = {
+ try {
+ Some(DynMethods.builder("getRolesFromUserAndGroups")
+ .hiddenImpl(SparkRangerAdminPlugin.getClass, classOf[String], classOf[JSet[String]])
+ .impl(SparkRangerAdminPlugin.getClass, classOf[String], classOf[JSet[String]])
+ .buildChecked)
+ } catch {
+ case _: Exception => None
+ }
+ }
+
+ private val setUserRolesMethod: Option[UnboundMethod] = {
+ try {
+ Some(DynMethods.builder("setUserRoles")
+ .hiddenImpl(classOf[AccessRequest], classOf[JSet[String]])
+ .impl(classOf[AccessRequest], classOf[JSet[String]])
+ .buildChecked)
+ } catch {
+ case _: Exception => None
+ }
+ }
+
private def getUserGroupsFromUgi(user: UserGroupInformation): JSet[String] = {
user.getGroupNames.toSet.asJava
}
- private def getUserGroupsFromUserStore(user: UserGroupInformation): Option[JSet[String]] = {
+ private lazy val userGroupMappingOpt: Option[JHashMap[String, JSet[String]]] = {
Review Comment:
> I'm not sure if this is safe to cache
`userGroupMapping` may be updated by [RangerUserStoreRefresher](https://github.com/apache/ranger/blob/master/agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerUserStoreRefresher.java), I will only cache reflect methods
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7079] Improve performance of AccessRequest initialization [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #7081:
URL: https://github.com/apache/kyuubi/pull/7081#discussion_r2191617146
##########
extensions/spark/kyuubi-spark-authz/src/main/scala/org/apache/kyuubi/plugin/spark/authz/ranger/AccessRequest.scala:
##########
@@ -68,17 +70,43 @@ object AccessRequest {
req
}
+ private val getRolesFromUserAndGroupsMethod: Option[UnboundMethod] =
+ getMethod(
+ SparkRangerAdminPlugin.getClass,
Review Comment:
```suggestion
classOf[SparkRangerAdminPlugin],
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7079] Improve performance of AccessRequest initialization [kyuubi]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #7081:
URL: https://github.com/apache/kyuubi/pull/7081#issuecomment-3704269512
Thanks for the PR! This PR is being closed due to inactivity. This isn't a judgement on the merit of the PR in any way. If this is still an issue with the latest version of Kyuubi, please reopen it and ask a committer to remove the Stale tag!
Thank you for using Kyuubi!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7079] Improve performance of authz rules [kyuubi]
Posted by "wForget (via GitHub)" <gi...@apache.org>.
wForget commented on code in PR #7081:
URL: https://github.com/apache/kyuubi/pull/7081#discussion_r2128389492
##########
extensions/spark/kyuubi-spark-authz/src/main/scala/org/apache/kyuubi/plugin/spark/authz/rule/RuleHelper.scala:
##########
@@ -47,6 +47,6 @@ trait RuleHelper extends Rule[LogicalPlan] {
plan.withNewChildren(newChildren)
}
- def ugi: UserGroupInformation = AuthZUtils.getAuthzUgi(spark.sparkContext)
Review Comment:
This change may affect test cases: https://github.com/apache/kyuubi/blob/cad5a392f3195d8f3b16c293a11ac54c2f090871/extensions/spark/kyuubi-spark-authz/src/test/scala/org/apache/kyuubi/plugin/spark/authz/ranger/IcebergCatalogRangerSparkExtensionSuite.scala#L68-L87
@pan3793 Is it possible to have multiple ugi in the spark engine?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7079] Improve performance of AccessRequest initialization [kyuubi]
Posted by "wForget (via GitHub)" <gi...@apache.org>.
wForget commented on code in PR #7081:
URL: https://github.com/apache/kyuubi/pull/7081#discussion_r2191917626
##########
extensions/spark/kyuubi-spark-authz/src/main/scala/org/apache/kyuubi/plugin/spark/authz/ranger/AccessRequest.scala:
##########
@@ -68,17 +70,43 @@ object AccessRequest {
req
}
+ private val getRolesFromUserAndGroupsMethod: Option[UnboundMethod] =
+ getMethod(
+ SparkRangerAdminPlugin.getClass,
+ "getRolesFromUserAndGroups",
+ classOf[String],
+ classOf[JSet[String]])
+
+ private val setUserRolesMethod: Option[UnboundMethod] =
+ getMethod(classOf[AccessRequest], "setUserRoles", classOf[JSet[String]])
+
private def getUserGroupsFromUgi(user: UserGroupInformation): JSet[String] = {
user.getGroupNames.toSet.asJava
}
+ private lazy val getUserStoreEnricherMethod: Option[UnboundMethod] =
+ getMethod(SparkRangerAdminPlugin.getClass, "getUserStoreEnricher")
+
+ private lazy val getRangerUserStoreMethod: Option[UnboundMethod] =
+ getMethod(
+ Class.forName("org.apache.ranger.plugin.contextenricher.RangerUserStoreEnricher"),
+ "getRangerUserStore")
+
+ private lazy val getUserGroupMappingMethod: Option[UnboundMethod] =
+ getMethod(
+ Class.forName("org.apache.ranger.plugin.util.RangerUserStore"),
Review Comment:
> it's located at `ranger-plugins-common`, should be always accessible?
I don't get your point, but the logic here seems to be consistent with previous.
https://github.com/apache/kyuubi/blob/4e40f9457dfcde9b2748eb6da877e1ee32193fca/extensions/spark/kyuubi-spark-authz/src/main/scala/org/apache/kyuubi/plugin/spark/authz/ranger/AccessRequest.scala#L79-L80
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7079] Improve performance of AccessRequest initialization [kyuubi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #7081:
URL: https://github.com/apache/kyuubi/pull/7081#issuecomment-2943726556
## [Codecov](https://app.codecov.io/gh/apache/kyuubi/pull/7081?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: Patch coverage is `0%` with `10 lines` in your changes missing coverage. Please review.
> Project coverage is 0.00%. Comparing base [(`cad5a39`)](https://app.codecov.io/gh/apache/kyuubi/commit/cad5a392f3195d8f3b16c293a11ac54c2f090871?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`c8401b5`)](https://app.codecov.io/gh/apache/kyuubi/commit/c8401b5c915aeda8e9d5e19903c3409e38c8c01f?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
| [Files with missing lines](https://app.codecov.io/gh/apache/kyuubi/pull/7081?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...uubi/plugin/spark/authz/ranger/AccessRequest.scala](https://app.codecov.io/gh/apache/kyuubi/pull/7081?src=pr&el=tree&filepath=extensions%2Fspark%2Fkyuubi-spark-authz%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fkyuubi%2Fplugin%2Fspark%2Fauthz%2Franger%2FAccessRequest.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZXh0ZW5zaW9ucy9zcGFyay9reXV1Ymktc3BhcmstYXV0aHovc3JjL21haW4vc2NhbGEvb3JnL2FwYWNoZS9reXV1YmkvcGx1Z2luL3NwYXJrL2F1dGh6L3Jhbmdlci9BY2Nlc3NSZXF1ZXN0LnNjYWxh) | 0.00% | [10 Missing :warning: ](https://app.codecov.io/gh/apache/kyuubi/pull/7081?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #7081 +/- ##
======================================
Coverage 0.00% 0.00%
======================================
Files 697 697
Lines 43203 43202 -1
Branches 5854 5856 +2
======================================
+ Misses 43203 43202 -1
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/kyuubi/pull/7081?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7079] Improve performance of AccessRequest initialization [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #7081:
URL: https://github.com/apache/kyuubi/pull/7081#discussion_r2191618378
##########
extensions/spark/kyuubi-spark-authz/src/main/scala/org/apache/kyuubi/plugin/spark/authz/ranger/AccessRequest.scala:
##########
@@ -68,17 +70,43 @@ object AccessRequest {
req
}
+ private val getRolesFromUserAndGroupsMethod: Option[UnboundMethod] =
+ getMethod(
+ SparkRangerAdminPlugin.getClass,
+ "getRolesFromUserAndGroups",
+ classOf[String],
+ classOf[JSet[String]])
+
+ private val setUserRolesMethod: Option[UnboundMethod] =
+ getMethod(classOf[AccessRequest], "setUserRoles", classOf[JSet[String]])
+
private def getUserGroupsFromUgi(user: UserGroupInformation): JSet[String] = {
user.getGroupNames.toSet.asJava
}
+ private lazy val getUserStoreEnricherMethod: Option[UnboundMethod] =
+ getMethod(SparkRangerAdminPlugin.getClass, "getUserStoreEnricher")
Review Comment:
```suggestion
getMethod(classOf[SparkRangerAdminPlugin], "getUserStoreEnricher")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7079] Improve performance of AccessRequest initialization [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #7081:
URL: https://github.com/apache/kyuubi/pull/7081#discussion_r2135198722
##########
extensions/spark/kyuubi-spark-authz/src/main/scala/org/apache/kyuubi/plugin/spark/authz/ranger/AccessRequest.scala:
##########
@@ -68,17 +69,39 @@ object AccessRequest {
req
}
+ private val getRolesFromUserAndGroupsMethod: Option[UnboundMethod] = {
+ try {
+ Some(DynMethods.builder("getRolesFromUserAndGroups")
+ .hiddenImpl(SparkRangerAdminPlugin.getClass, classOf[String], classOf[JSet[String]])
+ .impl(SparkRangerAdminPlugin.getClass, classOf[String], classOf[JSet[String]])
+ .buildChecked)
+ } catch {
+ case _: Exception => None
+ }
+ }
+
+ private val setUserRolesMethod: Option[UnboundMethod] = {
+ try {
+ Some(DynMethods.builder("setUserRoles")
+ .hiddenImpl(classOf[AccessRequest], classOf[JSet[String]])
+ .impl(classOf[AccessRequest], classOf[JSet[String]])
+ .buildChecked)
+ } catch {
+ case _: Exception => None
+ }
+ }
+
private def getUserGroupsFromUgi(user: UserGroupInformation): JSet[String] = {
user.getGroupNames.toSet.asJava
}
- private def getUserGroupsFromUserStore(user: UserGroupInformation): Option[JSet[String]] = {
+ private lazy val userGroupMappingOpt: Option[JHashMap[String, JSet[String]]] = {
Review Comment:
I'm not sure if this is safe to cache
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7079] Improve performance of AccessRequest initialization [kyuubi]
Posted by "wForget (via GitHub)" <gi...@apache.org>.
wForget commented on code in PR #7081:
URL: https://github.com/apache/kyuubi/pull/7081#discussion_r2128455153
##########
extensions/spark/kyuubi-spark-authz/src/main/scala/org/apache/kyuubi/plugin/spark/authz/rule/RuleHelper.scala:
##########
@@ -47,6 +47,6 @@ trait RuleHelper extends Rule[LogicalPlan] {
plan.withNewChildren(newChildren)
}
- def ugi: UserGroupInformation = AuthZUtils.getAuthzUgi(spark.sparkContext)
Review Comment:
Different sparkSession may have different authz ugi, I will revert this change first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7079] Improve performance of AccessRequest initialization [kyuubi]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #7081: [KYUUBI #7079] Improve performance of AccessRequest initialization
URL: https://github.com/apache/kyuubi/pull/7081
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7079] Improve performance of AccessRequest initialization [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #7081:
URL: https://github.com/apache/kyuubi/pull/7081#discussion_r2191622334
##########
extensions/spark/kyuubi-spark-authz/src/main/scala/org/apache/kyuubi/plugin/spark/authz/ranger/AccessRequest.scala:
##########
@@ -68,17 +70,43 @@ object AccessRequest {
req
}
+ private val getRolesFromUserAndGroupsMethod: Option[UnboundMethod] =
+ getMethod(
+ SparkRangerAdminPlugin.getClass,
+ "getRolesFromUserAndGroups",
+ classOf[String],
+ classOf[JSet[String]])
+
+ private val setUserRolesMethod: Option[UnboundMethod] =
+ getMethod(classOf[AccessRequest], "setUserRoles", classOf[JSet[String]])
+
private def getUserGroupsFromUgi(user: UserGroupInformation): JSet[String] = {
user.getGroupNames.toSet.asJava
}
+ private lazy val getUserStoreEnricherMethod: Option[UnboundMethod] =
+ getMethod(SparkRangerAdminPlugin.getClass, "getUserStoreEnricher")
+
+ private lazy val getRangerUserStoreMethod: Option[UnboundMethod] =
+ getMethod(
+ Class.forName("org.apache.ranger.plugin.contextenricher.RangerUserStoreEnricher"),
+ "getRangerUserStore")
+
+ private lazy val getUserGroupMappingMethod: Option[UnboundMethod] =
+ getMethod(
+ Class.forName("org.apache.ranger.plugin.util.RangerUserStore"),
Review Comment:
it's located at `ranger-plugins-common`, should be always accessible?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org