You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by GitBox <gi...@apache.org> on 2022/10/11 07:44:46 UTC
[GitHub] [incubator-kyuubi] zhouyifan279 opened a new issue, #3607: [Umbrella] RangerSparkExtension support {OWNER} variable in Ranger Policy
zhouyifan279 opened a new issue, #3607:
URL: https://github.com/apache/incubator-kyuubi/issues/3607
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-kyuubi/issues) and found no similar issues.
### Describe the proposal
Currently, if user does not have insert permission of all tables in a database, AccessControlException will be throw when user
insert into a newly created table.
RangerBasePlugin uses {OWNER} variable to remove this limitation:
At Ranger Admim side, set {OWNER} variable in Ranger Policy `Users` field.
At Ranger Plugin side, due to the above policy, current user gets the specified permissions of any table he creates(owns).
Some works need to be done to support this feature in Kyuubi RangerSparkExtension.
### Task list
- [] #1
- [] #2
- [] #3
### Are you willing to submit PR?
- [X] Yes. I can submit a PR independently to improve.
- [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
- [ ] No. I cannot submit a PR at this time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] bowenliang123 commented on issue #3607: [Umbrella] RangerSparkExtension support {OWNER} variable defined in Ranger Policy
Posted by GitBox <gi...@apache.org>.
bowenliang123 commented on issue #3607:
URL: https://github.com/apache/incubator-kyuubi/issues/3607#issuecomment-1284036299
Thanks for the investigation and explanation. Since the owner name is carried in `CatalogTable` in V1 and in `table` in V2Realtion, it's alright to use them without extra fetching action. And as of now we don't have to cache them in runtime, that's good enough for concrete implementation.
@zhouyifan279
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] bowenliang123 commented on issue #3607: [Umbrella] RangerSparkExtension support {OWNER} variable in Ranger Policy
Posted by GitBox <gi...@apache.org>.
bowenliang123 commented on issue #3607:
URL: https://github.com/apache/incubator-kyuubi/issues/3607#issuecomment-1277722485
This would be helpful for applying {OWNER} to policies.
But two problem to consider about this ,
1. is it investigating the owner for all tables even with no {OWNER} rules on them? This will cause a heavy cpu time to fetch these information, and additional cache will leave more memory footprints to it .
2. what is the proper caching and evicting strategy for caching table owners ? LTT or max cache counts will introducing worries for missed queries, whether they will be fetched again which could causing more action and load in 1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] zhouyifan279 commented on issue #3607: [Umbrella] RangerSparkExtension support {OWNER} variable defined in Ranger Policy
Posted by GitBox <gi...@apache.org>.
zhouyifan279 commented on issue #3607:
URL: https://github.com/apache/incubator-kyuubi/issues/3607#issuecomment-1283949844
@bowenliang123 , thanks for your comment. Here is my thoughts about your questions:
> 1. is it investigating the owner for all tables evn with no {OWNER} rules on them? This will cause a heavy CPU/RTT time to fetch this information, and the additional cache will leave more memory footprints to it.
For SQL query (SELECT & DML), we can always get the owner of table from `CatalogTable#owner` or `org.apache.spark.sql.connector.catalog.Table#properties.get("owner")`. No extra fetch is introduced.
For most SQL commands (DDL), table metadata is not fetched during SQL complie. We need to fetch table metadata.
In most cases, only one table metadata is fetched, there should not be much CPU/RTT overhead.
> 2\. what is the proper caching and evicting strategy for caching table owners? LTT or max cache counts will introduce worries for missed queries, whether they will be fetched again which could cause more action and load in 1.
Originally, I intended to cache table metadata because I wanted to fetch table metadata of `DataSourceV2Relation`.
After deeper investigation, I found `DataSourceV2Relation` carrying table metadata in `table` field. So cache is not needed anymore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] zhouyifan279 closed issue #3607: [Umbrella] RangerSparkExtension support {OWNER} variable defined in Ranger Policy
Posted by GitBox <gi...@apache.org>.
zhouyifan279 closed issue #3607: [Umbrella] RangerSparkExtension support {OWNER} variable defined in Ranger Policy
URL: https://github.com/apache/incubator-kyuubi/issues/3607
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org