[GitHub] [incubator-kyuubi] zhouyifan279 commented on issue #3607: [Umbrella] RangerSparkExtension support {OWNER} variable defined in Ranger Policy

[GitBox <gi...@apache.org>]

zhouyifan279 commented on issue #3607:
URL: https://github.com/apache/incubator-kyuubi/issues/3607#issuecomment-1283949844

   @bowenliang123 , thanks for your comment. Here is my thoughts about your questions:
   
   > 1. is it investigating the owner for all tables evn with no {OWNER} rules on them?  This will cause a heavy CPU/RTT time to fetch this information, and the additional cache will leave more memory footprints to it.
   
   For SQL query (SELECT & DML), we can always get the owner of table from `CatalogTable#owner` or `org.apache.spark.sql.connector.catalog.Table#properties.get("owner")`.  No extra fetch is introduced.
   For most SQL commands (DDL), table metadata is not fetched during SQL complie. We need to fetch table metadata.
   In most cases, only one table metadata is fetched, there should not be much CPU/RTT overhead.
   
   
   > 2\. what is the proper caching and evicting strategy for caching table owners? LTT or max cache counts will introduce worries for missed queries, whether they will be fetched again which could cause more action and load in 1.
   
   Originally, I intended to cache table metadata because I wanted to fetch table metadata of `DataSourceV2Relation`.
   After deeper investigation, I found `DataSourceV2Relation` carrying table metadata in `table` field. So cache is not needed anymore.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org