[GitHub] [spark] cloud-fan edited a comment on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system

[GitBox <gi...@apache.org>]

cloud-fan edited a comment on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system
URL: https://github.com/apache/spark/pull/23915#issuecomment-470335914
 
 
   The change itself LGTM(except one [comment](https://github.com/apache/spark/pull/23915#discussion_r262987904)) but I need a little more information to justify the design:
   1. why the `initialize` method takes the `name` parameter? Since one table catalog can be registered more than once withh different names, end users would expect the table catalog is not sensitive to the names. I think it's better if Spark can force the table catalog to be not sensitive to the names, i.e. do not give name when initializing table catalog.
   2. why the table catalog needs to report the name? Spark has a map of string to table catalog, when Spark gets a table catalog, the name should already be known.
   
   Interestingly the `name` doesn't exist in the original PR: https://github.com/apache/spark/pull/21306/files#diff-81c54123a7549b07a9d627353d9cbf95R49 . I'm wondering what has been hanged recently.
   
   EDIT:
   We discussed this problem in the ds v2 community meeting. The conclusion is, theoretically it's not needed, but makes it easier to use table catalog in Spark. We will justify it after we implement table catalog. We may decided to remove the `name` parameter and create a wrapper class in Spark, or leave it unchanged.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org