[GitHub] [spark] cloud-fan commented on issue #25291: [SPARK-28554][SQL] implement basic catalog functionalities for JDBC v2 with a DS v1 fallback API

[GitBox <gi...@apache.org>]

cloud-fan commented on issue #25291: [SPARK-28554][SQL] implement basic catalog functionalities for JDBC v2 with a DS v1 fallback API
URL: https://github.com/apache/spark/pull/25291#issuecomment-518340231
 
 
   I put JDBC v2 here because I want to give a concrete example and prove that my proposal works. I can surely create a new PR for API changes only, as soon as we agree on the approach.
   
   More specifically, JDBC v2 wants to
   1. implement catalog APIs (`TableCatalog`), so that users can register it as a catalog and access its tables directly.
   2. not implement `ScanBuilder`, `WriteBuilder` stuff right now, as it's a lot of work but the benefit is not very big.
   
   The proposal here is:
   1. have a special version of the v2 `Table` interface (`DataSourceV1Table`), which can return a v1 `BaseRelation`.
   2. have a special version of the v2 `TableCatalog` interface (`DataSourceV1TableCatalog`), which can return a `DataSourceV1Table`.
   
   How it works:
   1. JDBC v2 implements v2 APIs, so the analyzer treats it as a v2 source, and use v2 plans to process it.
   2. The v1 analyzer rule catches v2 plans and converts them to v1 plans, if the table is `DataSourceV1Table`.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org