[jira] [Created] (SPARK-24814) Relationship between catalog and datasources

["Bruce Robbins (JIRA)" <ji...@apache.org>]

Bruce Robbins created SPARK-24814:
-------------------------------------

             Summary: Relationship between catalog and datasources
                 Key: SPARK-24814
                 URL: https://issues.apache.org/jira/browse/SPARK-24814
             Project: Spark
          Issue Type: New Feature
          Components: SQL
    Affects Versions: 2.4.0
            Reporter: Bruce Robbins


This is somewhat related, though not identical to, Ryan Blue's SPIP on datasources and catalogs.

Here are the requirements (IMO) for fully implementing V2 datasources and their relationships to catalogs:
 # The global catalog should be configurable (the default can be HMS, but it should be overridable).
 # The default catalog (or an explicitly specified catalog in a query, once multiple catalogs are supported) can determine the V2 datasource to use for reading and writing the data.
 # Conversely, a V2 datasource can determine which catalog to use for resolution (e.g., if the user issues {{spark.read.format("acmex").table("mytable")}}, the acmex datasource would decide which catalog to use for resolving “mytable”).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org