You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Wenchen Fan (JIRA)" <ji...@apache.org> on 2018/07/22 14:45:00 UTC

[jira] [Created] (SPARK-24882) separate responsibilities of the data source v2 read API

Wenchen Fan created SPARK-24882:
-----------------------------------

             Summary: separate responsibilities of the data source v2 read API
                 Key: SPARK-24882
                 URL: https://issues.apache.org/jira/browse/SPARK-24882
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 2.4.0
            Reporter: Wenchen Fan
            Assignee: Wenchen Fan


Data source V2 is out for a while, see the SPIP [here|https://docs.google.com/document/d/1n_vUVbF4KD3gxTmkNEon5qdQ-Z8qU5Frf6WMQZ6jJVM/edit?usp=sharing]. We have already migrated most of the built-in streaming data sources to the V2 API, and the file source migration is in progress. During the migration, we found several problems and want to address them before we stabilize the V2 API.

To solve these problems, we need to separate responsibilities in the data source v2 read API. Details please see the attached google doc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org