You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Gengliang Wang (JIRA)" <ji...@apache.org> on 2018/04/22 13:48:00 UTC

[jira] [Created] (SPARK-24045) Create base class for file data source v2

Gengliang Wang created SPARK-24045:
--------------------------------------

             Summary: Create base class for file data source v2
                 Key: SPARK-24045
                 URL: https://issues.apache.org/jira/browse/SPARK-24045
             Project: Spark
          Issue Type: Sub-task
          Components: SQL
    Affects Versions: 2.2.1
            Reporter: Gengliang Wang


During File data source V2 migration,  we need to fall back to V1 implementation in following cases:
1. File datasource V2 might be implemented partially during migration.
  E.g. if [[ReadSupport]] is implemented while [[WriteSupport]] is not,
  write path should fall back to V1 implementation.

2. File datasource V2 implementations cause regression.
3. Catalog support is required, which is still under development for data source V2.

 

Create a base class FileDataSourceV2 and resolve the falling back problems.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org