You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Cheng Lian (JIRA)" <ji...@apache.org> on 2015/02/12 19:56:11 UTC

[jira] [Created] (SPARK-5777) Completes data source filter types and remove CatalystScan

Cheng Lian created SPARK-5777:
---------------------------------

             Summary: Completes data source filter types and remove CatalystScan
                 Key: SPARK-5777
                 URL: https://issues.apache.org/jira/browse/SPARK-5777
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 1.2.1, 1.2.0, 1.3.0
            Reporter: Cheng Lian
            Assignee: Cheng Lian


Currently the data source API only supports a fraction of common filters, esp. {{And}} is not supported yet. To workaround this issue and enable full filter push-down optimization in the Parquet data source, {{CatalystScan}} was introduced to receive full Catalyst filter expressions. This class should be removed, since in principle, data source implementations shouldn't touch Catalyst expressions (which are not part of the public developer API).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org