You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Cheng Lian (JIRA)" <ji...@apache.org> on 2016/06/16 05:34:05 UTC
[jira] [Updated] (SPARK-15983) Remove FileFormat.prepareRead
[ https://issues.apache.org/jira/browse/SPARK-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng Lian updated SPARK-15983:
-------------------------------
Description:
Interface method {{FileFormat.prepareRead()}} was added in [PR #12088|https://github.com/apache/spark/pull/12088] to handle a special case in the LibSVM data source.
However, the semantics of this interface method isn't intuitive: it returns a modified version of the data source options map. Considering that the LibSVM case can be easily handled using schema metadata inside {{inferSchema}}, we can remove this interface method to keep the {{FileFormat}} interface clean.
was:
Interface method {{FileFormat.prepareRead()}} was added in [PR #12088|https://github.com/apache/spark/pull/12088] to handle a special case in the LibSVM data source.
However, the semantics of this interface method isn't intuitive: it returns a modified version of the data source options map. Considering that the LibSVM case can be easily handled using schema metadata inside inferSchema, we can remove this interface method to keep the FileFormat interface clean.
> Remove FileFormat.prepareRead
> -----------------------------
>
> Key: SPARK-15983
> URL: https://issues.apache.org/jira/browse/SPARK-15983
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.0.0
> Reporter: Cheng Lian
> Assignee: Cheng Lian
>
> Interface method {{FileFormat.prepareRead()}} was added in [PR #12088|https://github.com/apache/spark/pull/12088] to handle a special case in the LibSVM data source.
> However, the semantics of this interface method isn't intuitive: it returns a modified version of the data source options map. Considering that the LibSVM case can be easily handled using schema metadata inside {{inferSchema}}, we can remove this interface method to keep the {{FileFormat}} interface clean.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org