You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Reynold Xin (JIRA)" <ji...@apache.org> on 2016/06/11 22:13:20 UTC
[jira] [Resolved] (SPARK-15585) Don't use null in data source
options to indicate default value
[ https://issues.apache.org/jira/browse/SPARK-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Reynold Xin resolved SPARK-15585.
---------------------------------
Resolution: Fixed
Assignee: Takeshi Yamamuro
Fix Version/s: 2.0.0
> Don't use null in data source options to indicate default value
> ---------------------------------------------------------------
>
> Key: SPARK-15585
> URL: https://issues.apache.org/jira/browse/SPARK-15585
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Reynold Xin
> Assignee: Takeshi Yamamuro
> Priority: Critical
> Fix For: 2.0.0
>
>
> See email: http://apache-spark-developers-list.1001551.n3.nabble.com/changed-behavior-for-csv-datasource-and-quoting-in-spark-2-0-0-SNAPSHOT-td17704.html
> We'd need to change DataFrameReader/DataFrameWriter in Python's csv/json/parquet/... functions to put the actual default option values as function parameters, rather than setting them to None. We can then in CSVOptions.getChar (and JSONOptions, etc) to actually return null if the value is null, rather than setting it to default value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org