You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2016/04/21 09:54:25 UTC
[jira] [Created] (SPARK-14800) Dealing with null as a value in
options for each internal data source
Hyukjin Kwon created SPARK-14800:
------------------------------------
Summary: Dealing with null as a value in options for each internal data source
Key: SPARK-14800
URL: https://issues.apache.org/jira/browse/SPARK-14800
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 2.0.0
Reporter: Hyukjin Kwon
Priority: Minor
This was found in https://github.com/apache/spark/pull/12494.
Currently, most of options in data sources throws {{NullPointerException}} when given value is {{null}}.
For example, the codes below:
{code}
sqlContext.read
.format("csv")
.option("compression", null)
{code}
throws an exception below:
{code}
java.lang.NullPointerException was thrown.
java.lang.NullPointerException
at org.apache.spark.sql.execution.datasources.csv.CSVOptions.getChar(CSVOptions.scala:32)
at org.apache.spark.sql.execution.datasources.csv.CSVOptions.<init>(CSVOptions.scala:68)
at org.apache.spark.sql.execution.datasources.csv.DefaultSource.prepareWrite(DefaultSource.scala:86)
...
{code}
while some of options such as {{nullValue}} in CSV data source accepts {{null}} .
So, the options throwing {{NullPointerException}} when it sets to {{null}} might have to be handled separately for each data source.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org