You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Herman van Hovell (JIRA)" <ji...@apache.org> on 2016/07/07 03:59:10 UTC
[jira] [Resolved] (SPARK-14839) Support for other types as option
in OPTIONS clause
[ https://issues.apache.org/jira/browse/SPARK-14839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Herman van Hovell resolved SPARK-14839.
---------------------------------------
Resolution: Resolved
Assignee: Hyukjin Kwon
Fix Version/s: 2.1.0
> Support for other types as option in OPTIONS clause
> ---------------------------------------------------
>
> Key: SPARK-14839
> URL: https://issues.apache.org/jira/browse/SPARK-14839
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.0.0
> Reporter: Hyukjin Kwon
> Assignee: Hyukjin Kwon
> Priority: Minor
> Fix For: 2.1.0
>
>
> This was found in https://github.com/apache/spark/pull/12494.
> Currently, Spark SQL does not support other types and {{null}} as a value of an options.
> For example,
> {code}
> CREATE ...
> USING csv
> OPTIONS (path "your-path", quote null)
> {code}
> throws an exception below
> {code}
> Unsupported SQL statement
> == SQL ==
> CREATE TEMPORARY TABLE carsTable (yearMade double, makeName string, modelName string, comments string, grp string) USING csv OPTIONS (path "your-path", quote null)
> org.apache.spark.sql.catalyst.parser.ParseException:
> Unsupported SQL statement
> == SQL ==
> CREATE TEMPORARY TABLE carsTable (yearMade double, makeName string, modelName string, comments string, grp string) USING csv OPTIONS (path "your-path", quote null)
> at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.nativeCommand(ParseDriver.scala:66)
> at org.apache.spark.sql.catalyst.parser.AbstractSqlParser$$anonfun$parsePlan$1.apply(ParseDriver.scala:56)
> at org.apache.spark.sql.catalyst.parser.AbstractSqlParser$$anonfun$parsePlan$1.apply(ParseDriver.scala:53)
> at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:86)
> at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:53)
> at org.apache.spark.sql.SQLContext.parseSql(SQLContext.scala:195)
> at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:764)
> ...
> {code}
> Currently, Scala API supports to take options with the types, {{String}}, {{Long}}, {{Double}} and {{Boolean}} and Python API also supports other types. I think in this way we can support data sources in a consistent way.
> It looks it is okay to to provide other types as arguments just like [Microsoft SQL|https://msdn.microsoft.com/en-us/library/ms190322.aspx] because [SQL-1992|http://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt] standard mentions options as below:
> {quote}
> An implementation remains conforming even if it provides user op-
> tions to process nonconforming SQL language or to process conform-
> ing SQL language in a nonconforming manner.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org