[jira] [Created] (SPARK-14839) Support for other types as option in OPTIONS clause

["Hyukjin Kwon (JIRA)" <ji...@apache.org>]

Hyukjin Kwon created SPARK-14839:
------------------------------------

             Summary: Support for other types as option in OPTIONS clause
                 Key: SPARK-14839
                 URL: https://issues.apache.org/jira/browse/SPARK-14839
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 2.0.0
            Reporter: Hyukjin Kwon
            Priority: Minor


This was found in https://github.com/apache/spark/pull/12494.

Currently, Spark SQL does not support other types and {{null}} as a value of an options. 

For example, 

{code}
...
CREATE ...
USING csv
OPTIONS (path "your-path", quote null)
{code}

throws an exception below

{code}
Unsupported SQL statement
== SQL ==
 CREATE TEMPORARY TABLE carsTable (yearMade double, makeName string, modelName string, comments string, grp string) USING csv OPTIONS (path "your-path", quote null)       
org.apache.spark.sql.catalyst.parser.ParseException: 
Unsupported SQL statement
== SQL ==
 CREATE TEMPORARY TABLE carsTable (yearMade double, makeName string, modelName string, comments string, grp string) USING csv OPTIONS (path "your-path", quote null)       
	at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.nativeCommand(ParseDriver.scala:66)
	at org.apache.spark.sql.catalyst.parser.AbstractSqlParser$$anonfun$parsePlan$1.apply(ParseDriver.scala:56)
	at org.apache.spark.sql.catalyst.parser.AbstractSqlParser$$anonfun$parsePlan$1.apply(ParseDriver.scala:53)
	at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:86)
	at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:53)
	at org.apache.spark.sql.SQLContext.parseSql(SQLContext.scala:195)
	at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:764)
...
{code}

Currently, Scala API supports to take options with the types, {{String}}, {{Long}}, {{Double}} and {{Boolean}} and Python API also supports other types. I think in this way we can support data sources in a consistent way.

It looks it is okay to  to provide other types as arguments just like [Microsoft SQL|https://msdn.microsoft.com/en-us/library/ms190322.aspx] because [SQL-1992|http://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt] standard mentions options as below:

{quote}
An implementation remains conforming even if it provides user op-
tions to process nonconforming SQL language or to process conform-
ing SQL language in a nonconforming manner.
{quote}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org