[jira] [Created] (SPARK-17492) Reading Cataloged Data Sources without Extending SchemaRelationProvider

["Xiao Li (JIRA)" <ji...@apache.org>]

Xiao Li created SPARK-17492:
-------------------------------

             Summary: Reading Cataloged Data Sources without Extending SchemaRelationProvider
                 Key: SPARK-17492
                 URL: https://issues.apache.org/jira/browse/SPARK-17492
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.1.0
            Reporter: Xiao Li


For data sources without extending `SchemaRelationProvider`, we expect users to not specify schemas when they creating tables. If the schema is input from users, an exception is issued. 

Since Spark 2.1, for any data source, to avoid infer the schema every time, we store the schema in the metastore catalog. Thus, when reading a cataloged data source table, the schema could be read from metastore catalog. In this case, we also got an exception. For example, 

{noformat}
sql(
  s"""
     |CREATE TABLE relationProvierWithSchema
     |USING org.apache.spark.sql.sources.SimpleScanSource
     |OPTIONS (
     |  From '1',
     |  To '10'
     |)
   """.stripMargin)
spark.table(tableName).show()
{noformat}
{noformat}
org.apache.spark.sql.sources.SimpleScanSource does not allow user-specified schemas.;
{noformat}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org