[jira] [Created] (SPARK-9685) "Unsupported dataType: char(X)" in Hive

["Ángel Álvarez (JIRA)" <ji...@apache.org>]

Ángel Álvarez created SPARK-9685:
------------------------------------

             Summary: "Unsupported dataType: char(X)" in Hive
                 Key: SPARK-9685
                 URL: https://issues.apache.org/jira/browse/SPARK-9685
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 1.5.0
            Reporter: Ángel Álvarez


I'm getting the following error when I try to read a Hive table with char(X) fields:

{code}
15/08/06 11:38:51 INFO parse.ParseDriver: Parse Completed
org.apache.spark.sql.types.DataTypeException: Unsupported dataType: char(8). If you have a struct and a field name of it has any special characters, please use backticks (`) to quote that field name, e.g. `x+y`. Please note that backtick itself is not supported in a field name.
        at org.apache.spark.sql.types.DataTypeParser$class.toDataType(DataTypeParser.scala:95)
        at org.apache.spark.sql.types.DataTypeParser$$anon$1.toDataType(DataTypeParser.scala:107)
        at org.apache.spark.sql.types.DataTypeParser$.parse(DataTypeParser.scala:111)
        at org.apache.spark.sql.hive.HiveMetastoreTypes$.toDataType(HiveMetastoreCatalog.scala:769)
        at org.apache.spark.sql.hive.MetastoreRelation$SchemaAttribute.toAttribute(HiveMetastoreCatalog.scala:742)
        at org.apache.spark.sql.hive.MetastoreRelation$$anonfun$44.apply(HiveMetastoreCatalog.scala:752)
        at org.apache.spark.sql.hive.MetastoreRelation$$anonfun$44.apply(HiveMetastoreCatalog.scala:752)
{code}

It seems there is no "char" DataType defined in the DataTypeParser class
{code}
  protected lazy val primitiveType: Parser[DataType] =
    "(?i)string".r ^^^ StringType |
    "(?i)float".r ^^^ FloatType |
    "(?i)(?:int|integer)".r ^^^ IntegerType |
    "(?i)tinyint".r ^^^ ByteType |
    "(?i)smallint".r ^^^ ShortType |
    "(?i)double".r ^^^ DoubleType |
    "(?i)(?:bigint|long)".r ^^^ LongType |
    "(?i)binary".r ^^^ BinaryType |
    "(?i)boolean".r ^^^ BooleanType |
    fixedDecimalType |
    "(?i)decimal".r ^^^ DecimalType.USER_DEFAULT |
    "(?i)date".r ^^^ DateType |
    "(?i)timestamp".r ^^^ TimestampType |
    varchar
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org