[jira] [Resolved] (SPARK-1922) hql query throws "RuntimeException: Unsupported dataType" if struct field of a table has a column with underscore in name

["Reynold Xin (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Reynold Xin resolved SPARK-1922.
--------------------------------

    Resolution: Fixed

> hql query throws "RuntimeException: Unsupported dataType" if struct field of a table has a column with underscore in name
> -------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-1922
>                 URL: https://issues.apache.org/jira/browse/SPARK-1922
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.0.0
>            Reporter: llai
>            Assignee: llai
>             Fix For: 1.1.0, 1.0.1
>
>
> If table A has a struct field A_strct <a_col: string>, when doing an hql query like "select", "RuntimeException: Unsupported dataType" is thrown.
> Running a query wiht "sbt/sbt hive/console":
> {code}
> scala> hql("SELECT utc_time, pkg FROM pkg_table where year=2014 and month=1 limit 10").collect().foreach(x => println(x(1)))
> {code}
> Console output:
> {code}
> 14/05/25 19:50:27 INFO parse.ParseDriver: Parsing command: SELECT utc_time, pkg FROM pkg_table where year=2014 and month=1 limit 10
> 14/05/25 19:50:27 INFO parse.ParseDriver: Parse Completed
> 14/05/25 19:50:28 INFO analysis.Analyzer: Max iterations (2) reached for batch MultiInstanceRelations
> 14/05/25 19:50:28 INFO analysis.Analyzer: Max iterations (2) reached for batch CaseInsensitiveAttributeReferences
> 14/05/25 19:50:28 INFO hive.metastore: Trying to connect to metastore with URI thrift://xxxxx
> 14/05/25 19:50:28 INFO hive.metastore: Waiting 1 seconds before next connection attempt.
> 14/05/25 19:50:29 INFO hive.metastore: Connected to metastore.
> java.lang.RuntimeException: Unsupported dataType: struct<adv_id:string,u_lat:double,u_lon:double>
> 	at scala.sys.package$.error(package.scala:27)
> 	at org.apache.spark.sql.hive.HiveMetastoreTypes$.toDataType(HiveMetastoreCatalog.scala:219)
> 	at org.apache.spark.sql.hive.MetastoreRelation$SchemaAttribute.toAttribute(HiveMetastoreCatalog.scala:273)
> 	at org.apache.spark.sql.hive.MetastoreRelation$$anonfun$8.apply(HiveMetastoreCatalog.scala:283)
> 	at org.apache.spark.sql.hive.MetastoreRelation$$anonfun$8.apply(HiveMetastoreCatalog.scala:283)
> 	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)