[jira] [Created] (SPARK-25554) Avro logical types get ignored in SchemaConverters.toSqlType

["Yanan Li (JIRA)" <ji...@apache.org>]

Yanan Li created SPARK-25554:
--------------------------------

             Summary: Avro logical types get ignored in SchemaConverters.toSqlType
                 Key: SPARK-25554
                 URL: https://issues.apache.org/jira/browse/SPARK-25554
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.3.0
         Environment: Below is the maven dependencies:
{code:java}
<dependency>
    <groupId>org.apache.avro</groupId>
    <artifactId>avro</artifactId>
    <version>1.8.2</version>
</dependency>

<dependency>
    <groupId>com.databricks</groupId>
    <artifactId>spark-avro_2.11</artifactId>
    <version>4.0.0</version>
</dependency>

<!-- spark denpendencies -->
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.11</artifactId>
    <version>2.3.0</version>
</dependency>
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-sql_2.11</artifactId>
    <version>2.3.0</version>
</dependency>
{code}
            Reporter: Yanan Li


Having Avro schema defined as follow:
{code:java}
{
   "namespace": "com.xxx.avro",
   "name": "Book",
   "type": "record",
   "fields": [{
         "name": "name",
         "type": ["null", "string"],
         "default": null
      }, {
         "name": "author",
         "type": ["null", "string"],
         "default": null
      }, {
         "name": "published_date",
         "type": ["null", {"type": "int", "logicalType": "date"}],
         "default": null
      }
   ]
}
{code}
Spark Schema converted from above Avro schema, logical type "date" gets ignored.
{code:java}
StructType(StructField(name,StringType,true),StructField(author,StringType,true),StructField(published_date,IntegerType,true))
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org