[jira] [Created] (SPARK-28032) DataFrame.saveAsTable( in AVRO format with Timestamps create bad Hive tables

["Mathew Wicks (JIRA)" <ji...@apache.org>]

Mathew Wicks created SPARK-28032:
------------------------------------

             Summary: DataFrame.saveAsTable( in AVRO format with Timestamps create bad Hive tables
                 Key: SPARK-28032
                 URL: https://issues.apache.org/jira/browse/SPARK-28032
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.4.3
         Environment: Spark 2.4.3

Hive 1.1.0
            Reporter: Mathew Wicks


I am not sure if it's my very old version of Hive (1.1.0), but when I use the following code, I end up with a table which Spark can read, but Hive cannot.

That is to say, when writing AVRO format tables, they cannot be read in Hive if they contain timestamp types.

*Hive error:*
{code:java}
Error while compiling statement: FAILED: UnsupportedOperationException timestamp is not supported.
{code}
*Spark Code:*
{code:java}
import java.sql.Timestamp
import spark.implicits._

val currentTime = new Timestamp(System.currentTimeMillis())
 
val df = Seq(
 (currentTime)
).toDF()

df.write.mode("overwrite").format("avro").saveAsTable("database.table_name")
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org