[jira] [Created] (SPARK-50364) Row.jsonValue is not able to serialize LocalDateTime

["Karim Ramadan (Jira)" <ji...@apache.org>]

Karim Ramadan created SPARK-50364:
-------------------------------------

             Summary: Row.jsonValue is not able to serialize LocalDateTime
                 Key: SPARK-50364
                 URL: https://issues.apache.org/jira/browse/SPARK-50364
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 3.5.3
            Reporter: Karim Ramadan


Trying to serialize to JSON a Row containing a column of type _TimestampNTZType_ ({_}LocalDateTime{_}) with methods:
{code:java}
def json: String = compact(jsonValue)
def prettyJson: String = pretty(render(jsonValue)) 
{code}
Will fail with:
{code:java}
[FAILED_ROW_TO_JSON] Failed to convert the row value '2018-05-14T12:13' of the class class java.time.LocalDateTime to the target SQL type "TIMESTAMPNTZTYPE" in the JSON format. SQLSTATE: 2203G
org.apache.spark.SparkIllegalArgumentException: [FAILED_ROW_TO_JSON] Failed to convert the row value '2018-05-14T12:13' of the class class java.time.LocalDateTime to the target SQL type "TIMESTAMPNTZTYPE" in the JSON format. SQLSTATE: 2203G
	at org.apache.spark.sql.Row.toJson$1(Row.scala:663)
	at org.apache.spark.sql.Row.toJson$1(Row.scala:651)
	at org.apache.spark.sql.Row.jsonValue(Row.scala:665)
	at org.apache.spark.sql.Row.jsonValue$(Row.scala:598)
	at org.apache.spark.sql.catalyst.expressions.GenericRow.jsonValue(rows.scala:28)
	at org.apache.spark.sql.RowJsonSuite.$anonfun$testJson$1(RowJsonSuite.scala:41)
{code}
This happens because the toJson method inside of the class is missing a serializer for the LocalDateTime type
{code:java}
    def toJson(value: Any, dataType: DataType): JValue = (value, dataType) match {
      case (null, _) => JNull
      case (b: Boolean, _) => JBool(b)
      case (b: Byte, _) => JLong(b)
      case (s: Short, _) => JLong(s)
      case (i: Int, _) => JLong(i)
      case (l: Long, _) => JLong(l)
...
{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org