You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Juliusz Sompolski (Jira)" <ji...@apache.org> on 2020/05/28 19:13:00 UTC
[jira] [Commented] (SPARK-31859) Thriftserver with
spark.sql.datetime.java8API.enabled=true
[ https://issues.apache.org/jira/browse/SPARK-31859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17119002#comment-17119002 ]
Juliusz Sompolski commented on SPARK-31859:
-------------------------------------------
Actually, it's already in HiveResult.toHiveString...
What happens is:
- its taken from SparkRow in SparkExecuteStatement.addNonNullColumnValue using "to += from.getAs[Timestamp](ordinal)". This appears to not complain that it's in fact an Instant, not Timestamp.
- in ColumnValue.timestampValue it gets turned into a String as value.toString(). That somehow also doesn't complain that the object is an Instant, not a Timestamp?
- this gets returned to the client as String, which complains that it cannot read it back into Timestamp..
I will fix it together with https://issues.apache.org/jira/browse/SPARK-31861
> Thriftserver with spark.sql.datetime.java8API.enabled=true
> ----------------------------------------------------------
>
> Key: SPARK-31859
> URL: https://issues.apache.org/jira/browse/SPARK-31859
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Juliusz Sompolski
> Priority: Major
>
> {code}
> test("spark.sql.datetime.java8API.enabled=true") {
> withJdbcStatement() { st =>
> st.execute("set spark.sql.datetime.java8API.enabled=true")
> val rs = st.executeQuery("select timestamp '2020-05-28 00:00:00'")
> rs.next()
> // scalastyle:off
> println(rs.getObject(1))
> }
> }
> {code}
> fails with
> {code}
> HiveThriftBinaryServerSuite:
> java.lang.IllegalArgumentException: Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff]
> at java.sql.Timestamp.valueOf(Timestamp.java:204)
> at org.apache.hive.jdbc.HiveBaseResultSet.evaluate(HiveBaseResultSet.java:444)
> at org.apache.hive.jdbc.HiveBaseResultSet.getColumnValue(HiveBaseResultSet.java:424)
> at org.apache.hive.jdbc.HiveBaseResultSet.getObject(HiveBaseResultSet.java:464
> {code}
> It seems it might be needed in HiveResult.toHiveString?
> cc [~maxgekk]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org