You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Tor Myklebust (JIRA)" <ji...@apache.org> on 2017/11/04 00:29:00 UTC
[jira] [Comment Edited] (SPARK-22441) JDBC REAL type is mapped to
Double instead of Float
[ https://issues.apache.org/jira/browse/SPARK-22441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16238628#comment-16238628 ]
Tor Myklebust edited comment on SPARK-22441 at 11/4/17 12:28 AM:
-----------------------------------------------------------------
I wrote this code a while ago, but I don't think REAL -> double was an accident.
The JDBC spec wasn't the reference for this implementation since a number of things it says about datatypes contradict the behaviour of actual databases. Which is to say that some database used REAL for 64-bit floats.
Postgres and H2 document that a REAL is a 32-bit float. MySQL docs say that "MySQL also treats REAL as a synonym for DOUBLE PRECISION." I haven't got a quick way to check, but I'd speculate that MySQL's JDBC connector advertises a MySQL REAL column as being of JDBC REAL type.
Re SQL FLOAT and DOUBLE types:
- H2 uses FLOAT and DOUBLE interchangeably according to its documentation.
- "PostgreSQL also supports the SQL-standard notations float and float(p) for specifying inexact numeric types. Here, p specifies the minimum acceptable precision in binary digits. PostgreSQL accepts float(1) to float(24) as selecting the real type, while float(25) to float(53) select double precision. Values of p outside the allowed range draw an error. float with no precision specified is taken to mean double precision."
- According to its docs, MySQL does something similar to Postgres for FLOAT.
Depending on how this is exposed via JDBC, perhaps JDBC FLOAT should map to double as well.
was (Author: tmyklebu):
I wrote this code a while ago, but I don't think REAL -> double was an accident.
The JDBC spec wasn't the reference for this implementation since a number of things it says about datatypes contradict the behaviour of actual databases. Which is to say that some database used REAL for 64-bit floats.
Postgres and H2 document that a REAL is a 32-bit float. MySQL docs say that "MySQL also treats REAL as a synonym for DOUBLE PRECISION." I haven't got a quick way to check, but I'd speculate that MySQL's JDBC connector advertises a MySQL REAL column as being of JDBC REAL type.
> JDBC REAL type is mapped to Double instead of Float
> ---------------------------------------------------
>
> Key: SPARK-22441
> URL: https://issues.apache.org/jira/browse/SPARK-22441
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.2.0
> Reporter: Hongbo
> Priority: Minor
>
> In [JDBC Specification|http://download.oracle.com/otn-pub/jcp/jdbc-4_1-mrel-eval-spec/jdbc4.1-fr-spec.pdf], REAL should be mapped to Float.
> But now, it's mapped to Double:
> [https://github.com/apache/spark/blob/bc7ca9786e162e33f29d57c4aacb830761b97221/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala#L220]
> Should it be changed according to the specification?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org