You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Shiv Prashant Sood (JIRA)" <ji...@apache.org> on 2019/06/24 22:09:00 UTC
[jira] [Created] (SPARK-28152) ShortType and FloatTypes are not
correctly mapped to right JDBC types when using JDBC connector
Shiv Prashant Sood created SPARK-28152:
------------------------------------------
Summary: ShortType and FloatTypes are not correctly mapped to right JDBC types when using JDBC connector
Key: SPARK-28152
URL: https://issues.apache.org/jira/browse/SPARK-28152
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.4.3, 3.0.0
Reporter: Shiv Prashant Sood
ShortType and FloatTypes are not correctly mapped to right JDBC types when using JDBC connector. This results in tables or spark data frame being created with unintended types.
Some example issue
* Write from df with column type results in a SQL table of with column type as INTEGER as opposed to SMALLINT. Thus a larger table that expected.
* read results in a dataframe with type INTEGER as opposed to ShortType
FloatTypes have a issue with read path. In the write path Spark data type 'FloatType' is correctly mapped to JDBC equivalent data type 'Real'. But in the read path when JDBC data types need to be converted to Catalyst data types ( getCatalystType) 'Real' gets incorrectly gets mapped to 'DoubleType' rather than 'FloatType'.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org