You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Michael Armbrust (JIRA)" <ji...@apache.org> on 2015/08/29 22:22:45 UTC
[jira] [Updated] (SPARK-10170) Writing from data frame into db2
database using jdbc data source api fails with error for string, and
boolean column types.
[ https://issues.apache.org/jira/browse/SPARK-10170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Armbrust updated SPARK-10170:
-------------------------------------
Target Version/s: 1.6.0
> Writing from data frame into db2 database using jdbc data source api fails with error for string, and boolean column types.
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-10170
> URL: https://issues.apache.org/jira/browse/SPARK-10170
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.4.1, 1.5.0
> Reporter: Suresh Thalamati
>
> Repro :
> -- start spark shell with classpath set to the db2 jdbc driver.
> SPARK_CLASSPATH=~/myjars/db2jcc.jar ./spark-shell
>
> // set connetion properties
> val properties = new java.util.Properties()
> properties.setProperty("user" , "user")
> properties.setProperty("password" , "password")
> // load the driver.
> Class.forName("com.ibm.db2.jcc.DB2Driver").newInstance
> // create data frame with a String type
> val empdf = sc.parallelize( Array((1,"John"), (2,"Mike"))).toDF("id", "name" )
> // write the data frame. this will fail with error.
> empdf.write.jdbc("jdbc:db2://bdvs150.svl.ibm.com:60000/SAMPLE:retrieveMessagesFromServerOnGetMessage=true;", "emp_data", properties)
> Error :
> com.ibm.db2.jcc.am.SqlSyntaxErrorException: TEXT
> at com.ibm.db2.jcc.am.fd.a(fd.java:679)
> at com.ibm.db2.jcc.am.fd.a(fd.java:60)
> ..
> // create data frame with String , and Boolean types
> val empdf = sc.parallelize( Array((1,"true".toBoolean ), (2, "false".toBoolean ))).toDF("id", "isManager")
> // write the data frame. this will fail with error.
> empdf.write.jdbc("jdbc:db2://<server>:<port> /SAMPLE:retrieveMessagesFromServerOnGetMessage=true;", "emp_data", properties)
> Error :
> com.ibm.db2.jcc.am.SqlSyntaxErrorException: TEXT
> at com.ibm.db2.jcc.am.fd.a(fd.java:679)
> at com.ibm.db2.jcc.am.fd.a(fd.java:60)
> Write is failing because by default JDBC data source implementation generating table schema with unsupported data types TEXT for String, and BIT1(1) for Boolean. I think String type should get mapped to CLOB/VARCHAR, and boolean type should be mapped to CHAR(1) for DB2 database.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org