[jira] [Commented] (SPARK-8386) DataFrame and JDBC regression

["Huaxin Gao (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947736#comment-14947736 ] 

Huaxin Gao commented on SPARK-8386:
-----------------------------------

I looked the code, it has this
  @deprecated("Use write.jdbc()", "1.4.0")
  def insertIntoJDBC(url: String, table: String, overwrite: Boolean): Unit = {
    val w = if (overwrite) write.mode(SaveMode.Overwrite) else write
    w.jdbc(url, table, new Properties)
  }
if overwrite is false, it doesn't set and the default is SaveMode.ErrorIfExists.
It seems to me that if the overwrite is false, the mode should be set to be Append

  def insertIntoJDBC(url: String, table: String, overwrite: Boolean): Unit = {
    val w = if (overwrite) write.mode(SaveMode.Overwrite) else write.mode(SaveMode.Append)
    w.jdbc(url, table, new Properties)
  }



> DataFrame and JDBC regression
> -----------------------------
>
>                 Key: SPARK-8386
>                 URL: https://issues.apache.org/jira/browse/SPARK-8386
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.4.0
>         Environment: RHEL 7.1
>            Reporter: Peter Haumer
>            Priority: Critical
>
> I have an ETL app that appends to a JDBC table new results found at each run.  In 1.3.1 I did this:
> testResultsDF.insertIntoJDBC(CONNECTION_URL, TABLE_NAME, false);
> When I do this now in 1.4 it complains that the "object" 'TABLE_NAME' already exists. I get this even if I switch the overwrite to true.  I also tried this now:
> testResultsDF.write().mode(SaveMode.Append).jdbc(CONNECTION_URL, TABLE_NAME, connectionProperties);
> getting the same error. It works running the first time creating the new table and adding data successfully. But, running it a second time it (the jdbc driver) will tell me that the table already exists. Even SaveMode.Overwrite will give me the same error. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org