[jira] [Created] (SPARK-22479) SaveIntoDataSourceCommand logs jdbc credentials

["Onur Satici (JIRA)" <ji...@apache.org>]

Onur Satici created SPARK-22479:
-----------------------------------

             Summary: SaveIntoDataSourceCommand logs jdbc credentials
                 Key: SPARK-22479
                 URL: https://issues.apache.org/jira/browse/SPARK-22479
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.2.0
            Reporter: Onur Satici


JDBC credentials are not redacted in plans including a 'SaveIntoDataSourceCommand'.

Steps to reproduce:
{code}
spark-shell --packages org.postgresql:postgresql:42.1.1
{code}

{code}
import org.apache.spark.sql.execution.QueryExecution
import org.apache.spark.sql.util.QueryExecutionListener
val listener = new QueryExecutionListener {
  override def onFailure(funcName: String, qe: QueryExecution, exception: Exception): Unit = {}
  override def onSuccess(funcName: String, qe: QueryExecution, duration: Long): Unit = {
    System.out.println(qe.toString())
  }
}
spark.listenerManager.register(listener)
spark.range(100).write.format("jdbc").option("url", "jdbc:postgresql:sparkdb").option("password", "pass").option("driver", "org.postgresql.Driver").option("dbtable", "test").save()
{code}
The above will yield the following plan:

{code}
== Parsed Logical Plan ==
SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver -> org.postgresql.Driver, url -> jdbc:postgresql:sparkdb, password -> pass), ErrorIfExists
   +- Range (0, 100, step=1, splits=Some(8))

== Analyzed Logical Plan ==
SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver -> org.postgresql.Driver, url -> jdbc:postgresql:sparkdb, password -> pass), ErrorIfExists
   +- Range (0, 100, step=1, splits=Some(8))

== Optimized Logical Plan ==
SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver -> org.postgresql.Driver, url -> jdbc:postgresql:sparkdb, password -> pass), ErrorIfExists
   +- Range (0, 100, step=1, splits=Some(8))

== Physical Plan ==
ExecutedCommand
   +- SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver -> org.postgresql.Driver, url -> jdbc:postgresql:sparkdb, password -> pass), ErrorIfExists
         +- Range (0, 100, step=1, splits=Some(8))
{code}








--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org