You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Felix Cheung (JIRA)" <ji...@apache.org> on 2017/11/17 05:30:00 UTC
[jira] [Updated] (SPARK-22479) SaveIntoDataSourceCommand logs jdbc
credentials
[ https://issues.apache.org/jira/browse/SPARK-22479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Felix Cheung updated SPARK-22479:
---------------------------------
Target Version/s: 2.3.0, 2.2.2
> SaveIntoDataSourceCommand logs jdbc credentials
> -----------------------------------------------
>
> Key: SPARK-22479
> URL: https://issues.apache.org/jira/browse/SPARK-22479
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.2.0
> Reporter: Onur Satici
> Assignee: Onur Satici
> Fix For: 2.3.0, 2.2.2
>
>
> JDBC credentials are not redacted in plans including a 'SaveIntoDataSourceCommand'.
> Steps to reproduce:
> {code}
> spark-shell --packages org.postgresql:postgresql:42.1.1
> {code}
> {code}
> import org.apache.spark.sql.execution.QueryExecution
> import org.apache.spark.sql.util.QueryExecutionListener
> val listener = new QueryExecutionListener {
> override def onFailure(funcName: String, qe: QueryExecution, exception: Exception): Unit = {}
> override def onSuccess(funcName: String, qe: QueryExecution, duration: Long): Unit = {
> System.out.println(qe.toString())
> }
> }
> spark.listenerManager.register(listener)
> spark.range(100).write.format("jdbc").option("url", "jdbc:postgresql:sparkdb").option("password", "pass").option("driver", "org.postgresql.Driver").option("dbtable", "test").save()
> {code}
> The above will yield the following plan:
> {code}
> == Parsed Logical Plan ==
> SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver -> org.postgresql.Driver, url -> jdbc:postgresql:sparkdb, password -> pass), ErrorIfExists
> +- Range (0, 100, step=1, splits=Some(8))
> == Analyzed Logical Plan ==
> SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver -> org.postgresql.Driver, url -> jdbc:postgresql:sparkdb, password -> pass), ErrorIfExists
> +- Range (0, 100, step=1, splits=Some(8))
> == Optimized Logical Plan ==
> SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver -> org.postgresql.Driver, url -> jdbc:postgresql:sparkdb, password -> pass), ErrorIfExists
> +- Range (0, 100, step=1, splits=Some(8))
> == Physical Plan ==
> ExecutedCommand
> +- SaveIntoDataSourceCommand jdbc, Map(dbtable -> test10, driver -> org.postgresql.Driver, url -> jdbc:postgresql:sparkdb, password -> pass), ErrorIfExists
> +- Range (0, 100, step=1, splits=Some(8))
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org