[jira] [Commented] (SPARK-37997) Allow query parameters to be passed into spark.read

["Plop (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-37997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517333#comment-17517333 ] 

Plop commented on SPARK-37997:
------------------------------

Is there a workaround for this?

> Allow query parameters to be passed into spark.read
> ---------------------------------------------------
>
>                 Key: SPARK-37997
>                 URL: https://issues.apache.org/jira/browse/SPARK-37997
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 3.2.0
>            Reporter: QFW
>            Priority: Minor
>
> This ticket is to allow query parameters to be used spark.read.
> While it is possible to inject some parameters into the query using string concatenation, this doesn't work for all data types, for example binaries. In this example, the parameter rowversion is a binary which needs to be passed into the sql query. 
> {code:java}
> _select_sql = f'SELECT * FROM dbo.Table WHERE RowVersion > {rowversion}'
> df = spark.read.format("jdbc") \
>     .option("url", "jdbc:sqlserver://databaseserver.database.windows.net;databaseName=databasename") \
>     .option("query", _select_sql) \
>     .option("username", "sql_username") \
>     .option("password", "sql_password") \
>     .option("driver", "com.microsoft.sqlserver.jdbc.SQLServerDriver") \
>     .load() {code}
> This results in the query looking this this...
> {code:java}
> SELECT * FROM dbo.Address WHERE RowVersion > bytearray(b'\x00\x00\x00\x00\x02\xdf=\xf5') {code}
> As far as I know, there is no way to do this currently.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org