You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "melin (Jira)" <ji...@apache.org> on 2023/05/24 15:45:00 UTC

[jira] [Updated] (SPARK-43776) [BUG] MySQL jdbc cursor has not taken effect

     [ https://issues.apache.org/jira/browse/SPARK-43776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

melin updated SPARK-43776:
--------------------------
    Description: 
JDBCRDD.scala:  stmt.setFetchSize(options.fetchSize)

 
JdbcRDD.scala:
{code:java}
if (url.startsWith("jdbc:mysql:"))
{ // setFetchSize(Integer.MIN_VALUE) is a mysql driver specific way to force // streaming results, rather than pulling entire resultset into memory. // See the below URL // dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-implementation-notes.html stmt.setFetchSize(Integer.MIN_VALUE) }
else
{ stmt.setFetchSize(100) }
{code}
 

 

  was:
JDBCRDD.scala:  stmt.setFetchSize(options.fetchSize)

 
JdbcRDD.scala:
```
if (url.startsWith("jdbc:mysql:")) {
// setFetchSize(Integer.MIN_VALUE) is a mysql driver specific way to force
// streaming results, rather than pulling entire resultset into memory.
// See the below URL
// dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-implementation-notes.html

stmt.setFetchSize(Integer.MIN_VALUE)
} else {
stmt.setFetchSize(100)
}
```


> [BUG] MySQL jdbc cursor has not taken effect
> --------------------------------------------
>
>                 Key: SPARK-43776
>                 URL: https://issues.apache.org/jira/browse/SPARK-43776
>             Project: Spark
>          Issue Type: New Feature
>          Components: SQL
>    Affects Versions: 3.4.1, 3.5.0
>            Reporter: melin
>            Priority: Major
>
> JDBCRDD.scala:  stmt.setFetchSize(options.fetchSize)
>  
> JdbcRDD.scala:
> {code:java}
> if (url.startsWith("jdbc:mysql:"))
> { // setFetchSize(Integer.MIN_VALUE) is a mysql driver specific way to force // streaming results, rather than pulling entire resultset into memory. // See the below URL // dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-implementation-notes.html stmt.setFetchSize(Integer.MIN_VALUE) }
> else
> { stmt.setFetchSize(100) }
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org