[GitHub] [spark] huaxingao commented on a diff in pull request #36405: [SPARK-39065][SQL] DS V2 Limit push-down should avoid out of memory

[GitBox <gi...@apache.org>]

huaxingao commented on code in PR #36405:
URL: https://github.com/apache/spark/pull/36405#discussion_r868169150


##########
docs/sql-data-sources-jdbc.md:
##########
@@ -286,6 +286,15 @@ logging into the data sources.
     <td>read</td>
   </tr>
 
+  <tr>
+    <td><code>maxPushDownLimit</code></td>

Review Comment:
   I feel it's a little bit overkill to add a new property. How likely will user have a very large LIMIT? Instead of adding a new property, can we just clearly document `pushDownLimit` to say the user needs to set `LIMIT` to a reasonable value when using `pushDownLimit`, otherwise an OOM might happen at the executor side if the `LIMIT` value if too large?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org