[GitHub] [spark] LuciferYang commented on a diff in pull request #39838: [SPARK-42270][SQL] Sort merge join may oom when right match rows are very large

["LuciferYang (via GitHub)" <gi...@apache.org>]

LuciferYang commented on code in PR #39838:
URL: https://github.com/apache/spark/pull/39838#discussion_r1096072047


##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -2131,6 +2131,14 @@ object SQLConf {
       .booleanConf
       .createWithDefault(true)
 
+  val SORT_MERGE_JOIN_BATCH_SIZE =

Review Comment:
   If I understand correctly, is this new configuration used to avoid OOM? I think there should add  a new case to show the original oom will not happen after this pr



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org