[GitHub] caneGuy commented on a change in pull request #23670: [SPARK-26601][SQL] Make broadcast-exchange thread pool configurable

[GitBox <gi...@apache.org>]

caneGuy commented on a change in pull request #23670: [SPARK-26601][SQL] Make broadcast-exchange thread pool configurable
URL: https://github.com/apache/spark/pull/23670#discussion_r257537551
 
 

 ##########
 File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala
 ##########
 @@ -132,4 +132,16 @@ object StaticSQLConf {
       .intConf
       .createWithDefault(1000)
 
+  val BROADCAST_EXCHANGE_MAX_THREAD_THREASHOLD =
+    buildStaticConf("spark.sql.broadcastExchange.maxThreadThreshold")
+      .doc("The maximum degree of parallelism to fetch and broadcast the table." +
+        "If we encounter memory issue like frequently full GC or OOM when broadcast table " +
+        "we can decrease this number in order to reduce memory usage." +
 
 Review comment:
   Done @maropu 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org