You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/12/04 08:07:30 UTC
[GitHub] [spark] cloud-fan commented on a change in pull request #30559: [SPARK-33617][SQL] Add default parallelism configuration for Spark SQL queries
cloud-fan commented on a change in pull request #30559:
URL: https://github.com/apache/spark/pull/30559#discussion_r535908701
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -386,6 +386,14 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
+ val DEFAULT_PARALLELISM = buildConf("spark.sql.default.parallelism")
+ .doc("The number of parallelism for Spark SQL, the default value is " +
+ "`spark.default.parallelism`.")
+ .version("3.1.0")
+ .intConf
+ .checkValue(_ > 0, "The value of spark.sql.default.parallelism must be positive.")
+ .createWithDefault(1)
Review comment:
shall we make this optional? so that we can fallback to `spark.default.parallelism` at caller side.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org