You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kyuubi.apache.org by GitBox <gi...@apache.org> on 2021/09/08 03:48:16 UTC
[GitHub] [incubator-kyuubi] yaooqinn commented on issue #1018: Set spark.sql.execution.topKSortFallbackThreshold to a reasonable value
yaooqinn commented on issue #1018:
URL: https://github.com/apache/incubator-kyuubi/issues/1018#issuecomment-914888754
```diff
diff --git a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/SparkSQLEngine.scala b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/SparkSQLEngine.scala
index 6968e07c..9c2832c5 100644
--- a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/SparkSQLEngine.scala
+++ b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/SparkSQLEngine.scala
@@ -96,6 +96,7 @@ object SparkSQLEngine extends Logging {
def createSpark(): SparkSession = {
val sparkConf = new SparkConf()
+ sparkConf.setIfMissing("spark.sql.execution.topKSortFallbackThreshold", "10000")
sparkConf.setIfMissing("spark.sql.legacy.castComplexTypesToString.enabled", "true")
sparkConf.setIfMissing("spark.master", "local")
sparkConf.setIfMissing("spark.ui.port", "0")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org