You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by hv...@apache.org on 2016/06/17 16:47:46 UTC
spark git commit: Remove non-obvious conf settings from TPCDS
benchmark
Repository: spark
Updated Branches:
refs/heads/master ef43b4ed8 -> 34d6c4cd1
Remove non-obvious conf settings from TPCDS benchmark
## What changes were proposed in this pull request?
My fault -- these 2 conf entries are mysteriously hidden inside the benchmark code and makes it non-obvious to disable whole stage codegen and/or the vectorized parquet reader.
PS: Didn't attach a JIRA as this change should otherwise be a no-op (both these conf are enabled by default in Spark)
## How was this patch tested?
N/A
Author: Sameer Agarwal <sa...@databricks.com>
Closes #13726 from sameeragarwal/tpcds-conf.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/34d6c4cd
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/34d6c4cd
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/34d6c4cd
Branch: refs/heads/master
Commit: 34d6c4cd113729fcc1d0bc1df8916d06b8854922
Parents: ef43b4e
Author: Sameer Agarwal <sa...@databricks.com>
Authored: Fri Jun 17 09:47:41 2016 -0700
Committer: Herman van Hovell <hv...@databricks.com>
Committed: Fri Jun 17 09:47:41 2016 -0700
----------------------------------------------------------------------
.../apache/spark/sql/execution/benchmark/TPCDSQueryBenchmark.scala | 2 --
1 file changed, 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/34d6c4cd/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/TPCDSQueryBenchmark.scala
----------------------------------------------------------------------
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/TPCDSQueryBenchmark.scala b/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/TPCDSQueryBenchmark.scala
index 398d8d9..957a1d6 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/TPCDSQueryBenchmark.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/TPCDSQueryBenchmark.scala
@@ -64,8 +64,6 @@ object TPCDSQueryBenchmark {
require(dataLocation.nonEmpty,
"please modify the value of dataLocation to point to your local TPCDS data")
val tableSizes = setupTables(dataLocation)
- spark.conf.set(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key, "true")
- spark.conf.set(SQLConf.WHOLESTAGE_CODEGEN_ENABLED.key, "true")
queries.foreach { name =>
val queryString = fileToString(new File(Thread.currentThread().getContextClassLoader
.getResource(s"tpcds/$name.sql").getFile))
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org