You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by mallman <gi...@git.apache.org> on 2018/09/11 15:18:52 UTC
[GitHub] spark pull request #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite....
GitHub user mallman opened a pull request:
https://github.com/apache/spark/pull/22394
[SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, move calls to `withSQLConf` inside calls to `test`
(Link to Jira: https://issues.apache.org/jira/browse/SPARK-25406)
## What changes were proposed in this pull request?
The current use of `withSQLConf` in `ParquetSchemaPruningSuite.scala` is incorrect. The desired configuration settings are not being set when running the test cases.
This PR fixes that defective usage and addresses the test failures that were previously masked by that defect.
## How was this patch tested?
I added code to relevant test cases to print the expected SQL configuration settings and found that the settings were not being set as expected. When I changed the order of calls to `test` and `withSQLConf` I found that the configuration settings were being set as expected.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/VideoAmp/spark-public spark-25406-fix_broken_schema_pruning_tests
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22394.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22394
----
commit 8cca76b7ec31c1c0cff1bd5b6772b43b49c30147
Author: Michael Allman <ms...@...>
Date: 2018-09-11T15:10:08Z
For ParquetSchemaPruningSuite.scala, move calls to `withSQLConf` inside
calls to `test`
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95992/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/22394
LGTM. Merged into master and branch 2.4. Thanks.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95951/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22394
**[Test build #95970 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95970/testReport)** for PR 22394 at commit [`c759aea`](https://github.com/apache/spark/commit/c759aeabc8b3fb3c426e432bff794deddef3e05e).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22394
**[Test build #95992 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95992/testReport)** for PR 22394 at commit [`1c89637`](https://github.com/apache/spark/commit/1c8963702f53c9d25cb741ef9c11bceef2c39188).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22394
**[Test build #95997 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95997/testReport)** for PR 22394 at commit [`1c89637`](https://github.com/apache/spark/commit/1c8963702f53c9d25cb741ef9c11bceef2c39188).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22394
**[Test build #96029 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96029/testReport)** for PR 22394 at commit [`1c89637`](https://github.com/apache/spark/commit/1c8963702f53c9d25cb741ef9c11bceef2c39188).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite....
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22394#discussion_r216875288
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala ---
@@ -245,28 +249,32 @@ class ParquetSchemaPruningSuite
checkAnswer(query.orderBy("id"), Row(1) :: Nil)
}
- private def testMixedCasePruning(testName: String)(testThunk: => Unit) {
- withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "true",
- SQLConf.CASE_SENSITIVE.key -> "true") {
- test(s"Spark vectorized reader - case-sensitive parser - mixed-case schema - $testName") {
- withMixedCaseData(testThunk)
+ private def testExactCasePruning(testName: String)(testThunk: => Unit) {
--- End diff --
testCaseSensitivePruning?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by dilipbiswal <gi...@git.apache.org>.
Github user dilipbiswal commented on the issue:
https://github.com/apache/spark/pull/22394
retest this please
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22394
retest this please
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/22394
retest this please.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by mallman <gi...@git.apache.org>.
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/22394
I'm working on fixing these test failures now. Hopefully I'll have something pushed soon.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite....
Posted by mallman <gi...@git.apache.org>.
Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/22394#discussion_r217052036
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala ---
@@ -245,28 +249,32 @@ class ParquetSchemaPruningSuite
checkAnswer(query.orderBy("id"), Row(1) :: Nil)
}
- private def testMixedCasePruning(testName: String)(testThunk: => Unit) {
- withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "true",
- SQLConf.CASE_SENSITIVE.key -> "true") {
- test(s"Spark vectorized reader - case-sensitive parser - mixed-case schema - $testName") {
- withMixedCaseData(testThunk)
+ private def testExactCasePruning(testName: String)(testThunk: => Unit) {
--- End diff --
The method names are meant to clarify the kind of queries being tested, not the setting of `SQLConf.CASE_SENSITIVE.key`. In this case, `testExactCasePruning` is supposed to mean that we're passing in a test in which the column names in the query are exactly the same as the column names in the relation.
It's not a very good name in that sense. I'll try to make it clearer and add a code comment to clarify.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3030/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96029/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3071/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22394
**[Test build #95951 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95951/testReport)** for PR 22394 at commit [`8cca76b`](https://github.com/apache/spark/commit/8cca76b7ec31c1c0cff1bd5b6772b43b49c30147).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95970/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22394
**[Test build #95951 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95951/testReport)** for PR 22394 at commit [`8cca76b`](https://github.com/apache/spark/commit/8cca76b7ec31c1c0cff1bd5b6772b43b49c30147).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96018/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite....
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22394#discussion_r216903678
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala ---
@@ -245,28 +249,32 @@ class ParquetSchemaPruningSuite
checkAnswer(query.orderBy("id"), Row(1) :: Nil)
}
- private def testMixedCasePruning(testName: String)(testThunk: => Unit) {
- withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "true",
- SQLConf.CASE_SENSITIVE.key -> "true") {
- test(s"Spark vectorized reader - case-sensitive parser - mixed-case schema - $testName") {
- withMixedCaseData(testThunk)
+ private def testExactCasePruning(testName: String)(testThunk: => Unit) {
+ test(s"Spark vectorized reader - case-sensitive parser - mixed-case schema - $testName") {
+ withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "true",
+ SQLConf.CASE_SENSITIVE.key -> "true") {
+ withMixedCaseData(testThunk)
}
}
- withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "false",
- SQLConf.CASE_SENSITIVE.key -> "false") {
- test(s"Parquet-mr reader - case-insensitive parser - mixed-case schema - $testName") {
+ test(s"Parquet-mr reader - case-sensitive parser - mixed-case schema - $testName") {
+ withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "false",
+ SQLConf.CASE_SENSITIVE.key -> "true") {
withMixedCaseData(testThunk)
}
}
- withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "true",
- SQLConf.CASE_SENSITIVE.key -> "false") {
- test(s"Spark vectorized reader - case-insensitive parser - mixed-case schema - $testName") {
- withMixedCaseData(testThunk)
+ testMixedCasePruning(testName)(testThunk)
+ }
+
+ private def testMixedCasePruning(testName: String)(testThunk: => Unit) {
--- End diff --
`testMixedCasePruning` looks previously testing case sensitive case too. Now, it looks not. Would you mind if I ask the reason why?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3052/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22394
**[Test build #95992 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95992/testReport)** for PR 22394 at commit [`1c89637`](https://github.com/apache/spark/commit/1c8963702f53c9d25cb741ef9c11bceef2c39188).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite....
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22394
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite....
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22394#discussion_r216903560
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala ---
@@ -245,28 +249,32 @@ class ParquetSchemaPruningSuite
checkAnswer(query.orderBy("id"), Row(1) :: Nil)
}
- private def testMixedCasePruning(testName: String)(testThunk: => Unit) {
- withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "true",
- SQLConf.CASE_SENSITIVE.key -> "true") {
- test(s"Spark vectorized reader - case-sensitive parser - mixed-case schema - $testName") {
- withMixedCaseData(testThunk)
+ private def testExactCasePruning(testName: String)(testThunk: => Unit) {
--- End diff --
This looks testing case insensitivity too?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite....
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22394#discussion_r216875122
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala ---
@@ -245,28 +249,32 @@ class ParquetSchemaPruningSuite
checkAnswer(query.orderBy("id"), Row(1) :: Nil)
}
- private def testMixedCasePruning(testName: String)(testThunk: => Unit) {
- withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "true",
- SQLConf.CASE_SENSITIVE.key -> "true") {
- test(s"Spark vectorized reader - case-sensitive parser - mixed-case schema - $testName") {
- withMixedCaseData(testThunk)
+ private def testExactCasePruning(testName: String)(testThunk: => Unit) {
+ test(s"Spark vectorized reader - case-sensitive parser - mixed-case schema - $testName") {
+ withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "true",
+ SQLConf.CASE_SENSITIVE.key -> "true") {
+ withMixedCaseData(testThunk)
}
}
- withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "false",
- SQLConf.CASE_SENSITIVE.key -> "false") {
- test(s"Parquet-mr reader - case-insensitive parser - mixed-case schema - $testName") {
+ test(s"Parquet-mr reader - case-sensitive parser - mixed-case schema - $testName") {
+ withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "false",
+ SQLConf.CASE_SENSITIVE.key -> "true") {
withMixedCaseData(testThunk)
}
}
- withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "true",
- SQLConf.CASE_SENSITIVE.key -> "false") {
- test(s"Spark vectorized reader - case-insensitive parser - mixed-case schema - $testName") {
- withMixedCaseData(testThunk)
+ testMixedCasePruning(testName)(testThunk)
+ }
+
+ private def testMixedCasePruning(testName: String)(testThunk: => Unit) {
+ test(s"Parquet-mr reader - case-insensitive parser - mixed-case schema - $testName") {
+ withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "false",
--- End diff --
The vectorized reader cases in both `testSchemaPruning` and `testExactCasePruning` are put ahead of Parquet-mr reader cases. Shall we follow it too in `testMixedCasePruning`?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by mallman <gi...@git.apache.org>.
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/22394
FYI @viirya @dbtsai @gatorsmile @HyukjinKwon
Can I get someone's review of this PR please? The unmasked failures appear to be false positives, so no changes to the tested code are required—just changes to the tests themselves.
Thanks.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3018/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3079/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22394
**[Test build #95997 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95997/testReport)** for PR 22394 at commit [`1c89637`](https://github.com/apache/spark/commit/1c8963702f53c9d25cb741ef9c11bceef2c39188).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by mallman <gi...@git.apache.org>.
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/22394
Retest this please.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22394
**[Test build #95952 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95952/testReport)** for PR 22394 at commit [`c759aea`](https://github.com/apache/spark/commit/c759aeabc8b3fb3c426e432bff794deddef3e05e).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22394
**[Test build #95952 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95952/testReport)** for PR 22394 at commit [`c759aea`](https://github.com/apache/spark/commit/c759aeabc8b3fb3c426e432bff794deddef3e05e).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22394
**[Test build #96018 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96018/testReport)** for PR 22394 at commit [`1c89637`](https://github.com/apache/spark/commit/1c8963702f53c9d25cb741ef9c11bceef2c39188).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22394
**[Test build #95970 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95970/testReport)** for PR 22394 at commit [`c759aea`](https://github.com/apache/spark/commit/c759aeabc8b3fb3c426e432bff794deddef3e05e).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95952/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22394
**[Test build #96029 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96029/testReport)** for PR 22394 at commit [`1c89637`](https://github.com/apache/spark/commit/1c8963702f53c9d25cb741ef9c11bceef2c39188).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by mallman <gi...@git.apache.org>.
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/22394
> Hey @mallman, let's just target to fix the problem in the JIRA without other refactorings.
The refactorings I've made address the problem directly. Hopefully that will be clearer with my most recent commit.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22394
Hey @mallman, let's just target to fix the problem in the JIRA without other refactorings.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3017/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite....
Posted by mallman <gi...@git.apache.org>.
Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/22394#discussion_r217055207
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala ---
@@ -245,28 +249,32 @@ class ParquetSchemaPruningSuite
checkAnswer(query.orderBy("id"), Row(1) :: Nil)
}
- private def testMixedCasePruning(testName: String)(testThunk: => Unit) {
- withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "true",
- SQLConf.CASE_SENSITIVE.key -> "true") {
- test(s"Spark vectorized reader - case-sensitive parser - mixed-case schema - $testName") {
- withMixedCaseData(testThunk)
+ private def testExactCasePruning(testName: String)(testThunk: => Unit) {
+ test(s"Spark vectorized reader - case-sensitive parser - mixed-case schema - $testName") {
+ withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "true",
+ SQLConf.CASE_SENSITIVE.key -> "true") {
+ withMixedCaseData(testThunk)
}
}
- withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "false",
- SQLConf.CASE_SENSITIVE.key -> "false") {
- test(s"Parquet-mr reader - case-insensitive parser - mixed-case schema - $testName") {
+ test(s"Parquet-mr reader - case-sensitive parser - mixed-case schema - $testName") {
+ withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "false",
+ SQLConf.CASE_SENSITIVE.key -> "true") {
withMixedCaseData(testThunk)
}
}
- withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "true",
- SQLConf.CASE_SENSITIVE.key -> "false") {
- test(s"Spark vectorized reader - case-insensitive parser - mixed-case schema - $testName") {
- withMixedCaseData(testThunk)
+ testMixedCasePruning(testName)(testThunk)
+ }
+
+ private def testMixedCasePruning(testName: String)(testThunk: => Unit) {
--- End diff --
Previously, this method ran `testThunk` with `SQLConf.CASE_SENSITIVE.key` set to true and false. That was a mistake and incorrect. For example, the query
select col1, col2.b from mixedcase
will fail if `SQLConf.CASE_SENSITIVE.key` is set to true. That mistake was causing 6 test cases to fail. Therefore, I moved the code that tests with a case-sensitive parser out of `testMixedCasePruning` into `testExactCasePruning` and included a call to `testMixedCasePruning` in `testExactCasePruning`.
I'll push a commit that refactors the method names and add code comments that will make this clearer.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite....
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22394#discussion_r216874181
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala ---
@@ -156,20 +156,24 @@ class ParquetSchemaPruningSuite
}
private def testSchemaPruning(testName: String)(testThunk: => Unit) {
- withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "true") {
- test(s"Spark vectorized reader - without partition data column - $testName") {
+ test(s"Spark vectorized reader - without partition data column - $testName") {
+ withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "true") {
--- End diff --
Yea, the call of `test` only registers the test function, it is not actually invoked within the `withSQLConf`. We shouldn't wrap `test` inside `withSQLConf`. Good catch.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22394
**[Test build #96018 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96018/testReport)** for PR 22394 at commit [`1c89637`](https://github.com/apache/spark/commit/1c8963702f53c9d25cb741ef9c11bceef2c39188).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3048/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95997/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite.scala, ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22394
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22394: [SPARK-25406][SQL] For ParquetSchemaPruningSuite....
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22394#discussion_r216875341
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala ---
@@ -245,28 +249,32 @@ class ParquetSchemaPruningSuite
checkAnswer(query.orderBy("id"), Row(1) :: Nil)
}
- private def testMixedCasePruning(testName: String)(testThunk: => Unit) {
- withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "true",
- SQLConf.CASE_SENSITIVE.key -> "true") {
- test(s"Spark vectorized reader - case-sensitive parser - mixed-case schema - $testName") {
- withMixedCaseData(testThunk)
+ private def testExactCasePruning(testName: String)(testThunk: => Unit) {
+ test(s"Spark vectorized reader - case-sensitive parser - mixed-case schema - $testName") {
+ withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "true",
+ SQLConf.CASE_SENSITIVE.key -> "true") {
+ withMixedCaseData(testThunk)
}
}
- withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "false",
- SQLConf.CASE_SENSITIVE.key -> "false") {
- test(s"Parquet-mr reader - case-insensitive parser - mixed-case schema - $testName") {
+ test(s"Parquet-mr reader - case-sensitive parser - mixed-case schema - $testName") {
+ withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "false",
+ SQLConf.CASE_SENSITIVE.key -> "true") {
withMixedCaseData(testThunk)
}
}
- withSQLConf(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "true",
- SQLConf.CASE_SENSITIVE.key -> "false") {
- test(s"Spark vectorized reader - case-insensitive parser - mixed-case schema - $testName") {
- withMixedCaseData(testThunk)
+ testMixedCasePruning(testName)(testThunk)
+ }
+
+ private def testMixedCasePruning(testName: String)(testThunk: => Unit) {
--- End diff --
testCaseInSensitivePruning?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org