You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by we...@apache.org on 2019/02/18 13:13:26 UTC
[spark] branch master updated:
[SPARK-26744][SPARK-26744][SQL][HOTFOX] Disable schema validation tests for
FileDataSourceV2 (partially revert )
This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new a0e81fc [SPARK-26744][SPARK-26744][SQL][HOTFOX] Disable schema validation tests for FileDataSourceV2 (partially revert )
a0e81fc is described below
commit a0e81fcfe8a6dbb246f8b170b6f5e203ab194d7e
Author: Hyukjin Kwon <gu...@apache.org>
AuthorDate: Mon Feb 18 21:13:00 2019 +0800
[SPARK-26744][SPARK-26744][SQL][HOTFOX] Disable schema validation tests for FileDataSourceV2 (partially revert )
## What changes were proposed in this pull request?
This PR partially revert SPARK-26744.
https://github.com/apache/spark/commit/60caa92deaf6941f58da82dcc0962ebf3a598ced and https://github.com/apache/spark/commit/4dce45a5992e6a89a26b5a0739b33cfeaf979208 were merged at similar time range independently. So the test failures were not caught.
- https://github.com/apache/spark/commit/60caa92deaf6941f58da82dcc0962ebf3a598ced happened to add a schema reading logic in writing path for overwrite mode as well.
- https://github.com/apache/spark/commit/4dce45a5992e6a89a26b5a0739b33cfeaf979208 added some tests with overwrite modes with migrated ORC v2.
And the tests looks starting to fail.
I guess the discussion won't be short (see https://github.com/apache/spark/pull/23606#discussion_r257675083) and this PR proposes to disable the tests added at https://github.com/apache/spark/commit/4dce45a5992e6a89a26b5a0739b33cfeaf979208 to unblock other PRs for now.
## How was this patch tested?
Existing tests.
Closes #23828 from HyukjinKwon/SPARK-26744.
Authored-by: Hyukjin Kwon <gu...@apache.org>
Signed-off-by: Wenchen Fan <we...@databricks.com>
---
.../test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala
index e0c0484..58522f7 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala
@@ -329,7 +329,7 @@ class FileBasedDataSourceSuite extends QueryTest with SharedSQLContext with Befo
test("SPARK-24204 error handling for unsupported Interval data types - csv, json, parquet, orc") {
withTempDir { dir =>
val tempDir = new File(dir, "files").getCanonicalPath
- Seq(true, false).foreach { useV1 =>
+ Seq(true).foreach { useV1 =>
val useV1List = if (useV1) {
"orc"
} else {
@@ -374,7 +374,7 @@ class FileBasedDataSourceSuite extends QueryTest with SharedSQLContext with Befo
}
test("SPARK-24204 error handling for unsupported Null data types - csv, parquet, orc") {
- Seq(true, false).foreach { useV1 =>
+ Seq(true).foreach { useV1 =>
val useV1List = if (useV1) {
"orc"
} else {
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org