You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by dongjoon-hyun <gi...@git.apache.org> on 2017/12/12 01:57:23 UTC
[GitHub] spark pull request #19948: [SPARK-19809][SQL][TEST] NullPointerException on ...
GitHub user dongjoon-hyun opened a pull request:
https://github.com/apache/spark/pull/19948
[SPARK-19809][SQL][TEST] NullPointerException on zero-size ORC file
## What changes were proposed in this pull request?
Until 2.2.1, Spark raises `RuntimeException` on zero-size ORC files. Usually, these zero-size ORC files are generated from 3rd-party apps like Flume. Since Apache ORC 1.4.1, ORC library supports this as an empty file correctly, ORC-162.
```scala
scala> sql("create table empty_orc(a int) stored as orc location '/tmp/empty_orc'")
$ touch /tmp/empty_orc/zero.orc
scala> sql("select * from empty_orc").show
java.lang.RuntimeException: serious problem
```
After [SPARK-22279](https://github.com/apache/spark/pull/19499), Apache Spark with the default configuration doesn't have this bug. Although Hive 1.2.1 library code path still has the problem, we had better have a test coverage on what we have now in order to prevent future regression on it.
## How was this patch tested?
Pass a newly added test case.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dongjoon-hyun/spark SPARK-19809-EMPTY-FILE
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19948.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19948
----
commit 492a30ff673372b04455cd3bb454701961c65760
Author: Dongjoon Hyun <do...@apache.org>
Date: 2017-12-12T01:42:10Z
[SPARK-19809][SQL][TEST] NullPointerException on zero-size ORC file
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19948
Merged to master.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19948
**[Test build #84747 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84747/testReport)** for PR 19948 at commit [`492a30f`](https://github.com/apache/spark/commit/492a30ff673372b04455cd3bb454701961c65760).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19948
**[Test build #84755 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84755/testReport)** for PR 19948 at commit [`492a30f`](https://github.com/apache/spark/commit/492a30ff673372b04455cd3bb454701961c65760).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19948: [SPARK-19809][SQL][TEST] NullPointerException on ...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/19948#discussion_r156534839
--- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ---
@@ -2172,4 +2172,21 @@ class SQLQuerySuite extends QueryTest with SQLTestUtils with TestHiveSingleton {
}
}
}
+
+ test("SPARK-19809 NullPointerException on zero-size ORC file") {
--- End diff --
I'll move this into `HiveOrcQuerySuite`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19948: [SPARK-19809][SQL][TEST] NullPointerException on ...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19948#discussion_r156538299
--- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ---
@@ -2172,4 +2172,21 @@ class SQLQuerySuite extends QueryTest with SQLTestUtils with TestHiveSingleton {
}
}
}
+
+ test("SPARK-19809 NullPointerException on zero-size ORC file") {
+ Seq("native", "hive").foreach { orcImpl =>
+ withSQLConf(SQLConf.ORC_IMPLEMENTATION.key -> orcImpl) {
+ withTempPath { dir =>
+ withTable("spark_19809") {
+ sql(s"CREATE TABLE spark_19809(a int) STORED AS ORC LOCATION '$dir'")
+ Files.touch(new File(s"${dir.getCanonicalPath}", "zero.orc"))
+
+ withSQLConf(HiveUtils.CONVERT_METASTORE_ORC.key -> "true") { // default since 2.3.0
--- End diff --
Add a comment in the test case?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19948
retest this please
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19948: [SPARK-19809][SQL][TEST] NullPointerException on ...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19948#discussion_r156518332
--- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ---
@@ -2172,4 +2172,21 @@ class SQLQuerySuite extends QueryTest with SQLTestUtils with TestHiveSingleton {
}
}
}
+
+ test("SPARK-19809 NullPointerException on zero-size ORC file") {
+ Seq("native", "hive").foreach { orcImpl =>
+ withSQLConf(SQLConf.ORC_IMPLEMENTATION.key -> orcImpl) {
+ withTempPath { dir =>
+ withTable("spark_19809") {
+ sql(s"CREATE TABLE spark_19809(a int) STORED AS ORC LOCATION '$dir'")
+ Files.touch(new File(s"${dir.getCanonicalPath}", "zero.orc"))
+
+ withSQLConf(HiveUtils.CONVERT_METASTORE_ORC.key -> "true") { // default since 2.3.0
+ checkAnswer(sql("SELECT * FROM spark_19809"), Seq.empty)
--- End diff --
Use `table("spark_19809")`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19948: [SPARK-19809][SQL][TEST] NullPointerException on ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19948
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/19948
Thank you, @HyukjinKwon and @gatorsmile . I'll.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/19948
@viirya .
The user will hit this issue only when `HiveUtils.CONVERT_METASTORE_ORC.key = false`.
This test case covers `HiveUtils.CONVERT_METASTORE_ORC.key -> true` for both `native` and `hive` ORC impl.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19948
@dongjoon-hyun Could you submit a follow-up PR to move the test case?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19948
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19948: [SPARK-19809][SQL][TEST] NullPointerException on ...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19948#discussion_r156538633
--- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ---
@@ -2172,4 +2172,21 @@ class SQLQuerySuite extends QueryTest with SQLTestUtils with TestHiveSingleton {
}
}
}
+
+ test("SPARK-19809 NullPointerException on zero-size ORC file") {
--- End diff --
Sure. That sounds fine to me.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19948
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84747/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19948: [SPARK-19809][SQL][TEST] NullPointerException on ...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19948#discussion_r156518416
--- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ---
@@ -2172,4 +2172,21 @@ class SQLQuerySuite extends QueryTest with SQLTestUtils with TestHiveSingleton {
}
}
}
+
+ test("SPARK-19809 NullPointerException on zero-size ORC file") {
+ Seq("native", "hive").foreach { orcImpl =>
+ withSQLConf(SQLConf.ORC_IMPLEMENTATION.key -> orcImpl) {
+ withTempPath { dir =>
+ withTable("spark_19809") {
+ sql(s"CREATE TABLE spark_19809(a int) STORED AS ORC LOCATION '$dir'")
+ Files.touch(new File(s"${dir.getCanonicalPath}", "zero.orc"))
+
+ withSQLConf(HiveUtils.CONVERT_METASTORE_ORC.key -> "true") { // default since 2.3.0
--- End diff --
Use both `true` and `false`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19948: [SPARK-19809][SQL][TEST] NullPointerException on ...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/19948#discussion_r156531525
--- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ---
@@ -2172,4 +2172,21 @@ class SQLQuerySuite extends QueryTest with SQLTestUtils with TestHiveSingleton {
}
}
}
+
+ test("SPARK-19809 NullPointerException on zero-size ORC file") {
--- End diff --
BTW, @gatorsmile . Which suite do you prefer? So far, this test case covers
- Both `native` and `hive`
- Also, `STORED AS` with `CONVERT_METASTORE_ORC=true`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19948
**[Test build #84747 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84747/testReport)** for PR 19948 at commit [`492a30f`](https://github.com/apache/spark/commit/492a30ff673372b04455cd3bb454701961c65760).
* This patch **fails SparkR unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19948
**[Test build #84755 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84755/testReport)** for PR 19948 at commit [`492a30f`](https://github.com/apache/spark/commit/492a30ff673372b04455cd3bb454701961c65760).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19948
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19948
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84755/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/19948
Thank you, @HyukjinKwon and @viirya .
Yes, @viirya .
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19948
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19948: [SPARK-19809][SQL][TEST] NullPointerException on ...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/19948#discussion_r156540759
--- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ---
@@ -2172,4 +2172,21 @@ class SQLQuerySuite extends QueryTest with SQLTestUtils with TestHiveSingleton {
}
}
}
+
+ test("SPARK-19809 NullPointerException on zero-size ORC file") {
+ Seq("native", "hive").foreach { orcImpl =>
+ withSQLConf(SQLConf.ORC_IMPLEMENTATION.key -> orcImpl) {
+ withTempPath { dir =>
+ withTable("spark_19809") {
+ sql(s"CREATE TABLE spark_19809(a int) STORED AS ORC LOCATION '$dir'")
+ Files.touch(new File(s"${dir.getCanonicalPath}", "zero.orc"))
+
+ withSQLConf(HiveUtils.CONVERT_METASTORE_ORC.key -> "true") { // default since 2.3.0
--- End diff --
Sure!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19948: [SPARK-19809][SQL][TEST] NullPointerException on ...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/19948#discussion_r156534414
--- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ---
@@ -2172,4 +2172,21 @@ class SQLQuerySuite extends QueryTest with SQLTestUtils with TestHiveSingleton {
}
}
}
+
+ test("SPARK-19809 NullPointerException on zero-size ORC file") {
+ Seq("native", "hive").foreach { orcImpl =>
+ withSQLConf(SQLConf.ORC_IMPLEMENTATION.key -> orcImpl) {
+ withTempPath { dir =>
+ withTable("spark_19809") {
+ sql(s"CREATE TABLE spark_19809(a int) STORED AS ORC LOCATION '$dir'")
+ Files.touch(new File(s"${dir.getCanonicalPath}", "zero.orc"))
+
+ withSQLConf(HiveUtils.CONVERT_METASTORE_ORC.key -> "true") { // default since 2.3.0
--- End diff --
Ur, this is for `convertMetastoreOrc=true` which bacame default in Spark 2.3.0.
`false` still has the issue of Hive 1.2.1 `OrcInputFormat.getSplits`, so I wrote like the following in PR description.
> After SPARK-22279, Apache Spark with the default configuration doesn't have this bug. Although Hive 1.2.1 library code path still has the problem, we had better have a test coverage on what we have now in order to prevent future regression on it.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19948
**[Test build #84758 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84758/testReport)** for PR 19948 at commit [`492a30f`](https://github.com/apache/spark/commit/492a30ff673372b04455cd3bb454701961c65760).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19948
So if `SQLConf.ORC_IMPLEMENTATION` is `hive`, we still hit this bug?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19948
retest this please
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19948
**[Test build #84758 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84758/testReport)** for PR 19948 at commit [`492a30f`](https://github.com/apache/spark/commit/492a30ff673372b04455cd3bb454701961c65760).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19948: [SPARK-19809][SQL][TEST] NullPointerException on ...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/19948#discussion_r156540742
--- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ---
@@ -2172,4 +2172,21 @@ class SQLQuerySuite extends QueryTest with SQLTestUtils with TestHiveSingleton {
}
}
}
+
+ test("SPARK-19809 NullPointerException on zero-size ORC file") {
+ Seq("native", "hive").foreach { orcImpl =>
+ withSQLConf(SQLConf.ORC_IMPLEMENTATION.key -> orcImpl) {
+ withTempPath { dir =>
+ withTable("spark_19809") {
+ sql(s"CREATE TABLE spark_19809(a int) STORED AS ORC LOCATION '$dir'")
+ Files.touch(new File(s"${dir.getCanonicalPath}", "zero.orc"))
+
+ withSQLConf(HiveUtils.CONVERT_METASTORE_ORC.key -> "true") { // default since 2.3.0
+ checkAnswer(sql("SELECT * FROM spark_19809"), Seq.empty)
--- End diff --
It's done.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19948
Hey @dongjoon-hyun, I am going to merge this but let's leave a comment about ^ in the JIRA too for clarification.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19948: [SPARK-19809][SQL][TEST] NullPointerException on ...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19948#discussion_r156517954
--- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ---
@@ -2172,4 +2172,21 @@ class SQLQuerySuite extends QueryTest with SQLTestUtils with TestHiveSingleton {
}
}
}
+
+ test("SPARK-19809 NullPointerException on zero-size ORC file") {
--- End diff --
This test case should not be put in SQLQuerySuite.scala
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19948: [SPARK-19809][SQL][TEST] NullPointerException on zero-si...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19948
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84758/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org