You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/10/13 09:41:59 UTC
[GitHub] [spark] AngersZhuuuu opened a new pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
AngersZhuuuu opened a new pull request #30027:
URL: https://github.com/apache/spark/pull/30027
### What changes were proposed in this pull request?
Improve error message on reading unexpected directory
### Why are the changes needed?
Improve error message on reading unexpected directory
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Ut
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710815176
**[Test build #129944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129944/testReport)** for PR 30027 at commit [`d5947e1`](https://github.com/apache/spark/commit/d5947e1dea2ecb28bab7e6a5abb1eda80ff2b5ef).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710747303
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34537/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #30027:
URL: https://github.com/apache/spark/pull/30027#discussion_r504402399
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveParquetSourceSuite.scala
##########
@@ -230,6 +230,13 @@ class HiveParquetSourceSuite extends ParquetPartitioningTest {
withTempPath { path =>
withTable("parq_tbl1", "parq_tbl2", "parq_tbl3",
"tbl1", "tbl2", "tbl3", "tbl4", "tbl5", "tbl6") {
+
+ def errorMsg(path: String): String = {
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710859856
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709411118
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129819/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710701400
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-707681926
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708649167
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #30027:
URL: https://github.com/apache/spark/pull/30027#discussion_r504350436
##########
File path: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala
##########
@@ -232,6 +232,11 @@ class HadoopRDD[K, V](
logWarning(s"${jobConf.get(FileInputFormat.INPUT_DIR)} doesn't exist and no" +
s" partitions returned from this path.", e)
Array.empty[Partition]
+ case e: IOException if e.getMessage.contains("Not a file") =>
Review comment:
> Nit: if e.getMessage.startsWith("Not a file:")
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-707625831
**[Test build #129743 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129743/testReport)** for PR 30027 at commit [`3155cf1`](https://github.com/apache/spark/commit/3155cf1d71a975ec90aba39fe527e5a98da5030e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708601578
**[Test build #129749 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129749/testReport)** for PR 30027 at commit [`2e76b2a`](https://github.com/apache/spark/commit/2e76b2a124cadfb7a73ba795f55bf35e48ffe4a9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-707681938
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/34349/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #30027:
URL: https://github.com/apache/spark/pull/30027#discussion_r504350493
##########
File path: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala
##########
@@ -232,6 +232,11 @@ class HadoopRDD[K, V](
logWarning(s"${jobConf.get(FileInputFormat.INPUT_DIR)} doesn't exist and no" +
s" partitions returned from this path.", e)
Array.empty[Partition]
+ case e: IOException if e.getMessage.contains("Not a file") =>
+ val path = e.getMessage.split(":").map(_.trim).apply(2)
+ throw new IOException(s"Path: ${path} is a directory, it is not" +
+ s" allowed for `serde` reader when" +
Review comment:
> The code path here is in core module. The reader may not be `serde` reader.
How about current?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710743730
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34537/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710760245
**[Test build #129940 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129940/testReport)** for PR 30027 at commit [`d5947e1`](https://github.com/apache/spark/commit/d5947e1dea2ecb28bab7e6a5abb1eda80ff2b5ef).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-711061231
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710701405
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129901/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708636880
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34355/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710747307
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709706678
**[Test build #129853 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129853/testReport)** for PR 30027 at commit [`2e76b2a`](https://github.com/apache/spark/commit/2e76b2a124cadfb7a73ba795f55bf35e48ffe4a9).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710759907
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-711060976
**[Test build #129944 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129944/testReport)** for PR 30027 at commit [`d5947e1`](https://github.com/apache/spark/commit/d5947e1dea2ecb28bab7e6a5abb1eda80ff2b5ef).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708947149
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #30027:
URL: https://github.com/apache/spark/pull/30027#discussion_r504398990
##########
File path: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala
##########
@@ -232,6 +232,11 @@ class HadoopRDD[K, V](
logWarning(s"${jobConf.get(FileInputFormat.INPUT_DIR)} doesn't exist and no" +
s" partitions returned from this path.", e)
Array.empty[Partition]
+ case e: IOException if e.getMessage.contains("Not a file") =>
+ val path = e.getMessage.split(":").map(_.trim).apply(2)
+ throw new IOException(s"Path: ${path} is a directory, it is not" +
+ s" allowed for `serde` reader when" +
Review comment:
How about:
```
s"Path: ${path} is a directory, which is not supported by the record reader when `mapreduce.input.fileinputformat.input.dir.recursive` is false."
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710808941
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #30027:
URL: https://github.com/apache/spark/pull/30027#discussion_r504399096
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveParquetSourceSuite.scala
##########
@@ -230,6 +230,13 @@ class HiveParquetSourceSuite extends ParquetPartitioningTest {
withTempPath { path =>
withTable("parq_tbl1", "parq_tbl2", "parq_tbl3",
"tbl1", "tbl2", "tbl3", "tbl4", "tbl5", "tbl6") {
+
+ def errorMsg(path: String): String = {
Review comment:
nit: `checkErrorMsg`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708695494
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708701738
**[Test build #129772 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129772/testReport)** for PR 30027 at commit [`2e76b2a`](https://github.com/apache/spark/commit/2e76b2a124cadfb7a73ba795f55bf35e48ffe4a9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710767230
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34545/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709584613
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34459/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710748841
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710748828
**[Test build #129932 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129932/testReport)** for PR 30027 at commit [`d5947e1`](https://github.com/apache/spark/commit/d5947e1dea2ecb28bab7e6a5abb1eda80ff2b5ef).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710738272
**[Test build #129932 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129932/testReport)** for PR 30027 at commit [`d5947e1`](https://github.com/apache/spark/commit/d5947e1dea2ecb28bab7e6a5abb1eda80ff2b5ef).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708695494
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709410434
**[Test build #129819 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129819/testReport)** for PR 30027 at commit [`2e76b2a`](https://github.com/apache/spark/commit/2e76b2a124cadfb7a73ba795f55bf35e48ffe4a9).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708946593
**[Test build #129772 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129772/testReport)** for PR 30027 at commit [`2e76b2a`](https://github.com/apache/spark/commit/2e76b2a124cadfb7a73ba795f55bf35e48ffe4a9).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709584639
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-711464895
ping @dongjoon-hyun
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708127310
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710117413
> The failure looks relevant. Could you check it with `hadoop-2.7`?
>
> ```
> - SPARK-25993 CREATE EXTERNAL TABLE with subdirectories *** FAILED ***
> Expected exception java.io.IOException to be thrown, but no exception was thrown (HiveParquetSourceSuite.scala:293)
> ```
Found the reason, added UT `test("SPARK-32069: Improve error message on reading unexpected directory") `
run before `HiveParquetSourceSuite`, in UT `SPARK-32069` set conf `sql("set mapreduce.input.fileinputformat.input.dir.recursive=true")` can impact latter UT(maybe set tp system props? I will check more).
Seems this behavior is different between hadoop-2.7 and hadoop-3.2.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709411107
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708953966
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710760245
**[Test build #129940 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129940/testReport)** for PR 30027 at commit [`d5947e1`](https://github.com/apache/spark/commit/d5947e1dea2ecb28bab7e6a5abb1eda80ff2b5ef).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708676750
Retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709550742
Retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710119210
**[Test build #129901 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129901/testReport)** for PR 30027 at commit [`d5947e1`](https://github.com/apache/spark/commit/d5947e1dea2ecb28bab7e6a5abb1eda80ff2b5ef).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709552058
**[Test build #129853 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129853/testReport)** for PR 30027 at commit [`2e76b2a`](https://github.com/apache/spark/commit/2e76b2a124cadfb7a73ba795f55bf35e48ffe4a9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708156797
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709038954
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34424/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710764198
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129940/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #30027:
URL: https://github.com/apache/spark/pull/30027#discussion_r504350520
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveParquetSourceSuite.scala
##########
@@ -287,7 +287,11 @@ class HiveParquetSourceSuite extends ParquetPartitioningTest {
val msg = intercept[IOException] {
sql("SELECT * FROM tbl1").show()
}.getMessage
- assert(msg.contains("Not a file:"))
+ assert(msg.contains(
Review comment:
> how about creating a method for checking the exception message?
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710737270
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708958258
**[Test build #129819 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129819/testReport)** for PR 30027 at commit [`2e76b2a`](https://github.com/apache/spark/commit/2e76b2a124cadfb7a73ba795f55bf35e48ffe4a9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709707028
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #30027:
URL: https://github.com/apache/spark/pull/30027#discussion_r504402354
##########
File path: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala
##########
@@ -232,6 +232,11 @@ class HadoopRDD[K, V](
logWarning(s"${jobConf.get(FileInputFormat.INPUT_DIR)} doesn't exist and no" +
s" partitions returned from this path.", e)
Array.empty[Partition]
+ case e: IOException if e.getMessage.contains("Not a file") =>
+ val path = e.getMessage.split(":").map(_.trim).apply(2)
+ throw new IOException(s"Path: ${path} is a directory, it is not" +
+ s" allowed for `serde` reader when" +
Review comment:
> How about:
>
> ```
> s"Path: ${path} is a directory, which is not supported by the record reader when `mapreduce.input.fileinputformat.input.dir.recursive` is false."
> ```
updated
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710166742
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34507/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-707689336
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709707028
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710859856
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-707689344
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129743/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710748841
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708701738
**[Test build #129772 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129772/testReport)** for PR 30027 at commit [`2e76b2a`](https://github.com/apache/spark/commit/2e76b2a124cadfb7a73ba795f55bf35e48ffe4a9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709059934
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710764834
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34545/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710843204
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34548/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710192355
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34507/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-707681926
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708840935
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34378/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709707036
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129853/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709059909
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34424/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709584639
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-707625831
**[Test build #129743 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129743/testReport)** for PR 30027 at commit [`3155cf1`](https://github.com/apache/spark/commit/3155cf1d71a975ec90aba39fe527e5a98da5030e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708694694
**[Test build #129749 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129749/testReport)** for PR 30027 at commit [`2e76b2a`](https://github.com/apache/spark/commit/2e76b2a124cadfb7a73ba795f55bf35e48ffe4a9).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710815176
**[Test build #129944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129944/testReport)** for PR 30027 at commit [`d5947e1`](https://github.com/apache/spark/commit/d5947e1dea2ecb28bab7e6a5abb1eda80ff2b5ef).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709715883
> The failure looks relevant. Could you check it with `hadoop-2.7`?
>
> ```
> - SPARK-25993 CREATE EXTERNAL TABLE with subdirectories *** FAILED ***
> Expected exception java.io.IOException to be thrown, but no exception was thrown (HiveParquetSourceSuite.scala:293)
> ```
Yea, checked a lot and run test with hadoop2.7 in local with success, and I have checked the code of hadoop-2.7. and hadoop-3.2. seems no reason for this test failed..
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710192463
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708947159
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129772/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-711061231
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #30027:
URL: https://github.com/apache/spark/pull/30027
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710192463
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710767239
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710764196
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710747307
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709059934
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709411107
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708958258
**[Test build #129819 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129819/testReport)** for PR 30027 at commit [`2e76b2a`](https://github.com/apache/spark/commit/2e76b2a124cadfb7a73ba795f55bf35e48ffe4a9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708649139
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34355/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708846245
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34378/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708154528
LGTM otherwise. Thanks for doing this work.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708114538
@dongjoon-hyun Seems jenkins down because of this pr? emmmmm
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710748848
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129932/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710738272
**[Test build #129932 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129932/testReport)** for PR 30027 at commit [`d5947e1`](https://github.com/apache/spark/commit/d5947e1dea2ecb28bab7e6a5abb1eda80ff2b5ef).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710859797
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34548/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709552058
**[Test build #129853 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129853/testReport)** for PR 30027 at commit [`2e76b2a`](https://github.com/apache/spark/commit/2e76b2a124cadfb7a73ba795f55bf35e48ffe4a9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708649167
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710767239
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #30027:
URL: https://github.com/apache/spark/pull/30027#discussion_r504046931
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveParquetSourceSuite.scala
##########
@@ -287,7 +287,11 @@ class HiveParquetSourceSuite extends ParquetPartitioningTest {
val msg = intercept[IOException] {
sql("SELECT * FROM tbl1").show()
}.getMessage
- assert(msg.contains("Not a file:"))
+ assert(msg.contains(
Review comment:
how about creating a method for checking the exception message?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708846257
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708947149
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708109066
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-707671944
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34349/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710764186
**[Test build #129940 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129940/testReport)** for PR 30027 at commit [`d5947e1`](https://github.com/apache/spark/commit/d5947e1dea2ecb28bab7e6a5abb1eda80ff2b5ef).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708601578
**[Test build #129749 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129749/testReport)** for PR 30027 at commit [`2e76b2a`](https://github.com/apache/spark/commit/2e76b2a124cadfb7a73ba795f55bf35e48ffe4a9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709576304
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34459/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708419035
Seems OK. Slightly hacky to rely on the message text, and I guess you lose the underlying exception, but, both of those seem worth the upside.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710764196
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-707688707
**[Test build #129743 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129743/testReport)** for PR 30027 at commit [`3155cf1`](https://github.com/apache/spark/commit/3155cf1d71a975ec90aba39fe527e5a98da5030e).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #30027:
URL: https://github.com/apache/spark/pull/30027#discussion_r503989771
##########
File path: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala
##########
@@ -232,6 +232,11 @@ class HadoopRDD[K, V](
logWarning(s"${jobConf.get(FileInputFormat.INPUT_DIR)} doesn't exist and no" +
s" partitions returned from this path.", e)
Array.empty[Partition]
+ case e: IOException if e.getMessage.contains("Not a file") =>
Review comment:
Nit: if e.getMessage.startsWith("Not a file:")
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710759885
> **[Test build #129932 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129932/testReport)** for PR 30027 at commit [`d5947e1`](https://github.com/apache/spark/commit/d5947e1dea2ecb28bab7e6a5abb1eda80ff2b5ef).
>
> * This patch **fails Spark unit tests**.
> * This patch merges cleanly.
> * This patch adds no public classes.
Failed caused by https://issues.apache.org/jira/browse/SPARK-33173
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-707689336
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-707623723
Ping @gengliangwang
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-708846257
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-707681912
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34349/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710701400
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #30027: [SPARK-32069][CORE][SQL] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #30027:
URL: https://github.com/apache/spark/pull/30027#discussion_r503990531
##########
File path: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala
##########
@@ -232,6 +232,11 @@ class HadoopRDD[K, V](
logWarning(s"${jobConf.get(FileInputFormat.INPUT_DIR)} doesn't exist and no" +
s" partitions returned from this path.", e)
Array.empty[Partition]
+ case e: IOException if e.getMessage.contains("Not a file") =>
+ val path = e.getMessage.split(":").map(_.trim).apply(2)
+ throw new IOException(s"Path: ${path} is a directory, it is not" +
+ s" allowed for `serde` reader when" +
Review comment:
The code path here is in core module. The reader may not be `serde` reader.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-710692930
Ya. It bites us frequently. So, we need to pass during the review phase for the suspected PR.
> Seems this behavior is different between hadoop-2.7 and hadoop-3.2.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30027: [SPARK-32069][CORE][SQL][test-maven][test-hadoop2.7] Improve error message on reading unexpected directory
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30027:
URL: https://github.com/apache/spark/pull/30027#issuecomment-709059943
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/34424/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org