You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by scwf <gi...@git.apache.org> on 2014/11/22 10:21:06 UTC
[GitHub] spark pull request: [SPARK-4552][SQL] query for empty parquet tabl...
GitHub user scwf opened a pull request:
https://github.com/apache/spark/pull/3413
[SPARK-4552][SQL] query for empty parquet table in spark sql hive get IllegalArgumentException
create table test_parquet(key int, value string) stored as parquet;
select * from test_parquet;
get error as follow
java.lang.IllegalArgumentException: Could not find Parquet metadata at path file:/user/hive/warehouse/test_parquet
at org.apache.spark.sql.parquet.ParquetTypesConverter$$anonfun$readMetaData$4.apply(ParquetTypes.scala:459)
at org.apache.spark.sql.parquet.ParquetTypesConverter$$anonfun$readMetaData$4.apply(ParquetTypes.scala:459)
at scala.Option.getOrElse(Option.scala:120)
at org.apache.spark.sql.parquet.ParquetTypesConverter$.readMetaData(ParquetTypes.sc
link to #3352
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/scwf/spark parquet-empty
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/3413.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3413
----
commit c38201dc9cabac67fa6f99ebc0d5bd2e6e5b16fa
Author: scwf <wa...@huawei.com>
Date: 2014-11-22T08:53:19Z
fix parquet empty issue
commit be3b00656957fea7dcd9e3e0f264e67430d3b39f
Author: scwf <wa...@huawei.com>
Date: 2014-11-22T08:53:26Z
Merge branch 'master' of https://github.com/apache/spark into parquet-1
commit 1e9cdfc9af06731aa4b18287e32c7ec82ac8aed1
Author: scwf <wa...@huawei.com>
Date: 2014-11-22T09:11:45Z
minor fix
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4552][SQL] query for empty parquet tabl...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3413#issuecomment-64074489
[Test build #23737 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23737/consoleFull) for PR 3413 at commit [`1e9cdfc`](https://github.com/apache/spark/commit/1e9cdfc9af06731aa4b18287e32c7ec82ac8aed1).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4552][SQL] Query for empty parquet tabl...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3413#issuecomment-64075701
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23739/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4552][SQL] Query for empty parquet tabl...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3413#issuecomment-64075697
[Test build #23739 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23739/consoleFull) for PR 3413 at commit [`9083f64`](https://github.com/apache/spark/commit/9083f644d23c0b9ddd8921488db06e64331da09d).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4552][SQL] Query for empty parquet tabl...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3413#issuecomment-64075291
[Test build #23737 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23737/consoleFull) for PR 3413 at commit [`1e9cdfc`](https://github.com/apache/spark/commit/1e9cdfc9af06731aa4b18287e32c7ec82ac8aed1).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4552][SQL] Query for empty parquet tabl...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3413#issuecomment-64077654
[Test build #23742 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23742/consoleFull) for PR 3413 at commit [`e6c7d8a`](https://github.com/apache/spark/commit/e6c7d8a6b3802ce32256aa31dd643147768a8f96).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4552][SQL] Query for empty parquet tabl...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3413#issuecomment-64079406
[Test build #23742 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23742/consoleFull) for PR 3413 at commit [`e6c7d8a`](https://github.com/apache/spark/commit/e6c7d8a6b3802ce32256aa31dd643147768a8f96).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4552][SQL] Query for empty parquet tabl...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3413#issuecomment-64079408
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23742/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4552][SQL] query for empty parquet tabl...
Posted by scwf <gi...@git.apache.org>.
Github user scwf commented on the pull request:
https://github.com/apache/spark/pull/3413#issuecomment-64074411
Need to add a test for this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4552][SQL] Query for empty parquet tabl...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3413#issuecomment-64074941
[Test build #23739 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23739/consoleFull) for PR 3413 at commit [`9083f64`](https://github.com/apache/spark/commit/9083f644d23c0b9ddd8921488db06e64331da09d).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4552][SQL] Query for empty parquet tabl...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3413#issuecomment-64075292
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23737/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4552][SQL] Query for empty parquet tabl...
Posted by marmbrus <gi...@git.apache.org>.
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/3413#issuecomment-65133627
Thanks for working on this! I think it might be better to make sure that we fix this as part of #3441. What do you think?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org