You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/03/12 19:28:55 UTC
[GitHub] [spark] dongjoon-hyun opened a new pull request #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
dongjoon-hyun opened a new pull request #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894
### What changes were proposed in this pull request?
This PR (SPARK-31136) is created to see how many tests are affected.
### Why are the changes needed?
This aims to comply the new community policy.
### Does this PR introduce any user-facing change?
Yes. This restores the existing behavior to avoid breaking the existing user pipeline.
### How was this patch tested?
Pass the Jenkins.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600458489
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119967/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598534823
I'm not looking at the number of failures. Since the UT had better be independent to the global conf changes, we had better add guards with `withSQLConf` at least.
The following is not a cost at all. The users already know that the vectorization is only enabled in case of native tables, do they? There exist user groups who have some reasons why they choose a vanilla Hive table. Not only for that, some users prefer `hive` ORC implementation to avoid Spark's `CHAR/VARCHAR` behavior difference.
> On the other hand, the "cost to maintain" is losing Spark's perf benefits:
Please see the old mail thread.
- https://lists.apache.org/thread.html/96b004331d9762e356053b5c8c97e953e398e489d15e1b49e775702f%40%3Cdev.spark.apache.org%3E
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600460129
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24694/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27894: [SPARK-31136] Revert
SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598472888
**[Test build #119729 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119729/testReport)** for PR 27894 at commit [`7765675`](https://github.com/apache/spark/commit/7765675190ddbd8ddc96eb4f136113682ee13e63).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27894: [SPARK-31136] Revert
SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598543190
Looks coherent to revert it per @marmbrus's policy and reverts made so far, although I personally are not really convinced that we should revert.
Reverting this change discourages users to leverage Spark's new features and edge optimizations, and let users more stick to Hive. What's your thought @marmbrus?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598502732
A lot of failures remains.
```
org.apache.spark.sql.SQLQueryTestSuite.sql 1.1 sec 1
org.apache.spark.sql.SQLQueryTestSuite.sql 3.2 sec 1
org.apache.spark.sql.connector.DataSourceV2SQLSuite.CreateTable: without USING clause 2 ms 1
org.apache.spark.sql.connector.DataSourceV2SQLSuite.CreateTableAsSelect: without USING clause 4 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.Test CTAS #3 2 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - basic 1 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - with database name 2 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - temporary 2 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - external 2 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - if not exists 2 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - comment 3 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - partitioned columns 3 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - clustered by 2 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - properties 1 ms 1
org.apache.spark.sql.sources.InsertSuite.SPARK-29174 Support LOCAL in INSERT OVERWRITE DIRECTORY to data source
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598444555
**[Test build #119728 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119728/testReport)** for PR 27894 at commit [`da991af`](https://github.com/apache/spark/commit/da991af705370d0f82e4212de4f5f9e21dee1d62).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598377847
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24451/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600460129
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24694/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598534823
I'm not looking at the number of failures. Since the UT had better be independent to the global conf changes, we had better add guards with `withSQLConf` at least.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598419125
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600578807
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119971/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598491197
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598419129
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119724/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598502732
A lot of failures are remaining.
```
org.apache.spark.sql.SQLQueryTestSuite.sql 1.1 sec 1
org.apache.spark.sql.SQLQueryTestSuite.sql 3.2 sec 1
org.apache.spark.sql.connector.DataSourceV2SQLSuite.CreateTable: without USING clause 2 ms 1
org.apache.spark.sql.connector.DataSourceV2SQLSuite.CreateTableAsSelect: without USING clause 4 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.Test CTAS #3 2 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - basic 1 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - with database name 2 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - temporary 2 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - external 2 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - if not exists 2 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - comment 3 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - partitioned columns 3 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - clustered by 2 ms 1
org.apache.spark.sql.execution.command.DDLParserSuite.create table - properties 1 ms 1
org.apache.spark.sql.sources.InsertSuite.SPARK-29174 Support LOCAL in INSERT OVERWRITE DIRECTORY to data source
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598445968
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24455/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598466623
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598473280
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24456/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598472888
**[Test build #119729 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119729/testReport)** for PR 27894 at commit [`7765675`](https://github.com/apache/spark/commit/7765675190ddbd8ddc96eb4f136113682ee13e63).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27894: [SPARK-31136] Revert
SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598418962
**[Test build #119724 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119724/testReport)** for PR 27894 at commit [`be16bf5`](https://github.com/apache/spark/commit/be16bf514dae5e52806de9decc6b37e37ad81a7b).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598432083
As expected, it fails. But, unfortunately, it failed at the first failure.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600578796
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600429754
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24691/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27894: [SPARK-31136] Revert
SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598491144
**[Test build #119729 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119729/testReport)** for PR 27894 at commit [`7765675`](https://github.com/apache/spark/commit/7765675190ddbd8ddc96eb4f136113682ee13e63).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598377847
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24451/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598544462
It just became a correctness issue due to the another example of `CHAR` type.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598419129
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119724/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598534823
I'm not looking at the number of failures. Since the UT had better be independent to the global conf changes, we had better add guards with `withSQLConf` at least.
The following is not a cost at all. The users already know that the vectorization is only enabled in case of native tables, don't they? There exist user groups who have some reasons why they choose a vanilla Hive table. Not only for that, some users prefer `hive` Parquet/ORC implementation to avoid Spark's `CHAR/VARCHAR` behavior difference.
> On the other hand, the "cost to maintain" is losing Spark's perf benefits:
Please see the old mail thread.
- https://lists.apache.org/thread.html/96b004331d9762e356053b5c8c97e953e398e489d15e1b49e775702f%40%3Cdev.spark.apache.org%3E
This will also break the query result on `CHAR` type.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600460119
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598445944
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27894: [SPARK-31136] Revert
SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600318272
**[Test build #119948 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119948/testReport)** for PR 27894 at commit [`1a68632`](https://github.com/apache/spark/commit/1a6863218577e4aa340c754539ec679644402e75).
* This patch **fails Spark unit tests**.
* This patch **does not merge cleanly**.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598534823
I'm not looking at the number of failures. Since the UT had better be independent to the global conf changes, we had better add guards with `withSQLConf` at least.
The following is not a cost at all. The users already know that the vectorization is only enabled in case of native tables, don't they? There exist user groups who have some reasons why they choose a vanilla Hive table. Not only for that, some users prefer `hive` ORC implementation to avoid Spark's `CHAR/VARCHAR` behavior difference.
> On the other hand, the "cost to maintain" is losing Spark's perf benefits:
Please see the old mail thread.
- https://lists.apache.org/thread.html/96b004331d9762e356053b5c8c97e953e398e489d15e1b49e775702f%40%3Cdev.spark.apache.org%3E
This will also break the query result on `CHAR` type.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598540663
Hi, All. I marked SPARK-31136 as a correctness issue and added the example.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600458489
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119967/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600578796
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598473275
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598491200
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119729/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27894: [SPARK-31136] Revert
SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600577959
**[Test build #119971 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119971/testReport)** for PR 27894 at commit [`9bff392`](https://github.com/apache/spark/commit/9bff39245269a60b32c9c4f9911b7d0c3a8b6b8a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598419125
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598377293
**[Test build #119724 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119724/testReport)** for PR 27894 at commit [`be16bf5`](https://github.com/apache/spark/commit/be16bf514dae5e52806de9decc6b37e37ad81a7b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600458469
Retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600429754
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24691/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598445944
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598473280
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24456/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598377838
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600249456
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24671/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27894: [SPARK-31136] Revert
SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598534428
I agree that we should evaluate the "cost to break", but looking at unit tests may not be a good idea. They heavily rely on internal assumptions and changing the table format will definitely break a lot of unit tests.
Ideally, table format only decides how the table is stored and should be a performance thing, but hive table is a bit different. IMO, the "cost to break" is losing hive compatibility a bit: Now the tables created without USING may not be readable to hive, and some hive specific commands like LOAD TABLE doesn't work for them.
On the other hand, the "cost to maintain" is losing Spark's perf benefits: Many users just run `CREATE TABLE` like they do in other databases, which creates a hive table before 3.0. This means all the features we build for our native readers are not available, like the vectorized reader, nested column pruning, nested field filter pushdown (@dbtsai is working on it), bucketed table, etc.
I think in this case the "cost to maintain" is more serious and we should accept this change. cc @marmbrus @srowen @maropu @viirya @HyukjinKwon
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600458484
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27894: [SPARK-31136] Revert
SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598543678
cc @rxin too
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27894: [SPARK-31136] Revert
SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600458374
**[Test build #119967 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119967/testReport)** for PR 27894 at commit [`9bff392`](https://github.com/apache/spark/commit/9bff39245269a60b32c9c4f9911b7d0c3a8b6b8a).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600429443
**[Test build #119967 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119967/testReport)** for PR 27894 at commit [`9bff392`](https://github.com/apache/spark/commit/9bff39245269a60b32c9c4f9911b7d0c3a8b6b8a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600460119
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27894: [SPARK-31136] Revert
SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600248835
**[Test build #119948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119948/testReport)** for PR 27894 at commit [`1a68632`](https://github.com/apache/spark/commit/1a6863218577e4aa340c754539ec679644402e75).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598466630
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119728/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598466623
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598534823
I'm not looking at the number of failures. Since the UT had better be independent to the global conf changes, we had better add guards with `withSQLConf` at least.
The following is not a cost at all. The users already know that the vectorization is only enabled in case of native tables, don't they? There exist user groups who have some reasons why they choose a vanilla Hive table. As a side note, some users prefer `hive` Parquet/ORC implementation to avoid Spark's `CHAR/VARCHAR` behavior difference.
> On the other hand, the "cost to maintain" is losing Spark's perf benefits:
Please see the old mail thread.
- https://lists.apache.org/thread.html/96b004331d9762e356053b5c8c97e953e398e489d15e1b49e775702f%40%3Cdev.spark.apache.org%3E
This will also break the query result on `CHAR` type.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27894: [SPARK-31136] Revert
SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600459688
**[Test build #119971 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119971/testReport)** for PR 27894 at commit [`9bff392`](https://github.com/apache/spark/commit/9bff39245269a60b32c9c4f9911b7d0c3a8b6b8a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27894: [SPARK-31136] Revert
SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598377293
**[Test build #119724 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119724/testReport)** for PR 27894 at commit [`be16bf5`](https://github.com/apache/spark/commit/be16bf514dae5e52806de9decc6b37e37ad81a7b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan edited a comment on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
cloud-fan edited a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598534428
I agree that we should evaluate the "cost to break", but looking at unit tests may not be a good idea. They heavily rely on internal assumptions and changing the table format will definitely break a lot of unit tests.
Ideally, table format only decides how the table is stored and should be a performance thing, but hive table is a bit different. IMO, the "cost to break" is losing hive compatibility a bit: Now the tables created without USING may not be readable to hive, and some hive specific commands like LOAD TABLE doesn't work for them.
On the other hand, the "cost to maintain" is losing Spark's perf benefits: Many users just run `CREATE TABLE` like they do in other databases, which creates a hive table before 3.0. This means all the features we build for our native readers are not available, like the vectorized reader, nested column pruning, nested field filter pushdown (@dbtsai is working on it), bucketed table, etc.
I think in this case the "cost to maintain" is more serious and we should accept that change and don't revert it. cc @marmbrus @srowen @maropu @viirya @HyukjinKwon
UPDATED to make my opinion more clear.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600578807
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119971/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27894: [SPARK-31136] Revert
SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598444555
**[Test build #119728 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119728/testReport)** for PR 27894 at commit [`da991af`](https://github.com/apache/spark/commit/da991af705370d0f82e4212de4f5f9e21dee1d62).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600318567
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119948/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600249447
Build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600249447
Build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600459688
**[Test build #119971 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119971/testReport)** for PR 27894 at commit [`9bff392`](https://github.com/apache/spark/commit/9bff39245269a60b32c9c4f9911b7d0c3a8b6b8a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27894: [SPARK-31136] Revert
SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598466558
**[Test build #119728 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119728/testReport)** for PR 27894 at commit [`da991af`](https://github.com/apache/spark/commit/da991af705370d0f82e4212de4f5f9e21dee1d62).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598534823
I'm not looking at the number of failures. Since the UT had better be independent to the global conf changes, we had better add guards with `withSQLConf` at least.
The following is not a cost at all. The users already know that the vectorization is only enabled in case of native tables, do they? There exist user groups who have some reasons why they choose a vanilla Hive table. Not only for that, some users prefer `hive` ORC implementation to avoid Spark's `CHAR/VARCHAR` behavior difference.
> On the other hand, the "cost to maintain" is losing Spark's perf benefits:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598445968
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24455/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598473275
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600318567
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119948/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27894: [SPARK-31136] Revert
SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600429443
**[Test build #119967 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119967/testReport)** for PR 27894 at commit [`9bff392`](https://github.com/apache/spark/commit/9bff39245269a60b32c9c4f9911b7d0c3a8b6b8a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598491197
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600318563
Build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598534823
I'm not looking at the number of failures. Since the UT had better be independent to the global conf changes, we had better add guards with `withSQLConf` at least.
The following is not a cost at all. The users already know that the vectorization is only enabled in case of native tables, do they? There exist user groups who have some reasons why they choose a vanilla Hive table. Not only for that, some users prefer `hive` ORC implementation to avoid Spark's `CHAR/VARCHAR` behavior difference.
> On the other hand, the "cost to maintain" is losing Spark's perf benefits:
Please see the old mail thread.
- https://lists.apache.org/thread.html/96b004331d9762e356053b5c8c97e953e398e489d15e1b49e775702f%40%3Cdev.spark.apache.org%3E
This will also break the query result on `CHAR` type.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598491200
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119729/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-601535039
Thank you all! I'm going to close this PR because this is discussed explicitly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598377838
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600248835
**[Test build #119948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119948/testReport)** for PR 27894 at commit [`1a68632`](https://github.com/apache/spark/commit/1a6863218577e4aa340c754539ec679644402e75).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598466630
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119728/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600458484
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600318563
Build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600429750
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598534823
I'm not looking at the number of failures. Since the UT had better be independent to the global conf changes, we had better add guards with `withSQLConf` at least.
The following is not a cost at all. The users already know that the vectorization is only enabled in case of native tables, do they? There exists user groups who has a reason why they choose a vanilla table. Not only for that, some users prefer `hive` ORC implementation to avoid Spark's `CHAR/VARCHAR` behavior difference.
> On the other hand, the "cost to maintain" is losing Spark's perf benefits:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600249456
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24671/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-600429750
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27894: [SPARK-31136]
Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598534823
> I'm not looking at the number of failures.
Since the UT had better be independent to the global conf changes, we had better add guards with `withSQLConf` at least.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #27894:
[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for
CREATE TABLE syntax
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on issue #27894: [SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
URL: https://github.com/apache/spark/pull/27894#issuecomment-598534823
I'm not looking at the number of failures. Since the UT had better be independent to the global conf changes, we had better add guards with `withSQLConf` at least.
The following is not a cost at all. The users already know that the vectorization is only enabled in case of native tables, do they? There exist user groups who has a reason why they choose a vanilla table. Not only for that, some users prefer `hive` ORC implementation to avoid Spark's `CHAR/VARCHAR` behavior difference.
> On the other hand, the "cost to maintain" is losing Spark's perf benefits:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org