You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/03/06 19:06:17 UTC
[GitHub] [spark] yuchenhuo opened a new pull request #27838: [SPARK-30902]
Allow ReplaceTableAsStatement to have none provider
yuchenhuo opened a new pull request #27838: [SPARK-30902] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
7. If you want to add a new configuration, please read the guideline first for naming configurations in
'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
-->
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
This is a follow up for https://github.com/apache/spark/pull/27650 where allow None provider for create table. Here we are doing the same thing for ReplaceTable. Although currently the ASTBuilder doesn't seem to allow `replace` without `USING` clause. This would allow `DataFrameWriterV2` to
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
### Does this PR introduce any user-facing change?
<!--
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If no, write 'No'.
-->
No.
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yuchenhuo commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
yuchenhuo commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r389934818
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -305,7 +305,8 @@ class ResolveSessionCatalog(
// session catalog and the table provider is not v2.
case c @ ReplaceTableStatement(
SessionCatalogAndTable(catalog, tbl), _, _, _, _, _, _, _, _, _) =>
- if (!isV2Provider(c.provider)) {
+ val provider = c.provider.getOrElse(conf.defaultDataSourceName)
Review comment:
Maybe `CatalogPlugin` should expose an interface for the default provider? It's not so clear to me what's the relationship between provider and catalog. Should every catalog have a provider?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-596007955
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119481/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yuchenhuo commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
yuchenhuo commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r389098945
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -305,7 +305,8 @@ class ResolveSessionCatalog(
// session catalog and the table provider is not v2.
case c @ ReplaceTableStatement(
SessionCatalogAndTable(catalog, tbl), _, _, _, _, _, _, _, _, _) =>
- if (!isV2Provider(c.provider)) {
+ val provider = c.provider.getOrElse(conf.defaultDataSourceName)
Review comment:
but this is the session catalog? Shouldn't it always use `defaultDataSourceName` to be coherent with sql?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] brkyvz commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
brkyvz commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r389097345
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -314,13 +315,14 @@ class ResolveSessionCatalog(
c.tableSchema,
// convert the bucket spec and add it as a transform
c.partitioning ++ c.bucketSpec.map(_.asTransform),
- convertTableProperties(c.properties, c.options, c.location, c.comment, Some(c.provider)),
+ convertTableProperties(c.properties, c.options, c.location, c.comment, c.provider),
orCreate = c.orCreate)
}
case c @ ReplaceTableAsSelectStatement(
SessionCatalogAndTable(catalog, tbl), _, _, _, _, _, _, _, _, _) =>
- if (!isV2Provider(c.provider)) {
+ val provider = c.provider.getOrElse(conf.defaultDataSourceName)
Review comment:
ditto
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595931080
**[Test build #119483 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119483/testReport)** for PR 27838 at commit [`94bcfb9`](https://github.com/apache/spark/commit/94bcfb99875738ce6a1eef19b389c7485ead3c87).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595922179
**[Test build #119481 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119481/testReport)** for PR 27838 at commit [`82eeb70`](https://github.com/apache/spark/commit/82eeb70e308444beac10218bc3f00105e1049ee9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-596026223
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119485/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27838: [SPARK-30902]
Allow ReplaceTableAsStatement to have none provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27838: [SPARK-30902] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595919468
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27838: [SPARK-30902]
Allow ReplaceTableAsStatement to have none provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27838: [SPARK-30902] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595916051
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-596007955
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119481/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-596007949
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595918978
**[Test build #119480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119480/testReport)** for PR 27838 at commit [`82eeb70`](https://github.com/apache/spark/commit/82eeb70e308444beac10218bc3f00105e1049ee9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] rdblue commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r389182195
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -305,7 +305,8 @@ class ResolveSessionCatalog(
// session catalog and the table provider is not v2.
case c @ ReplaceTableStatement(
SessionCatalogAndTable(catalog, tbl), _, _, _, _, _, _, _, _, _) =>
- if (!isV2Provider(c.provider)) {
+ val provider = c.provider.getOrElse(conf.defaultDataSourceName)
Review comment:
I think this should not use Spark's default. The default provider should be determined by the catalog, not by Spark. For example, if my default provider is `parquet` then how should a JDBC catalog interpret this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595929072
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595929072
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595931093
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] rdblue commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r389968253
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -305,7 +305,8 @@ class ResolveSessionCatalog(
// session catalog and the table provider is not v2.
case c @ ReplaceTableStatement(
SessionCatalogAndTable(catalog, tbl), _, _, _, _, _, _, _, _, _) =>
- if (!isV2Provider(c.provider)) {
+ val provider = c.provider.getOrElse(conf.defaultDataSourceName)
Review comment:
The provider doesn't determine whether the command should be run using v1 or v2, the table and catalog do. The provider for the session catalog matters, but not for any other catalog so I don't think changing either the `TableCatalog` or `CatalogPlugin` API is a good idea.
Since this is limited to the v2 session catalog, let's come up with a way to delegate to the v2 session catalog for this. That, or set a default.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595946154
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24214/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27838:
[SPARK-30902] Allow ReplaceTableAsStatement to have none provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27838: [SPARK-30902] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595919468
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27838: [SPARK-30902] Allow
ReplaceTableAsStatement to have none provider
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27838: [SPARK-30902] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595918978
**[Test build #119480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119480/testReport)** for PR 27838 at commit [`82eeb70`](https://github.com/apache/spark/commit/82eeb70e308444beac10218bc3f00105e1049ee9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] brkyvz commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
brkyvz commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r389096222
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
##########
@@ -2834,11 +2834,11 @@ class AstBuilder(conf: SQLConf) extends SqlBaseBaseVisitor[AnyRef] with Logging
case Some(query) =>
ReplaceTableAsSelectStatement(table, query, partitioning, bucketSpec, properties,
- provider, options, location, comment, orCreate = orCreate)
+ Some(provider), options, location, comment, orCreate = orCreate)
Review comment:
These should be `Option` not `Some`, `Some(null).get => null`, `Option(null) => None`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r389464985
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -305,7 +305,8 @@ class ResolveSessionCatalog(
// session catalog and the table provider is not v2.
case c @ ReplaceTableStatement(
SessionCatalogAndTable(catalog, tbl), _, _, _, _, _, _, _, _, _) =>
- if (!isV2Provider(c.provider)) {
+ val provider = c.provider.getOrElse(conf.defaultDataSourceName)
Review comment:
It's better to put the default provider in `V2SessionCatalog`, but the problem here is: this rule need to know the provider and see if it's v2, and then pick v1 or v2 command.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] brkyvz commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
brkyvz commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r389096222
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
##########
@@ -2834,11 +2834,11 @@ class AstBuilder(conf: SQLConf) extends SqlBaseBaseVisitor[AnyRef] with Logging
case Some(query) =>
ReplaceTableAsSelectStatement(table, query, partitioning, bucketSpec, properties,
- provider, options, location, comment, orCreate = orCreate)
+ Some(provider), options, location, comment, orCreate = orCreate)
Review comment:
These should be `Option` not `Some`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-596026218
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595948444
**[Test build #119485 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119485/testReport)** for PR 27838 at commit [`ffd7a21`](https://github.com/apache/spark/commit/ffd7a21ae2735d20fa8b8cac393d51ba37a5f3aa).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595946154
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24214/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595946147
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-596018904
**[Test build #119480 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119480/testReport)** for PR 27838 at commit [`82eeb70`](https://github.com/apache/spark/commit/82eeb70e308444beac10218bc3f00105e1049ee9).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-596019379
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119480/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-596019379
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119480/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595946147
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yuchenhuo commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
yuchenhuo commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r389120580
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -305,7 +305,8 @@ class ResolveSessionCatalog(
// session catalog and the table provider is not v2.
case c @ ReplaceTableStatement(
SessionCatalogAndTable(catalog, tbl), _, _, _, _, _, _, _, _, _) =>
- if (!isV2Provider(c.provider)) {
+ val provider = c.provider.getOrElse(conf.defaultDataSourceName)
Review comment:
The create table path is already doing this. Shall I change that too then? https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala#L281
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595929087
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24212/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27838:
[SPARK-30902] Allow ReplaceTableAsStatement to have none provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27838: [SPARK-30902] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595915499
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] rdblue commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r390000785
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -305,7 +305,8 @@ class ResolveSessionCatalog(
// session catalog and the table provider is not v2.
case c @ ReplaceTableStatement(
SessionCatalogAndTable(catalog, tbl), _, _, _, _, _, _, _, _, _) =>
- if (!isV2Provider(c.provider)) {
+ val provider = c.provider.getOrElse(conf.defaultDataSourceName)
Review comment:
Now that I understand that this problem is scoped to just how we handle the session catalog, I think it's fine to use the v1 default provider here. That's what the v1 and v2 session catalogs would use anyway.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-596885505
thanks, merging to master/3.0!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-596007483
**[Test build #119481 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119481/testReport)** for PR 27838 at commit [`82eeb70`](https://github.com/apache/spark/commit/82eeb70e308444beac10218bc3f00105e1049ee9).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yuchenhuo commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
yuchenhuo commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r390010125
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -305,7 +305,8 @@ class ResolveSessionCatalog(
// session catalog and the table provider is not v2.
case c @ ReplaceTableStatement(
SessionCatalogAndTable(catalog, tbl), _, _, _, _, _, _, _, _, _) =>
- if (!isV2Provider(c.provider)) {
+ val provider = c.provider.getOrElse(conf.defaultDataSourceName)
Review comment:
yup. @brkyvz @cloud-fan Good to go?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] brkyvz commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
brkyvz commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r389159001
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -305,7 +305,8 @@ class ResolveSessionCatalog(
// session catalog and the table provider is not v2.
case c @ ReplaceTableStatement(
SessionCatalogAndTable(catalog, tbl), _, _, _, _, _, _, _, _, _) =>
- if (!isV2Provider(c.provider)) {
+ val provider = c.provider.getOrElse(conf.defaultDataSourceName)
Review comment:
hmm... 🤔 @rdblue thoughts?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yuchenhuo commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
yuchenhuo commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r389186664
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -305,7 +305,8 @@ class ResolveSessionCatalog(
// session catalog and the table provider is not v2.
case c @ ReplaceTableStatement(
SessionCatalogAndTable(catalog, tbl), _, _, _, _, _, _, _, _, _) =>
- if (!isV2Provider(c.provider)) {
+ val provider = c.provider.getOrElse(conf.defaultDataSourceName)
Review comment:
Sounds reasonable, though I think the AST builder currently always requires a provider in the sql query? I can make the change for create and replace here to pass None in the properties if not specified. @cloud-fan thoughts?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595931093
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595928523
**[Test build #119483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119483/testReport)** for PR 27838 at commit [`94bcfb9`](https://github.com/apache/spark/commit/94bcfb99875738ce6a1eef19b389c7485ead3c87).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595922179
**[Test build #119481 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119481/testReport)** for PR 27838 at commit [`82eeb70`](https://github.com/apache/spark/commit/82eeb70e308444beac10218bc3f00105e1049ee9).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-596019374
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27838: [SPARK-30902]
Allow ReplaceTableAsStatement to have none provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27838: [SPARK-30902] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595915499
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] brkyvz commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
brkyvz commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-596823863
LGTM
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595929087
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24212/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27838:
[SPARK-30902] Allow ReplaceTableAsStatement to have none provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27838: [SPARK-30902] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595916051
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yuchenhuo commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
yuchenhuo commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r389998630
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -305,7 +305,8 @@ class ResolveSessionCatalog(
// session catalog and the table provider is not v2.
case c @ ReplaceTableStatement(
SessionCatalogAndTable(catalog, tbl), _, _, _, _, _, _, _, _, _) =>
- if (!isV2Provider(c.provider)) {
+ val provider = c.provider.getOrElse(conf.defaultDataSourceName)
Review comment:
Can we just say that the existing `defaultDataSourceName` is the default provider for v2 session catalog? Or shall I add another conf for that or add another interface for session catalog?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595948444
**[Test build #119485 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119485/testReport)** for PR 27838 at commit [`ffd7a21`](https://github.com/apache/spark/commit/ffd7a21ae2735d20fa8b8cac393d51ba37a5f3aa).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-596025959
**[Test build #119485 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119485/testReport)** for PR 27838 at commit [`ffd7a21`](https://github.com/apache/spark/commit/ffd7a21ae2735d20fa8b8cac393d51ba37a5f3aa).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] brkyvz commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
brkyvz commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r389097297
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -305,7 +305,8 @@ class ResolveSessionCatalog(
// session catalog and the table provider is not v2.
case c @ ReplaceTableStatement(
SessionCatalogAndTable(catalog, tbl), _, _, _, _, _, _, _, _, _) =>
- if (!isV2Provider(c.provider)) {
+ val provider = c.provider.getOrElse(conf.defaultDataSourceName)
Review comment:
We shouldn't do this change. Since the default provider of a catalog may be different
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r390077949
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -305,7 +305,8 @@ class ResolveSessionCatalog(
// session catalog and the table provider is not v2.
case c @ ReplaceTableStatement(
SessionCatalogAndTable(catalog, tbl), _, _, _, _, _, _, _, _, _) =>
- if (!isV2Provider(c.provider)) {
+ val provider = c.provider.getOrElse(conf.defaultDataSourceName)
Review comment:
yea good to go!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595931108
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119483/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] brkyvz commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
brkyvz commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r389958406
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -305,7 +305,8 @@ class ResolveSessionCatalog(
// session catalog and the table provider is not v2.
case c @ ReplaceTableStatement(
SessionCatalogAndTable(catalog, tbl), _, _, _, _, _, _, _, _, _) =>
- if (!isV2Provider(c.provider)) {
+ val provider = c.provider.getOrElse(conf.defaultDataSourceName)
Review comment:
A default provider doesn't make sense for CatalogPlugin, but it can make sense for a `TableCatalog`. V2SessionCatalog then can use the `conf.defaultDataSourceName`. WDYT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-596026223
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119485/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #27838: [SPARK-30902] Allow
ReplaceTableAsStatement to have none provider
Posted by GitBox <gi...@apache.org>.
gatorsmile commented on issue #27838: [SPARK-30902] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595918772
add to whitelist
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-596007949
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-596026218
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595931108
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119483/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yuchenhuo commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
yuchenhuo commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r389098554
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
##########
@@ -2834,11 +2834,11 @@ class AstBuilder(conf: SQLConf) extends SqlBaseBaseVisitor[AnyRef] with Logging
case Some(query) =>
ReplaceTableAsSelectStatement(table, query, partitioning, bucketSpec, properties,
- provider, options, location, comment, orCreate = orCreate)
+ Some(provider), options, location, comment, orCreate = orCreate)
Review comment:
Makes sense. Thanks! Though I think currently it wouldn't be null as I can't even do a replace table without the `USING` clause.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] brkyvz commented on a change in pull request #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
brkyvz commented on a change in pull request #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#discussion_r389111240
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -305,7 +305,8 @@ class ResolveSessionCatalog(
// session catalog and the table provider is not v2.
case c @ ReplaceTableStatement(
SessionCatalogAndTable(catalog, tbl), _, _, _, _, _, _, _, _, _) =>
- if (!isV2Provider(c.provider)) {
+ val provider = c.provider.getOrElse(conf.defaultDataSourceName)
Review comment:
I think that implementation needs to live within `V2SessionCatalog`, not here
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-596019374
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27838:
[SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none
provider
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27838: [SPARK-30902][SQL][FOLLOW-UP] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595928523
**[Test build #119483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119483/testReport)** for PR 27838 at commit [`94bcfb9`](https://github.com/apache/spark/commit/94bcfb99875738ce6a1eef19b389c7485ead3c87).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27838: [SPARK-30902]
Allow ReplaceTableAsStatement to have none provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27838: [SPARK-30902] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595919479
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24210/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27838:
[SPARK-30902] Allow ReplaceTableAsStatement to have none provider
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27838: [SPARK-30902] Allow ReplaceTableAsStatement to have none provider
URL: https://github.com/apache/spark/pull/27838#issuecomment-595919479
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24210/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org