You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/02/07 03:33:29 UTC
[GitHub] [spark] imback82 opened a new pull request #27482:
[WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
imback82 opened a new pull request #27482: [WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
-->
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
This PR proposes to support Hive-style `ALTER TABLE ... REPLACE COLUMNS ...` as described in https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Add/ReplaceColumns
The user now can do the following:
```SQL
CREATE TABLE t (col1 int, col2 int) USING Foo;
ALTER TABLE t REPLACE COLUMNS (col2 string COMMENT 'comment2', col3 int COMMENT 'comment3');
```
, which drops the existing columns `col1` and `col2`, and add new columns `col2` and `col3`.
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
This is a new DDL statement. Spark currently supports the Hive-style `ALTER TABLE ... CHANGE COLUMN ...`, so this new addition can be useful.
### Does this PR introduce any user-facing change?
<!--
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If no, write 'No'.
-->
Yes, adding a new DDL statement.
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
More tests to be added.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585242675
**[Test build #118294 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118294/testReport)** for PR 27482 at commit [`30785f5`](https://github.com/apache/spark/commit/30785f5db99d3c0dfeec22f6c249cfa03007b531).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585243561
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118294/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585093775
The problem is that we can't remove a SQL syntax that works in prior releases. Maybe we have to bear with it here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585313928
**[Test build #118313 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118313/testReport)** for PR 27482 at commit [`9cc277d`](https://github.com/apache/spark/commit/9cc277d0a26a5c930046a6445fd568b5bcde6268).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585133911
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585014946
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23035/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on issue #27482: [WIP][SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
imback82 commented on issue #27482: [WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-583222213
@cloud-fan This is WIP, but I have a couple of questions.
`REPLACE COLUMNS` needs to drop all the existing columns, so I am creating `Seq[TableChange]` which has `DeleteColumn`s followed by `AddColumn`s.
1) Can we assume that `TableCatalog.alterTable()` would apply the changes in the given order? (this is not documented).
2) Since it needs to drop all the existing columns, we need to look up the table before creating `AlterTable` logical plan. What I currently have is to call `loadTable` in `ResolveCatalogs`, which may not be ideal since we will do two look ups (another in `ResolveTables`). Another way is to register a callback to `AlterTable` which can be called after table is resolved. What do you think?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585726867
LGTM, merging to master!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585243561
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118294/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585014946
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23035/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585133921
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23052/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585081525
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118275/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585081514
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585133408
**[Test build #118294 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118294/testReport)** for PR 27482 at commit [`30785f5`](https://github.com/apache/spark/commit/30785f5db99d3c0dfeec22f6c249cfa03007b531).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27482:
[WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27482: [WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-583218830
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22774/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585436054
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#discussion_r380843983
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
##########
@@ -470,9 +470,15 @@ trait CheckAnalysis extends PredicateHelper {
}
}
+ val colsToDelete = mutable.Set.empty[Seq[String]]
Review comment:
Yes, working on it now!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585132919
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27482:
[WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27482: [WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-583275453
> apply the changes in the given order?
This is a good point. I think we should, can you open a PR to improve the doc?
> which may not be ideal since we will do two look ups
I think it's OK. We can clean it up later, thinking about how to resolve commands in general.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27482:
[WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27482: [WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-583275456
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118009/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#discussion_r378385876
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/connector/AlterTableTests.scala
##########
@@ -1073,4 +1073,20 @@ trait AlterTableTests extends SharedSparkSession {
assert(updated.properties === withDefaultOwnership(Map("provider" -> v2Format)).asJava)
}
}
+
+ test("AlterTable: replace columns") {
+ val t = s"${catalogAndNamespace}table_name"
+ withTable(t) {
+ sql(s"CREATE TABLE $t (col1 int, col2 int) USING $v2Format")
+ sql(s"ALTER TABLE $t REPLACE COLUMNS " +
+ "(col2 string COMMENT 'comment2', col3 int COMMENT 'comment3')")
Review comment:
The behavior of `REPLACE COLUMNS` is to drop all the existing columns first then add new columns. Thus, the comment will not be retained. I will update the test to reflect this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27482:
[WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27482: [WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-583275456
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118009/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585014942
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#discussion_r378155598
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/connector/AlterTableTests.scala
##########
@@ -1073,4 +1073,20 @@ trait AlterTableTests extends SharedSparkSession {
assert(updated.properties === withDefaultOwnership(Map("provider" -> v2Format)).asJava)
}
}
+
+ test("AlterTable: replace columns") {
+ val t = s"${catalogAndNamespace}table_name"
+ withTable(t) {
+ sql(s"CREATE TABLE $t (col1 int, col2 int) USING $v2Format")
+ sql(s"ALTER TABLE $t REPLACE COLUMNS " +
+ "(col2 string COMMENT 'comment2', col3 int COMMENT 'comment3')")
Review comment:
One question: if the `col2` already has comment but we don't specify new comment in `REPLACE COLUMNS`, will we retain the comment?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#discussion_r380458080
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
##########
@@ -470,9 +470,15 @@ trait CheckAnalysis extends PredicateHelper {
}
}
+ val colsToDelete = mutable.Set.empty[Seq[String]]
Review comment:
This causes conflicts when I backport https://github.com/apache/spark/pull/27584
I think the change in this file should go into 3.0 as well. Logically columns deleted should be skipped when checking name duplication for AddColumn.
@imback82 can you open a PR to backport https://github.com/apache/spark/pull/27584 with changes in this file?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27482:
[WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27482: [WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-583220174
**[Test build #118009 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118009/testReport)** for PR 27482 at commit [`e9a71b5`](https://github.com/apache/spark/commit/e9a71b595dd1101702f87b2458fb125560e72ff6).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585243549
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585314494
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23071/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27482:
[WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27482: [WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-583218830
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22774/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585133408
**[Test build #118294 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118294/testReport)** for PR 27482 at commit [`30785f5`](https://github.com/apache/spark/commit/30785f5db99d3c0dfeec22f6c249cfa03007b531).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
imback82 commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-584986695
We currently have the following for `ADD COLUMN`
```
| ALTER TABLE multipartIdentifier
ADD (COLUMN | COLUMNS)
columns=qualifiedColTypeWithPositionList #addTableColumns
| ALTER TABLE multipartIdentifier
ADD (COLUMN | COLUMNS)
'(' columns=qualifiedColTypeWithPositionList ')' #addTableColumns
```
But it seems that only the following is the sql standard:
```
| ALTER TABLE multipartIdentifier
ADD COLUMN?
column=qualifiedColTypeWithPosition
```
, and the following is Hive style:
```
| ALTER TABLE multipartIdentifier
ADD COLUMNS
'(' columns=qualifiedColTypeWithPositionList ')'
```
Should we fix this as well? (if so, we can combine hive style ADD and REPLACE grammar easily as well.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
imback82 commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-584981953
@cloud-fan this is now ready for review. Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585014604
**[Test build #118275 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118275/testReport)** for PR 27482 at commit [`30785f5`](https://github.com/apache/spark/commit/30785f5db99d3c0dfeec22f6c249cfa03007b531).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585243549
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27482:
[WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27482: [WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-583275437
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27482: [WIP][SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27482: [WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-583274883
**[Test build #118009 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118009/testReport)** for PR 27482 at commit [`e9a71b5`](https://github.com/apache/spark/commit/e9a71b595dd1101702f87b2458fb125560e72ff6).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class AlterTableReplaceColumnsStatement(`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585081525
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118275/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585314482
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585133911
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#discussion_r378155598
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/connector/AlterTableTests.scala
##########
@@ -1073,4 +1073,20 @@ trait AlterTableTests extends SharedSparkSession {
assert(updated.properties === withDefaultOwnership(Map("provider" -> v2Format)).asJava)
}
}
+
+ test("AlterTable: replace columns") {
+ val t = s"${catalogAndNamespace}table_name"
+ withTable(t) {
+ sql(s"CREATE TABLE $t (col1 int, col2 int) USING $v2Format")
+ sql(s"ALTER TABLE $t REPLACE COLUMNS " +
+ "(col2 string COMMENT 'comment2', col3 int COMMENT 'comment3')")
Review comment:
One question: if the `col2` already has comment but we don't specify new comment in `REPLACE COLUMNS`, shall we retain the comment? What's the behavior of Hive?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27482:
[WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27482: [WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-583275437
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585436066
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118313/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585314494
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23071/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585435332
**[Test build #118313 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118313/testReport)** for PR 27482 at commit [`9cc277d`](https://github.com/apache/spark/commit/9cc277d0a26a5c930046a6445fd568b5bcde6268).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585081514
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585314482
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585313928
**[Test build #118313 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118313/testReport)** for PR 27482 at commit [`9cc277d`](https://github.com/apache/spark/commit/9cc277d0a26a5c930046a6445fd568b5bcde6268).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27482:
[WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27482: [WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-583218827
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585081117
**[Test build #118275 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118275/testReport)** for PR 27482 at commit [`30785f5`](https://github.com/apache/spark/commit/30785f5db99d3c0dfeec22f6c249cfa03007b531).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27482:
[WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27482: [WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-583218827
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585133921
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23052/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27482:
[SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585014604
**[Test build #118275 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118275/testReport)** for PR 27482 at commit [`30785f5`](https://github.com/apache/spark/commit/30785f5db99d3c0dfeec22f6c249cfa03007b531).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585014942
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585436066
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118313/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27482: [WIP][SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27482: [WIP][SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-583220174
**[Test build #118009 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118009/testReport)** for PR 27482 at commit [`e9a71b5`](https://github.com/apache/spark/commit/e9a71b595dd1101702f87b2458fb125560e72ff6).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27482: [SPARK-30613][SQL]
Support Hive style REPLACE COLUMNS syntax
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27482: [SPARK-30613][SQL] Support Hive style REPLACE COLUMNS syntax
URL: https://github.com/apache/spark/pull/27482#issuecomment-585436054
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org