You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/04/06 00:03:40 UTC
[GitHub] [spark] tdas opened a new pull request #32058: [SPARK-34962][SQL] Explicit representation of * in UpdateAction and InsertAction in MergeIntoTable
tdas opened a new pull request #32058:
URL: https://github.com/apache/spark/pull/32058
### What changes were proposed in this pull request?
Change UpdateAction and InsertAction of MergeIntoTable to explicitly represent star,
### Why are the changes needed?
Currently, UpdateAction and InsertAction in the MergeIntoTable implicitly represent `update set *` and `insert *` with empty assignments. That means there is no way to differentiate between the representations of "update all columns" and "update no columns". For SQL MERGE queries, this inability does not matter because the SQL MERGE grammar that generated the MergeIntoTable plan does not allow "update no columns". However, other ways of generating the MergeIntoTable plan may not have that limitation, and may want to allow specifying "update no columns". For example, in the Delta Lake project we provide a type-safe Scala API for Merge, where it is perfectly valid to produce a Merge query with an update clause but no update assignments. Currently, we cannot use MergeIntoTable to represent this plan, thus complicating the generation, and resolution of merge query from scala API.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existing unit tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32058: [SPARK-34962][SQL] Explicit representation of * in UpdateAction and InsertAction in MergeIntoTable
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32058:
URL: https://github.com/apache/spark/pull/32058#issuecomment-813824169
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136922/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #32058: [SPARK-34962][SQL] Explicit representation of * in UpdateAction and InsertAction in MergeIntoTable
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32058:
URL: https://github.com/apache/spark/pull/32058#discussion_r607570776
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala
##########
@@ -420,13 +420,15 @@ case class DeleteAction(condition: Option[Expression]) extends MergeAction
case class UpdateAction(
condition: Option[Expression],
- assignments: Seq[Assignment]) extends MergeAction {
+ assignments: Seq[Assignment],
+ assignStar: Boolean) extends MergeAction {
override def children: Seq[Expression] = condition.toSeq ++ assignments
}
case class InsertAction(
condition: Option[Expression],
- assignments: Seq[Assignment]) extends MergeAction {
+ assignments: Seq[Assignment],
+ assignStar: Boolean) extends MergeAction {
Review comment:
shall we simply add a new implementation of `MergeAction`: `AssignStarAction`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] tdas closed pull request #32058: [SPARK-34962][SQL] Explicit representation of * in UpdateAction and InsertAction in MergeIntoTable
Posted by GitBox <gi...@apache.org>.
tdas closed pull request #32058:
URL: https://github.com/apache/spark/pull/32058
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32058: [SPARK-34962][SQL] Explicit representation of * in UpdateAction and InsertAction in MergeIntoTable
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32058:
URL: https://github.com/apache/spark/pull/32058#issuecomment-813750147
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41499/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32058: [SPARK-34962][SQL] Explicit representation of * in UpdateAction and InsertAction in MergeIntoTable
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32058:
URL: https://github.com/apache/spark/pull/32058#issuecomment-813824169
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136922/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32058: [SPARK-34962][SQL] Explicit representation of * in UpdateAction and InsertAction in MergeIntoTable
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32058:
URL: https://github.com/apache/spark/pull/32058#issuecomment-813731177
**[Test build #136922 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136922/testReport)** for PR 32058 at commit [`26b5104`](https://github.com/apache/spark/commit/26b5104a9283b070c11a13990aab8f0ac6ca687a).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32058: [SPARK-34962][SQL] Explicit representation of * in UpdateAction and InsertAction in MergeIntoTable
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32058:
URL: https://github.com/apache/spark/pull/32058#issuecomment-813750147
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41499/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] tdas commented on a change in pull request #32058: [SPARK-34962][SQL] Explicit representation of * in UpdateAction and InsertAction in MergeIntoTable
Posted by GitBox <gi...@apache.org>.
tdas commented on a change in pull request #32058:
URL: https://github.com/apache/spark/pull/32058#discussion_r607878704
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala
##########
@@ -420,13 +420,15 @@ case class DeleteAction(condition: Option[Expression]) extends MergeAction
case class UpdateAction(
condition: Option[Expression],
- assignments: Seq[Assignment]) extends MergeAction {
+ assignments: Seq[Assignment],
+ assignStar: Boolean) extends MergeAction {
override def children: Seq[Expression] = condition.toSeq ++ assignments
}
case class InsertAction(
condition: Option[Expression],
- assignments: Seq[Assignment]) extends MergeAction {
+ assignments: Seq[Assignment],
+ assignStar: Boolean) extends MergeAction {
Review comment:
in that case shouldnt it be `InsertStarAction(condition)` and `UpdateStarAction(condition)`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32058: [SPARK-34962][SQL] Explicit representation of * in UpdateAction and InsertAction in MergeIntoTable
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32058:
URL: https://github.com/apache/spark/pull/32058#issuecomment-813749548
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32058: [SPARK-34962][SQL] Explicit representation of * in UpdateAction and InsertAction in MergeIntoTable
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32058:
URL: https://github.com/apache/spark/pull/32058#issuecomment-813731177
**[Test build #136922 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136922/testReport)** for PR 32058 at commit [`26b5104`](https://github.com/apache/spark/commit/26b5104a9283b070c11a13990aab8f0ac6ca687a).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32058: [SPARK-34962][SQL] Explicit representation of * in UpdateAction and InsertAction in MergeIntoTable
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32058:
URL: https://github.com/apache/spark/pull/32058#issuecomment-813815644
**[Test build #136922 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136922/testReport)** for PR 32058 at commit [`26b5104`](https://github.com/apache/spark/commit/26b5104a9283b070c11a13990aab8f0ac6ca687a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org