You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/09/03 00:15:27 UTC
[GitHub] [spark] allisonwang-db opened a new pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
allisonwang-db opened a new pull request #33903:
URL: https://github.com/apache/spark/pull/33903
### What changes were proposed in this pull request?
This PR adds an additional check in the `CollapseProject` rule to prevent inlining expressions with correlated scalar subqueries.
### Why are the changes needed?
To avoid introducing unnecessary left outer joins when rewriting correlated subqueries. Projects should be collapsed after the scalar subqueries have been rewritten into left outer joins.
For example
```
// Before CollapseProject
Project [c1, s, (s * 10)]
+- Project [c1, scalar-subquery [c1] AS s]
: +- Aggregate [c1], [first(c2), c1]
: +- LocalRelation [c1, c2]
+- LocalRelation [c1, c2]
// After (scalar subqueries are inlined)
Project [c1, scalar-subquery [c1], (scalar-subquery [c1] * 10)]
: +- Aggregate [c1], [first(c2), c1]
: +- LocalRelation [c1, c2]
: +- Aggregate [c1], [first(c2), c1]
: +- LocalRelation [c1, c2]
+- LocalRelation [c1, c2]
```
Then the rule `RewriteCorrelatedScalarSubquery` rule will create two left outer joins instead of one. Also, since duplicate join attributes are not handled in this rule, this rewrite will break the structural integrity of the plan.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Unit test. Both test cases throw the following error before this PR:
```
After applying rule org.apache.spark.sql.catalyst.optimizer.RewriteCorrelatedScalarSubquery in batch Operator Optimization before Inferring Filters, the structural integrity of the plan is broken.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-926010296
**[Test build #143558 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143558/testReport)** for PR 33903 at commit [`f130a89`](https://github.com/apache/spark/commit/f130a89722f392da00cd9cfd88e5588866f036d6).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-925508631
@allisonwang-db can you fix the conficts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-926195439
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143558/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912421277
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47478/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-926460189
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912152132
**[Test build #142943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142943/testReport)** for PR 33903 at commit [`47e892d`](https://github.com/apache/spark/commit/47e892d3d0bef140613d9a88a8bbf37a8c8393f7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912571865
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142977/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-916878091
I've been thinking about it for a while, and this PR reminds me of the complaints to `CollapseProject` in the past. Correlated scalar subquery is not the only problem, `CollapseProject` may duplicate other expensive expressions and degrade the performance.
When `CollapseProject` was added, there was no whole stage codegen, and a `Project` node was quite expensive that we'd better try our best to eliminate it. Now we have whole stage codegen, and an extra `Project` node is really not an issue. Duplicated expressions are a bigger problem today. I think we can just define an allowlist of "cheap" expressions, and only allow to duplicate them in `CollapseProject`: https://github.com/apache/spark/pull/33958
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-925472748
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143521/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-925369367
**[Test build #143521 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143521/testReport)** for PR 33903 at commit [`9bbbdf2`](https://github.com/apache/spark/commit/9bbbdf2a113cf1604794443682d89c339eaf613d).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-926071195
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48067/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] allisonwang-db commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
allisonwang-db commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912680129
cc @cloud-fan @ueshin
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-925369367
**[Test build #143521 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143521/testReport)** for PR 33903 at commit [`9bbbdf2`](https://github.com/apache/spark/commit/9bbbdf2a113cf1604794443682d89c339eaf613d).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] allisonwang-db commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
allisonwang-db commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-914579551
@dongjoon-hyun There are two queries in the test case (instead of two test cases). I've updated the PR description. For (2) this PR can be considered as a bug fix too since under certain cases the rewrite can break plan structural integrity. We can backport this to other branches.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-926077809
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48067/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-925413309
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48030/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912922559
cc @sunchao
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912566664
**[Test build #142977 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142977/testReport)** for PR 33903 at commit [`47e892d`](https://github.com/apache/spark/commit/47e892d3d0bef140613d9a88a8bbf37a8c8393f7).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #33903:
URL: https://github.com/apache/spark/pull/33903
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-925410924
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48030/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-925393997
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48030/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-926191031
**[Test build #143558 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143558/testReport)** for PR 33903 at commit [`f130a89`](https://github.com/apache/spark/commit/f130a89722f392da00cd9cfd88e5588866f036d6).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-914911371
Thank you for the PR description updates, @allisonwang-db .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912367316
**[Test build #142943 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142943/testReport)** for PR 33903 at commit [`47e892d`](https://github.com/apache/spark/commit/47e892d3d0bef140613d9a88a8bbf37a8c8393f7).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912152132
**[Test build #142943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142943/testReport)** for PR 33903 at commit [`47e892d`](https://github.com/apache/spark/commit/47e892d3d0bef140613d9a88a8bbf37a8c8393f7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-926010296
**[Test build #143558 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143558/testReport)** for PR 33903 at commit [`f130a89`](https://github.com/apache/spark/commit/f130a89722f392da00cd9cfd88e5588866f036d6).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912370747
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-925472748
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143521/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-926077809
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48067/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912571865
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142977/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912389897
**[Test build #142977 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142977/testReport)** for PR 33903 at commit [`47e892d`](https://github.com/apache/spark/commit/47e892d3d0bef140613d9a88a8bbf37a8c8393f7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912426573
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47478/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-925374929
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143522/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912171035
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47444/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-925372325
**[Test build #143522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143522/testReport)** for PR 33903 at commit [`479e27a`](https://github.com/apache/spark/commit/479e27a1a505d3e8befef4c68dd446f6dc10281f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-925374929
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143522/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912922530
Hi, @allisonwang-db . I have two more questions.
1. According to the PR description, do we have another test case?
> Both test cases throw the following error before this PR
2. The affected version in JIRA is `3.3.0` because this is an improvement PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-925468420
**[Test build #143521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143521/testReport)** for PR 33903 at commit [`9bbbdf2`](https://github.com/apache/spark/commit/9bbbdf2a113cf1604794443682d89c339eaf613d).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-925372325
**[Test build #143522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143522/testReport)** for PR 33903 at commit [`479e27a`](https://github.com/apache/spark/commit/479e27a1a505d3e8befef4c68dd446f6dc10281f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-925374897
**[Test build #143522 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143522/testReport)** for PR 33903 at commit [`479e27a`](https://github.com/apache/spark/commit/479e27a1a505d3e8befef4c68dd446f6dc10281f).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912198824
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47444/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-926041258
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48067/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912426573
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47478/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912389897
**[Test build #142977 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142977/testReport)** for PR 33903 at commit [`47e892d`](https://github.com/apache/spark/commit/47e892d3d0bef140613d9a88a8bbf37a8c8393f7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-925413309
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48030/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33903: [SPARK-36656][SQL][TEST] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-926195439
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143558/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912187727
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47444/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912367970
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142943/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912198824
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47444/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912367970
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142943/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33903:
URL: https://github.com/apache/spark/pull/33903#issuecomment-912426540
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47478/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33903: [SPARK-36656][SQL] CollapseProject should not collapse correlated scalar subqueries
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #33903:
URL: https://github.com/apache/spark/pull/33903#discussion_r702246883
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala
##########
@@ -1902,4 +1902,28 @@ class SubquerySuite extends QueryTest with SharedSparkSession with AdaptiveSpark
assert(exchanges.size === 1)
}
}
+
+ test("SPARK-36656: Do not collapse projects with correlate scalar subqueries") {
+ withTempView("t1", "t2") {
+ Seq((0, 1), (1, 2)).toDF("c1", "c2").createOrReplaceTempView("t1")
+ Seq((0, 2), (0, 3)).toDF("c1", "c2").createOrReplaceTempView("t2")
+ val correctAnswer = Row(0, 2, 20) :: Row(1, null, null) :: Nil
+ checkAnswer(
+ sql(
+ """
+ |select c1, s, s * 10 from (
Review comment:
Could you capitalize the SQL keywords please, @allisonwang-db ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org