You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/11/10 11:27:49 UTC
[GitHub] [spark] AngersZhuuuu opened a new pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
AngersZhuuuu opened a new pull request #30315:
URL: https://github.com/apache/spark/pull/30315
### What changes were proposed in this pull request?
According to https://github.com/apache/spark/pull/28269#issuecomment-655365714
Create a basic class for `In` and `InSet`.
### Why are the changes needed?
No
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existed UT
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-725089380
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35503/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724741428
**[Test build #130873 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130873/testReport)** for PR 30315 at commit [`ef30e9e`](https://github.com/apache/spark/commit/ef30e9ee8fcf8c3f1c9cba078f24975f3bf8ff55).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-731894139
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36126/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-731921826
I guess this needs @cloud-fan 's approval since this is his request.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724692533
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35473/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-751973304
**[Test build #133476 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133476/testReport)** for PR 30315 at commit [`ed3530a`](https://github.com/apache/spark/commit/ed3530a560927fbbf78a142aa7aec98237b7a77c).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724696706
**[Test build #130871 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130871/testReport)** for PR 30315 at commit [`485ac47`](https://github.com/apache/spark/commit/485ac471e3796a11d1fd33b0d5e6096005d90396).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-751985404
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38065/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724741428
**[Test build #130873 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130873/testReport)** for PR 30315 at commit [`ef30e9e`](https://github.com/apache/spark/commit/ef30e9ee8fcf8c3f1c9cba078f24975f3bf8ff55).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724771756
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35480/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-731898811
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36126/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724913903
**[Test build #130873 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130873/testReport)** for PR 30315 at commit [`ef30e9e`](https://github.com/apache/spark/commit/ef30e9ee8fcf8c3f1c9cba078f24975f3bf8ff55).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724785835
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724644980
**[Test build #130865 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130865/testReport)** for PR 30315 at commit [`d0371ef`](https://github.com/apache/spark/commit/d0371ef8639a3b278c007308aacea18db5c65995).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724750235
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/130871/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724649057
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/130865/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-731898824
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724692577
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-725078337
**[Test build #130897 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130897/testReport)** for PR 30315 at commit [`a3675d9`](https://github.com/apache/spark/commit/a3675d92941b0db08d2b7a36e63d3076f200797e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724927569
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-731922019
> I guess this needs @cloud-fan 's approval since this is his request.
Yea, wait for his review
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-731970852
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724738914
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724746314
**[Test build #130875 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130875/testReport)** for PR 30315 at commit [`4f824ce`](https://github.com/apache/spark/commit/4f824ce8624ac31de58ac0b5928134ee720590cb).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-816319183
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-731921826
I guess this needs @cloud-fan 's approval.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-725227411
**[Test build #130897 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130897/testReport)** for PR 30315 at commit [`a3675d9`](https://github.com/apache/spark/commit/a3675d92941b0db08d2b7a36e63d3076f200797e).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `trait BaseIn extends Predicate `
* `case class In(value: Expression, override val list: Seq[Expression]) extends BaseIn `
* `case class InSet(value: Expression, override val hset: Set[Any]) extends BaseIn `
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-752001907
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38065/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724649051
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-725095406
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35503/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-725228239
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] closed pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #30315:
URL: https://github.com/apache/spark/pull/30315
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724738914
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #30315:
URL: https://github.com/apache/spark/pull/30315#discussion_r520600358
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala
##########
@@ -561,14 +548,84 @@ case class InSet(child: Expression, hset: Set[Any]) extends UnaryExpression with
}
override def sql: String = {
- val valueSQL = child.sql
+ val valueSQL = value.sql
+ val listSQL = list.map(_.sql).mkString(", ")
+ s"($valueSQL IN ($listSQL))"
+ }
+}
+
+/**
+ * Evaluates to `true` if `list` contains `value`.
+ */
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = "expr1 _FUNC_(expr2, expr3, ...) - Returns true if `expr` equals to any valN.",
+ arguments = """
+ Arguments:
+ * expr1, expr2, expr3, ... - the arguments must be same type.
+ """,
+ examples = """
+ Examples:
+ > SELECT 1 _FUNC_(1, 2, 3);
+ true
+ > SELECT 1 _FUNC_(2, 3, 4);
+ false
+ > SELECT named_struct('a', 1, 'b', 2) _FUNC_(named_struct('a', 1, 'b', 1), named_struct('a', 1, 'b', 3));
+ false
+ > SELECT named_struct('a', 1, 'b', 2) _FUNC_(named_struct('a', 1, 'b', 2), named_struct('a', 1, 'b', 3));
+ true
+ """,
+ since = "1.0.0")
+// scalastyle:on line.size.limit
+case class In(value: Expression, list: Seq[Expression]) extends BasicIn {
+
+ override def optimized: Boolean = false
+ override def children: Seq[Expression] = value +: list
+
+ override def checkInputDataTypes(): TypeCheckResult = {
+ val mismatchOpt = list.find(l => !DataType.equalsStructurally(l.dataType, value.dataType,
+ ignoreNullability = true))
+ if (mismatchOpt.isDefined) {
+ TypeCheckResult.TypeCheckFailure(s"Arguments must be same type but were: " +
+ s"${value.dataType.catalogString} != ${mismatchOpt.get.dataType.catalogString}")
+ } else {
+ TypeUtils.checkForOrderingExpr(value.dataType, s"function $prettyName")
+ }
+ }
+
+ override def toString: String = s"$value IN ${list.mkString("(", ",", ")")}"
+}
+
+/**
+ * Optimized version of In clause, when all filter values of In clause are
+ * static.
+ */
+case class InSet(value: Expression, list: Seq[Expression]) extends BasicIn {
Review comment:
> Could we avoid change `Set[Any]` to `Seq[Expression]`?
Yea, it seems achievableļ¼ updated
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724927569
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724750219
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-731969793
**[Test build #131523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131523/testReport)** for PR 30315 at commit [`5d90048`](https://github.com/apache/spark/commit/5d900486eb991ed9dc8fd0b362e1754d79d64494).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-752001907
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38065/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-732724353
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-725228239
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724926317
**[Test build #130875 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130875/testReport)** for PR 30315 at commit [`4f824ce`](https://github.com/apache/spark/commit/4f824ce8624ac31de58ac0b5928134ee720590cb).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-751992707
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38065/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724692577
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724649051
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724785795
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35480/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-732632958
**[Test build #131601 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131601/testReport)** for PR 30315 at commit [`5d90048`](https://github.com/apache/spark/commit/5d900486eb991ed9dc8fd0b362e1754d79d64494).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu edited a comment on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu edited a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-725076288
gentle ping @cloud-fan @viirya @dongjoon-hyun
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-732638290
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724750219
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724746314
**[Test build #130875 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130875/testReport)** for PR 30315 at commit [`4f824ce`](https://github.com/apache/spark/commit/4f824ce8624ac31de58ac0b5928134ee720590cb).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-732516826
**[Test build #131601 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131601/testReport)** for PR 30315 at commit [`5d90048`](https://github.com/apache/spark/commit/5d900486eb991ed9dc8fd0b362e1754d79d64494).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724696706
**[Test build #130871 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130871/testReport)** for PR 30315 at commit [`485ac47`](https://github.com/apache/spark/commit/485ac471e3796a11d1fd33b0d5e6096005d90396).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-725095419
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-725078337
**[Test build #130897 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130897/testReport)** for PR 30315 at commit [`a3675d9`](https://github.com/apache/spark/commit/a3675d92941b0db08d2b7a36e63d3076f200797e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-731970852
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724749884
**[Test build #130871 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130871/testReport)** for PR 30315 at commit [`485ac47`](https://github.com/apache/spark/commit/485ac471e3796a11d1fd33b0d5e6096005d90396).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-732638290
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724649026
**[Test build #130865 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130865/testReport)** for PR 30315 at commit [`d0371ef`](https://github.com/apache/spark/commit/d0371ef8639a3b278c007308aacea18db5c65995).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `trait BasicIn extends Predicate `
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-752050968
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133476/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-725076288
gentle ping @cloud-fan
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-731886661
**[Test build #131523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131523/testReport)** for PR 30315 at commit [`5d90048`](https://github.com/apache/spark/commit/5d900486eb991ed9dc8fd0b362e1754d79d64494).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-751973304
**[Test build #133476 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133476/testReport)** for PR 30315 at commit [`ed3530a`](https://github.com/apache/spark/commit/ed3530a560927fbbf78a142aa7aec98237b7a77c).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
wangyum commented on a change in pull request #30315:
URL: https://github.com/apache/spark/pull/30315#discussion_r520565188
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala
##########
@@ -561,14 +548,84 @@ case class InSet(child: Expression, hset: Set[Any]) extends UnaryExpression with
}
override def sql: String = {
- val valueSQL = child.sql
+ val valueSQL = value.sql
+ val listSQL = list.map(_.sql).mkString(", ")
+ s"($valueSQL IN ($listSQL))"
+ }
+}
+
+/**
+ * Evaluates to `true` if `list` contains `value`.
+ */
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = "expr1 _FUNC_(expr2, expr3, ...) - Returns true if `expr` equals to any valN.",
+ arguments = """
+ Arguments:
+ * expr1, expr2, expr3, ... - the arguments must be same type.
+ """,
+ examples = """
+ Examples:
+ > SELECT 1 _FUNC_(1, 2, 3);
+ true
+ > SELECT 1 _FUNC_(2, 3, 4);
+ false
+ > SELECT named_struct('a', 1, 'b', 2) _FUNC_(named_struct('a', 1, 'b', 1), named_struct('a', 1, 'b', 3));
+ false
+ > SELECT named_struct('a', 1, 'b', 2) _FUNC_(named_struct('a', 1, 'b', 2), named_struct('a', 1, 'b', 3));
+ true
+ """,
+ since = "1.0.0")
+// scalastyle:on line.size.limit
+case class In(value: Expression, list: Seq[Expression]) extends BasicIn {
+
+ override def optimized: Boolean = false
+ override def children: Seq[Expression] = value +: list
+
+ override def checkInputDataTypes(): TypeCheckResult = {
+ val mismatchOpt = list.find(l => !DataType.equalsStructurally(l.dataType, value.dataType,
+ ignoreNullability = true))
+ if (mismatchOpt.isDefined) {
+ TypeCheckResult.TypeCheckFailure(s"Arguments must be same type but were: " +
+ s"${value.dataType.catalogString} != ${mismatchOpt.get.dataType.catalogString}")
+ } else {
+ TypeUtils.checkForOrderingExpr(value.dataType, s"function $prettyName")
+ }
+ }
+
+ override def toString: String = s"$value IN ${list.mkString("(", ",", ")")}"
+}
+
+/**
+ * Optimized version of In clause, when all filter values of In clause are
+ * static.
+ */
+case class InSet(value: Expression, list: Seq[Expression]) extends BasicIn {
Review comment:
Could we avoid change `Set[Any]` to `Seq[Expression]`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-735399299
> I think that the merge is not only code side. We have many code place use the `In` and `InSet`, if we decide to merge them then the origin idea should like that how to promise wherever we use and whatever use `In` or `Inset` or `InBase` we can have the best performance.
Good suggestion, in this way we can avoid complex convert code and do all the thing in `InBase`
gentle ping @cloud-fan Hope for your suggestion.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-732516826
**[Test build #131601 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131601/testReport)** for PR 30315 at commit [`5d90048`](https://github.com/apache/spark/commit/5d900486eb991ed9dc8fd0b362e1754d79d64494).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-725095419
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724914976
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724676702
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35473/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724644980
**[Test build #130865 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130865/testReport)** for PR 30315 at commit [`d0371ef`](https://github.com/apache/spark/commit/d0371ef8639a3b278c007308aacea18db5c65995).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-732724345
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724720594
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35478/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-752044738
**[Test build #133476 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133476/testReport)** for PR 30315 at commit [`ed3530a`](https://github.com/apache/spark/commit/ed3530a560927fbbf78a142aa7aec98237b7a77c).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-731886661
**[Test build #131523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131523/testReport)** for PR 30315 at commit [`5d90048`](https://github.com/apache/spark/commit/5d900486eb991ed9dc8fd0b362e1754d79d64494).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724785835
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724738884
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35478/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-731898824
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30315: [WIP][SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-724914976
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ulysses-you commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
ulysses-you commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-732698682
I think that the merge is not only code side. We have many code place use the `In` and `InSet`, if we decide to merge them then the origin idea should like that how to promise wherever we use and whatever use `In` or `Inset` or `InBase` we can have the best performance.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30315: [SPARK-33388][SQL] Merge In and InSet predicate
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30315:
URL: https://github.com/apache/spark/pull/30315#issuecomment-752050968
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133476/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org