You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "bersprockets (via GitHub)" <gi...@apache.org> on 2024/03/23 21:29:04 UTC
[PR] [SPARK-47527][SQL] Normalize common expression ids during canonicalization of `With` expression [spark]
bersprockets opened a new pull request, #45677:
URL: https://github.com/apache/spark/pull/45677
### What changes were proposed in this pull request?
Add canonicalization logic to the `With` expression to normalize common expression ids in the `CommonExpressionDef` and `CommonExpressionRef` instances.
### Why are the changes needed?
This change increases the chance of a cache hit for a query using the `nullif`, `between`, or `count_if` expressions.
For example, without the fix the following does not use the cache:
```
create or replace temp view v1 as
select id from range(10);
create or replace temp view q1 as
select * from v1
where id between 2 and 4;
cache table q1;
explain select * from q1;
== Physical Plan ==
*(1) Filter ((id#51L >= 2) AND (id#51L <= 4))
+- *(1) Range (0, 10, step=1, splits=8)
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
New tests.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47527][SQL] Normalize common expression ids during canonicalization of `With` expression [spark]
Posted by "bersprockets (via GitHub)" <gi...@apache.org>.
bersprockets closed pull request #45677: [SPARK-47527][SQL] Normalize common expression ids during canonicalization of `With` expression
URL: https://github.com/apache/spark/pull/45677
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47527][SQL] Normalize common expression ids during canonicalization of `With` expression [spark]
Posted by "bersprockets (via GitHub)" <gi...@apache.org>.
bersprockets commented on PR #45677:
URL: https://github.com/apache/spark/pull/45677#issuecomment-2017053608
Looking at open PRs, this seems to be a dup of #45649. Will close for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org