You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "ahshahid (via GitHub)" <gi...@apache.org> on 2026/01/26 03:36:39 UTC
[PR] [WIP][SPARK-55185][SQL]. Fix for idempotency being broken, if InferFiltersFromCon… [spark]
ahshahid opened a new pull request, #53968:
URL: https://github.com/apache/spark/pull/53968
…straint rule is run as part of post optimzation batch with fixed ietrations
### What changes were proposed in this pull request?
Running PropagateEmptyRelation rule after PruneFilters rule, as part of the rules
operatorOptimizationRuleSet, of the Optimizer class.
### Why are the changes needed?
If InferFiltersFromConstraint rule is added to be run as part of the Batch "Operator Optimization after Inferring Filters", then the idempotency is not achieved in some cases.
The need for "InferFiltersFromConstraint" to be run as FixedPoint is needed for the following PR
[https://github.com/apache/spark/pull/53838](https://github.com/apache/spark/pull/53838)
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Added bug test
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55185][SQL]. Fix for idempotency being broken, if InferFiltersFromConstraint rule is run as part of post optimization batch with fixed iterations [spark]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #53968:
URL: https://github.com/apache/spark/pull/53968#issuecomment-4393283605
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [WIP][SPARK-55185][SQL]. Fix for idempotency being broken, if InferFiltersFromCon… [spark]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #53968:
URL: https://github.com/apache/spark/pull/53968#issuecomment-3797773104
## JIRA Issue Information
=== Bug SPARK-55185 ===
Summary: Adding rule InferFiltersFromConstraints to the Batch "Operator Optimization after Inferring Filters" causes idempotency break
Assignee: None
Status: Open
Affected: ["4.2.0","4.1.1"]
---
*This comment was automatically generated by GitHub Actions*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55185][SQL]. Fix for idempotency being broken, if InferFiltersFromConstraint rule is run as part of post optimization batch with fixed iterations [spark]
Posted by "ahshahid (via GitHub)" <gi...@apache.org>.
ahshahid commented on PR #53968:
URL: https://github.com/apache/spark/pull/53968#issuecomment-3801652634
The combined PR for SPARK-55185 and SPARK-55072 is
[PR-SPARK-55072_SPARK-55185](https://github.com/apache/spark/pull/53983)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55185][SQL]. Fix for idempotency being broken, if InferFiltersFromConstraint rule is run as part of post optimization batch with fixed iterations [spark]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #53968: [SPARK-55185][SQL]. Fix for idempotency being broken, if InferFiltersFromConstraint rule is run as part of post optimization batch with fixed iterations
URL: https://github.com/apache/spark/pull/53968
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org