You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "adriangb (via GitHub)" <gi...@apache.org> on 2026/05/15 21:53:19 UTC
[PR] Add OptionalFilterPhysicalExpr wrapper + proto support [datafusion]
adriangb opened a new pull request, #22234:
URL: https://github.com/apache/datafusion/pull/22234
## Which issue does this PR close?
- Part of #22144 (Adaptive filter pushdown), split into a reviewable stack. This is **PR 1 of 4**.
## Rationale for this change
Adaptive filter scheduling needs a way to mark a `PhysicalExpr` as a *performance hint* — a filter that may be dropped without affecting query correctness (e.g. a hash-join dynamic filter, whose predicate the join already enforces). This PR adds that marker type. It is deliberately inert: nothing reads it yet.
## What changes are included in this PR?
- `OptionalFilterPhysicalExpr`: a transparent `PhysicalExpr` wrapper that delegates every trait method to its inner expression.
- Proto support: a new `PhysicalOptionalFilterNode` message so physical plans containing the wrapper round-trip through `datafusion-proto`.
No caller wraps anything yet — the hash-join wrap and the parquet scheduler that consume the marker arrive later in the stack.
## Are these changes tested?
Yes — unit tests for the wrapper's delegation behavior and a proto round-trip test.
## Are there any user-facing changes?
New public type `OptionalFilterPhysicalExpr`. Purely additive; no behavior change.
---
Stacked PRs (review/merge in order):
1. **this PR** — OptionalFilterPhysicalExpr + proto
2. Per-conjunct pruning statistics
3. SelectivityTracker cost model
4. Adaptive parquet scan integration
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add OptionalFilterPhysicalExpr wrapper + proto support [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22234:
URL: https://github.com/apache/datafusion/pull/22234#issuecomment-4464037352
<!-- semver-check-comment -->
Thank you for opening this pull request!
Reviewer note: [cargo-semver-checks](https://github.com/obi1kenobi/cargo-semver-checks) reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).
<details>
<summary>Details</summary>
```
Cloning apache/main
Building datafusion-physical-expr-common v53.1.0 (current)
Built [ 28.811s] (current)
Parsing datafusion-physical-expr-common v53.1.0 (current)
Parsed [ 0.022s] (current)
Building datafusion-physical-expr-common v53.1.0 (baseline)
Built [ 18.807s] (baseline)
Parsing datafusion-physical-expr-common v53.1.0 (baseline)
Parsed [ 0.022s] (baseline)
Checking datafusion-physical-expr-common v53.1.0 -> v53.1.0 (no change; assume patch)
Checked [ 0.278s] 222 checks: 222 pass, 30 skip
Summary no semver update required
Finished [ 49.155s] datafusion-physical-expr-common
Building datafusion-proto v53.1.0 (current)
Built [ 52.143s] (current)
Parsing datafusion-proto v53.1.0 (current)
Parsed [ 0.140s] (current)
Building datafusion-proto v53.1.0 (baseline)
Built [ 52.274s] (baseline)
Parsing datafusion-proto v53.1.0 (baseline)
Parsed [ 0.138s] (baseline)
Checking datafusion-proto v53.1.0 -> v53.1.0 (no change; assume patch)
Checked [ 2.245s] 222 checks: 221 pass, 1 fail, 0 warn, 30 skip
--- failure enum_variant_added: enum variant added on exhaustive enum ---
Description:
A publicly-visible enum without #[non_exhaustive] has a new variant.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#enum-variant-new
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.47.0/src/lints/enum_variant_added.ron
Failed in:
variant ExprType:OptionalFilter in /home/runner/work/datafusion/datafusion/datafusion/proto/src/generated/prost.rs:1397
variant ExprType:OptionalFilter in /home/runner/work/datafusion/datafusion/datafusion/proto/src/generated/prost.rs:1397
Summary semver requires new major version: 1 major and 0 minor checks failed
Finished [ 109.275s] datafusion-proto
```
</details>
<!-- semver-check-comment -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add OptionalFilterPhysicalExpr wrapper + proto support [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #22234: Add OptionalFilterPhysicalExpr wrapper + proto support
URL: https://github.com/apache/datafusion/pull/22234
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add OptionalFilterPhysicalExpr wrapper + proto support [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22234:
URL: https://github.com/apache/datafusion/pull/22234#issuecomment-4976190317
Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org