You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "osipovartem (via GitHub)" <gi...@apache.org> on 2026/10/09 12:35:03 UTC
[PR] perf: resolve aliased field once during metadata inference [datafusion]
osipovartem opened a new pull request, #26156:
URL: https://github.com/apache/datafusion/pull/26156
## Which issue does this PR close?
Related to #24264. This isolates the alias single-resolution optimization from the broader, closed #24316.
## Rationale for this change
Deriving a field for nested aliases resolves each child twice: once through `metadata()` and once through `to_field()`. Since `metadata()` itself calls `to_field()`, the work grows exponentially with alias depth. This affects schema inference and planning; no row execution path changes.
## What changes are included in this PR?
Resolve the aliased child field once and read its metadata from that same field before applying explicit alias metadata. A 12-level regression test records 4,096 source-field lookups before the change and one after it, while checking metadata override, preserved metadata, output name, type, and nullability.
## What is the testing strategy for this PR?
- `cargo fmt --all --check`
- `cargo clippy --all-targets --all-features -- -D warnings`
- `cargo test -p datafusion-expr --lib` (282 passed)
- Full documented extended workspace command with initialized test-data submodules and Arrow/Parquet fixtures (passed)
- Independent read-only review: approved with no blocking findings.
## Are there any user-facing changes?
No schema or API behavior change; this reduces planning cost.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: resolve aliased field once during metadata inference [datafusion]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #26156:
URL: https://github.com/apache/datafusion/pull/26156#issuecomment-6082858275
## [Codecov](https://app.codecov.io/gh/apache/datafusion/pull/26156?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `86.11111%` with `5 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 82.77%. Comparing base ([`06aa131`](https://app.codecov.io/gh/apache/datafusion/commit/06aa131cda0d6829c517e77f3b8b08f3a24569f0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`982f93b`](https://app.codecov.io/gh/apache/datafusion/commit/982f93b0a4c0c72f52663d55b11f317a854e3ce6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/datafusion/pull/26156?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [datafusion/expr/src/expr\_schema.rs](https://app.codecov.io/gh/apache/datafusion/pull/26156?src=pr&el=tree&filepath=datafusion%2Fexpr%2Fsrc%2Fexpr_schema.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9leHByL3NyYy9leHByX3NjaGVtYS5ycw==) | 86.11% | [4 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/26156?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #26156 +/- ##
==========================================
- Coverage 82.77% 82.77% -0.01%
==========================================
Files 1147 1147
Lines 450909 450940 +31
Branches 450909 450940 +31
==========================================
+ Hits 373252 373267 +15
- Misses 54934 54946 +12
- Partials 22723 22727 +4
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/datafusion/pull/26156?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] perf: resolve aliased field once during metadata inference [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26156:
URL: https://github.com/apache/datafusion/pull/26156#issuecomment-6203611225
<!-- semver-check-comment -->
Thank you for opening this pull request!
Reviewer note: [cargo-semver-checks](https://github.com/obi1kenobi/cargo-semver-checks) reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).
<details>
<summary>Details</summary>
```
Cloning apache/main
Building datafusion-expr v55.1.0 (current)
Built [ 30.232s] (current)
Parsing datafusion-expr v55.1.0 (current)
Parsed [ 0.092s] (current)
Building datafusion-expr v55.1.0 (baseline)
Built [ 30.519s] (baseline)
Parsing datafusion-expr v55.1.0 (baseline)
Parsed [ 0.090s] (baseline)
Checking datafusion-expr v55.1.0 -> v55.1.0 (no change; assume patch)
Checked [ 1.441s] 229 checks: 227 pass, 2 fail, 0 warn, 31 skip
--- failure module_missing: pub module removed or renamed ---
Description:
A publicly-visible module cannot be imported by its prior path. A `pub use` may have been removed, or the module may have been renamed, removed, or made non-public.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.51.0/src/lints/module_missing.ron
Failed in:
mod datafusion_expr::logical_plan::materialized_cte, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/c156419899d49819f14fb51e7b948daf48020cbb/datafusion/expr/src/logical_plan/materialized_cte.rs:18
mod datafusion_expr::materialized_cte, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/c156419899d49819f14fb51e7b948daf48020cbb/datafusion/expr/src/logical_plan/materialized_cte.rs:18
--- failure struct_missing: pub struct removed or renamed ---
Description:
A publicly-visible struct cannot be imported by its prior path. A `pub use` may have been removed, or the struct itself may have been renamed or removed entirely.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.51.0/src/lints/struct_missing.ron
Failed in:
struct datafusion_expr::logical_plan::materialized_cte::MaterializedCteScan, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/c156419899d49819f14fb51e7b948daf48020cbb/datafusion/expr/src/logical_plan/materialized_cte.rs:144
struct datafusion_expr::materialized_cte::MaterializedCteScan, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/c156419899d49819f14fb51e7b948daf48020cbb/datafusion/expr/src/logical_plan/materialized_cte.rs:144
struct datafusion_expr::logical_plan::MaterializedCteScan, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/c156419899d49819f14fb51e7b948daf48020cbb/datafusion/expr/src/logical_plan/materialized_cte.rs:144
struct datafusion_expr::MaterializedCteScan, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/c156419899d49819f14fb51e7b948daf48020cbb/datafusion/expr/src/logical_plan/materialized_cte.rs:144
struct datafusion_expr::logical_plan::materialized_cte::MaterializedCteId, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/c156419899d49819f14fb51e7b948daf48020cbb/datafusion/expr/src/logical_plan/materialized_cte.rs:44
struct datafusion_expr::materialized_cte::MaterializedCteId, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/c156419899d49819f14fb51e7b948daf48020cbb/datafusion/expr/src/logical_plan/materialized_cte.rs:44
struct datafusion_expr::logical_plan::MaterializedCteId, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/c156419899d49819f14fb51e7b948daf48020cbb/datafusion/expr/src/logical_plan/materialized_cte.rs:44
struct datafusion_expr::MaterializedCteId, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/c156419899d49819f14fb51e7b948daf48020cbb/datafusion/expr/src/logical_plan/materialized_cte.rs:44
struct datafusion_expr::logical_plan::materialized_cte::MaterializedCte, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/c156419899d49819f14fb51e7b948daf48020cbb/datafusion/expr/src/logical_plan/materialized_cte.rs:69
struct datafusion_expr::materialized_cte::MaterializedCte, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/c156419899d49819f14fb51e7b948daf48020cbb/datafusion/expr/src/logical_plan/materialized_cte.rs:69
struct datafusion_expr::logical_plan::MaterializedCte, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/c156419899d49819f14fb51e7b948daf48020cbb/datafusion/expr/src/logical_plan/materialized_cte.rs:69
struct datafusion_expr::MaterializedCte, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/c156419899d49819f14fb51e7b948daf48020cbb/datafusion/expr/src/logical_plan/materialized_cte.rs:69
Summary semver requires new major version: 2 major and 0 minor checks failed
Finished [ 63.686s] datafusion-expr
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org