You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "RishavTiwari25 (via GitHub)" <gi...@apache.org> on 2026/10/13 07:17:50 UTC
[PR] fix: reference columns by qualified name in `DataFrame::fill_null` / `fill_nan` [datafusion]
RishavTiwari25 opened a new pull request, #26255:
URL: https://github.com/apache/datafusion/pull/26255
## Which issue does this PR close?
- Closes #25829.
## Rationale for this change
`DataFrame::fill_null` and `DataFrame::fill_nan` fail with `FieldNotFound` on any DataFrame that has a column whose name is not a plain lowercase identifier (for example `Name` or `a.b`), even when that column is not in the list being filled.
`fill_columns` rebuilt every column of the projection with `col(field.name())`. `col` parses its argument as a SQL identifier, so `Name` was normalized to `name` and `a.b` was split into table `a`, column `b`. Neither resolves against the schema.
## What changes are included in this PR?
`fill_columns` now iterates the schema's `(qualifier, field)` pairs and builds each projection entry as `Expr::Column(Column::from((qualifier, field)))`, the same way the other DataFrame methods in this file reference their columns. The fill logic itself is unchanged.
## What is the testing strategy for this PR?
New test `test_fill_null_and_nan_with_non_normalized_column_names` in `datafusion/core/tests/dataframe/mod.rs` builds a DataFrame with columns `Name`, `a.b` and `v`, and calls `fill_null` on `v` and `Name` and `fill_nan` on `a.b`. It fails on `main` with the `FieldNotFound { name: "name" }` error from the issue and passes with this change. The existing `fill_null` / `fill_nan` tests still pass.
## Are there any user-facing changes?
`fill_null` and `fill_nan` work on DataFrames with mixed-case or dotted column names. No API change.
I used an AI coding assistant while working on this change; I have read the diff and the test and can walk through them.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: reference columns by qualified name in `DataFrame::fill_null` / `fill_nan` [datafusion]
Posted by "RishavTiwari25 (via GitHub)" <gi...@apache.org>.
RishavTiwari25 commented on PR #26255:
URL: https://github.com/apache/datafusion/pull/26255#issuecomment-6143498752
Closing: #25857 by @limadog9 already makes the same change and was opened first. I missed it when I claimed the issue, sorry for the noise.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: reference columns by qualified name in `DataFrame::fill_null` / `fill_nan` [datafusion]
Posted by "RishavTiwari25 (via GitHub)" <gi...@apache.org>.
RishavTiwari25 closed pull request #26255: fix: reference columns by qualified name in `DataFrame::fill_null` / `fill_nan`
URL: https://github.com/apache/datafusion/pull/26255
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org