You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "mkleen (via GitHub)" <gi...@apache.org> on 2026/07/28 21:00:46 UTC
[PR] Improve error message for metadata conflict in schema [datafusion]
mkleen opened a new pull request, #23952:
URL: https://github.com/apache/datafusion/pull/23952
## Which issue does this PR close?
- Closes https://github.com/apache/datafusion/issues/23461
## Rationale for this change
Based on the discussion in https://github.com/apache/datafusion/issues/23461 we decided to just improves the error message, so the user knows there is a metadata conflict in the schema. It also reverts https://github.com/apache/datafusion/pull/23605, so `CrossJoinExec` has it's original behaviour. This changes a metadata conflict from a user perspective from:
```sql
select * from larger_table cross join smaller_table;
DataFusion error: Internal error: PhysicalOptimizer rule 'join_selection' failed. Schema mismatch. Expected original schema: Field { "a": Int32 }, Field { "b": Int32 }, got new schema: Field { "a": Int32 }, Field { "b": Int32 }
```
to:
```sql
select * from larger_table cross join smaller_table;
DataFusion error: PhysicalOptimizer rule 'join_selection' failed. Schema mismatch caused by Internal error: Schema metadata mismatch: Expected original metadata: {"metadata_key": "right"}, got metadata: {"metadata_key": "left"}.
```
## What changes are included in this PR?
- Improve error message for schema validation in physical_planner.rs
- Data creation in test_context.rs
- Reproduction in slt
- Revert behaviour of `CrossJoinExec`
## Are these changes tested?
Yes
## Are there any user-facing changes?
Yes, error message improved when the metadata has a conflict.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: Improve error message for metadata conflict in schema [datafusion]
Posted by "2010YOUY01 (via GitHub)" <gi...@apache.org>.
2010YOUY01 merged PR #23952:
URL: https://github.com/apache/datafusion/pull/23952
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: Improve error message for metadata conflict in schema [datafusion]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #23952:
URL: https://github.com/apache/datafusion/pull/23952#issuecomment-5120699246
## [Codecov](https://app.codecov.io/gh/apache/datafusion/pull/23952?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `97.29730%` with `1 line` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 80.75%. Comparing base ([`68d5874`](https://app.codecov.io/gh/apache/datafusion/commit/68d587468073c1fc84ffe00db66f35d506dcc828?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`a96c221`](https://app.codecov.io/gh/apache/datafusion/commit/a96c221bf6627c3b81bdf812669f0f526f1541d7?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 6 commits behind head on main.
| [Files with missing lines](https://app.codecov.io/gh/apache/datafusion/pull/23952?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [datafusion/core/src/physical\_planner.rs](https://app.codecov.io/gh/apache/datafusion/pull/23952?src=pr&el=tree&filepath=datafusion%2Fcore%2Fsrc%2Fphysical_planner.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9jb3JlL3NyYy9waHlzaWNhbF9wbGFubmVyLnJz) | 94.11% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23952?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #23952 +/- ##
========================================
Coverage 80.75% 80.75%
========================================
Files 1096 1096
Lines 373282 373522 +240
Branches 373282 373522 +240
========================================
+ Hits 301440 301642 +202
- Misses 53869 53894 +25
- Partials 17973 17986 +13
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/datafusion/pull/23952?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: Improve error message for metadata conflict in schema [datafusion]
Posted by "mkleen (via GitHub)" <gi...@apache.org>.
mkleen commented on PR #23952:
URL: https://github.com/apache/datafusion/pull/23952#issuecomment-5120833101
@2010YOUY01 Could you do a review please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: Improve error message for metadata conflict in schema [datafusion]
Posted by "mkleen (via GitHub)" <gi...@apache.org>.
mkleen commented on PR #23952:
URL: https://github.com/apache/datafusion/pull/23952#issuecomment-5129358574
> LGTM, thank you!
Thanks for the review!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org