You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "dwsmith1983 (via GitHub)" <gi...@apache.org> on 2026/09/15 09:53:19 UTC
[I] Delegate Delta physical schema preparation to DeltaParquetFileFormat.prepareSchemaForRead [datafusion-comet]
dwsmith1983 opened a new issue, #5949:
URL: https://github.com/apache/datafusion-comet/issues/5949
### What is the problem the feature request solves?
`toPhysical` in the Delta contrib duplicates `createPhysicalSchema` and the name-mode field-id stripping that `DeltaParquetFileFormat.prepareSchemaForRead` already implements in every supported Delta version.
### Describe the potential solution
Delegate to `prepareSchemaForRead` with the generated bookkeeping columns excluded, and keep a clear division between the physical read schemas and the logical output types in `common.fields` so native output adaptation can restore logical nested types before parent expressions run, with mapped complex predicates staying residual until they bind safely to the physical schema. The current ID-mode and mapped-nested fallbacks stay until that adaptation exists. Useful controls: upgrade and rename, `to_json` of mapped structs, missing nested children, and version-specific null-struct behavior; #5661 documents the support boundary.
### Additional context
Raised in the #5365 review (item 9); filed as a follow-up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Delegate Delta physical schema preparation to DeltaParquetFileFormat.prepareSchemaForRead [datafusion-comet]
Posted by "dwsmith1983 (via GitHub)" <gi...@apache.org>.
dwsmith1983 commented on issue #5949:
URL: https://github.com/apache/datafusion-comet/issues/5949#issuecomment-6078717826
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org