You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "LinSimon-901101 (via GitHub)" <gi...@apache.org> on 2026/10/16 20:20:37 UTC
[PR] fix: preserve Parquet root field IDs after nested pruning [datafusion-comet]
LinSimon-901101 opened a new pull request, #7081:
URL: https://github.com/apache/datafusion-comet/pull/7081
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: preserve Parquet root field IDs after nested pruning [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on PR #7081:
URL: https://github.com/apache/datafusion-comet/pull/7081#issuecomment-6252914863
Thanks for the fix, @LinSimon-901101. One docs follow-up: the Spark 3.4 limitation from #7031 is still listed in `docs/source/user-guide/latest/parquet.md` (the bullet "Spark 3.4: a struct matched by field id under a different name"), and it says such a struct reads as NULL after nested pruning. Once this merges that is no longer true, so could you remove that bullet in this PR?
The audit log in `docs/source/contributor-guide/data-source-audits/parquet.md` also lists #7031 under "Filed". That one is a historical record, so it is fine to leave as is unless you would rather note it as fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: preserve Parquet root field IDs after nested pruning [datafusion-comet]
Posted by "LinSimon-901101 (via GitHub)" <gi...@apache.org>.
LinSimon-901101 commented on PR #7081:
URL: https://github.com/apache/datafusion-comet/pull/7081#issuecomment-6265948174
Thanks @andygrove. I've removed the outdated limitation, kept the audit history, and synced with main. [[Fork CI](https://github.com/LinSimon-901101/datafusion-comet/actions/runs/39227343699)] now passes, including the previously failing Spark 3.4 and 3.5 `[exec]` jobs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: preserve Parquet root field IDs after nested pruning [datafusion-comet]
Posted by "LinSimon-901101 (via GitHub)" <gi...@apache.org>.
LinSimon-901101 commented on PR #7081:
URL: https://github.com/apache/datafusion-comet/pull/7081#issuecomment-6250816996
@rich7420 @andygrove Thanks for reviewing! Both Spark 3.4 and 3.5 `[exec]` jobs fail with `WRITE_ANCIENT_DATETIME`, which also occurs on `main` ([[3.4 logs](https://github.com/apache/datafusion-comet/actions/runs/39103849811/job/117606703125)], [[3.5 logs](https://github.com/apache/datafusion-comet/actions/runs/39103849811/job/117606703128)]). This appears to stem from the upstream window test’s ancient-date write configuration.
Do I need to make any changes to #7081?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: preserve Parquet root field IDs after nested pruning [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove merged PR #7081:
URL: https://github.com/apache/datafusion-comet/pull/7081
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org