[I] DataFusion ignores "column order" parquet statistics specification [datafusion]

["alamb (via GitHub)" <gi...@apache.org>]

alamb opened a new issue, #10586:
URL: https://github.com/apache/datafusion/issues/10586

   ### Describe the bug
   
   As @tustvold points out, there is a [`column_order` API](https://docs.rs/parquet/latest/parquet/file/metadata/struct.FileMetaData.html#method.column_order) defined in parquet that is currently entirely ignored by DataFusion 
   
   It is not entirely clear to me what the implications of ignoring this field are or what other parquet writers populate it with, but we should probably not ignore it
   
   
   ### To Reproduce
   
   _No response_
   
   ### Expected behavior
   
   _No response_
   
   ### Additional context
   
   To emphasise the point I made when this API was originally proposed, you need more than just the ParquetStatistics in order to correctly interpret the data. You need at least the FileMetadata to get the https://docs.rs/parquet/latest/parquet/file/metadata/struct.FileMetaData.html#method.column_order in order to be able to even interpret what the statistics mean for a given column.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org