You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/04/21 09:37:41 UTC

[GitHub] [arrow-datafusion] tustvold commented on issue #2293: Single File Per ParquetExec, AvroExec, etc...

tustvold commented on issue #2293:
URL: https://github.com/apache/arrow-datafusion/issues/2293#issuecomment-1104960286

   > Do you mean PartitionedFile for File, but removing the partition_values field?
   
   Yes, although removing the partition_values is likely follow up work
   
   > in ParquetExec's try_new method or somewhere related place in the physical plan?
   
   I would rather keep the translation logic out of the file format specific operators, but having a free function that can be called by `ListingTable` and potentially other things, such as your Spark translation layer, seems perfectly sensible to me. I just care about reducing the amount of smarts in the individual file format specific operators :smile: 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org