[GitHub] [iceberg] RussellSpitzer opened a new issue #1552: Spark Predicate Pushdown for Metadata Table Scans

[GitBox <gi...@apache.org>]

RussellSpitzer opened a new issue #1552:
URL: https://github.com/apache/iceberg/issues/1552


   As is there is no way to scan and return only partial results from a MetadataTable Scan. There are some use cases, like distributed planning, which would benefit from pushing down predicates to the Manifest reading itself to save time and bits over the wire.
   
   The goal here is that we should be able to pass in Metric or Partition predicates all the way down to the manifest scanning itself to prevent reading unnecessary files and serializing unnecessary rows. 
   
   
   I plan on taking a look at this once the distributed planning PR is merged.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org