You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/05/21 21:34:42 UTC

[GitHub] [arrow-datafusion] alamb commented on pull request #380: Support statistics pruning for formats other than parquet

alamb commented on pull request #380:
URL: https://github.com/apache/arrow-datafusion/pull/380#issuecomment-846274766


   > I think this is really cool, I think it would be also great to have this for in-memory tables.
   
   I agree -- I think all that is needed is to calculate the min/max statistics for each partition (or maybe even record batch) though we might have to be careful not to slow down queries where it wouldn't help. Maybe it could be opt in. Or perhaps we could compute the statistics "on demand" (after we have created a PruningPredicate)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org