You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Weston Pace (Jira)" <ji...@apache.org> on 2022/04/21 21:45:00 UTC
[jira] [Created] (ARROW-16275) [C++] Add support for pushdown projection of nested references
Weston Pace created ARROW-16275:
-----------------------------------
Summary: [C++] Add support for pushdown projection of nested references
Key: ARROW-16275
URL: https://issues.apache.org/jira/browse/ARROW-16275
Project: Apache Arrow
Issue Type: Improvement
Components: C++
Reporter: Weston Pace
Now that we support nested field references we should support pushdown predicates based on nested field references. For example:
{noformat}
dataset.to_table(filter=ds.field('values', 'one') > 200)
{noformat}
{{file_parquet.cc}} tests which row groups to include when scanning a parquet fragment using parquet statistics. At the moment it skips any non-leaf columns. That will need to change.
Second, even if we were able to detect and produce a guarantee based on nested references, it's not clear the simplification logic would be able to detect this and appropriately simplify. So there may be changes needed there too.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)