You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Weston Pace (Jira)" <ji...@apache.org> on 2022/04/21 21:45:00 UTC

[jira] [Created] (ARROW-16275) [C++] Add support for pushdown projection of nested references

Weston Pace created ARROW-16275:
-----------------------------------

             Summary: [C++] Add support for pushdown projection of nested references
                 Key: ARROW-16275
                 URL: https://issues.apache.org/jira/browse/ARROW-16275
             Project: Apache Arrow
          Issue Type: Improvement
          Components: C++
            Reporter: Weston Pace


Now that we support nested field references we should support pushdown predicates based on nested field references.  For example:

{noformat}
dataset.to_table(filter=ds.field('values', 'one') > 200)
{noformat}

{{file_parquet.cc}} tests which row groups to include when scanning a parquet fragment using parquet statistics.  At the moment it skips any non-leaf columns.  That will need to change.

Second, even if we were able to detect and produce a guarantee based on nested references, it's not clear the simplification logic would be able to detect this and appropriately simplify.  So there may be changes needed there too.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)