[jira] [Created] (ARROW-13263) [C++][Compute] Allow Fragments to attach guarantees to scanned batches

["Ben Kietzman (Jira)" <ji...@apache.org>]

Ben Kietzman created ARROW-13263:
------------------------------------

             Summary: [C++][Compute] Allow Fragments to attach guarantees to scanned batches
                 Key: ARROW-13263
                 URL: https://issues.apache.org/jira/browse/ARROW-13263
             Project: Apache Arrow
          Issue Type: Improvement
          Components: C++
            Reporter: Ben Kietzman


A Fragment may be able to attach guarantee expressions to individual batches beyond just its partition expression. For example, a parquet fragment can attach row group statistics. These guarantees can be leveraged by subsequent ExecNodes to optimize execution, for example to skip execution of unnecessary filter expressions.

In its simplest form this is probably an overload of ScanBatchesAsync which yields ExecBatches (which have a {{guarantee}} property) instead of RecordBatches. When transforming to an ExecBatch, (see {{expression.h::MakeExecBatch}} it'd be useful to also eagerly drop columns which are not referenced by any nodes in the graph- just in case the Fragment couldn't push the projection down any further.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)