[jira] [Created] (ARROW-17229) [C++] ReadRel is translated to a source node that emits unexpected fields

["David Li (Jira)" <ji...@apache.org>]

David Li created ARROW-17229:
--------------------------------

             Summary: [C++] ReadRel is translated to a source node that emits unexpected fields
                 Key: ARROW-17229
                 URL: https://issues.apache.org/jira/browse/ARROW-17229
             Project: Apache Arrow
          Issue Type: Improvement
          Components: C++
            Reporter: David Li


Currently, a Substrait plan with a RelRoot containing a ReadRel will contain extra, unexpected fields, namely {{__fragment_index}} et. al. Right now they are always included by default. There are a few things to be done:

* ReadRel's {{base_schema}} could be converted into a {{ScanOptions.dataset_schema}} to limit the fields read. (Also see ARROW-15585, these fields should be used for pushdown projection)
* The scanner always adds these extra fields - maybe it should be opt-in instead
* There's no way to manually insert a Project to "fix" things because as implemented, it can only add new columns



--
This message was sent by Atlassian Jira
(v8.20.10#820010)