[jira] [Created] (ARROW-16410) [C++] Scanner -> ScanNode

["Weston Pace (Jira)" <ji...@apache.org>]

Weston Pace created ARROW-16410:
-----------------------------------

             Summary: [C++] Scanner -> ScanNode
                 Key: ARROW-16410
                 URL: https://issues.apache.org/jira/browse/ARROW-16410
             Project: Apache Arrow
          Issue Type: Improvement
          Components: C++
            Reporter: Weston Pace
            Assignee: Weston Pace


This is going to be a parent issue for a number of changes I'd like to make to the scanner.

 * I'd like to remove the concept of the scanner from the public API
 * Related to that, ScanOptions will probably break into two classes.  QueryOptions will be the public facing half while ScanOptions will be options sent to the ScanNode.  Most users won't see ScanOptions as it will be internal.
    * For example, QueryOptions will have {{batch_readahead}} which represents how many "query engine batches" to readahead.  Since files & the query engine have different ideas of what constitutes a "batch" the related property in ScanOptions will be {{rows_to_readahead}}.
    * Another example is projection, in QueryOptions projection is column selection as well as custom projection expressions that a user wants to run.  In ScanOptions "projection" is the desired list of columns and the output type for each column, which controls casting and inference.
 * Partially related (and partially unrelated) to the above two items I would like to move the scanner away from AsyncGenerator and recast it as an execution engine node.
 * The {{Scanner}} class will become deprecated and eventually go away.  Some methods like {{Scanner::ToTable}} may move into a new {{QueryBuilder}} or {{Query}} object.




--
This message was sent by Atlassian Jira
(v8.20.7#820007)