You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Todd Farmer (Jira)" <ji...@apache.org> on 2022/08/26 16:06:00 UTC
[jira] [Commented] (ARROW-16410) [C++] Scanner -> ScanNode
[ https://issues.apache.org/jira/browse/ARROW-16410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585490#comment-17585490 ]
Todd Farmer commented on ARROW-16410:
-------------------------------------
This issue was last updated over 90 days ago, which may be an indication it is no longer being actively worked. To better reflect the current state, the issue is being unassigned per [project policy|https://arrow.apache.org/docs/dev/developers/bug_reports.html#issue-assignment. Please feel free to re-take assignment of the issue if it is being actively worked, or if you plan to start that work soon.
> [C++] Scanner -> ScanNode
> -------------------------
>
> Key: ARROW-16410
> URL: https://issues.apache.org/jira/browse/ARROW-16410
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Weston Pace
> Assignee: Weston Pace
> Priority: Major
>
> This is going to be a parent issue for a number of changes I'd like to make to the scanner.
> * I'd like to remove the concept of the scanner from the public API
> * Related to that, ScanOptions will probably break into two classes. QueryOptions will be the public facing half while ScanOptions will be options sent to the ScanNode. Most users won't see ScanOptions as it will be internal.
> * For example, QueryOptions will have {{batch_readahead}} which represents how many "query engine batches" to readahead. Since files & the query engine have different ideas of what constitutes a "batch" the related property in ScanOptions will be {{rows_to_readahead}}.
> * Another example is projection, in QueryOptions projection is column selection as well as custom projection expressions that a user wants to run. In ScanOptions "projection" is the desired list of columns and the output type for each column, which controls casting and inference.
> * Partially related (and partially unrelated) to the above two items I would like to move the scanner away from AsyncGenerator and recast it as an execution engine node.
> * The {{Scanner}} class will become deprecated and eventually go away. Some methods like {{Scanner::ToTable}} may move into a new {{QueryBuilder}} or {{Query}} object.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)