[jira] [Updated] (ARROW-15238) [C++] Create "engine" module for the query engine

["Weston Pace (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-15238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Weston Pace updated ARROW-15238:
--------------------------------
    Labels: query-engine  (was: )

> [C++] Create "engine" module for the query engine
> -------------------------------------------------
>
>                 Key: ARROW-15238
>                 URL: https://issues.apache.org/jira/browse/ARROW-15238
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: C++
>            Reporter: Weston Pace
>            Priority: Major
>              Labels: query-engine
>
> Circular dependencies are popping up in the query engine as the compute module is very low level.  For example, it would be nice if the default registry included the scan node and dataset write node.  We will want to be adding spillover support at some point and that will rely on parquet/dataset operations.
> We should create a dedicated engine module which includes the query plans, the nodes, etc.  This module would not contain the kernels or other low level compute primitives.  This way we could have something like...
> engine -> datasets (for scanning) -> parquet -> compute (for calculating statistics)
> The base ExecPlan itself could either go in compute or engine depending on which has the least amount of friction.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)