[jira] [Commented] (ARROW-14970) [C++][Compute] Replace ExecNode::InputReceived with ::MakeTask (Part 2)

["Weston Pace (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-14970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17452675#comment-17452675 ] 

Weston Pace commented on ARROW-14970:
-------------------------------------

We talked about this offline but I'll add it as a comment here for visibility.  I think this is a good idea but I'd like to avoid task-per-node (although that could be an acceptable stepping stone).  One way we can do this (inspired by work from [~aocsa]) is change InputReceived from:

{noformat}
void InputReceived(ExecNode* input, ExecBatch batch)
{noformat}

to something like...

{noformat}
void InputReceived(ExecNode* input, std::function<void(ExecBatch)> task)
{noformat}

A "terminating node" (e.g. sink node / pipeline breaker) could then submit the task.  A "non-terminating node" (e.g. filter/project/etc.) could wrap the task with its own work (function composition) and call InputReceived on the downstream node.

There are probably other ways to do this as well.

> [C++][Compute] Replace ExecNode::InputReceived with ::MakeTask (Part 2)
> -----------------------------------------------------------------------
>
>                 Key: ARROW-14970
>                 URL: https://issues.apache.org/jira/browse/ARROW-14970
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: C++
>    Affects Versions: 7.0.0
>            Reporter: Percy Camilo Triveño Aucahuasi
>            Assignee: Percy Camilo Triveño Aucahuasi
>            Priority: Major
>              Labels: compute
>
> In ARROW-13576 we made possible to defer computation of exec nodes but only for some implementations (e.g. Filter and Project) 
> This ticket is to continue that effort and make all the nodes submit their tasks at the moment they are operating the ExecNode::InputReceived
> Related tasks:
> https://issues.apache.org/jira/browse/ARROW-13576
> https://issues.apache.org/jira/browse/ARROW-14330
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)