You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@arrow.apache.org by "Antoine Pitrou (Jira)" <ji...@apache.org> on 2020/10/06 10:02:00 UTC

[jira] [Commented] (ARROW-10183) Create an async ParallelForEach that runs on an iterator

    [ https://issues.apache.org/jira/browse/ARROW-10183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17208626#comment-17208626 ] 

Antoine Pitrou commented on ARROW-10183:
----------------------------------------

So it's spawning a dedicated thread pool? Is there a desired use case for this feature?

> Create an async ParallelForEach that runs on an iterator
> --------------------------------------------------------
>
>                 Key: ARROW-10183
>                 URL: https://issues.apache.org/jira/browse/ARROW-10183
>             Project: Apache Arrow
>          Issue Type: Sub-task
>            Reporter: Weston Pace
>            Priority: Major
>
> This method should take in an iterator and spawn N threads to pull items off the iterator and start working on them.  It should return a future which will complete when all N threads have finished and the iterator is exhausted.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)