You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@arrow.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/02/20 01:12:00 UTC

[jira] [Updated] (ARROW-11590) [C++] Move CSV background generator to IO thread pool

     [ https://issues.apache.org/jira/browse/ARROW-11590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated ARROW-11590:
-----------------------------------
    Labels: pull-request-available  (was: )

> [C++] Move CSV background generator to IO thread pool
> -----------------------------------------------------
>
>                 Key: ARROW-11590
>                 URL: https://issues.apache.org/jira/browse/ARROW-11590
>             Project: Apache Arrow
>          Issue Type: Task
>          Components: C++
>            Reporter: Weston Pace
>            Assignee: Weston Pace
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> The current readahead accomplishes its task by using an async reentrant readahead.  We get away with this today because the background generator is async reentrant because it runs on a thread pool of size 1 and so the underlying thread pool can only iterate the underlying iterator synchronously.
> However, in order to move to the I/O thread pool we need a readahead operator that is does not pull reentrantly.
> This can be done either by submitting the next task as soon as the previous one is done or by creating a "serial concurrency" executor that wraps an underlying executor.  Both approaches will need a queue so this can be done after ARROW-11588



--
This message was sent by Atlassian Jira
(v8.3.4#803005)