You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Weston Pace (Jira)" <ji...@apache.org> on 2021/02/10 19:12:00 UTC
[jira] [Created] (ARROW-11590) [C++] Move CSV background generator
to IO thread pool
Weston Pace created ARROW-11590:
-----------------------------------
Summary: [C++] Move CSV background generator to IO thread pool
Key: ARROW-11590
URL: https://issues.apache.org/jira/browse/ARROW-11590
Project: Apache Arrow
Issue Type: Task
Components: C++
Reporter: Weston Pace
Assignee: Weston Pace
The current readahead accomplishes its task by using an async reentrant readahead. We get away with this today because the background generator is async reentrant because it runs on a thread pool of size 1 and so the underlying thread pool can only iterate the underlying iterator synchronously.
However, in order to move to the I/O thread pool we need a readahead operator that is does not pull reentrantly.
This can be done either by submitting the next task as soon as the previous one is done or by creating a "serial concurrency" executor that wraps an underlying executor. Both approaches will need a queue so this can be done after ARROW-11588
--
This message was sent by Atlassian Jira
(v8.3.4#803005)