You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@arrow.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/10/02 02:57:00 UTC

[jira] [Updated] (ARROW-14192) [C++][Dataset] Backpressure broken on ordered scans

     [ https://issues.apache.org/jira/browse/ARROW-14192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated ARROW-14192:
-----------------------------------
    Labels: pull-request-available query-engine  (was: query-engine)

> [C++][Dataset] Backpressure broken on ordered scans
> ---------------------------------------------------
>
>                 Key: ARROW-14192
>                 URL: https://issues.apache.org/jira/browse/ARROW-14192
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: C++
>            Reporter: Weston Pace
>            Assignee: Weston Pace
>            Priority: Major
>              Labels: pull-request-available, query-engine
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> ARROW-13611 adds a backpressure mechanism that works for unordered scans.  However, this backpressure is not properly applied on ordered (i.e. ScanBatches and not ScanBatchedUnordered) scans.  
> The fix will be to modify the merge generator used on ordered scans so that, while it still will read ahead somewhat on several files, it will never deliver batches except from the currently read file.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)