[jira] [Commented] (ARROW-9716) [Rust] [DataFusion] MergeExec should have concurrency limit

["Krisztian Szucs (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-9716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17178941#comment-17178941 ] 

Krisztian Szucs commented on ARROW-9716:
----------------------------------------

Depends on backward incompatible improvement https://github.com/apache/arrow/pull/7958 and https://github.com/apache/arrow/pull/7951 which also depends on the previous dependency.

> [Rust] [DataFusion] MergeExec  should have concurrency limit
> ------------------------------------------------------------
>
>                 Key: ARROW-9716
>                 URL: https://issues.apache.org/jira/browse/ARROW-9716
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Rust, Rust - DataFusion
>            Reporter: Andy Grove
>            Assignee: Andy Grove
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.0.1, 2.0.0
>
>          Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> MergeExec currently spins up one thread per input partition which causes apps to effectively hang if there are substantially more partitions than available cores.
> We can implement a configurable limit here pretty easily.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)