[jira] [Commented] (ARROW-15718) [R] Joining two datasets crashes if use_threads=FALSE

["Will Jones (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-15718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17519723#comment-17519723 ] 

Will Jones commented on ARROW-15718:
------------------------------------

A long-term solution might be implemented in something like https://github.com/apache/arrow/pull/12468, but that is blocked by ARROW-16072.

For 8.0.0 we should implement a workaround. I'll implement something simpler than what I started in https://github.com/apache/arrow/pull/12437

> [R] Joining two datasets crashes if use_threads=FALSE
> -----------------------------------------------------
>
>                 Key: ARROW-15718
>                 URL: https://issues.apache.org/jira/browse/ARROW-15718
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: C++, R
>    Affects Versions: 7.0.0
>            Reporter: Will Jones
>            Assignee: Will Jones
>            Priority: Major
>             Fix For: 8.0.0
>
>
> In ARROW-14908 we solved the case of joining a dataset to an in memory table, but did not solve joining two datasets.
> The previous solution was to add +1 to the thread count, because the hash join logic might be called by the scanner's IO thread. For joining more than 1 dataset, we might have more than 1 IO thread, so we either need to add a larger arbitrary number or find a way to make the state logic more resilient to unexpected threads.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)