[jira] [Updated] (ARROW-16389) [C++] Support hash-join on larger than memory datasets

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-16389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated ARROW-16389:
-----------------------------------
    Labels: pull-request-available  (was: )

> [C++] Support hash-join on larger than memory datasets
> ------------------------------------------------------
>
>                 Key: ARROW-16389
>                 URL: https://issues.apache.org/jira/browse/ARROW-16389
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: C++
>            Reporter: Weston Pace
>            Assignee: Sasha Krassovsky
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> The current implementation of the hash-join node current queues in memory the hashtable, the entire build side input, and the entire probe side input (e.g. the entire dataset).  This means the current implementation will run out of memory and crash if the input dataset is larger than the memory on the system.
> By spilling to disk when memory starts to fill up we can allow the hash-join node to process datasets larger than the available memory on the machine.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)