[jira] [Commented] (ARROW-14328) Request for enhancements to BufferAllocator.close()

["Hongze Zhang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-14328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429107#comment-17429107 ] 

Hongze Zhang commented on ARROW-14328:
--------------------------------------

A cleanup can be done at least by caching the allocated buffers and close them manually before allocator is closed, if you want to have a solution immediately based on current Arrow version.

Or I have a personal dev branch that tries to add leak detector/handler to Arrow Java, the branch includes a subfunction that cleanups allocated buffers on BufferAllocator.close(). Although it works only on root allocator.

[https://github.com/zhztheplayer/arrow-1/commits/wip-chunk-cleaner]

See an example UT:

[https://github.com/zhztheplayer/arrow-1/blob/49bb9d39d64caef9558ef8f6879aa391e74d205a/java/memory/memory-core/src/test/java/org/apache/arrow/memory/TestMemoryChunkCleaner.java#L244-L263]

> Request for enhancements to BufferAllocator.close()
> ---------------------------------------------------
>
>                 Key: ARROW-14328
>                 URL: https://issues.apache.org/jira/browse/ARROW-14328
>             Project: Apache Arrow
>          Issue Type: Wish
>            Reporter: Talia Trilling
>            Priority: Major
>
> We are using Arrow in a production environment and running into some issues with memory allocation. Specifically, we would like to be able to call `BufferAllocator.close()` and have that clean up the allocated memory even if there are existing data objects using it, rather than the method throwing an exception upon seeing that the allocator is still in use. Are there any plans to add support for releasing memory on demand? 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)