You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Wes McKinney (Jira)" <ji...@apache.org> on 2021/02/19 23:30:00 UTC
[jira] [Commented] (ARROW-3406) [C++] Create a caching memory pool
implementation
[ https://issues.apache.org/jira/browse/ARROW-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17287421#comment-17287421 ]
Wes McKinney commented on ARROW-3406:
-------------------------------------
This would probably better be a "buffer pool" in the context of query processing (if at all), where many smallish-buffers are allocated and discarded. Curious what other systems do
> [C++] Create a caching memory pool implementation
> -------------------------------------------------
>
> Key: ARROW-3406
> URL: https://issues.apache.org/jira/browse/ARROW-3406
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Affects Versions: 0.11.0
> Reporter: Antoine Pitrou
> Priority: Minor
> Labels: query-engine
>
> A caching memory pool implementation would be able to recycle freed memory blocks instead of returning them to the system immediately. Two different policies may be chosen:
> * either an unbounded cache
> * or a size-limited cache, perhaps with some kind of LRU mechanism
> Such a feature might help e.g. for CSV parsing, when reading and parsing data into temporary memory buffers.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)