You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2018/07/31 06:49:00 UTC
[jira] [Commented] (IMPALA-3200) Replace BufferedBlockMgr with new
buffer pool
[ https://issues.apache.org/jira/browse/IMPALA-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563202#comment-16563202 ]
ASF subversion and git services commented on IMPALA-3200:
---------------------------------------------------------
Commit 240fde62d532c7166fc613a97b38c199cec09f1f in impala's branch refs/heads/master from [~tarmstrong@cloudera.com]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=240fde6 ]
IMPALA-7333: remove MarkNeedsDeepCopy() in agg and BTS
This takes advantage of work (e.g. IMPALA-3200, IMPALA-5844)
to remove a couple of uses of the API.
Testing:
Ran core, ASAN and exhaustive builds.
Added unit tests to directly test the attaching behaviour.
Change-Id: I91ac53bacc00df4726c015a30ba5a2026aa4b5f5
Reviewed-on: http://gerrit.cloudera.org:8080/11007
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Replace BufferedBlockMgr with new buffer pool
> ---------------------------------------------
>
> Key: IMPALA-3200
> URL: https://issues.apache.org/jira/browse/IMPALA-3200
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.6.0
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Blocker
> Labels: resource-management
> Fix For: Impala 2.10.0
>
>
> We want to replace BufferedBlockMgr, a query-wide buffer pool, with a new BufferPool that is shared between all queries. The goals are:
> * Support for guaranteed reservations: i.e. if a reservation is granted, the buffer pool will fulfil it (unless the OS is unable to fulfill the buffer pool's memory requirements).
> * Simplified interaction between reservations and pins (if you have a reservation, you can pin, if you don't, you can't)
> * Support for increasing reservations (up to a planner-specified limit).
> * Support for smaller buffer sizes with similar performance (so we can reduce minimum memory requirement to execute spill-to-disk algorithm)
> * Support for larger buffers to support wide rows
> * Reduced reliance on TCMalloc, which isn't suited to management of large buffers (e.g. see IMPALA-2800)
> * Better transfer model for buffer pool pages, so we can implement transfer-of-ownership consistently for row batches (instead of mixing transfer with MarkNeedsToReturn()).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org