You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Xinyi Zou (Jira)" <ji...@apache.org> on 2021/10/22 06:33:00 UTC
[jira] [Comment Edited] (IMPALA-3201) Implement basic in-memory
buffer pool
[ https://issues.apache.org/jira/browse/IMPALA-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17432814#comment-17432814 ]
Xinyi Zou edited comment on IMPALA-3201 at 10/22/21, 6:32 AM:
--------------------------------------------------------------
{{"Eventually ReservationTrackers should become the main mechanism for}}
{{memory accounting, once all runtime memory is handled by the buffer}}
{{pool. In the meantime, query/process limits are enforced and memory}}
{{is reported through the preexisting MemTracker hierarchy."}}
The meaning of the above paragraph means that ReservationTracker will replace MemTracker in the future?
In the current code, query/process memory restrictions are enforced and AdmissionController collects the memory status of the impalad process through MemTracker.
Therefore, I feel that the ReservationTracker currently has no practical meaning. In other words, what reservationTracker can do that MemTracker can’t do.
Thanks : ) [~tarmstrong]
was (Author: xinyi zou):
{{"Eventually ReservationTrackers should become the main mechanism for}}
{{memory accounting, once all runtime memory is handled by the buffer}}
{{pool. In the meantime, query/process limits are enforced and memory}}
{{is reported through the preexisting MemTracker hierarchy."}}
The meaning of the above paragraph means that ReservationTracker will replace MemTracker in the future?
In the current code, query/process memory restrictions are enforced and AdmissionController collects the memory status of the impalad process through MemTracker.
Therefore, I feel that the ReservationTracker currently has no practical meaning.
What are the follow-up plans, Thanks : ) [~tarmstrong]
> Implement basic in-memory buffer pool
> -------------------------------------
>
> Key: IMPALA-3201
> URL: https://issues.apache.org/jira/browse/IMPALA-3201
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Affects Versions: Impala 2.6.0
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Labels: resource-management
> Fix For: Impala 2.8.0
>
>
> This is a subtask of IMPALA-3200. The first milestone we want to hit is a usable buffer pool that does not support spilling. This would include the actual buffer pool, minus the logic for flushing unpinned blocks to disk (if it runs out of buffers, we can just terminate queries).
> Initially we should just allocate memory from TCMalloc.
> It would also include the initial work to port over exec nodes to the new buffer pool: enough to run some queries using them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org