You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2018/07/31 06:49:00 UTC
[jira] [Commented] (IMPALA-5844) Fix management of FunctionContext
"local" allocations.
[ https://issues.apache.org/jira/browse/IMPALA-5844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563203#comment-16563203 ]
ASF subversion and git services commented on IMPALA-5844:
---------------------------------------------------------
Commit 240fde62d532c7166fc613a97b38c199cec09f1f in impala's branch refs/heads/master from [~tarmstrong@cloudera.com]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=240fde6 ]
IMPALA-7333: remove MarkNeedsDeepCopy() in agg and BTS
This takes advantage of work (e.g. IMPALA-3200, IMPALA-5844)
to remove a couple of uses of the API.
Testing:
Ran core, ASAN and exhaustive builds.
Added unit tests to directly test the attaching behaviour.
Change-Id: I91ac53bacc00df4726c015a30ba5a2026aa4b5f5
Reviewed-on: http://gerrit.cloudera.org:8080/11007
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Fix management of FunctionContext "local" allocations.
> ------------------------------------------------------
>
> Key: IMPALA-5844
> URL: https://issues.apache.org/jira/browse/IMPALA-5844
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Fix For: Impala 2.11.0
>
>
> FunctionContexts associated with expressions make two kind of allocations with very different lifecycles. One type of allocation is owned and managed by the expression, while the other "local" allocation is implicitly transferred to the Impala daemon after control flow returns from the expression. Both are currently allocated from the same pool.
> RowBatches returned from plan nodes may reference variable-length data stored in local allocations so this memory should be attached to the RowBatches.
> One approach here is:
> * Separate local and other allocations to allocate from different MemPools.
> * Manage local allocations in bulk by clearing, freeing, or transferring data from that MemPool, similar to other memory that would be allocated from a MemPool.
> I think there are some potential wrinkles here and work to set up correct MemPools for all places that use expressions but I think the high-level approach should work.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org