[jira] [Commented] (IMPALA-5844) Fix management of FunctionContext "local" allocations.

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-5844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16576543#comment-16576543 ] 

ASF subversion and git services commented on IMPALA-5844:
---------------------------------------------------------

Commit 4af3a7853e9d450675a61bff5b031a12e7a05172 in impala's branch refs/heads/master from [~tarmstrong@cloudera.com]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=4af3a78 ]

IMPALA-7333: remove MarkNeedsDeepCopy() in agg and BTS

This takes advantage of work (e.g. IMPALA-3200, IMPALA-5844)
to remove a couple of uses of the API.

Testing:
Ran core, ASAN and exhaustive builds.

Added unit tests to directly test the attaching behaviour.

Change-Id: I5f5b8a418d4816f603a64da6287ec392dbd4603f
Reviewed-on: http://gerrit.cloudera.org:8080/11156
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Fix management of FunctionContext "local" allocations.
> ------------------------------------------------------
>
>                 Key: IMPALA-5844
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5844
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Backend
>            Reporter: Tim Armstrong
>            Assignee: Tim Armstrong
>            Priority: Major
>             Fix For: Impala 2.11.0
>
>
> FunctionContexts associated with expressions make two kind of allocations with very different lifecycles. One type of allocation is owned and managed by the expression, while the other "local" allocation is implicitly transferred to the Impala daemon after control flow returns from the expression. Both are currently allocated from the same pool.
> RowBatches returned from plan nodes may reference variable-length data stored in local allocations so this memory should be attached to the RowBatches.
> One approach here is:
> * Separate local and other allocations to allocate from different MemPools. 
> * Manage local allocations in bulk by clearing, freeing, or transferring data from that MemPool, similar to other memory that would be allocated from a MemPool.
> I think there are some potential wrinkles here and work to set up correct MemPools for all places that use expressions but I think the high-level approach should work.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org