[jira] Commented: (PIG-1544) proactive-spill bags should share the memory alloted for it

["Olga Natkovich (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-1544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899162#action_12899162 ] 

Olga Natkovich commented on PIG-1544:
-------------------------------------

One way to do this is to only use InternalCacheBags for the bags that we are aware off upfront. Then we can have a visitor on the plan that counts the number of bags needed and divides memory accordingly.

> proactive-spill bags should share the memory alloted for it
> -----------------------------------------------------------
>
>                 Key: PIG-1544
>                 URL: https://issues.apache.org/jira/browse/PIG-1544
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Thejas M Nair
>
> Initially proactive spill bags were designed for use in (co)group (InternalCacheBag) and they knew the total number of proactive bags that were present, and shared the memory limit specified using the property pig.cachedbag.memusage .
> But the two proactive bag implementations were added later - InternalDistinctBag and InternalSortedBag are not aware of actual number of bags being used - their users always assume total-numbags = 3. 
> This needs to be fixed and all proactive-spill bags should share the memory-limit .

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.