[jira] [Commented] (PIG-2917) SpillableMemoryManager memory leak for WeakReference

["Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13454671#comment-13454671 ] 

Dmitriy V. Ryaboy commented on PIG-2917:
----------------------------------------

Haitao, I am curious what your script is doing that causes it to allocate that many bags?
I just created a ticket ripping out some unnecessary usage of Spillable objects; suspect your usage is pointing us at another place where we don't really need to be dealing with SMM in the first place.

Patch seems reasonable. Stylistically, I prefer the code below for iteration of Iterables:
{code}
for (WeakReference<Spillable> ref : spillables) {
  ...
}
{code}
                
> SpillableMemoryManager memory leak for WeakReference
> ----------------------------------------------------
>
>                 Key: PIG-2917
>                 URL: https://issues.apache.org/jira/browse/PIG-2917
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.11
>            Reporter: Haitao Yao
>         Attachments: aa.jpg, SpillableMemoryManager.patch
>
>
> Sine the SpillableMemoryManager subscribed the memory notifications, and hold a weak reference of all the spillable, if there's too many weak reference and the memory exceeds the threshold, the current code will not clear the spillable weak references.
> the details is illustrated in the attached screenshot of a MemoryAnalyzer.
> So we need to clear the spillables when the memory notifications is received.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira