[jira] [Updated] (FLUME-2197) Memory Channel has GC issues

["Roshan Naik (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLUME-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Roshan Naik updated FLUME-2197:
-------------------------------

    Attachment: spill ch - mem alloc.png
                mem ch - mem alloc.png

Hard to quantify the impact verbally. Attached images show GC activity of mem channel and  Spillable Channel (with overflow disabled). Spill Channel contains a modified implementation of mem channel and addresses this GC issue and some other locking issues.

 In boths runs, java was setup to have a minHeap & maxHeap at 3GB and 6GB respectively.   You can see the allocation graphs inching higher till max heap size is reached (after 6 mins) and then staying flat. 

Answer to your question lies in the number of peaks in each graph. Each peak indicates the start of a GC cycle. 


> Memory Channel has GC issues
> ----------------------------
>
>                 Key: FLUME-2197
>                 URL: https://issues.apache.org/jira/browse/FLUME-2197
>             Project: Flume
>          Issue Type: Bug
>            Reporter: Hari Shreedharan
>            Assignee: Roshan Naik
>         Attachments: mem ch - mem alloc.png, spill ch - mem alloc.png
>
>
> Due to the fact that we use a LinkedBlockingDeque as the backing queue for the MemoryChannel, we end up hitting GC issues more often than we should.



--
This message was sent by Atlassian JIRA
(v6.2#6252)