You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flume.apache.org by "gautham varada (JIRA)" <ji...@apache.org> on 2014/09/14 09:01:33 UTC

[jira] [Created] (FLUME-2465) Protections against OOM

gautham varada created FLUME-2465:
-------------------------------------

             Summary: Protections against OOM
                 Key: FLUME-2465
                 URL: https://issues.apache.org/jira/browse/FLUME-2465
             Project: Flume
          Issue Type: New Feature
          Components: Sinks+Sources
    Affects Versions: v1.5.0
         Environment: centos 6.3, open jdk 7
            Reporter: gautham varada
            Priority: Minor


I wanted to propose some kind of a mechanism in the Sinkrunner to prevent processing large messages to avoid OOM.  Invariably we all end up configuring multiple sinks to read messages in batches to drain the channel faster, this can potentially lead to OOM depending on the size of the message.

Can we implement some kind of a cyclic barrier (not sure if it needs to be in the sink runner) for the sink threads to wait once we hit a preconfigured threshold?





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)