You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "jeff (Updated) (JIRA)" <ji...@apache.org> on 2011/12/18 12:26:30 UTC

[jira] [Updated] (FLUME-895) ThriftEventSource sometime can't be closed when reconfig plugin , the spawn-threads num increase 1 per heartbeat

     [ https://issues.apache.org/jira/browse/FLUME-895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

jeff updated FLUME-895:
-----------------------

    Fix Version/s: v0.9.5
           Status: Patch Available  (was: Open)

The agent use the rpcsource translate method "thrift",  the ThriftEventSource close has a loop when the queue hase elements, the loop exit  depend on two conditions: close timeout (the default THRIFT_CLOSE_MAX_SLEEP is 10), and the events queue  size  of ThriftEventSource not changed in the period of loop.

So this bug occor by the second condition, it can't break at the queue be changed.
                
> ThriftEventSource sometime can't be closed when reconfig plugin ,  the spawn-threads num increase 1 per heartbeat 
> ------------------------------------------------------------------------------------------------------------------
>
>                 Key: FLUME-895
>                 URL: https://issues.apache.org/jira/browse/FLUME-895
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v0.9.3
>            Reporter: jeff
>            Priority: Critical
>             Fix For: v0.9.5
>
>
> In agent,I config as: rpcSource(333)| agentDFOChain("157.5.210.6:17876","157.5.210.5:17876")
> In collector,I config as: collector(17876)|myCustomplugin
> At agent mashine, I use a flume rpc client send 1000 event per sencond, after about five munite, i looked the agent monitor page, the metric "source.LazyOpenSource.ThriftEventSource.queueCapacity "  is 1000 ,which is the default size of the blocke queue in thrifteventsource,the "Source.LazyOpenSource.ThriftEventSource.enqueued" is 1357
> when I use flume shell commit reconfig agent: exec config agent_01 'rpcSource(333)' 'agentDFOChain("157.5.210.6:17876","157.5.210.5:17876")'
> I found the rpcsource cann't be reopen, the netstat command can't found  port 3333 is in listen, in log, I found the SpawningLogicalNode thread is spawn on every heartbeat
> [2011-12-12 08:03:15,670] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.handlers.thrift.ThriftEventSource 164] Closed server on port 3333...
> [2011-12-12 08:03:15,670] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.handlers.thrift.ThriftEventSource 167] Queue still has 898 elements ...
> [2011-12-12 08:03:18,614] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")
> [2011-12-12 08:03:18,614] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 164] Setting dfo failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:03:18,615] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 73] Setting failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:03:23,630] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")
> [2011-12-12 08:03:23,630] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 164] Setting dfo failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:03:23,661] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 73] Setting failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:03:28,643] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")
> [2011-12-12 08:03:28,644] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 164] Setting dfo failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:03:28,644] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 73] Setting failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:03:33,659] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")
> [2011-12-12 08:03:33,659] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 164] Setting dfo failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:03:33,659] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 73] Setting failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:03:38,665] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")
> [2011-12-12 08:03:38,704] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 164] Setting dfo failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:03:38,704] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 73] Setting failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:03:43,672] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")
> [2011-12-12 08:03:43,673] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 164] Setting dfo failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:03:43,673] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 73] Setting failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:03:48,680] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")
> [2011-12-12 08:03:48,680] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 164] Setting dfo failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:03:48,680] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 73] Setting failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:03:53,684] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")
> [2011-12-12 08:03:53,685] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 164] Setting dfo failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:03:53,764] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 73] Setting failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:03:58,688] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")
> [2011-12-12 08:03:58,688] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 164] Setting dfo failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:03:58,688] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 73] Setting failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:04:03,699] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")
> [2011-12-12 08:04:03,699] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 164] Setting dfo failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:04:03,699] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 73] Setting failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:04:08,732] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")
> [2011-12-12 08:04:08,771] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 164] Setting dfo failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:04:08,773] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 73] Setting failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:04:13,741] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")
> [2011-12-12 08:04:13,741] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 164] Setting dfo failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:04:13,742] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 73] Setting failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:04:18,755] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")
> [2011-12-12 08:04:18,755] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 164] Setting dfo failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:04:18,755] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 73] Setting failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:04:23,764] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")
> [2011-12-12 08:04:23,764] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 164] Setting dfo failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:04:23,819] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 73] Setting failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:04:28,772] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")
> [2011-12-12 08:04:28,773] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 164] Setting dfo failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:04:28,773] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 73] Setting failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:04:33,782] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")
> [2011-12-12 08:04:33,782] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 164] Setting dfo failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:04:33,782] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 73] Setting failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:04:38,792] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")
> [2011-12-12 08:04:38,792] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 164] Setting dfo failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:04:38,792] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 73] Setting failover chain to  < failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") ? diskFailover insistentAppend stubbornAppend insistentOpen failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)") >
> [2011-12-12 08:04:43,796] [INFO ] [SpawningLogicalNode pm_C04_JQ3_CNA03_E6000_a] [com.cloudera.flume.agent.AgentFailChainSink 135] Setting best effort failover chain to  failChain("{ lazyOpen => { stubbornAppend => %s } }  ","tsink(\"157.5.210.5\",17876)","tsink(\"157.5.210.6\",17876)")

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira