You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Tim Chng (JIRA)" <ji...@apache.org> on 2015/03/24 21:11:52 UTC
[jira] [Commented] (FLUME-2650) Sinkgroup round robin load balancer
hits first sink repeatedly
[ https://issues.apache.org/jira/browse/FLUME-2650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378497#comment-14378497 ]
Tim Chng commented on FLUME-2650:
---------------------------------
we have 64 sinks in our sinkgroup:
agent.sinkgroups.sinkGroup1.sinks = prodMapr1 prodMapr9 prodMapr17 prodMapr25 prodMapr33 prodMapr41 prodMapr49 prodMapr57 prodMapr2 prodMapr10 prodMapr18 prodMapr26 prodMapr34 prodMapr42 prodMapr50 prodMapr58 prodMapr3 prodMapr11 prodMapr19 prodMapr27 prodMapr35 prodMapr43 prodMapr51 prodMapr59 prodMapr4 prodMapr12 prodMapr20 prodMapr28 prodMapr36 prodMapr44 prodMapr52 prodMapr60 prodMapr5 prodMapr13 prodMapr21 prodMapr29 prodMapr37 prodMapr45 prodMapr53 prodMapr61 prodMapr6 prodMapr14 prodMapr22 prodMapr30 prodMapr38 prodMapr46 prodMapr54 prodMapr62 prodMapr7 prodMapr15 prodMapr23 prodMapr31 prodMapr39 prodMapr47 prodMapr55 prodMapr63 prodMapr8 prodMapr16 prodMapr24 prodMapr32 prodMapr40 prodMapr48 prodMapr56 prodMapr64
agent.sinkgroups.sinkGroup1.processor.type = load_balance
agent.sinkgroups.sinkGroup1.processor.backoff = true
agent.sinkgroups.sinkGroup1.processor.selector = random
agent.sinkgroups.sinkGroup1.processor.selector.maxTimeOut = 1000
The prodMapr1 sink is hit with 10X more events than all the other sinks
> Sinkgroup round robin load balancer hits first sink repeatedly
> --------------------------------------------------------------
>
> Key: FLUME-2650
> URL: https://issues.apache.org/jira/browse/FLUME-2650
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.5.0
> Environment: mapR flume v1.5
> Reporter: Tim Chng
>
> When a sinkgroup is configured with many sinks and the load balancer selector is set to round robin, the first sink received 10X more events than the other sinks in the sink group. When we changed the load balancer selector to random, than the events were balanced equally.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)