[jira] [Comment Edited] (FLUME-2069) Issue with Flume load balancing round robin

["Jayant Shekhar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLUME-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13676288#comment-13676288 ] 

Jayant Shekhar edited comment on FLUME-2069 at 6/5/13 8:09 PM:
---------------------------------------------------------------

Hi Osama, Can you also add the source and channels. Also, am interested in looking at how are the sinks connected to the channels.
                
      was (Author: jayants):
    Hi Osama, Can you also add the source and channels. Also, also interested in looking at how are the sinks connected to the channels.
                  
> Issue with Flume load balancing round robin
> -------------------------------------------
>
>                 Key: FLUME-2069
>                 URL: https://issues.apache.org/jira/browse/FLUME-2069
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v1.3.1
>            Reporter: Osama Awad
>            Priority: Blocker
>
> I am not sure if this is a bug. I have an http to hdfs scenario, 3 channels with 3 hdfs sinks, I have configured it to do load balancing with round robin, but when the request arrive it gets replicated into all the sinks instead of hitting the sink with round robin order, so I end up with the same data replicated by all sinks.
> here are my configs, I have not included the source configs here
> a1.sinks.k1.type = hdfs
> a1.sinks.k1.hdfs.path = /tmp/hadoop-oawad/dfs/name2/%y-%m-%d/%H%M/%S
> a1.sinks.k1.hdfs.fileType = DataStream
> a1.sinks.k1.hdfs.writeFormat = Text
> a1.sinks.k1.hdfs.filePrefix = events
> a1.sinks.k1.hdfs.batchSize = 1000
> a1.sinks.k2.type = hdfs
> a1.sinks.k2.hdfs.path = /tmp/hadoop-oawad/dfs/name2/%y-%m-%d/%H%M/%S
> a1.sinks.k2.hdfs.fileType = DataStream
> a1.sinks.k2.hdfs.writeFormat = Text
> a1.sinks.k2.hdfs.filePrefix = events
> a1.sinks.k2.hdfs.batchSize = 1000
> a1.sinks.k3.type = hdfs
> a1.sinks.k3.hdfs.path = /tmp/hadoop-oawad/dfs/name2/%y-%m-%d/%H%M/%S
> a1.sinks.k3.hdfs.fileType = DataStream
> a1.sinks.k3.hdfs.writeFormat = Text
> a1.sinks.k3.hdfs.filePrefix = events
> a1.sinks.k3.hdfs.batchSize = 1000
> a1.sinkgroups = g1
> a1.sinkgroups.g1.sinks = k1 k2 k3
> a1.sinkgroups.g1.processor.type = load_balance
> a1.sinkgroups.g1.processor.selector = round_robin
> am I missing something here?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira