Re: Complex topology in flume: reducing no. of channels and sinks

["Ketaki Gadre." <ke...@yahoo.co.in>]

Hi,

Just wanted to check if anybody has any suggestion on this.


Thanks,
Ketaki Gadre



On Friday, 29 November 2013 4:11 PM, Ketaki Gadre. <ke...@yahoo.co.in> wrote:
 
Hi,

In my use-case I need to send n log files on a machine (sender) to m different machines (receiver). These receiver machines process each file differently and there are multiple such senders.
As I read in the User Guide, currently Flume supports channel selector i.e. data can be multiplexed/replicated at the source level not at the channel level. This means that, to support above scenario, I will need to have n sources, n*m channels and n*m sinks in the flume agent at each sender machine. I want to reduce the number of channels as they will have replicated data and will take a lot of space.

Does the current version of Flume support multiplexing/replicating at channel level so that I can have reduce no. of channels? Also is there a way in which I can reduce no. of sinks as well in this scenario?

Can you please suggest?

Thanks,
Ketaki Gadre