You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "lizhenmxcz@163.com" <li...@163.com> on 2015/10/27 08:08:56 UTC
how to split one event to many
hi all:
i want to split one source log to many sinks, but i don't know how and where to split it. Thanks for regards.
here is a sample:
sources: a b,c,d
sinks(three): a,b
a,c
a,d
lizhenmxcz@163.com
Re: Re: how to split one event to many
Posted by Gonzalo Herreros <gh...@gmail.com>.
I don't think that is possible, Flume needs the Sinks defined beforehand
and 100 is not manageable.
If you really need that many destinations, create a custom Sink that
extends the one you are interested into and sends it to multiple
configurable destinations (you are likely to have other problems like
transactions or low performance).
Regards,
Gonzalo
On 27 October 2015 at 08:33, lizhenmxcz@163.com <li...@163.com> wrote:
>
> thank to Gonzalo,
> but that is just a sample, the one event maybe split to 10,100...
> ------------------------------
> lizhenmxcz@163.com
>
>
> *From:* Gonzalo Herreros <gh...@gmail.com>
> *Date:* 2015-10-27 16:18
> *To:* Flume <de...@flume.apache.org>
> *CC:* user <us...@flume.apache.org>
> *Subject:* Re: how to split one event to many
> Create a channel for each sink and then link the source to the 3 channels
> instead of just one
>
> Regards,
> Gonzalo
>
> On 27 October 2015 at 07:08, lizhenmxcz@163.com <li...@163.com>
> wrote:
>
>>
>> hi all:
>> i want to split one source log to many sinks, but i don't know how
>> and where to split it. Thanks for regards.
>> here is a sample:
>> sources: a b,c,d
>> sinks(three): a,b
>> a,c
>> a,d
>>
>>
>> lizhenmxcz@163.com
>>
>
>
Re: Re: how to split one event to many
Posted by Gonzalo Herreros <gh...@gmail.com>.
I don't think that is possible, Flume needs the Sinks defined beforehand
and 100 is not manageable.
If you really need that many destinations, create a custom Sink that
extends the one you are interested into and sends it to multiple
configurable destinations (you are likely to have other problems like
transactions or low performance).
Regards,
Gonzalo
On 27 October 2015 at 08:33, lizhenmxcz@163.com <li...@163.com> wrote:
>
> thank to Gonzalo,
> but that is just a sample, the one event maybe split to 10,100...
> ------------------------------
> lizhenmxcz@163.com
>
>
> *From:* Gonzalo Herreros <gh...@gmail.com>
> *Date:* 2015-10-27 16:18
> *To:* Flume <de...@flume.apache.org>
> *CC:* user <us...@flume.apache.org>
> *Subject:* Re: how to split one event to many
> Create a channel for each sink and then link the source to the 3 channels
> instead of just one
>
> Regards,
> Gonzalo
>
> On 27 October 2015 at 07:08, lizhenmxcz@163.com <li...@163.com>
> wrote:
>
>>
>> hi all:
>> i want to split one source log to many sinks, but i don't know how
>> and where to split it. Thanks for regards.
>> here is a sample:
>> sources: a b,c,d
>> sinks(three): a,b
>> a,c
>> a,d
>>
>>
>> lizhenmxcz@163.com
>>
>
>
Re: Re: how to split one event to many
Posted by "lizhenmxcz@163.com" <li...@163.com>.
thank to Gonzalo,
but that is just a sample, the one event maybe split to 10,100...
lizhenmxcz@163.com
From: Gonzalo Herreros
Date: 2015-10-27 16:18
To: Flume
CC: user
Subject: Re: how to split one event to many
Create a channel for each sink and then link the source to the 3 channels instead of just one
Regards,
Gonzalo
On 27 October 2015 at 07:08, lizhenmxcz@163.com <li...@163.com> wrote:
hi all:
i want to split one source log to many sinks, but i don't know how and where to split it. Thanks for regards.
here is a sample:
sources: a b,c,d
sinks(three): a,b
a,c
a,d
lizhenmxcz@163.com
Re: Re: how to split one event to many
Posted by "lizhenmxcz@163.com" <li...@163.com>.
thank to Gonzalo,
but that is just a sample, the one event maybe split to 10,100...
lizhenmxcz@163.com
From: Gonzalo Herreros
Date: 2015-10-27 16:18
To: Flume
CC: user
Subject: Re: how to split one event to many
Create a channel for each sink and then link the source to the 3 channels instead of just one
Regards,
Gonzalo
On 27 October 2015 at 07:08, lizhenmxcz@163.com <li...@163.com> wrote:
hi all:
i want to split one source log to many sinks, but i don't know how and where to split it. Thanks for regards.
here is a sample:
sources: a b,c,d
sinks(three): a,b
a,c
a,d
lizhenmxcz@163.com
Re: how to split one event to many
Posted by Gonzalo Herreros <gh...@gmail.com>.
Create a channel for each sink and then link the source to the 3 channels
instead of just one
Regards,
Gonzalo
On 27 October 2015 at 07:08, lizhenmxcz@163.com <li...@163.com> wrote:
>
> hi all:
> i want to split one source log to many sinks, but i don't know how and
> where to split it. Thanks for regards.
> here is a sample:
> sources: a b,c,d
> sinks(three): a,b
> a,c
> a,d
>
>
> lizhenmxcz@163.com
>
Re: how to split one event to many
Posted by Gonzalo Herreros <gh...@gmail.com>.
Create a channel for each sink and then link the source to the 3 channels
instead of just one
Regards,
Gonzalo
On 27 October 2015 at 07:08, lizhenmxcz@163.com <li...@163.com> wrote:
>
> hi all:
> i want to split one source log to many sinks, but i don't know how and
> where to split it. Thanks for regards.
> here is a sample:
> sources: a b,c,d
> sinks(three): a,b
> a,c
> a,d
>
>
> lizhenmxcz@163.com
>