You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Abhijeet Pathak <Ab...@kpitcummins.com> on 2012/12/19 09:23:48 UTC
Configuring failover sync in Flume-ng
Hi,
I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc], 1 sink [AsyncHbase]) on one server.
However, I am not able to find documentation on how to configure failover sinks for such scenario.
I want to run same sink as backup/failover sink on different server.
How can that be configured?
Thanks,
Abhijeet
Re: Configuring failover sync in Flume-ng
Posted by Alexander Alten-Lorenz <wg...@gmail.com>.
YOUR_NAME.sinks = HOSTSINK-1 HOSTSINK-2
HOSTSINK-1.type=
HOSTSINK-1.bind=HOSTNAME-1
HOSTSINK-1.port=needed by JDBC?
HOSTSINK-1.channels=YOUR_CHANNELS
the same for the other HOSTSINK and deploy the config to the hosts and startup. Should work.
Best,
Alex
On Dec 19, 2012, at 10:01 AM, Abhijeet Pathak <Ab...@kpitcummins.com> wrote:
> Yes, I saw that.
>
> But I'm not sure how to tell flume that the failover sink is running on "another" machine.
>
> -----Original Message-----
> From: Alexander Lorenz [mailto:wget.null@gmail.com]
> Sent: Wednesday, December 19, 2012 1:59 PM
> To: user@flume.apache.org
> Subject: Re: Configuring failover sync in Flume-ng
>
> Hi,
>
> Failover:
> http://flume.apache.org/FlumeUserGuide.html#failover-sink-processor
>
> Just define more as one JDBC channel and add them to the failover config (see the examples).
>
> Best
> - Alex
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
> On Dec 19, 2012, at 9:23 AM, Abhijeet Pathak <Ab...@kpitcummins.com> wrote:
>
>> Hi,
>>
>> I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc], 1 sink [AsyncHbase]) on one server.
>>
>> However, I am not able to find documentation on how to configure failover sinks for such scenario.
>>
>> I want to run same sink as backup/failover sink on different server.
>> How can that be configured?
>>
>> Thanks,
>> Abhijeet
>>
>
RE: Configuring failover sync in Flume-ng
Posted by Abhijeet Pathak <Ab...@kpitcummins.com>.
Yes, I saw that.
But I'm not sure how to tell flume that the failover sink is running on "another" machine.
-----Original Message-----
From: Alexander Lorenz [mailto:wget.null@gmail.com]
Sent: Wednesday, December 19, 2012 1:59 PM
To: user@flume.apache.org
Subject: Re: Configuring failover sync in Flume-ng
Hi,
Failover:
http://flume.apache.org/FlumeUserGuide.html#failover-sink-processor
Just define more as one JDBC channel and add them to the failover config (see the examples).
Best
- Alex
--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF
On Dec 19, 2012, at 9:23 AM, Abhijeet Pathak <Ab...@kpitcummins.com> wrote:
> Hi,
>
> I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc], 1 sink [AsyncHbase]) on one server.
>
> However, I am not able to find documentation on how to configure failover sinks for such scenario.
>
> I want to run same sink as backup/failover sink on different server.
> How can that be configured?
>
> Thanks,
> Abhijeet
>
Re: Configuring failover sync in Flume-ng
Posted by Alexander Lorenz <wg...@gmail.com>.
Hi,
Failover:
http://flume.apache.org/FlumeUserGuide.html#failover-sink-processor
Just define more as one JDBC channel and add them to the failover config (see the examples).
Best
- Alex
--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF
On Dec 19, 2012, at 9:23 AM, Abhijeet Pathak <Ab...@kpitcummins.com> wrote:
> Hi,
>
> I've working setup of flume-ng agent (1 source [tail], 1 channel[jdbc], 1 sink [AsyncHbase]) on one server.
>
> However, I am not able to find documentation on how to configure failover sinks for such scenario.
>
> I want to run same sink as backup/failover sink on different server.
> How can that be configured?
>
> Thanks,
> Abhijeet
>