You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by rafeeq s <ra...@gmail.com> on 2014/08/03 14:45:13 UTC
How to use kafka as flume source.
Hi,
We are planning to use kafka as *flume source*. Please advice me, how to
use kafka as source in flume.
please share if there is any best example of *flume- kafka source- hdfs
sink*.
Regards,
Rafeeq S
*(“What you do is what matters, not what you think or say or plan.” )*
Re: How to use kafka as flume source.
Posted by Gwen Shapira <gs...@cloudera.com>.
Just to update (better late than never!):
The Kafka source & sink for Flume were updated to latest Kafka version
and improved a bit (offsets are now committed after data is written to
Flume channel).
If you build Flume from trunk, you'll get these.
Gwen
On Sun, Aug 3, 2014 at 10:31 AM, Andrew Ehrlich <eh...@gmail.com> wrote:
> The author may or may not have upgraded the project to be compatible with
> 0.8.1. You could message the author directly to ask.
>
>
> On Sun, Aug 3, 2014 at 6:53 AM, rafeeq s <ra...@gmail.com> wrote:
>
>> Thanks.
>> But above link example uses flume-ng <https://github.com/apache/flume> to
>> communicate with *kafka 0.7,2 <http://kafka.apache.org/07/quickstart.html
>> >*.
>>
>> I am using *kafka 0.8.1*. Please advice an example of flume-ng with kafka
>> 0.8.1.
>>
>> Thanks in advance
>>
>> Regards,
>>
>> Rafeeq S
>> *(“What you do is what matters, not what you think or say or plan.” )*
>>
>>
>>
>> On Sun, Aug 3, 2014 at 4:40 PM, Sharninder <sh...@gmail.com> wrote:
>>
>> > https://github.com/baniuyao/flume-ng-kafka-source
>> >
>> >
>> >
>> > On Sun, Aug 3, 2014 at 6:15 PM, rafeeq s <ra...@gmail.com> wrote:
>> >
>> > > Hi,
>> > >
>> > > We are planning to use kafka as *flume source*. Please advice me, how
>> to
>> > > use kafka as source in flume.
>> > >
>> > > please share if there is any best example of *flume- kafka source- hdfs
>> > > sink*.
>> > >
>> > > Regards,
>> > >
>> > > Rafeeq S
>> > > *("What you do is what matters, not what you think or say or plan." )*
>> > >
>> >
>>
Re: How to use kafka as flume source.
Posted by Andrew Ehrlich <eh...@gmail.com>.
The author may or may not have upgraded the project to be compatible with
0.8.1. You could message the author directly to ask.
On Sun, Aug 3, 2014 at 6:53 AM, rafeeq s <ra...@gmail.com> wrote:
> Thanks.
> But above link example uses flume-ng <https://github.com/apache/flume> to
> communicate with *kafka 0.7,2 <http://kafka.apache.org/07/quickstart.html
> >*.
>
> I am using *kafka 0.8.1*. Please advice an example of flume-ng with kafka
> 0.8.1.
>
> Thanks in advance
>
> Regards,
>
> Rafeeq S
> *(“What you do is what matters, not what you think or say or plan.” )*
>
>
>
> On Sun, Aug 3, 2014 at 4:40 PM, Sharninder <sh...@gmail.com> wrote:
>
> > https://github.com/baniuyao/flume-ng-kafka-source
> >
> >
> >
> > On Sun, Aug 3, 2014 at 6:15 PM, rafeeq s <ra...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > We are planning to use kafka as *flume source*. Please advice me, how
> to
> > > use kafka as source in flume.
> > >
> > > please share if there is any best example of *flume- kafka source- hdfs
> > > sink*.
> > >
> > > Regards,
> > >
> > > Rafeeq S
> > > *("What you do is what matters, not what you think or say or plan." )*
> > >
> >
>
Re: How to use kafka as flume source.
Posted by rafeeq s <ra...@gmail.com>.
Thanks.
But above link example uses flume-ng <https://github.com/apache/flume> to
communicate with *kafka 0.7,2 <http://kafka.apache.org/07/quickstart.html>*.
I am using *kafka 0.8.1*. Please advice an example of flume-ng with kafka
0.8.1.
Thanks in advance
Regards,
Rafeeq S
*(“What you do is what matters, not what you think or say or plan.” )*
On Sun, Aug 3, 2014 at 4:40 PM, Sharninder <sh...@gmail.com> wrote:
> https://github.com/baniuyao/flume-ng-kafka-source
>
>
>
> On Sun, Aug 3, 2014 at 6:15 PM, rafeeq s <ra...@gmail.com> wrote:
>
> > Hi,
> >
> > We are planning to use kafka as *flume source*. Please advice me, how to
> > use kafka as source in flume.
> >
> > please share if there is any best example of *flume- kafka source- hdfs
> > sink*.
> >
> > Regards,
> >
> > Rafeeq S
> > *("What you do is what matters, not what you think or say or plan." )*
> >
>
Re: How to use kafka as flume source.
Posted by Sharninder <sh...@gmail.com>.
https://github.com/baniuyao/flume-ng-kafka-source
On Sun, Aug 3, 2014 at 6:15 PM, rafeeq s <ra...@gmail.com> wrote:
> Hi,
>
> We are planning to use kafka as *flume source*. Please advice me, how to
> use kafka as source in flume.
>
> please share if there is any best example of *flume- kafka source- hdfs
> sink*.
>
> Regards,
>
> Rafeeq S
> *("What you do is what matters, not what you think or say or plan." )*
>