You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Oleg Ruchovets <or...@gmail.com> on 2013/04/28 13:29:54 UTC
kafka --> hadoop question
Hi ,
I am looking for simple way to transfer from kafka to hadoop.
I found such solutions in Github:
https://github.com/linkedin/camus
https://github.com/miniway/kafka-hadoop-consumer
https://github.com/kafka-dev/kafka/tree/master/contrib/hadoop-consumer
Question:
What solution is stable and what framework do you use in your project?
Is there another kafka --> hadoop solution that I didn't find and it is
worth an attention?
I am looking something simple and stable. I just need to transfer kafka
content to hdfs. There is no plan to use ETL at this stage.
Please advice.
Thanks
Oleg.
Re: kafka --> hadoop question
Posted by Oleg Ruchovets <or...@gmail.com>.
Sounds good :-)
On Sun, Apr 28, 2013 at 8:24 PM, Neha Narkhede <ne...@gmail.com>wrote:
> You can ask that question on the camus mailing list.
>
> Thanks,
> Neha
> On Apr 28, 2013 10:14 AM, "Oleg Ruchovets" <or...@gmail.com> wrote:
>
> > Thank you Neha.
> > Is Camus stable enough? Is anybody already (except LinkedIn :-) )
> used
> > in in production?
> > Thanks
> > Oleg.
> >
> >
> > On Sun, Apr 28, 2013 at 5:17 PM, Neha Narkhede <neha.narkhede@gmail.com
> > >wrote:
> >
> > > Camus is a solution that is well supported and currently used at
> > LinkedIn.
> > >
> > > Thanks,
> > > Neha
> > > On Apr 28, 2013 4:30 AM, "Oleg Ruchovets" <or...@gmail.com>
> wrote:
> > >
> > > > Hi ,
> > > > I am looking for simple way to transfer from kafka to hadoop.
> > > >
> > > > I found such solutions in Github:
> > > > https://github.com/linkedin/camus
> > > > https://github.com/miniway/kafka-hadoop-consumer
> > > >
> > > >
> https://github.com/kafka-dev/kafka/tree/master/contrib/hadoop-consumer
> > > >
> > > >
> > > > Question:
> > > > What solution is stable and what framework do you use in your
> > > project?
> > > > Is there another kafka --> hadoop solution that I didn't find and
> > it
> > > is
> > > > worth an attention?
> > > >
> > > > I am looking something simple and stable. I just need to transfer
> kafka
> > > > content to hdfs. There is no plan to use ETL at this stage.
> > > >
> > > > Please advice.
> > > >
> > > > Thanks
> > > > Oleg.
> > > >
> > >
> >
>
Re: kafka --> hadoop question
Posted by Neha Narkhede <ne...@gmail.com>.
You can ask that question on the camus mailing list.
Thanks,
Neha
On Apr 28, 2013 10:14 AM, "Oleg Ruchovets" <or...@gmail.com> wrote:
> Thank you Neha.
> Is Camus stable enough? Is anybody already (except LinkedIn :-) ) used
> in in production?
> Thanks
> Oleg.
>
>
> On Sun, Apr 28, 2013 at 5:17 PM, Neha Narkhede <neha.narkhede@gmail.com
> >wrote:
>
> > Camus is a solution that is well supported and currently used at
> LinkedIn.
> >
> > Thanks,
> > Neha
> > On Apr 28, 2013 4:30 AM, "Oleg Ruchovets" <or...@gmail.com> wrote:
> >
> > > Hi ,
> > > I am looking for simple way to transfer from kafka to hadoop.
> > >
> > > I found such solutions in Github:
> > > https://github.com/linkedin/camus
> > > https://github.com/miniway/kafka-hadoop-consumer
> > >
> > > https://github.com/kafka-dev/kafka/tree/master/contrib/hadoop-consumer
> > >
> > >
> > > Question:
> > > What solution is stable and what framework do you use in your
> > project?
> > > Is there another kafka --> hadoop solution that I didn't find and
> it
> > is
> > > worth an attention?
> > >
> > > I am looking something simple and stable. I just need to transfer kafka
> > > content to hdfs. There is no plan to use ETL at this stage.
> > >
> > > Please advice.
> > >
> > > Thanks
> > > Oleg.
> > >
> >
>
Re: kafka --> hadoop question
Posted by Oleg Ruchovets <or...@gmail.com>.
Thank you Neha.
Is Camus stable enough? Is anybody already (except LinkedIn :-) ) used
in in production?
Thanks
Oleg.
On Sun, Apr 28, 2013 at 5:17 PM, Neha Narkhede <ne...@gmail.com>wrote:
> Camus is a solution that is well supported and currently used at LinkedIn.
>
> Thanks,
> Neha
> On Apr 28, 2013 4:30 AM, "Oleg Ruchovets" <or...@gmail.com> wrote:
>
> > Hi ,
> > I am looking for simple way to transfer from kafka to hadoop.
> >
> > I found such solutions in Github:
> > https://github.com/linkedin/camus
> > https://github.com/miniway/kafka-hadoop-consumer
> >
> > https://github.com/kafka-dev/kafka/tree/master/contrib/hadoop-consumer
> >
> >
> > Question:
> > What solution is stable and what framework do you use in your
> project?
> > Is there another kafka --> hadoop solution that I didn't find and it
> is
> > worth an attention?
> >
> > I am looking something simple and stable. I just need to transfer kafka
> > content to hdfs. There is no plan to use ETL at this stage.
> >
> > Please advice.
> >
> > Thanks
> > Oleg.
> >
>
Re: kafka --> hadoop question
Posted by Neha Narkhede <ne...@gmail.com>.
Camus is a solution that is well supported and currently used at LinkedIn.
Thanks,
Neha
On Apr 28, 2013 4:30 AM, "Oleg Ruchovets" <or...@gmail.com> wrote:
> Hi ,
> I am looking for simple way to transfer from kafka to hadoop.
>
> I found such solutions in Github:
> https://github.com/linkedin/camus
> https://github.com/miniway/kafka-hadoop-consumer
>
> https://github.com/kafka-dev/kafka/tree/master/contrib/hadoop-consumer
>
>
> Question:
> What solution is stable and what framework do you use in your project?
> Is there another kafka --> hadoop solution that I didn't find and it is
> worth an attention?
>
> I am looking something simple and stable. I just need to transfer kafka
> content to hdfs. There is no plan to use ETL at this stage.
>
> Please advice.
>
> Thanks
> Oleg.
>