You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Komal Thombare <ko...@tcs.com> on 2014/06/02 10:43:50 UTC
Storm-kafka Integration
Hi,
I am trying to integrate storm with kafka, but did not get any proper documentation to do so.
Can anyone please help me for getting started with the integration.
Thanks and Regards,
Komal Thombare
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
Re: Storm-kafka Integration
Posted by Komal Thombare <ko...@tcs.com>.
Only Storm
Thanks and Regards,
Komal Thombare
Tata Consultancy Services Limited
Ph:- 086-55388772
Mail-to: komal.thombare@tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty. IT Services
Business Solutions
Consulting
________________________________________
-----Andres Gomez wrote: -----
To: user@storm.incubator.apache.org
From: Andres Gomez <an...@gmail.com>
Date: 06/02/2014 02:17PM
Subject: Re: Storm-kafka Integration
Do you use trindent or only storm???
Regards,
Andres
El 02/06/2014, a las 10:43, Komal Thombare <ko...@tcs.com> escribió:
Hi,
I am trying to integrate storm with kafka, but did not get any proper documentation to do so.
Can anyone please help me for getting started with the integration.
Thanks and Regards,
Komal Thombare
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
Re: Storm-kafka Integration
Posted by Andres Gomez <an...@gmail.com>.
Do you use trindent or only storm???
Regards,
Andres
El 02/06/2014, a las 10:43, Komal Thombare <ko...@tcs.com> escribió:
> Hi,
>
> I am trying to integrate storm with kafka, but did not get any proper documentation to do so.
> Can anyone please help me for getting started with the integration.
>
> Thanks and Regards,
>
> Komal Thombare
>
> =====-----=====-----=====
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
Re: Storm-kafka Integration
Posted by Anis Nasir <aa...@gmail.com>.
Dear Komal,
I used the storm-kafka version mentioned in the link.
http://anisnasir.wordpress.com/2014/05/25/apache-storm-kafka-a-nice-choice/
Regards
Anis
On Mon, Jun 2, 2014 at 1:49 PM, Komal Thombare <ko...@tcs.com>
wrote:
> Hi Andres,
>
> Still i am getting the same error.
>
>
> Thanks and Regards,
>
> Komal Thombare
>
>
> -----Andres Gomez wrote: -----
>
> To: user@storm.incubator.apache.org
> From: Andres Gomez <an...@gmail.com>
> Date: 06/02/2014 04:29PM
>
> Subject: Re: Storm-kafka Integration
>
> Hi again,
>
> I advise you to use the repo kafka happens to you:
>
>
> https://github.com/apache/incubator-storm/tree/master/external/storm-kafka
>
>
> 1. make a clone of this project
> 2. mvm package — generate a jar of project
> 3. mvn install — install the project on local repository
>
> Regards,
>
> Andres
>
> El 02/06/2014, a las 12:56, Komal Thombare <ko...@tcs.com>
> escribió:
>
> Hi,
>
> I have downloaded the storm-kafka jar from the below link:
> http://repo1.maven.org/maven2/com/n3twork/storm/storm-kafka/20140521/
>
> Now I have made changes in the storm word count topology and used
> KafkaSpout.
> While running topology in Local mode I get following error:
>
> java.lang.NoClassDefFoundError: org/apache/curator/RetryPolicy
> at storm.kafka.KafkaSpout.open(KafkaSpout.java:85)
> ~[storm-kafka-20140521.jar:na]
> at
> backtype.storm.daemon.executor$eval5100$fn__5101$fn__5116.invoke(executor.clj:519)
> ~[na:na]
> at backtype.storm.util$async_loop$fn__390.invoke(util.clj:431) ~[na:na]
> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
> at java.lang.Thread.run(Thread.java:636) [na:1.6.0_17]
> Caused by: java.lang.ClassNotFoundException: org.apache.curator.RetryPolicy
> at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
> ~[na:1.6.0_17]
> at java.security.AccessController.doPrivileged(Native Method)
> ~[na:1.6.0_17]
> at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
> ~[na:1.6.0_17]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:319) ~[na:1.6.0_17]
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
> ~[na:1.6.0_17]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:264) ~[na:1.6.0_17]
> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:332)
> ~[na:1.6.0_17]
>
> Though the curator-client-1.0.1.jar is present in the lib of storm and I
> have included same in my buildpath.
>
>
> Thanks and Regards,
>
> Komal Thombare
>
>
> -----Andres Gomez wrote: -----
>
> To: user@storm.incubator.apache.org
> From: Andres Gomez <an...@gmail.com>
> Date: 06/02/2014 02:33PM
> Subject: Re: Storm-kafka Integration
>
> You should make a clone of this project:
>
> https://github.com/apache/incubator-storm/tree/master/external/storm-kafka
>
> and do “mvn install”, I suposse you use kafka 0.8.+
>
> and then you do this:
>
> SpoutConfig spoutConfig = new SpoutConfig();
>
> spoutConfig.zkServers = “localhost”; //zookeeper Host
> spoutConfig.zkPort = 2181; //zookeeper port
> spoutConfig.zkRoot = “/kafkaStorm/“; // zookeeper path
> spoutConfig.id = “storm”; // zookeeper id
>
>
> // This config is a example!!
>
> KafkaSpout spout = new KafkaSpout(spoutConfig);
>
>
> El 02/06/2014, a las 10:56, Joe Stein <jo...@stealth.ly> escribió:
>
> Please take a look at
> http://www.michael-noll.com/blog/2014/05/27/kafka-storm-integration-example-tutorial/#state-of-the-integration-game
>
> There is a github project too
> https://github.com/miguno/kafka-storm-starter
>
> This covers latest Storm, Kafka and Avro.
>
> /*******************************************
> Joe Stein
> Founder, Principal Consultant
> Big Data Open Source Security LLC
> http://www.stealth.ly
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> ********************************************/
>
>
> On Mon, Jun 2, 2014 at 4:55 AM, Komal Thombare <ko...@tcs.com>
> wrote:
>
>> Hi Deepak,
>>
>> Yes i have. I have also got the storm-contrib source code, but then I am
>> unaware of how to compile it.
>>
>>
>> Thanks and Regards,
>>
>> Komal Thombare
>> Tata Consultancy Services Limited
>> Ph:- 086-55388772
>> Mail-to: komal.thombare@tcs.com
>> Website: http://www.tcs.com
>> ____________________________________________
>> Experience certainty. IT Services
>> Business Solutions
>> Consulting
>> ________________________________________
>>
>> -----Deepak Sharma wrote: -----
>>
>> To: user <us...@storm.incubator.apache.org>
>> From: Deepak Sharma <de...@gmail.com>
>> Date: 06/02/2014 02:22PM
>> Subject: Re: Storm-kafka Integration
>>
>>
>> Hi Komal
>> Have you looked at KafkaSpout?
>>
>> Thanks
>> Deepak
>>
>>
>> On Mon, Jun 2, 2014 at 2:13 PM, Komal Thombare <ko...@tcs.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I am trying to integrate storm with kafka, but did not get any proper
>>> documentation to do so.
>>> Can anyone please help me for getting started with the integration.
>>>
>>> Thanks and Regards,
>>>
>>> Komal Thombare
>>>
>>> =====-----=====-----=====
>>> Notice: The information contained in this e-mail
>>> message and/or attachments to it may contain
>>> confidential or privileged information. If you are
>>> not the intended recipient, any dissemination, use,
>>> review, distribution, printing or copying of the
>>> information contained in this e-mail message
>>> and/or attachments to it are strictly prohibited. If
>>> you have received this communication in error,
>>> please notify us by reply e-mail or telephone and
>>> immediately and permanently delete the message
>>> and any attachments. Thank you
>>>
>>>
>>
>>
>> --
>> Thanks
>> Deepak
>> www.bigdatabig.com
>> www.keosha.net
>>
>>
>
>
>
Re: Storm-kafka Integration
Posted by Komal Thombare <ko...@tcs.com>.
Hi Andres,
Still i am getting the same error.
Thanks and Regards,
Komal Thombare
-----Andres Gomez wrote: -----
To: user@storm.incubator.apache.org
From: Andres Gomez <an...@gmail.com>
Date: 06/02/2014 04:29PM
Subject: Re: Storm-kafka Integration
Hi again,
I advise you to use the repo kafka happens to you:
https://github.com/apache/incubator-storm/tree/master/external/storm-kafka
1. make a clone of this project
2. mvm package — generate a jar of project
3. mvn install — install the project on local repository
Regards,
Andres
El 02/06/2014, a las 12:56, Komal Thombare <ko...@tcs.com> escribió:
Hi,
I have downloaded the storm-kafka jar from the below link:
http://repo1.maven.org/maven2/com/n3twork/storm/storm-kafka/20140521/
Now I have made changes in the storm word count topology and used KafkaSpout.
While running topology in Local mode I get following error:
java.lang.NoClassDefFoundError: org/apache/curator/RetryPolicy
at storm.kafka.KafkaSpout.open(KafkaSpout.java:85) ~[storm-kafka-20140521.jar:na]
at backtype.storm.daemon.executor$eval5100$fn__5101$fn__5116.invoke(executor.clj:519) ~[na:na]
at backtype.storm.util$async_loop$fn__390.invoke(util.clj:431) ~[na:na]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
at java.lang.Thread.run(Thread.java:636) [na:1.6.0_17]
Caused by: java.lang.ClassNotFoundException: org.apache.curator.RetryPolicy
at java.net.URLClassLoader$1.run(URLClassLoader.java:217) ~[na:1.6.0_17]
at java.security.AccessController.doPrivileged(Native Method) ~[na:1.6.0_17]
at java.net.URLClassLoader.findClass(URLClassLoader.java:205) ~[na:1.6.0_17]
at java.lang.ClassLoader.loadClass(ClassLoader.java:319) ~[na:1.6.0_17]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) ~[na:1.6.0_17]
at java.lang.ClassLoader.loadClass(ClassLoader.java:264) ~[na:1.6.0_17]
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:332) ~[na:1.6.0_17]
Though the curator-client-1.0.1.jar is present in the lib of storm and I have included same in my buildpath.
Thanks and Regards,
Komal Thombare
-----Andres Gomez wrote: -----
To: user@storm.incubator.apache.org
From: Andres Gomez <an...@gmail.com>
Date: 06/02/2014 02:33PM
Subject: Re: Storm-kafka Integration
You should make a clone of this project:
https://github.com/apache/incubator-storm/tree/master/external/storm-kafka
and do “mvn install”, I suposse you use kafka 0.8.+
and then you do this:
SpoutConfig spoutConfig = new SpoutConfig();
spoutConfig.zkServers = “localhost”; //zookeeper Host
spoutConfig.zkPort = 2181; //zookeeper port
spoutConfig.zkRoot = “/kafkaStorm/“; // zookeeper path
spoutConfig.id = “storm”; // zookeeper id
// This config is a example!!
KafkaSpout spout = new KafkaSpout(spoutConfig);
El 02/06/2014, a las 10:56, Joe Stein <jo...@stealth.ly> escribió:
Please take a look at http://www.michael-noll.com/blog/2014/05/27/kafka-storm-integration-example-tutorial/#state-of-the-integration-game
There is a github project too https://github.com/miguno/kafka-storm-starter
This covers latest Storm, Kafka and Avro.
/*******************************************
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop
********************************************/
On Mon, Jun 2, 2014 at 4:55 AM, Komal Thombare <ko...@tcs.com> wrote:
Hi Deepak,
Yes i have. I have also got the storm-contrib source code, but then I am unaware of how to compile it.
Thanks and Regards,
Komal Thombare
Tata Consultancy Services Limited
Ph:- 086-55388772
Mail-to: komal.thombare@tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty. IT Services
Business Solutions
Consulting
________________________________________
-----Deepak Sharma wrote: -----
To: user <us...@storm.incubator.apache.org>
From: Deepak Sharma <de...@gmail.com>
Date: 06/02/2014 02:22PM
Subject: Re: Storm-kafka Integration
Hi Komal
Have you looked at KafkaSpout?
Thanks
Deepak
On Mon, Jun 2, 2014 at 2:13 PM, Komal Thombare <ko...@tcs.com> wrote:
Hi,
I am trying to integrate storm with kafka, but did not get any proper documentation to do so.
Can anyone please help me for getting started with the integration.
Thanks and Regards,
Komal Thombare
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
--
Thanks
Deepak
www.bigdatabig.com
www.keosha.net
Re: Storm-kafka Integration
Posted by Andres Gomez <an...@gmail.com>.
Hi again,
I advise you to use the repo kafka happens to you:
> https://github.com/apache/incubator-storm/tree/master/external/storm-kafka
1. make a clone of this project
2. mvm package — generate a jar of project
3. mvn install — install the project on local repository
Regards,
Andres
El 02/06/2014, a las 12:56, Komal Thombare <ko...@tcs.com> escribió:
> Hi,
>
> I have downloaded the storm-kafka jar from the below link:
> http://repo1.maven.org/maven2/com/n3twork/storm/storm-kafka/20140521/
>
> Now I have made changes in the storm word count topology and used KafkaSpout.
> While running topology in Local mode I get following error:
>
> java.lang.NoClassDefFoundError: org/apache/curator/RetryPolicy
> at storm.kafka.KafkaSpout.open(KafkaSpout.java:85) ~[storm-kafka-20140521.jar:na]
> at backtype.storm.daemon.executor$eval5100$fn__5101$fn__5116.invoke(executor.clj:519) ~[na:na]
> at backtype.storm.util$async_loop$fn__390.invoke(util.clj:431) ~[na:na]
> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
> at java.lang.Thread.run(Thread.java:636) [na:1.6.0_17]
> Caused by: java.lang.ClassNotFoundException: org.apache.curator.RetryPolicy
> at java.net.URLClassLoader$1.run(URLClassLoader.java:217) ~[na:1.6.0_17]
> at java.security.AccessController.doPrivileged(Native Method) ~[na:1.6.0_17]
> at java.net.URLClassLoader.findClass(URLClassLoader.java:205) ~[na:1.6.0_17]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:319) ~[na:1.6.0_17]
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) ~[na:1.6.0_17]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:264) ~[na:1.6.0_17]
> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:332) ~[na:1.6.0_17]
>
> Though the curator-client-1.0.1.jar is present in the lib of storm and I have included same in my buildpath.
>
>
> Thanks and Regards,
>
> Komal Thombare
>
>
> -----Andres Gomez wrote: -----
> To: user@storm.incubator.apache.org
> From: Andres Gomez <an...@gmail.com>
> Date: 06/02/2014 02:33PM
> Subject: Re: Storm-kafka Integration
>
> You should make a clone of this project:
>
> https://github.com/apache/incubator-storm/tree/master/external/storm-kafka
>
> and do “mvn install”, I suposse you use kafka 0.8.+
>
> and then you do this:
>
> SpoutConfig spoutConfig = new SpoutConfig();
>
> spoutConfig.zkServers = “localhost”; //zookeeper Host
> spoutConfig.zkPort = 2181; //zookeeper port
> spoutConfig.zkRoot = “/kafkaStorm/“; // zookeeper path
> spoutConfig.id = “storm”; // zookeeper id
>
> // This config is a example!!
>
> KafkaSpout spout = new KafkaSpout(spoutConfig);
>
>
> El 02/06/2014, a las 10:56, Joe Stein <jo...@stealth.ly> escribió:
>
>> Please take a look at http://www.michael-noll.com/blog/2014/05/27/kafka-storm-integration-example-tutorial/#state-of-the-integration-game
>>
>> There is a github project too https://github.com/miguno/kafka-storm-starter
>>
>> This covers latest Storm, Kafka and Avro.
>>
>> /*******************************************
>> Joe Stein
>> Founder, Principal Consultant
>> Big Data Open Source Security LLC
>> http://www.stealth.ly
>> Twitter: @allthingshadoop
>> ********************************************/
>>
>>
>> On Mon, Jun 2, 2014 at 4:55 AM, Komal Thombare <ko...@tcs.com> wrote:
>> Hi Deepak,
>>
>> Yes i have. I have also got the storm-contrib source code, but then I am unaware of how to compile it.
>>
>>
>> Thanks and Regards,
>>
>> Komal Thombare
>> Tata Consultancy Services Limited
>> Ph:- 086-55388772
>> Mail-to: komal.thombare@tcs.com
>> Website: http://www.tcs.com
>> ____________________________________________
>> Experience certainty. IT Services
>> Business Solutions
>> Consulting
>> ________________________________________
>>
>> -----Deepak Sharma wrote: -----
>> To: user <us...@storm.incubator.apache.org>
>> From: Deepak Sharma <de...@gmail.com>
>> Date: 06/02/2014 02:22PM
>> Subject: Re: Storm-kafka Integration
>>
>>
>> Hi Komal
>> Have you looked at KafkaSpout?
>>
>> Thanks
>> Deepak
>>
>>
>> On Mon, Jun 2, 2014 at 2:13 PM, Komal Thombare <ko...@tcs.com> wrote:
>> Hi,
>>
>> I am trying to integrate storm with kafka, but did not get any proper documentation to do so.
>> Can anyone please help me for getting started with the integration.
>>
>> Thanks and Regards,
>>
>> Komal Thombare
>>
>> =====-----=====-----=====
>> Notice: The information contained in this e-mail
>> message and/or attachments to it may contain
>> confidential or privileged information. If you are
>> not the intended recipient, any dissemination, use,
>> review, distribution, printing or copying of the
>> information contained in this e-mail message
>> and/or attachments to it are strictly prohibited. If
>> you have received this communication in error,
>> please notify us by reply e-mail or telephone and
>> immediately and permanently delete the message
>> and any attachments. Thank you
>>
>>
>>
>>
>>
>> --
>> Thanks
>> Deepak
>> www.bigdatabig.com
>> www.keosha.net
>>
>
Re: Storm-kafka Integration
Posted by Komal Thombare <ko...@tcs.com>.
Hi,
I have downloaded the storm-kafka jar from the below link:
http://repo1.maven.org/maven2/com/n3twork/storm/storm-kafka/20140521/
Now I have made changes in the storm word count topology and used KafkaSpout.
While running topology in Local mode I get following error:
java.lang.NoClassDefFoundError: org/apache/curator/RetryPolicy
at storm.kafka.KafkaSpout.open(KafkaSpout.java:85) ~[storm-kafka-20140521.jar:na]
at backtype.storm.daemon.executor$eval5100$fn__5101$fn__5116.invoke(executor.clj:519) ~[na:na]
at backtype.storm.util$async_loop$fn__390.invoke(util.clj:431) ~[na:na]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
at java.lang.Thread.run(Thread.java:636) [na:1.6.0_17]
Caused by: java.lang.ClassNotFoundException: org.apache.curator.RetryPolicy
at java.net.URLClassLoader$1.run(URLClassLoader.java:217) ~[na:1.6.0_17]
at java.security.AccessController.doPrivileged(Native Method) ~[na:1.6.0_17]
at java.net.URLClassLoader.findClass(URLClassLoader.java:205) ~[na:1.6.0_17]
at java.lang.ClassLoader.loadClass(ClassLoader.java:319) ~[na:1.6.0_17]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) ~[na:1.6.0_17]
at java.lang.ClassLoader.loadClass(ClassLoader.java:264) ~[na:1.6.0_17]
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:332) ~[na:1.6.0_17]
Though the curator-client-1.0.1.jar is present in the lib of storm and I have included same in my buildpath.
Thanks and Regards,
Komal Thombare
-----Andres Gomez wrote: -----
To: user@storm.incubator.apache.org
From: Andres Gomez <an...@gmail.com>
Date: 06/02/2014 02:33PM
Subject: Re: Storm-kafka Integration
You should make a clone of this project:
https://github.com/apache/incubator-storm/tree/master/external/storm-kafka
and do “mvn install”, I suposse you use kafka 0.8.+
and then you do this:
SpoutConfig spoutConfig = new SpoutConfig();
spoutConfig.zkServers = “localhost”; //zookeeper Host
spoutConfig.zkPort = 2181; //zookeeper port
spoutConfig.zkRoot = “/kafkaStorm/“; // zookeeper path
spoutConfig.id = “storm”; // zookeeper id
// This config is a example!!
KafkaSpout spout = new KafkaSpout(spoutConfig);
El 02/06/2014, a las 10:56, Joe Stein <jo...@stealth.ly> escribió:
Please take a look at http://www.michael-noll.com/blog/2014/05/27/kafka-storm-integration-example-tutorial/#state-of-the-integration-game
There is a github project too https://github.com/miguno/kafka-storm-starter
This covers latest Storm, Kafka and Avro.
/*******************************************
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop
********************************************/
On Mon, Jun 2, 2014 at 4:55 AM, Komal Thombare <ko...@tcs.com> wrote:
Hi Deepak,
Yes i have. I have also got the storm-contrib source code, but then I am unaware of how to compile it.
Thanks and Regards,
Komal Thombare
Tata Consultancy Services Limited
Ph:- 086-55388772
Mail-to: komal.thombare@tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty. IT Services
Business Solutions
Consulting
________________________________________
-----Deepak Sharma wrote: -----
To: user <us...@storm.incubator.apache.org>
From: Deepak Sharma <de...@gmail.com>
Date: 06/02/2014 02:22PM
Subject: Re: Storm-kafka Integration
Hi Komal
Have you looked at KafkaSpout?
Thanks
Deepak
On Mon, Jun 2, 2014 at 2:13 PM, Komal Thombare <ko...@tcs.com> wrote:
Hi,
I am trying to integrate storm with kafka, but did not get any proper documentation to do so.
Can anyone please help me for getting started with the integration.
Thanks and Regards,
Komal Thombare
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
--
Thanks
Deepak
www.bigdatabig.com
www.keosha.net
Re: Storm-kafka Integration
Posted by Andres Gomez <an...@gmail.com>.
You should make a clone of this project:
https://github.com/apache/incubator-storm/tree/master/external/storm-kafka
and do “mvn install”, I suposse you use kafka 0.8.+
and then you do this:
SpoutConfig spoutConfig = new SpoutConfig();
spoutConfig.zkServers = “localhost”; //zookeeper Host
spoutConfig.zkPort = 2181; //zookeeper port
spoutConfig.zkRoot = “/kafkaStorm/“; // zookeeper path
spoutConfig.id = “storm”; // zookeeper id
// This config is a example!!
KafkaSpout spout = new KafkaSpout(spoutConfig);
El 02/06/2014, a las 10:56, Joe Stein <jo...@stealth.ly> escribió:
> Please take a look at http://www.michael-noll.com/blog/2014/05/27/kafka-storm-integration-example-tutorial/#state-of-the-integration-game
>
> There is a github project too https://github.com/miguno/kafka-storm-starter
>
> This covers latest Storm, Kafka and Avro.
>
> /*******************************************
> Joe Stein
> Founder, Principal Consultant
> Big Data Open Source Security LLC
> http://www.stealth.ly
> Twitter: @allthingshadoop
> ********************************************/
>
>
> On Mon, Jun 2, 2014 at 4:55 AM, Komal Thombare <ko...@tcs.com> wrote:
> Hi Deepak,
>
> Yes i have. I have also got the storm-contrib source code, but then I am unaware of how to compile it.
>
>
> Thanks and Regards,
>
> Komal Thombare
> Tata Consultancy Services Limited
> Ph:- 086-55388772
> Mail-to: komal.thombare@tcs.com
> Website: http://www.tcs.com
> ____________________________________________
> Experience certainty. IT Services
> Business Solutions
> Consulting
> ________________________________________
>
> -----Deepak Sharma wrote: -----
> To: user <us...@storm.incubator.apache.org>
> From: Deepak Sharma <de...@gmail.com>
> Date: 06/02/2014 02:22PM
> Subject: Re: Storm-kafka Integration
>
>
> Hi Komal
> Have you looked at KafkaSpout?
>
> Thanks
> Deepak
>
>
> On Mon, Jun 2, 2014 at 2:13 PM, Komal Thombare <ko...@tcs.com> wrote:
> Hi,
>
> I am trying to integrate storm with kafka, but did not get any proper documentation to do so.
> Can anyone please help me for getting started with the integration.
>
> Thanks and Regards,
>
> Komal Thombare
>
> =====-----=====-----=====
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
>
>
>
> --
> Thanks
> Deepak
> www.bigdatabig.com
> www.keosha.net
>
Re: Storm-kafka Integration
Posted by Joe Stein <jo...@stealth.ly>.
Please take a look at
http://www.michael-noll.com/blog/2014/05/27/kafka-storm-integration-example-tutorial/#state-of-the-integration-game
There is a github project too https://github.com/miguno/kafka-storm-starter
This covers latest Storm, Kafka and Avro.
/*******************************************
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/
On Mon, Jun 2, 2014 at 4:55 AM, Komal Thombare <ko...@tcs.com>
wrote:
> Hi Deepak,
>
> Yes i have. I have also got the storm-contrib source code, but then I am
> unaware of how to compile it.
>
>
> Thanks and Regards,
>
> Komal Thombare
> Tata Consultancy Services Limited
> Ph:- 086-55388772
> Mail-to: komal.thombare@tcs.com
> Website: http://www.tcs.com
> ____________________________________________
> Experience certainty. IT Services
> Business Solutions
> Consulting
> ________________________________________
>
> -----Deepak Sharma wrote: -----
>
> To: user <us...@storm.incubator.apache.org>
> From: Deepak Sharma <de...@gmail.com>
> Date: 06/02/2014 02:22PM
> Subject: Re: Storm-kafka Integration
>
>
> Hi Komal
> Have you looked at KafkaSpout?
>
> Thanks
> Deepak
>
>
> On Mon, Jun 2, 2014 at 2:13 PM, Komal Thombare <ko...@tcs.com>
> wrote:
>
>> Hi,
>>
>> I am trying to integrate storm with kafka, but did not get any proper
>> documentation to do so.
>> Can anyone please help me for getting started with the integration.
>>
>> Thanks and Regards,
>>
>> Komal Thombare
>>
>> =====-----=====-----=====
>> Notice: The information contained in this e-mail
>> message and/or attachments to it may contain
>> confidential or privileged information. If you are
>> not the intended recipient, any dissemination, use,
>> review, distribution, printing or copying of the
>> information contained in this e-mail message
>> and/or attachments to it are strictly prohibited. If
>> you have received this communication in error,
>> please notify us by reply e-mail or telephone and
>> immediately and permanently delete the message
>> and any attachments. Thank you
>>
>>
>
>
> --
> Thanks
> Deepak
> www.bigdatabig.com
> www.keosha.net
>
>
Re: Storm-kafka Integration
Posted by Komal Thombare <ko...@tcs.com>.
Hi Deepak,
Yes i have. I have also got the storm-contrib source code, but then I am unaware of how to compile it.
Thanks and Regards,
Komal Thombare
Tata Consultancy Services Limited
Ph:- 086-55388772
Mail-to: komal.thombare@tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty. IT Services
Business Solutions
Consulting
________________________________________
-----Deepak Sharma wrote: -----
To: user <us...@storm.incubator.apache.org>
From: Deepak Sharma <de...@gmail.com>
Date: 06/02/2014 02:22PM
Subject: Re: Storm-kafka Integration
Hi Komal
Have you looked at KafkaSpout?
Thanks
Deepak
On Mon, Jun 2, 2014 at 2:13 PM, Komal Thombare <ko...@tcs.com> wrote:
Hi,
I am trying to integrate storm with kafka, but did not get any proper documentation to do so.
Can anyone please help me for getting started with the integration.
Thanks and Regards,
Komal Thombare
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
--
Thanks
Deepak
www.bigdatabig.com
www.keosha.net
Re: Storm-kafka Integration
Posted by Deepak Sharma <de...@gmail.com>.
Hi Komal
Have you looked at KafkaSpout?
Thanks
Deepak
On Mon, Jun 2, 2014 at 2:13 PM, Komal Thombare <ko...@tcs.com>
wrote:
> Hi,
>
> I am trying to integrate storm with kafka, but did not get any proper
> documentation to do so.
> Can anyone please help me for getting started with the integration.
>
> Thanks and Regards,
>
> Komal Thombare
>
> =====-----=====-----=====
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
--
Thanks
Deepak
www.bigdatabig.com
www.keosha.net