You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@storm.apache.org by Binita Bharati <bi...@gmail.com> on 2014/03/04 14:45:18 UTC

Working Kafka Spout example

Hi ,

I am using the following leningen dependencies :

   [storm "0.9.0.1"]
   [org.apache.kafka/kafka_2.9.2 "0.8.0"]
   [storm/storm-kafka "0.9.0-wip16a-scala292"]


Here is the sample code snippet: http://pastebin.com/E585PyfY

The Bolt is not receiving any data from the Kafka Spout.
Any ideas why ?


Thanks
Binita

Re: Working Kafka Spout example

Posted by "Dan DeCapria, CivicScience" <da...@civicscience.com>.

Hi Binita,

I have had success with integrating current Kafka 0.8.1 with Storm 0.9.0.1;
this includes both a KafkaSpout and a KafkaWriterBolt implementation.

Maven dependency links for wurstmeister
0.4.0<https://clojars.org/net.wurstmeister.storm/storm-kafka-0.8-plus>,
and the git repo for Kafka
0.8.1<https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=shortlog;h=refs/heads/0.8.1>.
 For completeness, I had Gradle compile Kafka 0.8.1 per all versions of
Scala (although I think only 2.9.2 is required). There are minor
modifications to KafkaSpout between the wurstmeister 0.2.0 and 0.4.0
versions: basically, your SpoutConfig() object now takes
"spoutConfig.startOffsetTime = kafka.api.OffsetRequest.EarliestTime();" or
some variant.

Hope this helps,

-Dan

On Tue, Mar 4, 2014 at 9:17 AM, Danijel Schiavuzzi
<da...@schiavuzzi.com>wrote:

> Hi Binita,
>
> You seem to be using Kafka 0.8, in which case you must use the Kafka spout
> from this project: https://github.com/wurstmeister/storm-kafka-0.8-plus/
>
> The current storm-kafka KafkaSpout doesn't support the new Kafka version
> 0.8.
>
> Best regards,
>
> Danijel Schiavuzzi
> www.schiavuzzi.com
>
>
> On Tue, Mar 4, 2014 at 2:45 PM, Binita Bharati <bi...@gmail.com>wrote:
>
>> Hi ,
>>
>> I am using the following leningen dependencies :
>>
>>    [storm "0.9.0.1"]
>>    [org.apache.kafka/kafka_2.9.2 "0.8.0"]
>>    [storm/storm-kafka "0.9.0-wip16a-scala292"]
>>
>>
>> Here is the sample code snippet: http://pastebin.com/E585PyfY
>>
>> The Bolt is not receiving any data from the Kafka Spout.
>> Any ideas why ?
>>
>>
>> Thanks
>> Binita
>>
>
>
>
> --
> Danijel Schiavuzzi
>
> E: danijel@schiavuzzi.com
> W: www.schiavuzzi.com
> T: +385989035562
> Skype: danijels7
>

Re: Working Kafka Spout example

Posted by Danijel Schiavuzzi <da...@schiavuzzi.com>.

Hi Binita,

You seem to be using Kafka 0.8, in which case you must use the Kafka spout
from this project: https://github.com/wurstmeister/storm-kafka-0.8-plus/

The current storm-kafka KafkaSpout doesn't support the new Kafka version
0.8.

Best regards,

Danijel Schiavuzzi
www.schiavuzzi.com


On Tue, Mar 4, 2014 at 2:45 PM, Binita Bharati <bi...@gmail.com>wrote:

> Hi ,
>
> I am using the following leningen dependencies :
>
>    [storm "0.9.0.1"]
>    [org.apache.kafka/kafka_2.9.2 "0.8.0"]
>    [storm/storm-kafka "0.9.0-wip16a-scala292"]
>
>
> Here is the sample code snippet: http://pastebin.com/E585PyfY
>
> The Bolt is not receiving any data from the Kafka Spout.
> Any ideas why ?
>
>
> Thanks
> Binita
>



-- 
Danijel Schiavuzzi

E: danijel@schiavuzzi.com
W: www.schiavuzzi.com
T: +385989035562
Skype: danijels7