You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@storm.apache.org by Ankur Garg <an...@gmail.com> on 2015/09/29 07:39:07 UTC

Spring AMQP Integration with Storm

Hi,

I want to consume the messages in my Storm Spout from a rabbitMq Queue.

Now , we are using Spring AMQP to send and receive messages from RabbitMq
asynchronously.

Spring AMQP provides mechanism(either creating a listener or using
annotation @RabbitListner) to read message from the queue aysnchronously.

The problem is I can have a Listener to read the message from the Queue.
But how do I send this message to my Storm Spout which is running on storm
cluster ?

The topology will start a cluster, but in my nextTuple() method of my spout
, I need to read message from this Queue. Can Spring AMQP be used here ?

I have a listener configured to read message from the queue:

@RabbitListener(queues = "queueName")
public void processMessage(QueueMessage message) {

}

How can the above message received at the listener be sent to my spout
running on a cluster .

Alternatively , how can a spout's nextTuple() method have this method
inside it ? Is it possible

Is there any integration there between Spring AMQP and Storm?


Thanks

Ankur

Re: Spring AMQP Integration with Storm

Posted by Ankur Garg <an...@gmail.com>.

Hi Stephen,

I want to use SpringAMQP . All examples you have quoted are not using
Spring nywhere which we are using heavily in our project .

If nothing works out , then I will have to use the example implementations
:(



On Tue, Sep 29, 2015 at 5:52 PM, Stephen Powis <sp...@salesforce.com>
wrote:

> I'd recommend making use of a spout that others have already built and
> battle tested.  A quick google search shows several including this one:
> https://github.com/ppat/storm-rabbitmq
>
> Unless you have a very special use case, I'm not sure re-inventing the
> wheel is worth the time and effort.
>
> Stephen
>
> On Tue, Sep 29, 2015 at 1:39 AM, Ankur Garg <an...@gmail.com> wrote:
>
>> Hi,
>>
>> I want to consume the messages in my Storm Spout from a rabbitMq Queue.
>>
>> Now , we are using Spring AMQP to send and receive messages from RabbitMq
>> asynchronously.
>>
>> Spring AMQP provides mechanism(either creating a listener or using
>> annotation @RabbitListner) to read message from the queue aysnchronously.
>>
>> The problem is I can have a Listener to read the message from the Queue.
>> But how do I send this message to my Storm Spout which is running on storm
>> cluster ?
>>
>> The topology will start a cluster, but in my nextTuple() method of my
>> spout , I need to read message from this Queue. Can Spring AMQP be used
>> here ?
>>
>> I have a listener configured to read message from the queue:
>>
>> @RabbitListener(queues = "queueName")
>> public void processMessage(QueueMessage message) {
>>
>> }
>>
>> How can the above message received at the listener be sent to my spout
>> running on a cluster .
>>
>> Alternatively , how can a spout's nextTuple() method have this method
>> inside it ? Is it possible
>>
>> Is there any integration there between Spring AMQP and Storm?
>>
>>
>> Thanks
>>
>> Ankur
>>
>>
>>
>>
>

Re: Spring AMQP Integration with Storm

Posted by Stephen Powis <sp...@salesforce.com>.

I'd recommend making use of a spout that others have already built and
battle tested.  A quick google search shows several including this one:
https://github.com/ppat/storm-rabbitmq

Unless you have a very special use case, I'm not sure re-inventing the
wheel is worth the time and effort.

Stephen

On Tue, Sep 29, 2015 at 1:39 AM, Ankur Garg <an...@gmail.com> wrote:

> Hi,
>
> I want to consume the messages in my Storm Spout from a rabbitMq Queue.
>
> Now , we are using Spring AMQP to send and receive messages from RabbitMq
> asynchronously.
>
> Spring AMQP provides mechanism(either creating a listener or using
> annotation @RabbitListner) to read message from the queue aysnchronously.
>
> The problem is I can have a Listener to read the message from the Queue.
> But how do I send this message to my Storm Spout which is running on storm
> cluster ?
>
> The topology will start a cluster, but in my nextTuple() method of my
> spout , I need to read message from this Queue. Can Spring AMQP be used
> here ?
>
> I have a listener configured to read message from the queue:
>
> @RabbitListener(queues = "queueName")
> public void processMessage(QueueMessage message) {
>
> }
>
> How can the above message received at the listener be sent to my spout
> running on a cluster .
>
> Alternatively , how can a spout's nextTuple() method have this method
> inside it ? Is it possible
>
> Is there any integration there between Spring AMQP and Storm?
>
>
> Thanks
>
> Ankur
>
>
>
>