Transactions and redelivery

[Geoffrey Arnold <ge...@geoffreyarnold.com>]

Hi All,

When a message is rolled back on a session, it appears that no other  
messages will be delivered to the consumer's listener for that session  
until the original message is consumed (committed).  I believe that  
this is due to the session attempting to maintain message order,  
however this can quickly exhaust message processing when the number of  
concurrent consumers is capped.

What strategies are you using to handle this type of situation?

Thanks,
Geoff.

Re: Transactions and redelivery

[Geoffrey Arnold <ge...@gmail.com>]

Thanks for the information, Mark!

We ended up rolling our own redelivery logic similar to your setup.   
We have asynchronous listeners which, on recoverable error, create a  
copy of the message and place it on a separate redelivery queue.  The  
message copy has an additional property which specifies the redelivery  
time (current time + predefined delay).  We then have a synchronous  
listener on the redelivery queue which runs every 5 seconds and uses a  
message selector to pull messages off of the queue which have a  
redelivery time < the current time and place them back onto their  
original queue.

Thanks again for your insights.

Geoff.

On Oct 16, 2009, at 3:19 PM, magellings wrote:

>
> Yes.  The way we are doing it is adding a bit of metadata into the  
> message to
> keep a redelivery count.  This is added only when we want to do a
> redelivery.
>
> Our framework sits on top of the NMS ActiveMQ provider (.NET).  It  
> parses
> the metadata from the message.  This redelivery logic is abstracted  
> away
> from the many consumers using our framework and their main logic.  It
> executes a redelivery policy before delegating the work to the  
> consumer to
> actually process the message.
>
> If the redelivery count is exhausted according to the redelivery  
> policy, we
> simply acknowledge the message without re-cloning, hence  
> redelivering it.
>
> As I've been coding it I wonder how necessary it really is to have  
> this
> functionality though.  If you have enough consumers in your cluster  
> it seems
> the transactional mode would be sufficient.  It just so happens our  
> previous
> messaging framework retried messages by placing them at the end of  
> the queue
> so this is more of legacy support than anything.
>
> Regards
>
>
> Geoffrey Arnold-2 wrote:
>>
>> Hey Mark,
>>
>> We ended up opting to put the message back on the end of the queue
>> instead of rolling back, but this leads to repeated reprocessing of
>> the message.  My guess is that you have solved this with metadata:
>>
>>> Our framework is sophisticated enough to
>>> manage redeliveries in this way by adding some redeliveredCnt
>>> metadata to
>>> the message each time it is cloned.
>>
>> Can you elaborate?
>>
>> Thanks,
>> Geoff.
>>
>>
>
> -- 
> View this message in context: http://www.nabble.com/Transactions-and-redelivery-tp25915300p25930878.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Re: Transactions and redelivery

[magellings <ma...@qg.com>]

Yes.  The way we are doing it is adding a bit of metadata into the message to
keep a redelivery count.  This is added only when we want to do a
redelivery.

Our framework sits on top of the NMS ActiveMQ provider (.NET).  It parses
the metadata from the message.  This redelivery logic is abstracted away
from the many consumers using our framework and their main logic.  It
executes a redelivery policy before delegating the work to the consumer to
actually process the message.  

If the redelivery count is exhausted according to the redelivery policy, we
simply acknowledge the message without re-cloning, hence redelivering it.

As I've been coding it I wonder how necessary it really is to have this
functionality though.  If you have enough consumers in your cluster it seems
the transactional mode would be sufficient.  It just so happens our previous
messaging framework retried messages by placing them at the end of the queue
so this is more of legacy support than anything.

Regards


Geoffrey Arnold-2 wrote:
> 
> Hey Mark,
> 
> We ended up opting to put the message back on the end of the queue  
> instead of rolling back, but this leads to repeated reprocessing of  
> the message.  My guess is that you have solved this with metadata:
> 
>> Our framework is sophisticated enough to
>> manage redeliveries in this way by adding some redeliveredCnt  
>> metadata to
>> the message each time it is cloned.
> 
> Can you elaborate?
> 
> Thanks,
> Geoff.
> 
> 

-- 
View this message in context: http://www.nabble.com/Transactions-and-redelivery-tp25915300p25930878.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Transactions and redelivery

[Geoffrey Arnold <ge...@gmail.com>]

Hey Mark,

We ended up opting to put the message back on the end of the queue  
instead of rolling back, but this leads to repeated reprocessing of  
the message.  My guess is that you have solved this with metadata:

> Our framework is sophisticated enough to
> manage redeliveries in this way by adding some redeliveredCnt  
> metadata to
> the message each time it is cloned.

Can you elaborate?

Thanks,
Geoff.

Re: Transactions and redelivery

[magellings <ma...@qg.com>]

We are sort of in the same boat.

Transactions generally are not just used for redelivery purposes.  They also
are performance efficient as you can batch.

http://activemq.apache.org/should-i-use-transactions.html

What we are doing to avoid hang ups in clients on redeliveries is to use
ClientAcknowledgement and clone a message (placing it at the end of the
queue) before we Acknowledge it.  Our framework is sophisticated enough to
manage redeliveries in this way by adding some redeliveredCnt metadata to
the message each time it is cloned.




Geoffrey Arnold wrote:
> 
> Hi All,
> 
> When a message is rolled back on a session, it appears that no other  
> messages will be delivered to the consumer's listener for that session  
> until the original message is consumed (committed).  I believe that  
> this is due to the session attempting to maintain message order,  
> however this can quickly exhaust message processing when the number of  
> concurrent consumers is capped.
> 
> What strategies are you using to handle this type of situation?
> 
> Thanks,
> Geoff.
> 
> 

-- 
View this message in context: http://www.nabble.com/Transactions-and-redelivery-tp25915300p25916092.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.