Coping with forgetful subscribers

[Jeff Wartes <jw...@whitepages.com>]

Given a queue and the assumption that messages are idempotent, I'm looking for a way to have a given consumer go 'back in time', and re-play some period or even all available messages prior to resuming normal consumption. Possibly as much as the last 24 hours worth.
The trick is that the consumer may have already consumed some of these messages, but forgotten that it had done so past a certain point in time. (This is by design, and I can get into the reasons, but that's going to be a longer post)

We can assume that these bouts of amnesia coincide with the consumer disconnecting and reconnecting again, so something like subscription recovery<http://activemq.apache.org/subscription-recovery-policy.html> looks close, but this seems to be intended for cases where:

*         We're not going very far back in time. (does this feature store messages in something other than memory if it's a durable queue?)

*         The consumer never actually got the messages that the broker is re-sending. (Does the broker blindly re-send, or does it only send messages it thinks the consumer missed?)

Another thought was to keep a "historical queue" of messages that never has any subscribers, but receives a copy of every message with a durability of 24 hours. The idea would be that between the amnesiac disconnect and reconnect, we insert some steps, such that:

1.       The subscriber stops listening to its normal queue (disconnect)

2.       We forget that we got the last six hours of messages (amnesia strikes)

3.       We clone the "historical queue" including all messages

4.       We consume all messages from the cloned queue (get back to where we were)

5.       The subscriber resumes listening to its normal queue (reconnect)
This would allow unlimited replay of messages while still consuming in the normal sense. However, I have not come across documentation that allows a client to request a queue be "cloned", complete with unconsumed messages.

Is this possible with ActiveMQ, or do I need to find another solution for message-replay?

Re: Coping with forgetful subscribers

[Jeff Wartes <jw...@whitepages.com>]

I have, and agree that this would be an excellent way to maintain a sliding
window of message history. However, when it comes time to use that history,
you don't want to consume the messages in such a way that they aren't
available again. Otherwise, if you have two bouts of amnesia before the
message "history" has refilled enough to cover, you might fail to replay
certain messages. (You could solve this by having multiple history queues,
of course, but that seems... inelegant.)

I hadn't thought about playing with message acks though. I had assumed that
if you fail to acknowledge receipt of a message, you won't get the next one.
The thought would be that the subscriber would just never acknowledge
messages, so that whenever it reconnects, it gets a full replay?



Gary Tully wrote:
> 
> Have a read of http://activemq.apache.org/virtual-destinations.html
> I can imagine a variant of a filtered destination that sets an message
> expiry duration such that it maintains a time limited mirror of a
> queue. Then a composite destination that also pulls from the mirror.
> If you implement your own destination interceptor you can have
> complete control of where messages end up w.r.t to destinations.
> 
> I also wonder about ack modes, could the amnesia include forgetting to
> ack or commit, that seems easier as all of the received but un acked
> messages will still be available for redelivery once the consumer
> disconnects.
> 

-- 
View this message in context: http://old.nabble.com/Coping-with-forgetful-subscribers-tp29292239p29298648.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Coping with forgetful subscribers

[Gary Tully <ga...@gmail.com>]

Have a read of http://activemq.apache.org/virtual-destinations.html
I can imagine a variant of a filtered destination that sets an message
expiry duration such that it maintains a time limited mirror of a
queue. Then a composite destination that also pulls from the mirror.
If you implement your own destination interceptor you can have
complete control of where messages end up w.r.t to destinations.

I also wonder about ack modes, could the amnesia include forgetting to
ack or commit, that seems easier as all of the received but un acked
messages will still be available for redelivery once the consumer
disconnects.

On 29 July 2010 02:15, Jeff Wartes <jw...@whitepages.com> wrote:
>
> Given a queue and the assumption that messages are idempotent, I'm looking for a way to have a given consumer go 'back in time', and re-play some period or even all available messages prior to resuming normal consumption. Possibly as much as the last 24 hours worth.
> The trick is that the consumer may have already consumed some of these messages, but forgotten that it had done so past a certain point in time. (This is by design, and I can get into the reasons, but that's going to be a longer post)
>
> We can assume that these bouts of amnesia coincide with the consumer disconnecting and reconnecting again, so something like subscription recovery<http://activemq.apache.org/subscription-recovery-policy.html> looks close, but this seems to be intended for cases where:
>
> *         We're not going very far back in time. (does this feature store messages in something other than memory if it's a durable queue?)
>
> *         The consumer never actually got the messages that the broker is re-sending. (Does the broker blindly re-send, or does it only send messages it thinks the consumer missed?)
>
> Another thought was to keep a "historical queue" of messages that never has any subscribers, but receives a copy of every message with a durability of 24 hours. The idea would be that between the amnesiac disconnect and reconnect, we insert some steps, such that:
>
> 1.       The subscriber stops listening to its normal queue (disconnect)
>
> 2.       We forget that we got the last six hours of messages (amnesia strikes)
>
> 3.       We clone the "historical queue" including all messages
>
> 4.       We consume all messages from the cloned queue (get back to where we were)
>
> 5.       The subscriber resumes listening to its normal queue (reconnect)
> This would allow unlimited replay of messages while still consuming in the normal sense. However, I have not come across documentation that allows a client to request a queue be "cloned", complete with unconsumed messages.
>
> Is this possible with ActiveMQ, or do I need to find another solution for message-replay?
>
>



-- 
http://blog.garytully.com

Open Source Integration
http://fusesource.com