Durable subscription with retroactive subscription recovery policy

[Jose Luna <j-...@rocketmail.com>]

Hello,

We have a large number (thousands) of durable subscribers that subscribe to a set of topics.  These consumers will be connecting/disconnecting frequently, and we must ensure that they receive messages sent while they are disconnected (hence durable subscriptions).  These durable subscriptions will be created after important messages have been sent, so we must also ensure that the consumers receive messages retroactively, according to our subscription recovery policy.

Unfortunately, the retroactive consumer behavior + durable subscriptions do not mix well.   When connecting a durable subscription while using the activemq.retroactive header (we use STOMP), we receive all retroactive messages every time we connect.  It looks like from JIRA AMQ-1549 (https://issues.apache.org/activemq/browse/AMQ-1549) that the durable subscriptions with retroactive behaved as desired in the 4.X release of activemq.  The bug is open and assigned, with fixed version marked as 5.3 (although no work has been done yet).

However, there's a bit of a conflicting report here: http://www.nabble.com/Durable-subscriptions-to19688854.html .  Bruce and James seem to agree that subscription recovery policy is only for non-durable subscriptions.   So I'm wondering which is accurate:  Is this a bug since in 5.X that just hasn't been worked on yet? Or is this the desired behavior with no plans to change it? Of course, my vote is in for the former rather than the latter.  I think it would be desirable in general (not just my case) for durable subscriptions to play nicely with the subscription recovery policy.

That said, I'm also wondering if there's any advice for my particular case.   I'd love to work on a patch, but we are a small team up against a tight deadline and we have no experience with the Activemq source -- I'm not sure this is something we can tackle right now.  If we aren't able to get the desired behavior from activemq, each consumer could send the retroactive header only the very first time they connect.  However, there's no clear way (that I can see) to tell when all retroactive messages have been received.  So the consumer may make the first connection retroactive, but how does it know that it successfully received all retroactive messages and shouldn't use the retroactive header next time?

Thank you for your attention,

JLuna



      

Re: Durable subscription with subscription recovery policy

[Jose Luna <j-...@rocketmail.com>]

----- Original Message ----

> From: Jose Luna <j-...@rocketmail.com>
> To: users@activemq.apache.org
> Sent: Thursday, April 30, 2009 3:04:33 PM
> Subject: Durable subscription with subscription recovery policy
> 
> 
> Hello,
> 
> I haven't received a response after a few days, so I'd like to try again. I'll 
> try to be brief this time:
> 
> Our consumers disconnect/reconnect frequently.  We need durable subscriptions to 
> work with the subscription recovery policy.  Currently (Activemq 5.2), durable 
> subscriptions with the "activemq.retroactive" header (STOMP) will receive all 
> retroactive messages every time the durable subscriber reconnects.
> 
> A bug report can be found at https://issues.apache.org/activemq/browse/AMQ-1549 
> .  The bug is open and assigned, with fixed
> version marked as 5.3 (although no work has been done yet).
> 
> However, in this thread 
> http://www.nabble.com/Durable-subscriptions-to19688854.html,  Bruce and James 
> seems to say that subscription recovery policy is
> only for non-durable subscriptions.  
> 
> So, are durable subscriptions meant to work with subscription recovery 
> policies?  It used to work this way in 4.x, but I'm not sure if it was an 
> intentional change for 5.x.
> 
> I'm looking for two kinds of advice: 
> 1.) How we might patch this ourselves.  We're a small development team on a 
> tight deadline, but we might be able to take a crack at it.
> 
> 2.) Any advice on how to otherwise circumvent this problem.  The obvious 
> solution is to use the retroactive header only on the very first connection, but 
> how can we know when the last retroactive message is received?
> 
> I hope reposting is ok, I couldn't find any mailing list rules.  Here's the more 
> verbose version that I originally wrote:
> http://www.nabble.com/Durable-subscription-with-retroactive-subscription-recovery-policy-td23258698ef2356.html
> 
> Thanks for your time,
> 
> JLuna


Just in case anyone has a similar problem finds this post, we ended up modifying ActiveMQ to send an advisory message after all past messages have been recovered for the subscription.  (The Advisory message goes only to the consumer, not to the topic for all consumers to see.)  This allows the consumer to know that it has received all past messages and shouldn't use the retroactive header in future.  This is undoubtedly somewhat "hackish" way to do it, but it's the quickest thing we could come up with given a short deadline.  The diff is attached.

JLuna



      

Durable subscription with subscription recovery policy

[Jose Luna <j-...@rocketmail.com>]

Hello,

I haven't received a response after a few days, so I'd like to try again. I'll try to be brief this time:

Our consumers disconnect/reconnect frequently.  We need durable subscriptions to work with the subscription recovery policy.  Currently (Activemq 5.2), durable subscriptions with the "activemq.retroactive" header (STOMP) will receive all retroactive messages every time the durable subscriber reconnects.

A bug report can be found at https://issues.apache.org/activemq/browse/AMQ-1549 .  The bug is open and assigned, with fixed
version marked as 5.3 (although no work has been done yet).

However, in this thread http://www.nabble.com/Durable-subscriptions-to19688854.html,  Bruce and James seems to say that subscription recovery policy is
only for non-durable subscriptions.  

So, are durable subscriptions meant to work with subscription recovery policies?  It used to work this way in 4.x, but I'm not sure if it was an intentional change for 5.x.

I'm looking for two kinds of advice: 
1.) How we might patch this ourselves.  We're a small development team on a tight deadline, but we might be able to take a crack at it.

2.) Any advice on how to otherwise circumvent this problem.  The obvious solution is to use the retroactive header only on the very first connection, but how can we know when the last retroactive message is received?

I hope reposting is ok, I couldn't find any mailing list rules.  Here's the more verbose version that I originally wrote:
http://www.nabble.com/Durable-subscription-with-retroactive-subscription-recovery-policy-td23258698ef2356.html

Thanks for your time,

JLuna