Re: MessageListener not delivering any more messages at some point

[Jeremias Maerki <de...@jeremias-maerki.ch>]

We've had to wait until today to have another incident. To answer your
questions:
- the two (expected) connections (one per tenant) are active.
- the number of consumers for the destination is correct: again two.
- A non-durable subscription exists for each client. Again two in total.

Basically, we have two clients listening on the same queue and picking
messages using a selector. Each client is a tenant/instance of an
application that share a common service that produces the notifications.

So, if I can trust the JMX output, the broker knows about the two
consumers and tells me they are active.

We're rolling out 5.6.0 to the production environment soon. We'll see if
that changes anything.


Jeremias Maerki


On 04.06.2012 15:48:28 Raul Kripalani wrote:
> Definitely upgrade to 5.5.1 or later.
> 
> What does your JMX instrumentation tell you when the problem occurs?
> 
>    - Is the connection active?
>    - Does the # of consumers in the destination match what you expect?
>    - Does a subscription for that client exist?
> 
> My questions boil down to: at that point, does the broker know about the
> (allegedly) active consumer?
> 
> Regards,
> 
>  *Raúl Kripalani*
> Principal Consultant | FuseSource Corp.
> raul@fusesource.com | fusesource.com <http://www.fusesource.com/>
> skype: raul.fuse | twitter: @raulvk <http://twitter.com/raulvk>,
> @fusenews<http://twitter.com/fusenews>
> 
> <http://twitter.com/fusenews>
> 
> On 30 May 2012 10:49, Torsten Mielke <to...@fusesource.com> wrote:
> 
> > Hhm, SMX 4.4.1 deploys ActiveMQ 5.5.1 just nicely into its OSGi container.
> >
> > On May 30, 2012, at 11:44 AM, Jeremias Maerki wrote:
> >
> > > Hi Torsten,
> > >
> > > thanks. Yes, I've made experiments with 5.5.1 on the server side but had
> > > trouble getting it up reliably in my OSGi environment so I'm hesitant to
> > > upgrade on the server side. However, the problem resides mainly in the
> > > client AFAICS, and that one uses 5.5.1 already. I guess I'll take a peek
> > > at 5.6.0 to see if it looks better there.
> > >
> > >
> > > Jeremias Maerki
> > >
> > >
> > > On 30.05.2012 11:30:04 Torsten Mielke wrote:
> > >> Hello,
> > >>
> > >> ActiveMQ 5.3.0 is really old. Have you thought of upgrading to the
> > latest broker release 5.5.1?
> > >> Chances aren't too bad that your problem has been resolved.
> > >>
> > >>
> > >>
> > >> Torsten Mielke
> > >> torsten@fusesource.com
> > >> tmielke@blogspot.com
> > >>
> > >>
> > >> On May 30, 2012, at 11:25 AM, Jeremias Maerki wrote:
> > >>
> > >>> No ideas? As a work-around I'm now thinking about running a scheduled
> > >>> task that restarts the MessageListener every hour. That's a work-around
> > >>> that shouldn't really be necessary, right? A colleague told me
> > yesterday
> > >>> that he did exactly that in a personal project of his. So, I'm probably
> > >>> not the only one with this problem, although I couldn't find anything
> > >>> related to that on the net.
> > >>>
> > >>> Thanks,
> > >>> Jeremias Maerki
> > >>>
> > >>>
> > >>> On 25.05.2012 12:23:54 Jeremias Maerki wrote:
> > >>>> Hi,
> > >>>>
> > >>>> I've got an ActiveMQ 5.3.0 broker running in one JVM.
> > >>>>
> > >>>>       <transportConnectors>
> > >>>>           <transportConnector name="openwire"
> > uri="tcp://localhost:19616"/>
> > >>>>       </transportConnectors>
> > >>>>
> > >>>> Another application (different JVM but currently on the same machine)
> > >>>> connects to that broker to receive messages from a single queue
> > through
> > >>>> a MessageListener:
> > >>>>
> > >>>>
> > failover://(tcp://localhost:19616?keepAlive=true)?startupMaxReconnectAttempts=5
> > >>>>
> > >>>> The message frequency is rather low (5-500 per hour) and can drop to
> > >>>> zero over multiple hours (during the night). Every now and then, the
> > >>>> MessageListener simply doesn't get any more messages and we have to
> > >>>> restart the application for a reconnect despite the failover protocol
> > >>>> (as seen above). We've tried various connection URI parameters but so
> > >>>> far, the issue keeps popping up every few weeks. I somehow doubt it
> > has
> > >>>> to do with some timeout because, yesterday, it happened during the day
> > >>>> during normal business, not in the night after hours of inactivity.
> > When
> > >>>> the problem happens, the message producer can still happily add new
> > >>>> messages which are then simply piled up.
> > >>>>
> > >>>> We first thought that the broker would stop accepting connections at
> > >>>> some point but we now have a monitoring agent for Nagios that
> > regularly
> > >>>> tries to connect to ActiveMQ to check if anything is wrong on that
> > side,
> > >>>> but that proved to be in vain. So, everything is solid on the broker
> > >>>> side.
> > >>>>
> > >>>> The client used to run ActiveMQ 5.3.0, too (just the
> > >>>> ActiveMQConnectionFactory, no broker). We also tried 5.5.1 on the
> > client
> > >>>> side but nothing has changed.
> > >>>>
> > >>>> I was wondering if anyone has a good idea on this problem.
> > >>>>
> > >>>> Thanks a lot,
> > >>>> Jeremias Maerki
> > >>>
> > >>
> > >>
> > >>
> > >>
> > >
> >
> > Torsten Mielke
> > torsten@fusesource.com
> > tmielke@blogspot.com
> >
> >
> >
> >