You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Jason Chaffee <jc...@ebates.com> on 2010/11/15 22:16:15 UTC
messages stuck in queues
I am seeing issues where my messages are getting stuck in queues. I
check the MBeans and it says they are inflight, so the consumer hasn't
picked them. In my case, the consumer is a POJO method in the same JVM.
Also, it appears retries are not happening. It just gets stuck.
Before I attach my configuration, etc. I am wondering if anyone has
seen this or has idea of what might be happening?
Thanks.
Jason
Re: messages stuck in queues
Posted by Willem Jiang <wi...@gmail.com>.
It looks like more than Camel issue. as there are more than on broker
involved.
Did you try to send this issue to ActiveMQ user list?
On 11/16/10 12:48 PM, Jason Chaffee wrote:
> Sure have.
>
> Here is what I found.
>
> One of m brokers (on another machine) was hitting it's store usage limit so it stopped allow producers to send it messages. The two other brokers then had their queues getting backed up because they could not route from their queues to to this one.
>
> I fixed the usage limit configuration and it allow new messages to flow, but the old ones were still stuck, even though I have retries configured to run for a few days. I had to actually restart all of them to get them to empty out.
>
> A very painful experience. Not sure if this an ActiveMQ or a Camel issue because it basically stopped routing the messages and we use Camel for that.
>
> Jason
>
>
> -----Original Message-----
> From: Taariq Levack [mailto:taariql@itaro.co.za]
> Sent: Mon 11/15/2010 8:46 PM
> To: users@camel.apache.org
> Subject: RE: messages stuck in queues
>
> Have you enabled the tracer?
> http://camel.apache.org/tracer.html
>
> It may or may not be useful for this problem to read this sort of output...
> INFO TraceInterceptor - ID-davsclaus-local-54403-1246038742624-0-1>>> from(direct:start) --> MyProcessor , Pattern:InOnly, Headers:{from=Claus}, BodyType:String, Body:This is Copenhagen calling
> INFO TraceInterceptor - ID-davsclaus-local-54403-1246038742624-0-1>>> MyProcessor --> mock:a , Pattern:InOnly, Headers:{from=Claus}, BodyType:String, Body:This is Copenhagen calling
> INFO TraceInterceptor - ID-davsclaus-local-54403-1246038742624-0-1>>> mock:a --> mock:b , Pattern:InOnly, Headers:{from=Claus}, BodyType:String, Body:This is Copenhagen calling
>
> Taariq
>
> -----Original Message-----
> From: Jason Chaffee [mailto:jchaffee@ebates.com]
> Sent: 15 November 2010 23:16
> To: users@camel.apache.org
> Subject: messages stuck in queues
>
> I am seeing issues where my messages are getting stuck in queues. I check the MBeans and it says they are inflight, so the consumer hasn't picked them. In my case, the consumer is a POJO method in the same JVM.
> Also, it appears retries are not happening. It just gets stuck.
>
>
>
> Before I attach my configuration, etc. I am wondering if anyone has seen this or has idea of what might be happening?
>
>
>
> Thanks.
>
>
>
> Jason
>
>
>
--
Willem
----------------------------------
FuseSource
Web: http://www.fusesource.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.javaeye.com (Chinese)
Twitter: willemjiang
RE: messages stuck in queues
Posted by Jason Chaffee <jc...@ebates.com>.
Sure have.
Here is what I found.
One of m brokers (on another machine) was hitting it's store usage limit so it stopped allow producers to send it messages. The two other brokers then had their queues getting backed up because they could not route from their queues to to this one.
I fixed the usage limit configuration and it allow new messages to flow, but the old ones were still stuck, even though I have retries configured to run for a few days. I had to actually restart all of them to get them to empty out.
A very painful experience. Not sure if this an ActiveMQ or a Camel issue because it basically stopped routing the messages and we use Camel for that.
Jason
-----Original Message-----
From: Taariq Levack [mailto:taariql@itaro.co.za]
Sent: Mon 11/15/2010 8:46 PM
To: users@camel.apache.org
Subject: RE: messages stuck in queues
Have you enabled the tracer?
http://camel.apache.org/tracer.html
It may or may not be useful for this problem to read this sort of output...
INFO TraceInterceptor - ID-davsclaus-local-54403-1246038742624-0-1 >>> from(direct:start) --> MyProcessor , Pattern:InOnly, Headers:{from=Claus}, BodyType:String, Body:This is Copenhagen calling
INFO TraceInterceptor - ID-davsclaus-local-54403-1246038742624-0-1 >>> MyProcessor --> mock:a , Pattern:InOnly, Headers:{from=Claus}, BodyType:String, Body:This is Copenhagen calling
INFO TraceInterceptor - ID-davsclaus-local-54403-1246038742624-0-1 >>> mock:a --> mock:b , Pattern:InOnly, Headers:{from=Claus}, BodyType:String, Body:This is Copenhagen calling
Taariq
-----Original Message-----
From: Jason Chaffee [mailto:jchaffee@ebates.com]
Sent: 15 November 2010 23:16
To: users@camel.apache.org
Subject: messages stuck in queues
I am seeing issues where my messages are getting stuck in queues. I check the MBeans and it says they are inflight, so the consumer hasn't picked them. In my case, the consumer is a POJO method in the same JVM.
Also, it appears retries are not happening. It just gets stuck.
Before I attach my configuration, etc. I am wondering if anyone has seen this or has idea of what might be happening?
Thanks.
Jason
RE: messages stuck in queues
Posted by Taariq Levack <ta...@itaro.co.za>.
Have you enabled the tracer?
http://camel.apache.org/tracer.html
It may or may not be useful for this problem to read this sort of output...
INFO TraceInterceptor - ID-davsclaus-local-54403-1246038742624-0-1 >>> from(direct:start) --> MyProcessor , Pattern:InOnly, Headers:{from=Claus}, BodyType:String, Body:This is Copenhagen calling
INFO TraceInterceptor - ID-davsclaus-local-54403-1246038742624-0-1 >>> MyProcessor --> mock:a , Pattern:InOnly, Headers:{from=Claus}, BodyType:String, Body:This is Copenhagen calling
INFO TraceInterceptor - ID-davsclaus-local-54403-1246038742624-0-1 >>> mock:a --> mock:b , Pattern:InOnly, Headers:{from=Claus}, BodyType:String, Body:This is Copenhagen calling
Taariq
-----Original Message-----
From: Jason Chaffee [mailto:jchaffee@ebates.com]
Sent: 15 November 2010 23:16
To: users@camel.apache.org
Subject: messages stuck in queues
I am seeing issues where my messages are getting stuck in queues. I check the MBeans and it says they are inflight, so the consumer hasn't picked them. In my case, the consumer is a POJO method in the same JVM.
Also, it appears retries are not happening. It just gets stuck.
Before I attach my configuration, etc. I am wondering if anyone has seen this or has idea of what might be happening?
Thanks.
Jason