You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by RuralHunter <ru...@gmail.com> on 2016/06/01 01:06:25 UTC
Negative message count and dead messages in queue
Hi,
We are using activemq 5.10.2 on our production. I always see negative
message count on some busy queues with many producers and consumers in admin
ui. This is not a big problem but another issue is important. There are some
stuck messages left in some of my queues. The messages are produced several
days ago and it seems they will never be consumed. Though new messages are
produced and consumed in the same queue without problem. Those stuck
messages seems are dead in the queue. This again leads the storage full
problem. The old logs can not be removed because of those dead messages.
Is there anyway I can investigate the cause of the problem?
--
View this message in context: http://activemq.2283324.n4.nabble.com/Negative-message-count-and-dead-messages-in-queue-tp4712505.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Negative message count and dead messages in queue
Posted by RuralHunter <ru...@gmail.com>.
Yes, in web console. I can see those messages. They are produced several days
ago. When there is no new message, they stick in the queue. When new
messages are produced, they are consumed quickly but those old messages are
still there and no consumer can get them.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Negative-message-count-and-dead-messages-in-queue-tp4712505p4712560.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Negative message count and dead messages in queue
Posted by RuralHunter <ru...@gmail.com>.
We only ran 5.13.3 in production for several days and didn't see this issue.
But we saw another issue which we didn't see in 5.10.2:
Some old data file(2 days ago) is reserved because of some queue. But when I
check the queue in admin ui, I didn't see any old message. The new messsages
are in and out all the time in that queue. If I delete the queue in admin
ui, the old data file is cleaned up soon.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Negative-message-count-and-dead-messages-in-queue-tp4712505p4712765.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Negative message count and dead messages in queue
Posted by Tim Bain <tb...@alumni.duke.edu>.
This feels very familiar, like there was a bug fixed recently or a mailing
list conversion where someone else had the same problem. I've done some
quick searching both places and didn't find anything obvious. I keep
thinking I'll have some time to do a better search, but it hasn't happened;
sorry.
I'm curious whether it still happens after your recent upgrade to 5.13.3.
(Once you fix the OOM, which I'm sorry, I don't know anything about.) If
so, do you have any reliable way for someone who isn't you to reproduce the
problem? That would let someone attach a debugger and step through to see
what's happening.
Tim
On Jun 3, 2016 12:00 AM, "RuralHunter" <ru...@gmail.com> wrote:
> I can view those dead messages in web console by clicking the queue name,
> not
> the count. When I restart, they disappears.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Negative-message-count-and-dead-messages-in-queue-tp4712505p4712591.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: Negative message count and dead messages in queue
Posted by RuralHunter <ru...@gmail.com>.
I can view those dead messages in web console by clicking the queue name, not
the count. When I restart, they disappears.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Negative-message-count-and-dead-messages-in-queue-tp4712505p4712591.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Negative message count and dead messages in queue
Posted by Tim Bain <tb...@alumni.duke.edu>.
To be clear, are you saying that you can see them by browsing the queue via
the web console, or are you saying that the counts are non-zero?
Also, what happens if you restart the broker?
On Thu, Jun 2, 2016 at 8:22 AM, RuralHunter <ru...@gmail.com> wrote:
> And no, no selector is used, just simple JMS consumer.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Negative-message-count-and-dead-messages-in-queue-tp4712505p4712561.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: Negative message count and dead messages in queue
Posted by RuralHunter <ru...@gmail.com>.
And no, no selector is used, just simple JMS consumer.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Negative-message-count-and-dead-messages-in-queue-tp4712505p4712561.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Negative message count and dead messages in queue
Posted by Tim Bain <tb...@alumni.duke.edu>.
I'm asking what evidence points to the existence of those messages in the
first place. Counts in the web console, or something else?
Also, do your consumers use any selectors?
On Jun 2, 2016 12:42 AM, "RuralHunter" <ru...@gmail.com> wrote:
> Because new messages are produced and consumed all the time in the same
> queue. Just those messages are left there and it seems no consumers can get
> them.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Negative-message-count-and-dead-messages-in-queue-tp4712505p4712545.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: Negative message count and dead messages in queue
Posted by RuralHunter <ru...@gmail.com>.
Because new messages are produced and consumed all the time in the same
queue. Just those messages are left there and it seems no consumers can get
them.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Negative-message-count-and-dead-messages-in-queue-tp4712505p4712545.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Negative message count and dead messages in queue
Posted by Tim Bain <tb...@alumni.duke.edu>.
How are you determining that there are dead messages?
On May 31, 2016 7:44 PM, "RuralHunter" <ru...@gmail.com> wrote:
> Hi,
>
> We are using activemq 5.10.2 on our production. I always see negative
> message count on some busy queues with many producers and consumers in
> admin
> ui. This is not a big problem but another issue is important. There are
> some
> stuck messages left in some of my queues. The messages are produced several
> days ago and it seems they will never be consumed. Though new messages are
> produced and consumed in the same queue without problem. Those stuck
> messages seems are dead in the queue. This again leads the storage full
> problem. The old logs can not be removed because of those dead messages.
> Is there anyway I can investigate the cause of the problem?
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Negative-message-count-and-dead-messages-in-queue-tp4712505.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>