You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by David Sitsky <si...@nuix.com> on 2008/02/08 17:09:44 UTC
Contention in Queue.lock() via IndirectMessageReference.lock()
In my application, I have a large number of threads in the broker, which
are calling dispatchPending() on their subscriptions, but are unable to
make fast progress, as can be seen in the following stack trace:
"ActiveMQ Transport: tcp:///192.168.222.75:56495" daemon prio=4
tid=0x4be82000 n
id=0x1028 waiting for monitor entry [0x54fdf000..0x54fdfb00]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.activemq.broker.region.Queue.lock(Queue.java:166)
- locked <0x0a60bf58> (a java.lang.Object)
at
org.apache.activemq.broker.region.IndirectMessageReference.lock(IndirectMessageReference.java:143)
at
org.apache.activemq.broker.region.QueueSubscription.canDispatch(QueueSubscription.java:119)
at
org.apache.activemq.broker.region.PrefetchSubscription.dispatchPending(PrefetchSubscription.java:466)
- locked <0x0b55bd78> (a java.lang.Object)
at
org.apache.activemq.broker.region.PrefetchSubscription.acknowledge(PrefetchSubscription.java:347)
at
org.apache.activemq.broker.region.AbstractRegion.acknowledge(AbstractRegion.java:342)
at
org.apache.activemq.broker.region.RegionBroker.acknowledge(RegionBroker.java:461)
at
org.apache.activemq.broker.TransactionBroker.acknowledge(TransactionBroker.java:194)
at
org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:73)
at
org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:73)
at
org.apache.activemq.broker.MutableBrokerFilter.acknowledge(MutableBrokerFilter.java:87)
at
org.apache.activemq.broker.TransportConnection.processMessageAck(TransportConnection.java:440)
at
org.apache.activemq.command.MessageAck.visit(MessageAck.java:196)
....
I am trying to understand why IndirectMessageReference.lock() would
require synchronisation on the entire Queue object it is associated with?
This is why the performance in my application is grinding... each time a
prefetch/queue subscription is testing to see if they can dispatch a
message, we are effectively serialising on the Queue object.
Any ideas on how we can fix this? I have some ideas, but thought the
experts are better qualified. :)
This became evident when I had 20 consumers hammering away. With 2,
you'd never see this.
Cheers,
David
Re: Contention in Queue.lock() via IndirectMessageReference.lock()
Posted by David Sitsky <si...@nuix.com>.
FWIW - since I don't use exclusive consumers in my application, I
commented out the first three lines in IndirectMessageReference.lock()
below, and that bottleneck has been removed.
if (!regionDestination.lock(this, subscription)) {
return false;
}
Anyway - hopefully this is of interest to somebody. Perhaps we can
optimise this so that if we don't use exclusive consumers to make the
above call a no-op?
Cheers,
David
David Sitsky wrote:
> In my application, I have a large number of threads in the broker, which
> are calling dispatchPending() on their subscriptions, but are unable to
> make fast progress, as can be seen in the following stack trace:
>
> "ActiveMQ Transport: tcp:///192.168.222.75:56495" daemon prio=4
> tid=0x4be82000 n
> id=0x1028 waiting for monitor entry [0x54fdf000..0x54fdfb00]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.activemq.broker.region.Queue.lock(Queue.java:166)
> - locked <0x0a60bf58> (a java.lang.Object)
> at
> org.apache.activemq.broker.region.IndirectMessageReference.lock(IndirectMessageReference.java:143)
>
> at
> org.apache.activemq.broker.region.QueueSubscription.canDispatch(QueueSubscription.java:119)
>
> at
> org.apache.activemq.broker.region.PrefetchSubscription.dispatchPending(PrefetchSubscription.java:466)
>
> - locked <0x0b55bd78> (a java.lang.Object)
> at
> org.apache.activemq.broker.region.PrefetchSubscription.acknowledge(PrefetchSubscription.java:347)
>
> at
> org.apache.activemq.broker.region.AbstractRegion.acknowledge(AbstractRegion.java:342)
>
> at
> org.apache.activemq.broker.region.RegionBroker.acknowledge(RegionBroker.java:461)
>
> at
> org.apache.activemq.broker.TransactionBroker.acknowledge(TransactionBroker.java:194)
>
> at
> org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:73)
> at
> org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:73)
> at
> org.apache.activemq.broker.MutableBrokerFilter.acknowledge(MutableBrokerFilter.java:87)
>
> at
> org.apache.activemq.broker.TransportConnection.processMessageAck(TransportConnection.java:440)
>
> at
> org.apache.activemq.command.MessageAck.visit(MessageAck.java:196)
> ....
>
> I am trying to understand why IndirectMessageReference.lock() would
> require synchronisation on the entire Queue object it is associated with?
>
> This is why the performance in my application is grinding... each time a
> prefetch/queue subscription is testing to see if they can dispatch a
> message, we are effectively serialising on the Queue object.
>
> Any ideas on how we can fix this? I have some ideas, but thought the
> experts are better qualified. :)
>
> This became evident when I had 20 consumers hammering away. With 2,
> you'd never see this.
>
> Cheers,
> David
Re: Contention in Queue.lock() via IndirectMessageReference.lock()
Posted by Rob Davies <ra...@gmail.com>.
Its there for message exclusive consumers - will look at changing ;)
On Feb 8, 2008, at 4:09 PM, David Sitsky wrote:
> In my application, I have a large number of threads in the broker,
> which are calling dispatchPending() on their subscriptions, but are
> unable to make fast progress, as can be seen in the following stack
> trace:
>
> "ActiveMQ Transport: tcp:///192.168.222.75:56495" daemon prio=4
> tid=0x4be82000 n
> id=0x1028 waiting for monitor entry [0x54fdf000..0x54fdfb00]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.activemq.broker.region.Queue.lock(Queue.java:166)
> - locked <0x0a60bf58> (a java.lang.Object)
> at
> org
> .apache
> .activemq
> .broker
> .region.IndirectMessageReference.lock(IndirectMessageReference.java:
> 143)
> at
> org
> .apache
> .activemq
> .broker.region.QueueSubscription.canDispatch(QueueSubscription.java:
> 119)
> at
> org
> .apache
> .activemq
> .broker
> .region
> .PrefetchSubscription.dispatchPending(PrefetchSubscription.java:466)
> - locked <0x0b55bd78> (a java.lang.Object)
> at
> org
> .apache
> .activemq
> .broker
> .region.PrefetchSubscription.acknowledge(PrefetchSubscription.java:
> 347)
> at
> org
> .apache
> .activemq
> .broker.region.AbstractRegion.acknowledge(AbstractRegion.java:342)
> at
> org
> .apache
> .activemq.broker.region.RegionBroker.acknowledge(RegionBroker.java:
> 461)
> at
> org
> .apache
> .activemq
> .broker.TransactionBroker.acknowledge(TransactionBroker.java:194)
> at
> org
> .apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:73)
> at
> org
> .apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:73)
> at
> org
> .apache
> .activemq
> .broker.MutableBrokerFilter.acknowledge(MutableBrokerFilter.java:87)
> at
> org
> .apache
> .activemq
> .broker
> .TransportConnection.processMessageAck(TransportConnection.java:440)
> at
> org.apache.activemq.command.MessageAck.visit(MessageAck.java:196)
> ....
>
> I am trying to understand why IndirectMessageReference.lock() would
> require synchronisation on the entire Queue object it is associated
> with?
>
> This is why the performance in my application is grinding... each
> time a prefetch/queue subscription is testing to see if they can
> dispatch a message, we are effectively serialising on the Queue
> object.
>
> Any ideas on how we can fix this? I have some ideas, but thought
> the experts are better qualified. :)
>
> This became evident when I had 20 consumers hammering away. With 2,
> you'd never see this.
>
> Cheers,
> David