PooledConnections Poor Performance

[ks <va...@yahoo.com>]

I used Spring's CachingConnectionFactory and found that it uses only one
connection. When we run performance tests, threads are blocked to send
message ( even when it is async). 

So I tried using PooledConnectionFactory. Now threads are blocked more time
for getting connection ( creating the connection and also acquiring the
connection). With the pooling I would expect only first set of requests to
take time. But here it also took more time to get the created connection. 

	
		
			
		
	

Any suggestions?

--
View this message in context: http://activemq.2283324.n4.nabble.com/PooledConnections-Poor-Performance-tp3446142p3446142.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: PooledConnections Poor Performance

[Reynald Borer <re...@gmail.com>]

Hi,

Indeed PooledConnectionFactory is the way to go if you want to have the best performances with ActiveMQ.

Can you tell us a little bit more on how you do send your messages to be so badly impacted by the lock on the pool? I suppose your are using the JmsTemplate from Spring to send your messages, right? In that case, yes there is a massive overhead with this template as each call on send() will borrow a connection from the pool (1 round-trip to the broker if this is a new connection), create a session (1 round-trip to the broker) and then create a message producer (1 round-trip to the broker too). Then once the message is sent, all those elements are closed (of course the connection is given back to the pool so it remains open).

So I would tend to say that you should try to drop JmsTemplate and replace it by your own implementation that will keep the producers connected as long as there is something to send. Depending on your needs, you might consider using a pool of threads that will consume messages to send from a BlockingQueue, while your business code will simply add messages to this queue. But of course this depends on your business needs.

Maybe you can try to combine CachingConnectionFactory and PooledConnectionFactory so that the CachingConnectionFactory keep the session and producer in cache (that is open), this will allow you to keep using JmsTemplate. I don't know how this will work together, I have never tested this setup myself. Maybe someone else on the mailing-list have an experience to share.

Have you also considered using non persistent messages, or is it a requirement for you to have persistent messages? Messages are persistent by default, and of course each message is sent synchronously to the broker, which has to do a disk sync before sending back the acknowledgment. This lower the throughput of the broker, of course.

Regards,
Reynald


On Wednesday, April 13, 2011 at 19:07 , ks wrote: 
> Thanks. I did increase my session size to 100. So it cached sessions. The
> problem was with Single connection Object. All the threads were trying to
> use the same connection. Connection Object uses this Mutex before sending.
> So essentially this is a blocker. 
> 
>  java.lang.Thread.State: BLOCKED
>  at
> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:39)
>  - waiting to lock <1770d26> (a java.lang.Object) owned by
> "catalina-exec-110" t@210
>  at
> org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
>  at
> org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1214)
>  at
> org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1208)
>  at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1643)
> 
> So I wanted to have more connections and I started with
> PooledConnectionFactory from activeMQ and configured to have 50 connections.
> This did not help either. The lock to create the pooled connection and
> obtain a connection has major impact on throughput. 
> 
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/PooledConnections-Poor-Performance-tp3446142p3447752.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> 

Re: PooledConnections Poor Performance

[ks <va...@yahoo.com>]

Thanks. I did increase my session size to 100. So it cached sessions. The
problem was with Single connection Object. All the threads were trying to
use the same connection. Connection Object uses this Mutex before sending.
So essentially this is a blocker. 

  java.lang.Thread.State: BLOCKED
	at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:39)
	- waiting to lock <1770d26> (a java.lang.Object) owned by
"catalina-exec-110" t@210
	at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
	at
org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1214)
	at
org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1208)
	at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1643)

So I wanted to have more connections and I started with
PooledConnectionFactory from activeMQ and configured to have 50 connections.
This did not help either. The lock to create the pooled connection and
obtain a connection has major impact on throughput. 

--
View this message in context: http://activemq.2283324.n4.nabble.com/PooledConnections-Poor-Performance-tp3446142p3447752.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: PooledConnections Poor Performance

[Reynald Borer <re...@gmail.com>]

Hi,

By default, CachingConnectionFactory only caches a single session, as explained on the JavaDoc of the class (http://static.springsource.org/spring/docs/2.5.x/api/org/springframework/jms/connection/CachingConnectionFactory.html):

"By default, only one single Session will be cached, with further requested Sessions being created and disposed on demand. Consider raising the "sessionCacheSize" value in case of a high-concurrency environment."

So you should consider increasing the session cache size to see if it help.

Cheers,
Reynald

On Wednesday, April 13, 2011 at 4:51 , ks wrote: 
> I used Spring's CachingConnectionFactory and found that it uses only one
> connection. When we run performance tests, threads are blocked to send
> message ( even when it is async). 
> 
> So I tried using PooledConnectionFactory. Now threads are blocked more time
> for getting connection ( creating the connection and also acquiring the
> connection). With the pooling I would expect only first set of requests to
> take time. But here it also took more time to get the created connection. 
> 
> 
> 
> 
> 
> 
> 
> Any suggestions?
> 
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/PooledConnections-Poor-Performance-tp3446142p3446142.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>