You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Frizz <fr...@googlemail.com> on 2016/02/22 17:06:26 UTC
No performance improvement using PooledConnectionFactory (AMQ 5.12.1).
Hi there,
I am experimenting with a Pooled Connection Factory - but I don't see any
performance gain (even worse: using PooledConnectionFactory is slower than
ActiveMQConnectionFactory).
What I do: Sending a few 100.000 messages in several threads (using
CompletableFuture). Ond run with a ActiveMQConnectionFactory, the other
with PooledConnectionFactory:
1st Run:
ConnectionFactory cf = new
ActiveMQConnectionFactory("tcp://localhost:61616");
Connection connection = cf.createConnection();
connection.start();
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
producer = session.createProducer(null);
final Topic topic = session.createTopic(topicName);
final ObjectMessage message = session.createObjectMessage(myObject);
producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
producer.send(topic, message);
2nd Run:
PooledConnectionFactory pcf = new
PooledConnectionFactory("tcp://localhost:61616");
pcf.setMaxConnections(8);
Connection connection = pcf.createConnection();
connection.start();
...
Am I setting up the PooledConnectionFactory the wrong way? Am I missing
something?
Re: No performance improvement using PooledConnectionFactory (AMQ 5.12.1).
Posted by Tim Bain <tb...@alumni.duke.edu>.
It's also possible that 8 isn't the optimal pool size for your test driver,
and that a different (presumably larger) value would give better
throughput. Tuning the pool size of any pooling technology (ActiveMQ,
JDBC, etc.) is an iterative and experimental process, and your first guess
often won't be the optimal one.
On Feb 25, 2016 9:26 PM, "artnaseef" <ar...@artnaseef.com> wrote:
> I agree with Robbie's comments here. The statement that the application
> only
> creates the connections once got me confused at first, but looking at the
> original post, the pseudo-code there clearly creates the connection once on
> each producer.
>
> To verify, start up the application and use one of the following methods to
> check the number of actual connections to the broker:
>
> * ActiveMQ webconsole
> * netstat -an | grep 61616 | grep ESTABLISHED # on the broker box
> * JConsole, VisualVM, or the like (view the connection mbeans, or the
> connection count on the broker mbean)
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/No-performance-improvement-using-PooledConnectionFactory-AMQ-5-12-1-tp4708033p4708432.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: No performance improvement using PooledConnectionFactory (AMQ
5.12.1).
Posted by artnaseef <ar...@artnaseef.com>.
I agree with Robbie's comments here. The statement that the application only
creates the connections once got me confused at first, but looking at the
original post, the pseudo-code there clearly creates the connection once on
each producer.
To verify, start up the application and use one of the following methods to
check the number of actual connections to the broker:
* ActiveMQ webconsole
* netstat -an | grep 61616 | grep ESTABLISHED # on the broker box
* JConsole, VisualVM, or the like (view the connection mbeans, or the
connection count on the broker mbean)
--
View this message in context: http://activemq.2283324.n4.nabble.com/No-performance-improvement-using-PooledConnectionFactory-AMQ-5-12-1-tp4708033p4708432.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: No performance improvement using PooledConnectionFactory (AMQ 5.12.1).
Posted by Robbie Gemmell <ro...@gmail.com>.
Ok, so it looks like points from the earlier replies of both Tim and
myself would seem to apply then.
You arent creating and closing lots of connections repeatedly over
time, which is about the only situation a pool will help improve
performance in.
The non-pooled factory doesnt give you back the same connection each
time, it creates a new one each time you ask, so it would seem your
test will be alternating between using 250 real connections, and only
using 8 with the added overhead associated in implementing the
pooling.
Robbie
On 23 February 2016 at 12:47, Frizz <fr...@googlemail.com> wrote:
> On Tue, Feb 23, 2016 at 12:25 PM, Robbie Gemmell <ro...@gmail.com>
> wrote:
>
>> I can't say for sure from your summarised description and partial
>> code, but it seems like you may only ever create 1 connection object
>> with the factory. If so, explicitly creating multiple connections and
>> splitting your producers across them would likely improve performance.
>>
>
>
> I am calling "Connection connection =
> pooledConnectionFactory.createConnection(); connection.start();" for every
> Thread / Producer - so 250 times.
>
> At least for my pooled connection is should result in 8 "real" connections,
> no?
>
>
> Here is my test class:
>
> public class ProducerMT {
>
> private static final long NUM_MESSAGES_PER_PRODUCER = 1000;
> private final static int NUM_PRODUCERS = 250;
>
> private static String connectionUrl = "tcp://localhost:61616";
> private MessageProducer producer = null;
> private Session session = null;
>
> private static ConnectionFactory factory = null;
>
> public static void setPooledConnectionFactory() {
> PooledConnectionFactory pcf = new
> PooledConnectionFactory(connectionUrl);
> pcf.setMaxConnections(8);
> pcf.setMaximumActiveSessionPerConnection(500);
> factory = pcf;
> }
>
>
> public void sendMessage(String topicName, Event event) {
>
> if(producer == null) {
>
> try {
> Connection connection = factory.createConnection();
> connection.start();
> session = connection.createSession(false,
> Session.AUTO_ACKNOWLEDGE);
> producer = session.createProducer(null);
>
> } catch (JMSException exp) {
> System.err.println("Error creating
> connection/session/producer: " + exp);
> exp.printStackTrace();
> }
> } else
>
> try {
> final Topic topic = session.createTopic(topicName);
> final ObjectMessage message =
> session.createObjectMessage(event);
>
> producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
> producer.send(topic, message);
> } catch (JMSException exp) {
> System.err.println("Error sending message: " + exp);
> exp.printStackTrace();
> }
>
> }
>
>
> private CompletableFuture<Integer> sendMessages(int eventId) {
> CompletableFuture<Integer> cf = CompletableFuture.supplyAsync(
> () -> {
> int i;
> for (i = 0; i < NUM_MESSAGES_PER_PRODUCER; i++) {
> sendMessage("topic_" + eventId, new Event());
> }
> return i;
> });
> return cf;
> }
>
> public static void main(String[] args) {
>
> setPooledConnectionFactory();
>
> List<CompletableFuture<Integer>> returnValues = new ArrayList<>();
>
> ProducerMT producers[] = new ProducerMT[NUM_PRODUCERS];
> for (int i = 0; i < NUM_PRODUCERS; i++) {
> producers[i] = new ProducerMT();
> }
>
> long startTime = System.nanoTime();
> try {
> for (int i = 0; i < NUM_PRODUCERS; i++) {
> returnValues.add(producers[i].sendMessages(i));
> }
>
> for (int i = 0; i < NUM_PRODUCERS; i++) {
> returnValues.get(i).get();
> }
>
> } catch (InterruptedException | ExecutionException e) {
> e.printStackTrace();
> }
>
> long elapsedTime = System.nanoTime() - startTime;
> double seconds = (double)elapsedTime / 1000000000.0;
>
> }
> }
Re: No performance improvement using PooledConnectionFactory (AMQ 5.12.1).
Posted by Frizz <fr...@googlemail.com>.
On Tue, Feb 23, 2016 at 12:25 PM, Robbie Gemmell <ro...@gmail.com>
wrote:
> I can't say for sure from your summarised description and partial
> code, but it seems like you may only ever create 1 connection object
> with the factory. If so, explicitly creating multiple connections and
> splitting your producers across them would likely improve performance.
>
I am calling "Connection connection =
pooledConnectionFactory.createConnection(); connection.start();" for every
Thread / Producer - so 250 times.
At least for my pooled connection is should result in 8 "real" connections,
no?
Here is my test class:
public class ProducerMT {
private static final long NUM_MESSAGES_PER_PRODUCER = 1000;
private final static int NUM_PRODUCERS = 250;
private static String connectionUrl = "tcp://localhost:61616";
private MessageProducer producer = null;
private Session session = null;
private static ConnectionFactory factory = null;
public static void setPooledConnectionFactory() {
PooledConnectionFactory pcf = new
PooledConnectionFactory(connectionUrl);
pcf.setMaxConnections(8);
pcf.setMaximumActiveSessionPerConnection(500);
factory = pcf;
}
public void sendMessage(String topicName, Event event) {
if(producer == null) {
try {
Connection connection = factory.createConnection();
connection.start();
session = connection.createSession(false,
Session.AUTO_ACKNOWLEDGE);
producer = session.createProducer(null);
} catch (JMSException exp) {
System.err.println("Error creating
connection/session/producer: " + exp);
exp.printStackTrace();
}
} else
try {
final Topic topic = session.createTopic(topicName);
final ObjectMessage message =
session.createObjectMessage(event);
producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
producer.send(topic, message);
} catch (JMSException exp) {
System.err.println("Error sending message: " + exp);
exp.printStackTrace();
}
}
private CompletableFuture<Integer> sendMessages(int eventId) {
CompletableFuture<Integer> cf = CompletableFuture.supplyAsync(
() -> {
int i;
for (i = 0; i < NUM_MESSAGES_PER_PRODUCER; i++) {
sendMessage("topic_" + eventId, new Event());
}
return i;
});
return cf;
}
public static void main(String[] args) {
setPooledConnectionFactory();
List<CompletableFuture<Integer>> returnValues = new ArrayList<>();
ProducerMT producers[] = new ProducerMT[NUM_PRODUCERS];
for (int i = 0; i < NUM_PRODUCERS; i++) {
producers[i] = new ProducerMT();
}
long startTime = System.nanoTime();
try {
for (int i = 0; i < NUM_PRODUCERS; i++) {
returnValues.add(producers[i].sendMessages(i));
}
for (int i = 0; i < NUM_PRODUCERS; i++) {
returnValues.get(i).get();
}
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
long elapsedTime = System.nanoTime() - startTime;
double seconds = (double)elapsedTime / 1000000000.0;
}
}
Re: No performance improvement using PooledConnectionFactory (AMQ 5.12.1).
Posted by Robbie Gemmell <ro...@gmail.com>.
On 23 February 2016 at 10:53, Frizz <fr...@googlemail.com> wrote:
> On Mon, Feb 22, 2016 at 5:32 PM, Timothy Bish <ta...@gmail.com> wrote:
>
>> ... and for the case of many senders you might
>> indeed see a performance drop since you are now reducing the number of
>> connections to what looks like eight, so all those producers are now
>> restricted to eight connection to shovel all their data through.
>>
>
> But without a PooledConnection, the number of connections would be = 1, no?
>
> More details: My test application creates 250 MessageProducers, each one
> running in its own thread. Connection/Session/MessageProducer are only
> created once - and never closed. Messages are sent in a loop per thread, as
> fast as possible.
>
> I don't see why a PooledConnectionFactory (connections > 1) does not result
> in an increased number of messages/second - compared to a
> non-PooledConnectionFactory (connections == 1).
The pooled factory enables re-use of underlying connections when
applications/frameworks repeatedly create+close connection objects
over time, which is inefficient because creating the underlying
connection is relatively expensive. If you create connection objects
and keep them open forever the pool can't actually help as the problem
behaviour it caters for isnt actually occurring.
I can't say for sure from your summarised description and partial
code, but it seems like you may only ever create 1 connection object
with the factory. If so, explicitly creating multiple connections and
splitting your producers across them would likely improve performance.
Re: No performance improvement using PooledConnectionFactory (AMQ 5.12.1).
Posted by Frizz <fr...@googlemail.com>.
On Mon, Feb 22, 2016 at 5:32 PM, Timothy Bish <ta...@gmail.com> wrote:
> ... and for the case of many senders you might
> indeed see a performance drop since you are now reducing the number of
> connections to what looks like eight, so all those producers are now
> restricted to eight connection to shovel all their data through.
>
But without a PooledConnection, the number of connections would be = 1, no?
More details: My test application creates 250 MessageProducers, each one
running in its own thread. Connection/Session/MessageProducer are only
created once - and never closed. Messages are sent in a loop per thread, as
fast as possible.
I don't see why a PooledConnectionFactory (connections > 1) does not result
in an increased number of messages/second - compared to a
non-PooledConnectionFactory (connections == 1).
Re: No performance improvement using PooledConnectionFactory (AMQ
5.12.1).
Posted by Timothy Bish <ta...@gmail.com>.
Without seeing the full code it is hard to comment on the impact the
pool would have in your specific use case. If your threads aren't
trying to create a connection each time then you wouldn't see a
performance improvement, and for the case of many senders you might
indeed see a performance drop since you are now reducing the number of
connections to what looks like eight, so all those producers are now
restricted to eight connection to shovel all their data through.
On 02/22/2016 11:06 AM, Frizz wrote:
> Hi there,
>
> I am experimenting with a Pooled Connection Factory - but I don't see any
> performance gain (even worse: using PooledConnectionFactory is slower than
> ActiveMQConnectionFactory).
>
> What I do: Sending a few 100.000 messages in several threads (using
> CompletableFuture). Ond run with a ActiveMQConnectionFactory, the other
> with PooledConnectionFactory:
>
> 1st Run:
> ConnectionFactory cf = new
> ActiveMQConnectionFactory("tcp://localhost:61616");
> Connection connection = cf.createConnection();
> connection.start();
> session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
> producer = session.createProducer(null);
> final Topic topic = session.createTopic(topicName);
> final ObjectMessage message = session.createObjectMessage(myObject);
> producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
> producer.send(topic, message);
>
> 2nd Run:
> PooledConnectionFactory pcf = new
> PooledConnectionFactory("tcp://localhost:61616");
> pcf.setMaxConnections(8);
> Connection connection = pcf.createConnection();
> connection.start();
> ...
>
> Am I setting up the PooledConnectionFactory the wrong way? Am I missing
> something?
>
--
Tim Bish
twitter: @tabish121
blog: http://timbish.blogspot.com/