You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "Francesco Nigro (Jira)" <ji...@apache.org> on 2021/10/09 14:22:00 UTC
[jira] [Updated] (ARTEMIS-3522) Implement performance tools to
evaluate throughput and Response Under Load performance of Artemis
[ https://issues.apache.org/jira/browse/ARTEMIS-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Francesco Nigro updated ARTEMIS-3522:
-------------------------------------
Description:
There are many performance benchmarks around eg [SoftwareMill MqPerf|https://softwaremill.com/mqperf/] that could be used to test performance of Artemis in specific scenario, but none is both simple and easy to be composed with ad-hoc env setup scripts to perform a wide range of different performance tests against the broker.
This JIRA aim to provide CLI commands that could be used as building blocks to perform:
* all-out throughput tests
* responsiveness under load tests (with no [coordinated omission|http://highscalability.com/blog/2015/10/5/your-load-generator-is-probably-lying-to-you-take-the-red-pi.html]) ie fixed throughput (per producer) load
* scalability tests
The effort of this JIRA should produce CLI commands similar to [Apache Pulsar Perf|https://pulsar.apache.org/docs/en/performance-pulsar-perf/] that could be composed to create complete performance benchmark pipelines (eg using [qDup|https://github.com/Hyperfoil/qDup] and [Horreum|https://github.com/Hyperfoil/Horreum] on a CI/CD) or used as-it-is by users to quickly check performance of the broker.
Requirements:
* support AMQP and Core protocol
* cross JVMs with microseconds time measurement granularity
* support parsable output format
* suitable to perform scale tests
The last requirement can be achieved both by using MessageListeners and async producers with [JMS 2!https://javaee.github.io/jms-spec/pages/JMS20FinalRelease] although it seems that both [qpid JMS!https://github.com/apache/qpid-jms] and Artemis Core protocols blocks the producer for different reasons ie the former on [jmsConnection::send|https://github.com/apache/qpid-jms/blob/1622de679c3c6763db54e9ac506ef2412fbc4481/qpid-jms-client/src/main/java/org/apache/qpid/jms/JmsConnection.java#L773], awaiting Netty threads to unblock it on [AmqpFixedProducer::doSend|https://github.com/apache/qpid-jms/blob/1622de679c3c6763db54e9ac506ef2412fbc4481/qpid-jms-client/src/main/java/org/apache/qpid/jms/provider/amqp/AmqpFixedProducer.java#L169], while the latter on [ClientProducerImpl::sendRegularMessage|https://github.com/apache/activemq-artemis/blob/e364961c8f035613f3ce4e3bdb3430a17efb0ffd/artemis-core-client/src/main/java/org/apache/activemq/artemis/core/client/impl/ClientProducerImpl.java#L284-L294].
This seems odd because [JMS 2's CompletionListener|https://docs.oracle.com/javaee/7/api/javax/jms/CompletionListener.html] should save any previous send operation to ever block and it seems duty of the user (despite being tedious and error-prone) to track the amount of in-flight messages and limit it accordly (that's what [Reactive Messaging's Emitter!https://smallrye.io/smallrye-reactive-messaging/smallrye-reactive-messaging/2/emitter/emitter.html#emitter-overflow] does for a similar use case ie be reactive but still saving a producer thread to ever block.
If JMS 2 both impl cannot be turned into non-blocking then there're just 2 options:
# using the blocking variant: it means that scalability tests requires using machines with high core numbers
# using https://github.com/eclipse/microprofile-reactive-messaging losing the ability to use local transactions (and maybe other JMS features)
The first option seems fine, if not that due to it, if the number of producers would be > available cores, the load generator would benchmark OS ability to context switch threads instead of just producing load against the broker, to not mention that the simplicity of this CLI commands to be used by "simple" users (and devs too!) would be (further) limited by the spec of the machine.
was:
There are many performance benchmarks around eg [SoftwareMill MqPerf|https://softwaremill.com/mqperf/] that could be used to test performance of Artemis in specific scenario, but none is both simple and easy to be composed with ad-hoc env setup scripts to perform a wide range of different performance tests against the broker.
This JIRA aim to provide CLI commands that could be used as building blocks to perform:
* all-out throughput tests
* responsiveness under load tests (with no [coordinated omission|http://highscalability.com/blog/2015/10/5/your-load-generator-is-probably-lying-to-you-take-the-red-pi.html]) ie fixed throughput (per producer) load
* scalability tests
The effort of this JIRA should produce CLI commands similar to [Apache Pulsar Perf|https://pulsar.apache.org/docs/en/performance-pulsar-perf/] that could be composed to create complete performance benchmark pipelines (eg using [qDup|https://github.com/Hyperfoil/qDup] and [Horreum|https://github.com/Hyperfoil/Horreum] on a CI/CD) or used as-it-is by users to quickly check performance of the broker.
Requirements:
* support AMQP and Core protocol
* cross JVMs with microseconds time measurement granularity
* support parsable output format
* suitable to perform scale tests
The last requirement can be achieved both by using MessageListeners and async producers with [JMS 2!https://javaee.github.io/jms-spec/pages/JMS20FinalRelease] although it seems that both [qpid JMS!https://github.com/apache/qpid-jms] and Artemis Core protocols blocks the producer for different reasons ie the former on [jmsConnection::send|https://github.com/apache/qpid-jms/blob/1622de679c3c6763db54e9ac506ef2412fbc4481/qpid-jms-client/src/main/java/org/apache/qpid/jms/JmsConnection.java#L773], awaiting Netty threads to unblock it on [AmqpFixedProducer::doSend|https://github.com/apache/qpid-jms/blob/1622de679c3c6763db54e9ac506ef2412fbc4481/qpid-jms-client/src/main/java/org/apache/qpid/jms/provider/amqp/AmqpFixedProducer.java#L169], while the latter on [ClientProducerImpl::sendRegularMessage!https://github.com/apache/activemq-artemis/blob/e364961c8f035613f3ce4e3bdb3430a17efb0ffd/artemis-core-client/src/main/java/org/apache/activemq/artemis/core/client/impl/ClientProducerImpl.java#L284-L294].
This seems odd because [JMS 2's CompletionListener|https://docs.oracle.com/javaee/7/api/javax/jms/CompletionListener.html] should save any previous send operation to ever block and it seems duty of the user (despite being tedious and error-prone) to track the amount of in-flight messages and limit it accordly (that's what [Reactive Messaging's Emitter!https://smallrye.io/smallrye-reactive-messaging/smallrye-reactive-messaging/2/emitter/emitter.html#emitter-overflow] does for a similar use case ie be reactive but still saving a producer thread to ever block.
If JMS 2 both impl cannot be turned into non-blocking then there're just 2 options:
# using the blocking variant: it means that scalability tests requires using machines with high core numbers
# using https://github.com/eclipse/microprofile-reactive-messaging losing the ability to use local transactions (and maybe other JMS features)
The first option seems fine, if not that due to it, if the number of producers would be > available cores, the load generator would benchmark OS ability to context switch threads instead of just producing load against the broker, to not mention that the simplicity of this CLI commands to be used by "simple" users (and devs too!) would be (further) limited by the spec of the machine.
> Implement performance tools to evaluate throughput and Response Under Load performance of Artemis
> -------------------------------------------------------------------------------------------------
>
> Key: ARTEMIS-3522
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3522
> Project: ActiveMQ Artemis
> Issue Type: New Feature
> Components: Broker, JMS
> Reporter: Francesco Nigro
> Assignee: Francesco Nigro
> Priority: Major
>
> There are many performance benchmarks around eg [SoftwareMill MqPerf|https://softwaremill.com/mqperf/] that could be used to test performance of Artemis in specific scenario, but none is both simple and easy to be composed with ad-hoc env setup scripts to perform a wide range of different performance tests against the broker.
> This JIRA aim to provide CLI commands that could be used as building blocks to perform:
> * all-out throughput tests
> * responsiveness under load tests (with no [coordinated omission|http://highscalability.com/blog/2015/10/5/your-load-generator-is-probably-lying-to-you-take-the-red-pi.html]) ie fixed throughput (per producer) load
> * scalability tests
> The effort of this JIRA should produce CLI commands similar to [Apache Pulsar Perf|https://pulsar.apache.org/docs/en/performance-pulsar-perf/] that could be composed to create complete performance benchmark pipelines (eg using [qDup|https://github.com/Hyperfoil/qDup] and [Horreum|https://github.com/Hyperfoil/Horreum] on a CI/CD) or used as-it-is by users to quickly check performance of the broker.
> Requirements:
> * support AMQP and Core protocol
> * cross JVMs with microseconds time measurement granularity
> * support parsable output format
> * suitable to perform scale tests
> The last requirement can be achieved both by using MessageListeners and async producers with [JMS 2!https://javaee.github.io/jms-spec/pages/JMS20FinalRelease] although it seems that both [qpid JMS!https://github.com/apache/qpid-jms] and Artemis Core protocols blocks the producer for different reasons ie the former on [jmsConnection::send|https://github.com/apache/qpid-jms/blob/1622de679c3c6763db54e9ac506ef2412fbc4481/qpid-jms-client/src/main/java/org/apache/qpid/jms/JmsConnection.java#L773], awaiting Netty threads to unblock it on [AmqpFixedProducer::doSend|https://github.com/apache/qpid-jms/blob/1622de679c3c6763db54e9ac506ef2412fbc4481/qpid-jms-client/src/main/java/org/apache/qpid/jms/provider/amqp/AmqpFixedProducer.java#L169], while the latter on [ClientProducerImpl::sendRegularMessage|https://github.com/apache/activemq-artemis/blob/e364961c8f035613f3ce4e3bdb3430a17efb0ffd/artemis-core-client/src/main/java/org/apache/activemq/artemis/core/client/impl/ClientProducerImpl.java#L284-L294].
> This seems odd because [JMS 2's CompletionListener|https://docs.oracle.com/javaee/7/api/javax/jms/CompletionListener.html] should save any previous send operation to ever block and it seems duty of the user (despite being tedious and error-prone) to track the amount of in-flight messages and limit it accordly (that's what [Reactive Messaging's Emitter!https://smallrye.io/smallrye-reactive-messaging/smallrye-reactive-messaging/2/emitter/emitter.html#emitter-overflow] does for a similar use case ie be reactive but still saving a producer thread to ever block.
> If JMS 2 both impl cannot be turned into non-blocking then there're just 2 options:
> # using the blocking variant: it means that scalability tests requires using machines with high core numbers
> # using https://github.com/eclipse/microprofile-reactive-messaging losing the ability to use local transactions (and maybe other JMS features)
> The first option seems fine, if not that due to it, if the number of producers would be > available cores, the load generator would benchmark OS ability to context switch threads instead of just producing load against the broker, to not mention that the simplicity of this CLI commands to be used by "simple" users (and devs too!) would be (further) limited by the spec of the machine.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)