[jira] [Work logged] (ARTEMIS-3522) Implement performance tools to evaluate throughput and Response Under Load performance of Artemis

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARTEMIS-3522?focusedWorklogId=690785&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-690785 ]

ASF GitHub Bot logged work on ARTEMIS-3522:
-------------------------------------------

                Author: ASF GitHub Bot
            Created on: 06/Dec/21 06:36
            Start Date: 06/Dec/21 06:36
    Worklog Time Spent: 10m 
      Work Description: franz1981 commented on pull request #3857:
URL: https://github.com/apache/activemq-artemis/pull/3857#issuecomment-986481350


   After working on integrating this with CI I'm improving:
   
   - documentation
   - error handling (now reported as proper statistics)
   - telemetry of load generator itself


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 690785)
    Time Spent: 40m  (was: 0.5h)

> Implement performance tools to evaluate throughput and Response Under Load performance of Artemis
> -------------------------------------------------------------------------------------------------
>
>                 Key: ARTEMIS-3522
>                 URL: https://issues.apache.org/jira/browse/ARTEMIS-3522
>             Project: ActiveMQ Artemis
>          Issue Type: New Feature
>          Components: Broker, JMS
>            Reporter: Francesco Nigro
>            Assignee: Francesco Nigro
>            Priority: Major
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> There are many performance benchmarks around eg [SoftwareMill MqPerf|https://softwaremill.com/mqperf/] that could be used to test performance of Artemis in specific scenario, but none is both simple and easy to be composed with ad-hoc env setup scripts to perform a wide range of different performance tests against the broker.
> This JIRA aim to provide CLI commands that could be used as building blocks to perform:
> * all-out throughput tests
> * responsiveness under load tests (with no [coordinated omission|http://highscalability.com/blog/2015/10/5/your-load-generator-is-probably-lying-to-you-take-the-red-pi.html]) ie fixed throughput (per producer) load
> * scalability tests
> The effort of this JIRA should produce CLI commands similar to [Apache Pulsar Perf|https://pulsar.apache.org/docs/en/performance-pulsar-perf/] that could be composed to create complete performance benchmark pipelines (eg using [qDup|https://github.com/Hyperfoil/qDup] and [Horreum|https://github.com/Hyperfoil/Horreum] on a CI/CD) or used as-it-is by users to quickly check performance of the broker.
> Requirements:
> * support AMQP and Core protocol
> * cross JVMs with microseconds time measurement granularity
> *  support parsable output format 
> * suitable to perform scale tests
> The last requirement can be achieved both by using MessageListeners and async producers available on [JMS 2|https://javaee.github.io/jms-spec/pages/JMS20FinalRelease] although both [qpid JMS|https://github.com/apache/qpid-jms] and Artemis Core protocols blocks the producer caller thread ie the former on [jmsConnection::send|https://github.com/apache/qpid-jms/blob/1622de679c3c6763db54e9ac506ef2412fbc4481/qpid-jms-client/src/main/java/org/apache/qpid/jms/JmsConnection.java#L773], awaiting Netty threads to unblock it on [AmqpFixedProducer::doSend|https://github.com/apache/qpid-jms/blob/1622de679c3c6763db54e9ac506ef2412fbc4481/qpid-jms-client/src/main/java/org/apache/qpid/jms/provider/amqp/AmqpFixedProducer.java#L169], while the latter on [ClientProducerImpl::sendRegularMessage|https://github.com/apache/activemq-artemis/blob/e364961c8f035613f3ce4e3bdb3430a17efb0ffd/artemis-core-client/src/main/java/org/apache/activemq/artemis/core/client/impl/ClientProducerImpl.java#L284-L294].
> This seems odd because [JMS 2's CompletionListener|https://docs.oracle.com/javaee/7/api/javax/jms/CompletionListener.html] should save any previous send operation to ever block and the user should take care (despite being tedious and error-prone) to track the amount of in-flight messages and limit it accordly (ie [Reactive Messaging's Emitter|https://smallrye.io/smallrye-reactive-messaging/smallrye-reactive-messaging/2/emitter/emitter.html#emitter-overflow] abstract it with its overflow policies to save blocking the caller thread). 
> If JMS 2 both impl cannot be turned into non-blocking then there're just 2 options:
> # using the blocking variant: it means that scalability tests requires using machines with high core numbers 
> #  using [Reactive Messaging|https://github.com/eclipse/microprofile-reactive-messaging], but losing the ability to use local transactions (and maybe other JMS features)
> With the first option the number of producers threads can easily be much more then the available cores, causing the load generator to benchmark OS (or the runtime) ability to context switch threads instead of the broker. That's why a non-blocking approach should be preferred.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)