[jira] [Closed] (ARTEMIS-2072) Significant performance degradation introduced after 2.4 when using auto create queues

["clebert suconic (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARTEMIS-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

clebert suconic closed ARTEMIS-2072.
------------------------------------

> Significant performance degradation introduced after 2.4 when using auto create queues
> --------------------------------------------------------------------------------------
>
>                 Key: ARTEMIS-2072
>                 URL: https://issues.apache.org/jira/browse/ARTEMIS-2072
>             Project: ActiveMQ Artemis
>          Issue Type: Bug
>          Components: AMQP
>    Affects Versions: 2.5.0, 2.6.0, 2.6.1, 2.6.2
>            Reporter: Marcel Meulemans
>            Assignee: Justin Bertram
>            Priority: Major
>             Fix For: 2.7.0, 2.6.4
>
>         Attachments: implement-caching-of-binding-queries.patch
>
>
> We use Artemis with an AMQP acceptor, auto create queues enabled, 10000+ queues and are producing/consuming around 500 message per second. In this setup commit [f3ace6afd726dc8e3c1c58c76e3fad3d5cfa357d|https://github.com/apache/activemq-artemis/commit/f3ace6afd726dc8e3c1c58c76e3fad3d5cfa357d] introduced a major performance degradation (on a 2 core GCE machine the cpu usage went from around 40% to above 200%, i.e. not enough cpu to handle the load). After investigation we found that the main cause of the performance degradation is caused by calling {{serverSession.executeBindingQuery()}} twice for every message being put into a queue. This function is very expensive because it does a matching (optionally wildcard matching) on the address of a message with *every* existing binding (in our case this results in 20000+ regex matches for every message being put into a queue):
> [https://github.com/apache/activemq-artemis/blob/master/artemis-server/src/main/java/org/apache/activemq/artemis/core/postoffice/impl/SimpleAddressManager.java#L129]
> It seems that commit 1d1d6c8b4686f869df0ca5fc09c20128f8481cff tries to resolve the issue a bit by adding a query cache, but we never hit this cache (maybe because it caches only the last query? Anyway in our situation the code always ends up performing the query in the postoffice addressmanager). 
> We created a [^implement-caching-of-binding-queries.patch] (attached) that implements caching of the bindings query results in the address manager implemenation (in SimpleAddressManager so it works also for the WildcardAddressManager) and this reduces the cpu load to *below* the level of before the commit that introduced the problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)