[jira] [Updated] (AMQ-8247) Consumer performance issues with many topics

["Rob Ambalu (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AMQ-8247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rob Ambalu updated AMQ-8247:
----------------------------
    Attachment: diff.txt

> Consumer performance issues with many topics
> --------------------------------------------
>
>                 Key: AMQ-8247
>                 URL: https://issues.apache.org/jira/browse/AMQ-8247
>             Project: ActiveMQ
>          Issue Type: Improvement
>            Reporter: Rob Ambalu
>            Priority: Major
>         Attachments: diff.out, diff.txt
>
>
> activemq-cpp has a performance issue when subscribing to many topics.  When I try to subscribe to market data by ticker, with 1 topic per ticker, it starts to bog down as I add more Consumers.  Digging into it, I see the issue is in how consumers are managed in ActiveMQSessionKernel.cpp.  It maintains all consumers in a linked list, and then for *every message* that is received, it does a linear O(n) scan to find the matching consumer to call back into.
> My application has close to 10k topics subscribed, every message does up to 10k scan.  This is completely unusable.
> The fix is simple, maintain consumers by HashMap not a LinkedList.  I dont have access to create a pull-request in github, but I already have the diffs and will attach it to this PR.
>  
> Some notes:
>  * adding an identical consumer will silently fail, but this is the same behavior as it is today ( duplicates will add to the LinkedList of consumers, but upon lookup only the first one will be returned... )
>  * Added slight optim to ConsumerId::getHashCode() because it was too unnecessarily inefficient to bare
>  * I tested this on my use case and my application can easily keep up without issue now



--
This message was sent by Atlassian Jira
(v8.3.4#803005)