[jira] [Created] (AMQ-8247) Consumer performance issues with many topics

["Rob Ambalu (Jira)" <ji...@apache.org>]

Rob Ambalu created AMQ-8247:
-------------------------------

             Summary: Consumer performance issues with many topics
                 Key: AMQ-8247
                 URL: https://issues.apache.org/jira/browse/AMQ-8247
             Project: ActiveMQ
          Issue Type: Improvement
            Reporter: Rob Ambalu
         Attachments: diff.out

activemq-cpp has a performance issue when subscribing to many topics.  When I try to subscribe to market data by ticker, with 1 topic per ticker, it starts to bog down as I add more Consumers.  Digging into it, I see the issue is in how consumers are managed in ActiveMQSessionKernel.cpp.  It maintains all consumers in a linked list, and then for *every message* that is received, it does a linear O(n) scan to find the matching consumer to call back into.

My application has close to 10k topics subscribed, every message does up to 10k scan.  This is completely unusable.

The fix is simple, maintain consumers by HashMap not a LinkedList.  I dont have access to create a pull-request in github, but I already have the diffs and will attach it to this PR.

 

Some notes:
 * adding an identical consumer will silently fail, but this is the same behavior as it is today ( duplicates will add to the LinkedList of consumers, but upon lookup only the first one will be returned... )
 * Added slight optim to ConsumerId::getHashCode() because it was too unnecessarily inefficient to bare
 * I tested this on my use case and my application can easily keep up without issue now



--
This message was sent by Atlassian Jira
(v8.3.4#803005)