[jira] [Created] (AMQ-7136) Improve speed of KahaDB durable subscription pending metrics recovery

["Christopher L. Shannon (JIRA)" <ji...@apache.org>]

Christopher L. Shannon created AMQ-7136:
-------------------------------------------

             Summary: Improve speed of KahaDB durable subscription pending metrics recovery
                 Key: AMQ-7136
                 URL: https://issues.apache.org/jira/browse/AMQ-7136
             Project: ActiveMQ
          Issue Type: Improvement
          Components: KahaDB
    Affects Versions: 5.15.8
            Reporter: Christopher L. Shannon
            Assignee: Christopher L. Shannon
             Fix For: 5.16.0, 5.15.9


There is a flag (off by default) in KahaDB to track pending metrics for durable subscriptions.  The recovery for the metrics requires iterating over the order index for the topic on broker start in order to compute the pending message size for each subscription.  Currently this is done inefficiently because all the messages that exist need to be iterated over for every subscription on the Topic.  The starting point of the iterating is where the subscriptions left off so if there are many subscriptions with a lot of messages to consume this can be very slow as the messages are iterated over several times.

A much better and faster way is to simply iterate over the messages one time for the destination (starting at the first message that needs to be acked across all subscriptions) and then adding the size to the counter for each subscription where that message is pending.

Testing recovery on a Topic with a couple million messages and 10 durable subscriptions has shown a pretty significant improvement in start up time of KahaDB with this change (on my machine the recovery went down from 10+ seconds to under 2 seconds)

Ultimately the fastest way to recover would be to write the pending metrics to the index just like AMQ-7132 introduced for destinations but this is at least an improvement over the current method without having to do another index change.  If we decide there is a need we can start writing the metrics to the index in the same pattern as AMQ-7132 in a future update.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)