[jira] [Commented] (AMQ-7136) Improve speed of KahaDB durable subscription pending metrics recovery

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AMQ-7136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16743323#comment-16743323 ] 

ASF subversion and git services commented on AMQ-7136:
------------------------------------------------------

Commit c3714457f11633231a6e925f09028686db04e423 in activemq's branch refs/heads/master from Christopher L. Shannon (cshannon)
[ https://gitbox.apache.org/repos/asf?p=activemq.git;h=c371445 ]

AMQ-7136 - Improve recovery of durable subscription metrics in KahaDB

Updated metrics recovery to only have to iterate over the order index 1
time to recovery the pending metrics for the subscriptions instead of
making a pass over the index once per subscription


> Improve speed of KahaDB durable subscription pending metrics recovery
> ---------------------------------------------------------------------
>
>                 Key: AMQ-7136
>                 URL: https://issues.apache.org/jira/browse/AMQ-7136
>             Project: ActiveMQ
>          Issue Type: Improvement
>          Components: KahaDB
>    Affects Versions: 5.15.8
>            Reporter: Christopher L. Shannon
>            Assignee: Christopher L. Shannon
>            Priority: Major
>             Fix For: 5.16.0, 5.15.9
>
>
> There is a flag (off by default) in KahaDB to track pending metrics for durable subscriptions.  The recovery for the metrics requires iterating over the order index for the topic on broker start in order to compute the pending message size for each subscription.  Currently this is done inefficiently because all the messages that exist need to be iterated over for every subscription on the Topic.  The starting point of the iterating is where the subscriptions left off so if there are many subscriptions with a lot of messages to consume this can be very slow as the messages are iterated over several times.
> A much better and faster way is to simply iterate over the messages one time for the destination (starting at the first message that needs to be acked across all subscriptions) and then adding the size to the counter for each subscription where that message is pending.
> Testing recovery on a Topic with a couple million messages and 10 durable subscriptions has shown a pretty significant improvement in start up time of KahaDB with this change (on my machine the recovery went down from 10+ seconds to under 2 seconds)
> Ultimately the fastest way to recover would be to write the pending metrics to the index just like AMQ-7132 introduced for destinations but this is at least an improvement over the current method without having to do another index change.  If we decide there is a need we can start writing the metrics to the index in the same pattern as AMQ-7132 in a future update.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)