[jira] [Resolved] (AMQ-5920) Improve performance of virtual topic fanout

["Gary Tully (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AMQ-5920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gary Tully resolved AMQ-5920.
-----------------------------
    Resolution: Fixed

option 1 and 2 completed.

> Improve performance of virtual topic fanout
> -------------------------------------------
>
>                 Key: AMQ-5920
>                 URL: https://issues.apache.org/jira/browse/AMQ-5920
>             Project: ActiveMQ
>          Issue Type: Improvement
>          Components: Broker
>    Affects Versions: 5.11.0
>            Reporter: Gary Tully
>            Assignee: Gary Tully
>              Labels: kahadb, perfomance, virtualTopic
>             Fix For: 5.12.0
>
>
> Virtual topics provide a nice alternative to durable subs. Each durable sub is modeled as a separate queue.
> There are performance implications however, because a message has to be sent to each of the (fanout) queues. For a durable subs, there is a single message in the journal and just index updates for each sub.
> To improve performance there are three ways to improve the comparison between virtual topics and durable subs.
>    # avoid the disk sync associated with enqueue
>    # do parallel enqueues to get the store batching writes
>    # introduce message references in the journal to reduce the disk io
> For 1, introducing a transaction (either client side or automatically, broker side) ensures there is a single disk sync on commit.
> For 2, using an executor to do sends in parallel allows the journal to batch as seen in AMQ-5077
> For 3, the implementation is a lot more involved; for recovery there needs to be a journal write per destination and reading the journal will require two reads because of the indirection. Tracking gc needs to be handled to ensure the referenced entry is maintained. In short this is a lot of work and will only be visible for large (> 8k) messages where the cost of a large v small journal write is noticeable. The messageId dataLocator provides an entry point for this work but considering that 1 & 2 combined can give a 3x improvement I don't think it is worth the effort (and added complexity) at the moment.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)