[jira] [Commented] (AMQ-5200) Dispatch of messages within JMSXGroupID is affected by maxPageSize

["Matt Pavlovich (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AMQ-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304187#comment-17304187 ] 

Matt Pavlovich commented on AMQ-5200:
-------------------------------------

This is a runway problem that is best addressed by using broker filtering and separation of destinations. If the need for consumer filtering exceeds the ability to tune the maxPageSize (say, even if set to 128,000) then an alternate pattern to Message Groups should be used.

 If there are 1 Billion messages for JMSXGroupID='a' followed by 1 message for JMSXGroupID='b', the broker is forced to spend time scanning over the entire message store. This is the same reason why using 'priority' is not ideal in a distributed messaging architecture. It is too inefficient to be practical.

Recommend looking at [Virtual Destinations|https://activemq.apache.org/virtual-destinations] and filtering based on server-side selector, or using a Camel route to consume all messages and filter to separate destinations.

> Dispatch of messages within JMSXGroupID is affected by maxPageSize
> ------------------------------------------------------------------
>
>                 Key: AMQ-5200
>                 URL: https://issues.apache.org/jira/browse/AMQ-5200
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.8.0, 5.9.0, 5.9.1
>            Reporter: Matt Nathan
>            Assignee: Matt Pavlovich
>            Priority: Major
>
> When you have multiple consumers for a single queue and are using JMSXGroupID on each message, when one of the consumers is slow the in-memory messages held in the Queue in the broker will eventually get filled up with messages that only the slow consumer can handle. This means that no other messages are delivered as only the in-memory collection is checked for messages awaiting dispatch.
> The size of the in-memory collection is controlled by the maxPageSize parameter.
> Additionally (and the cause of much confusion in our setup) the expired message scheduled task, which runs every 30s by default, will page in more than maxPageSize messages, causing additional non-slow messages to be available for dispatch. This results in what appeared to be a 30s bursting of messages sent to all non-slow consumers.
> https://issues.apache.org/jira/browse/AMQ-2217 - the same issue but talks about message selectors instead of JMSXGroupID
> http://stackoverflow.com/q/23810807/317404 - my question on StackOverflow



--
This message was sent by Atlassian Jira
(v8.3.4#803005)