You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "Matthew Stratton (JIRA)" <ji...@apache.org> on 2018/10/17 14:56:00 UTC
[jira] [Updated] (AMQ-7078) possibly unintended competing consumer
behavior with virtual topic consumer queues using wildcards
[ https://issues.apache.org/jira/browse/AMQ-7078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthew Stratton updated AMQ-7078:
----------------------------------
Description:
I spoke with [~gtully] over irc about this, attached a log of that and a unit test demonstrating what I mean. Verified in 5.15.6, likely an issue as far back as you go, but hard to verify due to a related bug AMQ-6443.
If you use virtual topic consumer queues using wild wildcards and non-unique consumer queue prefixes (the part before "VirtualTopic" in the queue name) you can end up with duplicate messages in the wildcard queue while taking messages from nonwildcard queues. E.g. "Consumer.foo.VirtualTopic.>" and "Consumer.foo.VirtualTopic.bar".
"Consumer.foo.VirtualTopic.>" gets dispatched messages sent to the topic "VirtualTopic.bar" but consumers of "Consumer.foo.VirtualTopic.>" also become consumers of *any virtual topic consumer queue that matches the wildcard pattern*. So consumers of wildcard virtual topic queues can receive messages multiple times while depriving others of the messages they expect. If 5 messages are sent to "VirtualTopic.bar", 5 messages are dispatched to both "Consumer.foo.VirtualTopic.>" and "Consumer.foo.VirtualTopic.bar", this is correct so far. But the "Consumer.foo.VirtualTopic.>" consumer will end up with more than 5 messages (being detected as duplicated if the client supports it) while the "Consumer.foo.VirtualTopic.bar" ends up with less than it expects.
Possibly related to https://issues.apache.org/jira/browse/AMQ-6643, though i think the fix (and me trying to verify its working as expected) simply made the behavior obvious. I dont think the AMQ-6643 fix introduced this behavior. I dont think this is a bug, but an unintended interaction of wildcards between virtualtopic destination mapping and queue wildcards in general. The later would need to explicitly guard against the case.
There is an obvious and simple client side work around (dont use the same prefix) but at the very least I think this should be called out in documentation as it surprised me and I am quite familiar with how AMQ and virtual topics work. It makes sense in hindsight, but the behavior is very subtle if you arent paying attention and requires you to connect some dots. Most of my amq clients are outside of my direct control and managed by people with little to no amq/jms experience so the simple client side work around is less simple.
was:
I spoke with [~gtully] over irc about this, attached a log of that and a unit test demonstrating what I mean attached. Verified in 5.15.6, likely an issue as far back as you go, but hard to verify due to a related bug AMQ-6443.
If you use virtual topic consumer queues using wild wildcards and non-unique consumer queue prefixes (the part before "VirtualTopic" in the queue name) you can end up with duplicate messages in the wildcard queue while taking messages from nonwildcard queues. E.g. "Consumer.foo.VirtualTopic.>" and "Consumer.foo.VirtualTopic.bar".
"Consumer.foo.VirtualTopic.>" gets dispatched messages sent to the topic "VirtualTopic.bar" but consumers of "Consumer.foo.VirtualTopic.>" also become consumers of *any virtual topic consumer queue that matches the wildcard pattern*. So consumers of wildcard virtual topic queues can receive messages multiple times while depriving others of the messages they expect. If 5 messages are sent to "VirtualTopic.bar", 5 messages are dispatched to both "Consumer.foo.VirtualTopic.>" and "Consumer.foo.VirtualTopic.bar", this is correct so far. But the "Consumer.foo.VirtualTopic.>" consumer will end up with more than 5 messages (being detected as duplicated if the client supports it) while the "Consumer.foo.VirtualTopic.bar" ends up with less than it expects.
Possibly related to https://issues.apache.org/jira/browse/AMQ-6643, though i think the fix (and me trying to verify its working as expected) simply made the behavior obvious. I dont think the AMQ-6643 fix introduced this behavior. I dont think this is a bug, but an unintended behavior of wildcards between virtualtopic destination mapping and queue wildcards in general. The later would need to explicitly guard against the case.
There is an obvious and simple client side work around (dont use the same prefix) but at the very least I think this should be called out in documentation as it surprised me and I am quite familiar with how AMQ and virtual topics work. It makes sense in hindsight, but the behavior is very subtle if you arent paying attention and requires you to connect some dots. Most of my amq clients are outside of my direct control and managed by people with little to no amq/jms experience so the simple client side work around is less simple.
> possibly unintended competing consumer behavior with virtual topic consumer queues using wildcards
> --------------------------------------------------------------------------------------------------
>
> Key: AMQ-7078
> URL: https://issues.apache.org/jira/browse/AMQ-7078
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.15.6
> Reporter: Matthew Stratton
> Priority: Major
> Attachments: VirtualTopicWildcardMisrouting.java, gtully_irc_conversation.txt
>
>
> I spoke with [~gtully] over irc about this, attached a log of that and a unit test demonstrating what I mean. Verified in 5.15.6, likely an issue as far back as you go, but hard to verify due to a related bug AMQ-6443.
> If you use virtual topic consumer queues using wild wildcards and non-unique consumer queue prefixes (the part before "VirtualTopic" in the queue name) you can end up with duplicate messages in the wildcard queue while taking messages from nonwildcard queues. E.g. "Consumer.foo.VirtualTopic.>" and "Consumer.foo.VirtualTopic.bar".
> "Consumer.foo.VirtualTopic.>" gets dispatched messages sent to the topic "VirtualTopic.bar" but consumers of "Consumer.foo.VirtualTopic.>" also become consumers of *any virtual topic consumer queue that matches the wildcard pattern*. So consumers of wildcard virtual topic queues can receive messages multiple times while depriving others of the messages they expect. If 5 messages are sent to "VirtualTopic.bar", 5 messages are dispatched to both "Consumer.foo.VirtualTopic.>" and "Consumer.foo.VirtualTopic.bar", this is correct so far. But the "Consumer.foo.VirtualTopic.>" consumer will end up with more than 5 messages (being detected as duplicated if the client supports it) while the "Consumer.foo.VirtualTopic.bar" ends up with less than it expects.
> Possibly related to https://issues.apache.org/jira/browse/AMQ-6643, though i think the fix (and me trying to verify its working as expected) simply made the behavior obvious. I dont think the AMQ-6643 fix introduced this behavior. I dont think this is a bug, but an unintended interaction of wildcards between virtualtopic destination mapping and queue wildcards in general. The later would need to explicitly guard against the case.
> There is an obvious and simple client side work around (dont use the same prefix) but at the very least I think this should be called out in documentation as it surprised me and I am quite familiar with how AMQ and virtual topics work. It makes sense in hindsight, but the behavior is very subtle if you arent paying attention and requires you to connect some dots. Most of my amq clients are outside of my direct control and managed by people with little to no amq/jms experience so the simple client side work around is less simple.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)