You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Jeremy Gooch <go...@yahoo.co.uk.INVALID> on 2017/04/10 09:06:20 UTC
One subscription, multiple consumers.
Hello,
Using Qpid Proton (mainly Python), I know you can have multiple consumers against one queue and they operate in a race condition.
But can the same happen with a subscription? In other words, how do I configure multiple consumers against one subscription so that messages are distributed amongst them? This has come up as a requirement to support an auto-scaling client, who wants to make multiple connections to a durable subscription but not receive duplicate messages.
Thanks,
J.
Jeremy Gooch http://goochgooch.co.uk
Re: One subscription, multiple consumers.
Posted by Timothy Bish <ta...@gmail.com>.
On 04/11/2017 12:42 PM, Robbie Gemmell wrote:
> For the most part topic subscriptions only ever had a single
> subscriber asociated with them, until at some point someone thought it
> would be useful to support more. Some systems can do that, others
> cant, others still have specific mechanisms to achieve the same
> effect.
>
> This is all completely independent of the concepts of sessions and
> connections. You could have a single session with lots of subscribers
> on it, or lots of sessions with a single subscriber, but that doesnt
> particularly influence whether they can share a single subscription or
> not, the messaging system itself does. Various messaging systems now
> support doing that in their own way, hence my original question what
> you were using. In ActiveMQ 5.x its abilities are closely related to
> JMS 1.1, and I beleive its only method of achieving the shared
> subscribers effect is Virtual Destinations, effectively turning each
> subscriber for a given subscription into a specific queue consumer as
> you note.
>
> We recently mapped out a mechanism to support shared subscriptions
> over AMQP 1.0 such that the JMS 2.0 shared subscriptions functionlity
> could be supported for the Qpid JMS client. I'm only aware of 2
> brokers that already implement support for the precise mechanism it
> uses, ActiveMQ Artemis (I think 2.x is needed) and the upcoming Qpid
> Broker-J 7.0.0. The Qpid C++ broker also supports a variant of shared
> susbcriptions over AMQP 1.0, but does not currently support the
> precise mechanism required by the JMS 2.0 client. You could similarly
> use those mechanisms from other AMQP 1.0 clients, or alternatively
> there may be other broker-specific mechanisms for achieving the same
> effect, but as far as ActiveMQ 5.x is concerned, I beleive your only
> option is Virtual Topics.
That is correct, you'd need to use Virtual Topic subscriptions for this
scenario.
>
> Robbie
>
> On 11 April 2017 at 16:57, Jeremy Gooch <go...@yahoo.co.uk.invalid> wrote:
>> If I understand this correctly, the limitation with regard to only a single consumer on one topic is in order to maintain compliance with the JMS specification. ActiveMQ's Virtual Destinations give a workaround for this where a producer will publish to a topic as normal whilst allowing a consumer to connect as if to a queue, thereby enabling the consumer to operate multiple connections to increase its message consumption rate.
>>
>> The AMQP model has the concept of sessions within a connection. Is it possible therefore to have multiple sessions within a connection against a single subscription without using Virtual Destinations? We could then use a client with some sort of concurrency feature to operate multiple sessions and increase message consumption.
>> J.
>>
>> Jeremy Gooch http://goochgooch.co.uk
>>
>> From: Robbie Gemmell <ro...@gmail.com>
>> To: "users@qpid.apache.org" <us...@qpid.apache.org>
>> Sent: Tuesday, 11 April 2017, 14:49
>> Subject: Re: One subscription, multiple consumers.
>>
>> With ActiveMQ 5 I think your only option would be Virtual Topics,
>> http://activemq.apache.org/virtual-destinations.html
>>
>> On 11 April 2017 at 13:17, Jeremy Gooch <go...@yahoo.co.uk.invalid> wrote:
>>> It's...
>>> - ActiveMQ 5.14- Python 3 (running in Docker)- Latest version of Qpid Proton (installed via pip3)
>>> J.
>>> Jeremy Gooch http://goochgooch.co.uk
>>>
>>> From: Robbie Gemmell <ro...@gmail.com>
>>> To: "users@qpid.apache.org" <us...@qpid.apache.org>
>>> Sent: Monday, 10 April 2017, 15:55
>>> Subject: Re: One subscription, multiple consumers.
>>>
>>> Answers here are likely to be quite dependent on which server, and
>>> which version of it, that you are actually using.
>>>
>>> On 10 April 2017 at 10:06, Jeremy Gooch <go...@yahoo.co.uk.invalid> wrote:
>>>> Hello,
>>>> Using Qpid Proton (mainly Python), I know you can have multiple consumers against one queue and they operate in a race condition.
>>>> But can the same happen with a subscription? In other words, how do I configure multiple consumers against one subscription so that messages are distributed amongst them? This has come up as a requirement to support an auto-scaling client, who wants to make multiple connections to a durable subscription but not receive duplicate messages.
>>>> Thanks,
>>>> J.
>>>>
>>>> Jeremy Gooch http://goochgooch.co.uk
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>>> For additional commands, e-mail: users-help@qpid.apache.org
>>>
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
--
Tim Bish
twitter: @tabish121
blog: http://timbish.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: One subscription, multiple consumers.
Posted by Robbie Gemmell <ro...@gmail.com>.
For the most part topic subscriptions only ever had a single
subscriber asociated with them, until at some point someone thought it
would be useful to support more. Some systems can do that, others
cant, others still have specific mechanisms to achieve the same
effect.
This is all completely independent of the concepts of sessions and
connections. You could have a single session with lots of subscribers
on it, or lots of sessions with a single subscriber, but that doesnt
particularly influence whether they can share a single subscription or
not, the messaging system itself does. Various messaging systems now
support doing that in their own way, hence my original question what
you were using. In ActiveMQ 5.x its abilities are closely related to
JMS 1.1, and I beleive its only method of achieving the shared
subscribers effect is Virtual Destinations, effectively turning each
subscriber for a given subscription into a specific queue consumer as
you note.
We recently mapped out a mechanism to support shared subscriptions
over AMQP 1.0 such that the JMS 2.0 shared subscriptions functionlity
could be supported for the Qpid JMS client. I'm only aware of 2
brokers that already implement support for the precise mechanism it
uses, ActiveMQ Artemis (I think 2.x is needed) and the upcoming Qpid
Broker-J 7.0.0. The Qpid C++ broker also supports a variant of shared
susbcriptions over AMQP 1.0, but does not currently support the
precise mechanism required by the JMS 2.0 client. You could similarly
use those mechanisms from other AMQP 1.0 clients, or alternatively
there may be other broker-specific mechanisms for achieving the same
effect, but as far as ActiveMQ 5.x is concerned, I beleive your only
option is Virtual Topics.
Robbie
On 11 April 2017 at 16:57, Jeremy Gooch <go...@yahoo.co.uk.invalid> wrote:
> If I understand this correctly, the limitation with regard to only a single consumer on one topic is in order to maintain compliance with the JMS specification. ActiveMQ's Virtual Destinations give a workaround for this where a producer will publish to a topic as normal whilst allowing a consumer to connect as if to a queue, thereby enabling the consumer to operate multiple connections to increase its message consumption rate.
>
> The AMQP model has the concept of sessions within a connection. Is it possible therefore to have multiple sessions within a connection against a single subscription without using Virtual Destinations? We could then use a client with some sort of concurrency feature to operate multiple sessions and increase message consumption.
> J.
>
> Jeremy Gooch http://goochgooch.co.uk
>
> From: Robbie Gemmell <ro...@gmail.com>
> To: "users@qpid.apache.org" <us...@qpid.apache.org>
> Sent: Tuesday, 11 April 2017, 14:49
> Subject: Re: One subscription, multiple consumers.
>
> With ActiveMQ 5 I think your only option would be Virtual Topics,
> http://activemq.apache.org/virtual-destinations.html
>
> On 11 April 2017 at 13:17, Jeremy Gooch <go...@yahoo.co.uk.invalid> wrote:
>> It's...
>> - ActiveMQ 5.14- Python 3 (running in Docker)- Latest version of Qpid Proton (installed via pip3)
>> J.
>> Jeremy Gooch http://goochgooch.co.uk
>>
>> From: Robbie Gemmell <ro...@gmail.com>
>> To: "users@qpid.apache.org" <us...@qpid.apache.org>
>> Sent: Monday, 10 April 2017, 15:55
>> Subject: Re: One subscription, multiple consumers.
>>
>> Answers here are likely to be quite dependent on which server, and
>> which version of it, that you are actually using.
>>
>> On 10 April 2017 at 10:06, Jeremy Gooch <go...@yahoo.co.uk.invalid> wrote:
>>> Hello,
>>> Using Qpid Proton (mainly Python), I know you can have multiple consumers against one queue and they operate in a race condition.
>>> But can the same happen with a subscription? In other words, how do I configure multiple consumers against one subscription so that messages are distributed amongst them? This has come up as a requirement to support an auto-scaling client, who wants to make multiple connections to a durable subscription but not receive duplicate messages.
>>> Thanks,
>>> J.
>>>
>>> Jeremy Gooch http://goochgooch.co.uk
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: One subscription, multiple consumers.
Posted by Jeremy Gooch <go...@yahoo.co.uk.INVALID>.
If I understand this correctly, the limitation with regard to only a single consumer on one topic is in order to maintain compliance with the JMS specification. ActiveMQ's Virtual Destinations give a workaround for this where a producer will publish to a topic as normal whilst allowing a consumer to connect as if to a queue, thereby enabling the consumer to operate multiple connections to increase its message consumption rate.
The AMQP model has the concept of sessions within a connection. Is it possible therefore to have multiple sessions within a connection against a single subscription without using Virtual Destinations? We could then use a client with some sort of concurrency feature to operate multiple sessions and increase message consumption.
J.
Jeremy Gooch http://goochgooch.co.uk
From: Robbie Gemmell <ro...@gmail.com>
To: "users@qpid.apache.org" <us...@qpid.apache.org>
Sent: Tuesday, 11 April 2017, 14:49
Subject: Re: One subscription, multiple consumers.
With ActiveMQ 5 I think your only option would be Virtual Topics,
http://activemq.apache.org/virtual-destinations.html
On 11 April 2017 at 13:17, Jeremy Gooch <go...@yahoo.co.uk.invalid> wrote:
> It's...
> - ActiveMQ 5.14- Python 3 (running in Docker)- Latest version of Qpid Proton (installed via pip3)
> J.
> Jeremy Gooch http://goochgooch.co.uk
>
> From: Robbie Gemmell <ro...@gmail.com>
> To: "users@qpid.apache.org" <us...@qpid.apache.org>
> Sent: Monday, 10 April 2017, 15:55
> Subject: Re: One subscription, multiple consumers.
>
> Answers here are likely to be quite dependent on which server, and
> which version of it, that you are actually using.
>
> On 10 April 2017 at 10:06, Jeremy Gooch <go...@yahoo.co.uk.invalid> wrote:
>> Hello,
>> Using Qpid Proton (mainly Python), I know you can have multiple consumers against one queue and they operate in a race condition.
>> But can the same happen with a subscription? In other words, how do I configure multiple consumers against one subscription so that messages are distributed amongst them? This has come up as a requirement to support an auto-scaling client, who wants to make multiple connections to a durable subscription but not receive duplicate messages.
>> Thanks,
>> J.
>>
>> Jeremy Gooch http://goochgooch.co.uk
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: One subscription, multiple consumers.
Posted by Robbie Gemmell <ro...@gmail.com>.
With ActiveMQ 5 I think your only option would be Virtual Topics,
http://activemq.apache.org/virtual-destinations.html
On 11 April 2017 at 13:17, Jeremy Gooch <go...@yahoo.co.uk.invalid> wrote:
> It's...
> - ActiveMQ 5.14- Python 3 (running in Docker)- Latest version of Qpid Proton (installed via pip3)
> J.
> Jeremy Gooch http://goochgooch.co.uk
>
> From: Robbie Gemmell <ro...@gmail.com>
> To: "users@qpid.apache.org" <us...@qpid.apache.org>
> Sent: Monday, 10 April 2017, 15:55
> Subject: Re: One subscription, multiple consumers.
>
> Answers here are likely to be quite dependent on which server, and
> which version of it, that you are actually using.
>
> On 10 April 2017 at 10:06, Jeremy Gooch <go...@yahoo.co.uk.invalid> wrote:
>> Hello,
>> Using Qpid Proton (mainly Python), I know you can have multiple consumers against one queue and they operate in a race condition.
>> But can the same happen with a subscription? In other words, how do I configure multiple consumers against one subscription so that messages are distributed amongst them? This has come up as a requirement to support an auto-scaling client, who wants to make multiple connections to a durable subscription but not receive duplicate messages.
>> Thanks,
>> J.
>>
>> Jeremy Gooch http://goochgooch.co.uk
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: One subscription, multiple consumers.
Posted by Jeremy Gooch <go...@yahoo.co.uk.INVALID>.
It's...
- ActiveMQ 5.14- Python 3 (running in Docker)- Latest version of Qpid Proton (installed via pip3)
J.
Jeremy Gooch http://goochgooch.co.uk
From: Robbie Gemmell <ro...@gmail.com>
To: "users@qpid.apache.org" <us...@qpid.apache.org>
Sent: Monday, 10 April 2017, 15:55
Subject: Re: One subscription, multiple consumers.
Answers here are likely to be quite dependent on which server, and
which version of it, that you are actually using.
On 10 April 2017 at 10:06, Jeremy Gooch <go...@yahoo.co.uk.invalid> wrote:
> Hello,
> Using Qpid Proton (mainly Python), I know you can have multiple consumers against one queue and they operate in a race condition.
> But can the same happen with a subscription? In other words, how do I configure multiple consumers against one subscription so that messages are distributed amongst them? This has come up as a requirement to support an auto-scaling client, who wants to make multiple connections to a durable subscription but not receive duplicate messages.
> Thanks,
> J.
>
> Jeremy Gooch http://goochgooch.co.uk
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: One subscription, multiple consumers.
Posted by Robbie Gemmell <ro...@gmail.com>.
Answers here are likely to be quite dependent on which server, and
which version of it, that you are actually using.
On 10 April 2017 at 10:06, Jeremy Gooch <go...@yahoo.co.uk.invalid> wrote:
> Hello,
> Using Qpid Proton (mainly Python), I know you can have multiple consumers against one queue and they operate in a race condition.
> But can the same happen with a subscription? In other words, how do I configure multiple consumers against one subscription so that messages are distributed amongst them? This has come up as a requirement to support an auto-scaling client, who wants to make multiple connections to a durable subscription but not receive duplicate messages.
> Thanks,
> J.
>
> Jeremy Gooch http://goochgooch.co.uk
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org