You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Gordon Sim <gs...@redhat.com> on 2013/10/04 13:23:25 UTC
AMQP 1.0 and creating nodes on the fly
A common convenience is to allow queues or topics to be created on
demand, i.e. having them come into existence when a link is attached.
This is useful where the messaging infrastructure is supposed to be a
'hidden' part of the system, and manual configuration is not desirable.
The AMQP 1.0 specification has a 'dynamic' field defined for sources and
targets used on attach, which if set causes the broker to create the
node on demand. This works well for the intended case of 'temporary
reply queues', but not where the application wants to choose the name[1].
At present the qpid::messaging library over 1.0 treats a node name of
'#' as a request for a broker assigned temporary queue. If the name is
not '#' and the create option is specified, it will request a qpidd
specific capability, create-on-demand[2].
With ActiveMQ and ApolloMQ, when attaching to a node it is always
created on demand. The configuration for it is determined by matching
the name against a set of preconfigured rules (this part is similar to
RabbitMQ 'policies').
It would be great to have some convention that worked with different 1.0
brokers.
Ultimately if a common management protocol emerges, it could address
this as well as many other things of course. However I think for this
specific and limited case there is benefit in seeking consensus on
something simple in the short term.
Though I see some value in not creating nodes on demand in some
cases[3], I like the ApolloMQ approach for those cases where the
convenience of doing so outweighs disadvantages.
I propose adding some mechanism for configuring qpidd to support the
same behaviour for specific portions of the node namespace. This would
be something like the RabbitMQ policies, with the ability to define a
create-on-demand property for a given policy. Nodes matching the pttern
for that policy would then be created on demand and would have any other
settings in the policy applied to them.
One other aspect of this that is important is how to determine if the
node should be a 'queue' or a 'topic', as the two most common node
types. One suggestion would be to have brokers recognise two specific
capabilities for these types. The 'queue' capability implies the ability
to distribute messages between consumers in some fashion, and to store
messages when no consumers are available. The 'topic' capability would
distribute all messages to all subscribers (i.e. non-competing
consumers) and would drop messages if there were no subscribers.
Any thoughts, comments, complaints, alternative suggestions etc? I'd
like to get agreement on something that is simple but useful for users
and not difficult for different brokers to implement to improve the
chances of a de-facto standard emerging.
--Gordon
[1] The specification explicitly prohibits the name of the
source/target being sent if the dynamic flag is set. The node name is
specified as being assigned by the broker. There is also a field for
passing desired properties of the node and/or communicating back the
actual properties. Sending the dynamic node properties if the dynamic
flag is not set is also prohibited.
[2] However if particular node properties have been requested it will
also send those, which strictly speaking is in violation of the
specification. In any case, having the configuration specified by the
first link to attach is error prone. The qpid::messaging implementation
does have the ability to assert that the actual properties match the
requested ones to highlight any errors early, but again that requires
sending the node properties even when the dynamic flag is not set.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: AMQP 1.0 and creating nodes on the fly
Posted by Gordon Sim <gs...@redhat.com>.
On 10/23/2013 09:03 PM, Gordon Sim wrote:
> On 10/09/2013 03:14 PM, Gordon Sim wrote:
>> I propose to add a mechanism to the 1.0 support in the Qpid c++
>> broker to allow nodes to be created on demand through broker
>> configuration alone. Though the details of this would be different from
>> ActiveMQ and ApolloMQ, the same behaviour would then be available if
>> configured, which would aid in moving applications between these.
>
> For anyone interested, I've put an early patch up for review and comment
> that implements this for qpidd: https://reviews.apache.org/r/14882/
I've now updated that patch based on initial feedback. Non-durable
queues created through matching a given policy will by default be
automatically deleted if both unused (i.e. no senders or receivers
attached) *and* if the queue is empty. Since the queue will be recreated
if it is needed later, this has no impact on application.
I'm going to be exploring whether I can do something similar for
topics/exchanges.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: AMQP 1.0 and creating nodes on the fly
Posted by Gordon Sim <gs...@redhat.com>.
On 10/09/2013 03:14 PM, Gordon Sim wrote:
> I propose to add a mechanism to the 1.0 support in the Qpid c++
> broker to allow nodes to be created on demand through broker
> configuration alone. Though the details of this would be different from
> ActiveMQ and ApolloMQ, the same behaviour would then be available if
> configured, which would aid in moving applications between these.
For anyone interested, I've put an early patch up for review and comment
that implements this for qpidd: https://reviews.apache.org/r/14882/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: AMQP 1.0 and creating nodes on the fly
Posted by Gordon Sim <gs...@redhat.com>.
On 10/09/2013 01:25 PM, Rob Godfrey wrote:
> OK - I think I'm confused as to what you are proposing then. I thought you
> were looking for a way to standardise without client libraries having to
> send properties / capabilities.
>
> Perhaps it would be helpful to write out what you are thinking in some sort
> of pseudo code or something? There are obviously a lot of different places
> that capabilities exist and a lot of different names. I can now no longer
> tell if we are agreeing, disagreeing or talking about completely different
> scenarios :-)
My apologies! I think I included too many different points in my
original email with the intention of adding some context.
I'm really making two separate proposals:
(1) I propose to add a mechanism to the 1.0 support in the Qpid c++
broker to allow nodes to be created on demand through broker
configuration alone. Though the details of this would be different from
ActiveMQ and ApolloMQ, the same behaviour would then be available if
configured, which would aid in moving applications between these. As
part of this proposal, I would also then see if I can persuade other 1.0
broker implementers to offer something similar (with the understanding
that the details of configuration will all be different[1]).
(2) The second proposal is to try and get agreement on two capabilities
that describe the basic functionality of 'queues' and 'topics' as
commonly understood (e.g. as described by JMS).
A 'queue' capability implies that message will (by default) be
distributed between competing consumers (i.e. it supports a distribution
mode of 'move' by default, though it may also support 'copy' and behaves
as a 'distribution node' as described by section 3.3 of the spec) and
will be stored pending the availability of a consumer to deliver it to.
A 'topic' capability will distribute messages to all interested
subscribers, i.e. non-competing consumers, i.e. a distribution mode of
'copy' and unless it offers some caching/reply capability (which I'm not
concerned with here), it would drop messages for which there were no
interested subscribers.
These capabilities could then be requested and/or advertised in the
source/target of an attach to indicate the sort of basic behaviour
expected or provided by a node.
--Gordon
[1] E.g. one broker might have a simple broker wide toggle as to whether
nodes are created on demand or not, one might provide a way to define
policies that match by name and determine whether they can be created on
demand or not, another might create on demand by default unless
restricted through permissions etc.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: AMQP 1.0 and creating nodes on the fly
Posted by Rob Godfrey <ro...@gmail.com>.
On 9 October 2013 13:40, Gordon Sim <gs...@redhat.com> wrote:
> On 10/09/2013 12:10 PM, Rob Godfrey wrote:
>
>> On 9 October 2013 11:24, Gordon Sim <gs...@redhat.com> wrote:
>>
>>> On 10/08/2013 03:42 PM, Rob Godfrey wrote:
>>>
>>>> The second case of defining some namespace pattern within the broker
>>>> wherein any unrecognized names will lead to node creation seems like a
>>>> broker specific feature with no need (or requirement?) for
>>>> standardisation.
>>>> Am I missing something here?
>>>>
>>>>
>>> I personally much prefer the second approach. Its simpler and more
>>> flexible for broker implementers to do in different ways (according to
>>> their existing configuration mechanisms and details of the way their code
>>> already works). It also takes the 'decision' out of the client, which
>>> seems
>>> preferable from a 'philosophical' pint of view, and means that client
>>> libraries need not be affected.
>>>
>>>
>>> But whether an attempt to send to/receive from an address which does not
>> exist should result in either a failure or the creation of a new node is
>> not just a broker/service concern - I'd actually think that it was an
>> application (and therefore client) concern.
>>
>
> I agree I think it is an application concern, but I see it as a
> deployment/configuration concern, rather than a development concern. Being
> able to control that entirely through broker configuration is valuable.
>
>
> I'm certainly not suggesting any official 'standardisation'. I'm merely
>>> hoping to find a simple, practical, 'bottom-up' consensus that would make
>>> switching between AMQP brokers easier for users.
>>>
>>> If more brokers were to support some way of having nodes created on
>>> demand
>>> purely based on broker side configuration (the details of which would be
>>> broker specific), that would in my view be useful to anyone looking to
>>> try
>>> out applications against different brokers.
>>>
>>> As such, I'm keen to implement something like that in qpidd and would
>>> also
>>> be keen to start talking to some of the other broker maintainers to see
>>> if
>>> I could persuade others to do the same (if they have not already).
>>>
>>>
>>> So while I don't object to brokers doing this, I'm really not keen on
>> trying to use names to identify the properties of nodes to be created
>>
>
> If you mean the address of the node, then I agree with you. I'm not keen
> on that and mentioned it merely as what is currently relied on in the
> absence of anything else.
>
> Hence my preference for capabilities, which is a defined extension point
> that clearly (in my view) fits this situation quite well and can be used
> whether or not the node is to be dynamically created and orthogonal to any
> naming scheme for the node itself.
>
>
OK - I think I'm confused as to what you are proposing then. I thought you
were looking for a way to standardise without client libraries having to
send properties / capabilities.
Perhaps it would be helpful to write out what you are thinking in some sort
of pseudo code or something? There are obviously a lot of different places
that capabilities exist and a lot of different names. I can now no longer
tell if we are agreeing, disagreeing or talking about completely different
scenarios :-)
-- Rob
> [...]
>
> Yes, we need to define what node-properties terms like "queue" and
>>>> "topic"
>>>> actually mean.
>>>>
>>>>
>>> The two sentences above attempt to do that in a minimal way. To me they
>>> capture the essential capabilities that people expect when thinking of a
>>> queue or topic. I would of course be eager to here alternative
>>> suggestions,
>>> whether it be entirely different mechanisms, or just different capability
>>> names or descriptions. That's the purpose of the original email really.
>>>
>>>
>>> So as above I'm pretty negative on just using names
>>
>
> Again, I'm *not* proposing using *node* names to indicate the type of
> node, I'm proposing using named *capabilities*.
>
> The distribution mode gets close, but is not quite adequate. Since I think
> 'queue' and 'topic' are basic capabilities for a source or target that
> would are supported by all current brokers and reasonably well understood,
> that is what I suggested.
>
> [...]
>
> I wouldn't want to tie consensus on a very simple thing to a larger more
>>> comprehensive standardisation effort for management. Neither would I want
>>> to get in your way there however. If you have different names or
>>> descriptions that would align better with what you are doing, please feel
>>> free to suggest them.
>>>
>>>
>>> But I wouldn't want to define a "simple" thing that (potentially)
>> conflicts
>> with the work that is being done through the official standards, or gives
>> the impression of a standardisation which in fact is not standard.
>>
>
> All I'm suggesting with named capabilities is exactly what has been done
> for filters already.
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.**org<us...@qpid.apache.org>
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: AMQP 1.0 and creating nodes on the fly
Posted by Gordon Sim <gs...@redhat.com>.
On 10/09/2013 12:10 PM, Rob Godfrey wrote:
> On 9 October 2013 11:24, Gordon Sim <gs...@redhat.com> wrote:
>> On 10/08/2013 03:42 PM, Rob Godfrey wrote:
>>> The second case of defining some namespace pattern within the broker
>>> wherein any unrecognized names will lead to node creation seems like a
>>> broker specific feature with no need (or requirement?) for
>>> standardisation.
>>> Am I missing something here?
>>>
>>
>> I personally much prefer the second approach. Its simpler and more
>> flexible for broker implementers to do in different ways (according to
>> their existing configuration mechanisms and details of the way their code
>> already works). It also takes the 'decision' out of the client, which seems
>> preferable from a 'philosophical' pint of view, and means that client
>> libraries need not be affected.
>>
>>
> But whether an attempt to send to/receive from an address which does not
> exist should result in either a failure or the creation of a new node is
> not just a broker/service concern - I'd actually think that it was an
> application (and therefore client) concern.
I agree I think it is an application concern, but I see it as a
deployment/configuration concern, rather than a development concern.
Being able to control that entirely through broker configuration is
valuable.
>> I'm certainly not suggesting any official 'standardisation'. I'm merely
>> hoping to find a simple, practical, 'bottom-up' consensus that would make
>> switching between AMQP brokers easier for users.
>>
>> If more brokers were to support some way of having nodes created on demand
>> purely based on broker side configuration (the details of which would be
>> broker specific), that would in my view be useful to anyone looking to try
>> out applications against different brokers.
>>
>> As such, I'm keen to implement something like that in qpidd and would also
>> be keen to start talking to some of the other broker maintainers to see if
>> I could persuade others to do the same (if they have not already).
>>
>>
> So while I don't object to brokers doing this, I'm really not keen on
> trying to use names to identify the properties of nodes to be created
If you mean the address of the node, then I agree with you. I'm not keen
on that and mentioned it merely as what is currently relied on in the
absence of anything else.
Hence my preference for capabilities, which is a defined extension point
that clearly (in my view) fits this situation quite well and can be used
whether or not the node is to be dynamically created and orthogonal to
any naming scheme for the node itself.
[...]
>>> Yes, we need to define what node-properties terms like "queue" and
>>> "topic"
>>> actually mean.
>>>
>>
>> The two sentences above attempt to do that in a minimal way. To me they
>> capture the essential capabilities that people expect when thinking of a
>> queue or topic. I would of course be eager to here alternative suggestions,
>> whether it be entirely different mechanisms, or just different capability
>> names or descriptions. That's the purpose of the original email really.
>>
>>
> So as above I'm pretty negative on just using names
Again, I'm *not* proposing using *node* names to indicate the type of
node, I'm proposing using named *capabilities*.
The distribution mode gets close, but is not quite adequate. Since I
think 'queue' and 'topic' are basic capabilities for a source or target
that would are supported by all current brokers and reasonably well
understood, that is what I suggested.
[...]
>> I wouldn't want to tie consensus on a very simple thing to a larger more
>> comprehensive standardisation effort for management. Neither would I want
>> to get in your way there however. If you have different names or
>> descriptions that would align better with what you are doing, please feel
>> free to suggest them.
>>
>>
> But I wouldn't want to define a "simple" thing that (potentially) conflicts
> with the work that is being done through the official standards, or gives
> the impression of a standardisation which in fact is not standard.
All I'm suggesting with named capabilities is exactly what has been done
for filters already.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: AMQP 1.0 and creating nodes on the fly
Posted by Rob Godfrey <ro...@gmail.com>.
On 9 October 2013 11:24, Gordon Sim <gs...@redhat.com> wrote:
> On 10/08/2013 03:42 PM, Rob Godfrey wrote:
>
>> On 4 October 2013 13:23, Gordon Sim <gs...@redhat.com> wrote:
>>
>> A common convenience is to allow queues or topics to be created on
>>> demand,
>>> i.e. having them come into existence when a link is attached. This is
>>> useful where the messaging infrastructure is supposed to be a 'hidden'
>>> part
>>> of the system, and manual configuration is not desirable.
>>>
>> [...]
>
> It would be great to have some convention that worked with different 1.0
>>> brokers.
>>>
>> [...]
>
> So - there seem to be two different things here. One is the notion of an
>> AMQP property/capability for "create-on-demand" that AMQP "server"
>> containers might implement such that if they see a link attach to an
>> address which does not exist, and the given node property is specified,
>> then the server will (subject to any authorisation rules) create the node.
>>
>
> I was really just describing what qpid::messaging currently does when
> mentioning the create-on-demand capability. I should probably have left
> that to a footnote. I'm not that keen on this approach and it was really
> just an attempt to smooth the transition by supporting a 'legacy' feature.
>
>
> This idea seems like something we should propose and register and then
>> attempt to standardise in the OASIS AMQP specifications.
>>
>> The second case of defining some namespace pattern within the broker
>> wherein any unrecognized names will lead to node creation seems like a
>> broker specific feature with no need (or requirement?) for
>> standardisation.
>> Am I missing something here?
>>
>
> I personally much prefer the second approach. Its simpler and more
> flexible for broker implementers to do in different ways (according to
> their existing configuration mechanisms and details of the way their code
> already works). It also takes the 'decision' out of the client, which seems
> preferable from a 'philosophical' pint of view, and means that client
> libraries need not be affected.
>
>
But whether an attempt to send to/receive from an address which does not
exist should result in either a failure or the creation of a new node is
not just a broker/service concern - I'd actually think that it was an
application (and therefore client) concern.
> I'm certainly not suggesting any official 'standardisation'. I'm merely
> hoping to find a simple, practical, 'bottom-up' consensus that would make
> switching between AMQP brokers easier for users.
>
> If more brokers were to support some way of having nodes created on demand
> purely based on broker side configuration (the details of which would be
> broker specific), that would in my view be useful to anyone looking to try
> out applications against different brokers.
>
> As such, I'm keen to implement something like that in qpidd and would also
> be keen to start talking to some of the other broker maintainers to see if
> I could persuade others to do the same (if they have not already).
>
>
So while I don't object to brokers doing this, I'm really not keen on
trying to use names to identify the properties of nodes to be created and
furthermore trying to get some standardisation based on this - rather than
aiding interop this seems like something that will cause breakage when
people move from a broker that supports some naming pattern to one which
does not. The idea of capability negotiation in AMQP is that this sort of
functionality van be requested and the absence of it easily detected with
an informative failure. using naming patterns runs the risk of not having
informative failure as well as leading to contradictions between the
official standardisations in the fields of addressing and node behaviour,
and "unofficial" standardisations being promoted by individual projects.
>
> One other aspect of this that is important is how to determine if the node
>>> should be a 'queue' or a 'topic', as the two most common node types. One
>>> suggestion would be to have brokers recognise two specific capabilities
>>> for
>>> these types. The 'queue' capability implies the ability to distribute
>>> messages between consumers in some fashion, and to store messages when no
>>> consumers are available. The 'topic' capability would distribute all
>>> messages to all subscribers (i.e. non-competing consumers) and would drop
>>> messages if there were no subscribers.
>>>
>>>
>>> Yes, we need to define what node-properties terms like "queue" and
>> "topic"
>> actually mean.
>>
>
> The two sentences above attempt to do that in a minimal way. To me they
> capture the essential capabilities that people expect when thinking of a
> queue or topic. I would of course be eager to here alternative suggestions,
> whether it be entirely different mechanisms, or just different capability
> names or descriptions. That's the purpose of the original email really.
>
>
So as above I'm pretty negative on just using names unless we can square it
with the work already ongoing in the global addressing work at OASIS and
the suggested work on standardising the expression of topic hierarchies /
subscriptions that is being mooted there at present. Node properties are
the mechanism that were essentially designed for this purpose, it therefore
seems like we should use them. If this is an issue for client libraries,
then maybe we can help those who have client libraries to enhance them.
I'm not suggesting that implementers should be prevented from defining
namespaces in which unknown addresses get auto created and based on the
name the created node would act as a queue or as a topic... but suggesting
this as a standard would likely cause more problems than it solves unless
the proposal is integrated into the existing OASIS work.
>
> Obviously there exist a number of sub-behaviours also (like
>> the ability to support message priorities for instance).
>>
>
> Yes, but I wouldn't want to tie these all together. All the current 1.0
> brokers support the basic distinction between 'queue' and 'topic'. I'd like
> to get some consensus amongst the different communities around a way of
> recognising that. Two simple capabilities seemed like a good way to me, but
> I'm eager to hear other ideas.
>
>
(At present the most common approach is to use different conventions for
> the name/address of the node, e.g. topic://my-topic or /queues/my-queue
> etc).
>
>
> This definitely
>> overlaps with management as we would want to have some commonality between
>> names used in node properties and the names used for creation of nodes
>> through the management spec.
>>
>
> I wouldn't want to tie consensus on a very simple thing to a larger more
> comprehensive standardisation effort for management. Neither would I want
> to get in your way there however. If you have different names or
> descriptions that would align better with what you are doing, please feel
> free to suggest them.
>
>
But I wouldn't want to define a "simple" thing that (potentially) conflicts
with the work that is being done through the official standards, or gives
the impression of a standardisation which in fact is not standard.
>
> Any thoughts, comments, complaints, alternative suggestions etc? I'd like
>>> to get agreement on something that is simple but useful for users and not
>>> difficult for different brokers to implement to improve the chances of a
>>> de-facto standard emerging.
>>>
>> [...]
>
> Where's [3]? :-)
>>
>
> Oops, forgot to add that or remove the reference. The point is simply that
> sometimes getting a 'node not found' error is a good thing as it highlights
> configuration errors simply and clearly rather than having processes 'talk
> past' each other.
>
>
>
Yeah - in fact this sort of problem is something that I often see with
users of the JMS 0-8 client when using BURL addresses - since it auto
creates the queues there is no error when a client has been misconfigured.
-- Rob
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.**org<us...@qpid.apache.org>
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: AMQP 1.0 and creating nodes on the fly
Posted by Gordon Sim <gs...@redhat.com>.
On 10/08/2013 03:42 PM, Rob Godfrey wrote:
> On 4 October 2013 13:23, Gordon Sim <gs...@redhat.com> wrote:
>
>> A common convenience is to allow queues or topics to be created on demand,
>> i.e. having them come into existence when a link is attached. This is
>> useful where the messaging infrastructure is supposed to be a 'hidden' part
>> of the system, and manual configuration is not desirable.
[...]
>> It would be great to have some convention that worked with different 1.0
>> brokers.
[...]
> So - there seem to be two different things here. One is the notion of an
> AMQP property/capability for "create-on-demand" that AMQP "server"
> containers might implement such that if they see a link attach to an
> address which does not exist, and the given node property is specified,
> then the server will (subject to any authorisation rules) create the node.
I was really just describing what qpid::messaging currently does when
mentioning the create-on-demand capability. I should probably have left
that to a footnote. I'm not that keen on this approach and it was really
just an attempt to smooth the transition by supporting a 'legacy' feature.
> This idea seems like something we should propose and register and then
> attempt to standardise in the OASIS AMQP specifications.
>
> The second case of defining some namespace pattern within the broker
> wherein any unrecognized names will lead to node creation seems like a
> broker specific feature with no need (or requirement?) for standardisation.
> Am I missing something here?
I personally much prefer the second approach. Its simpler and more
flexible for broker implementers to do in different ways (according to
their existing configuration mechanisms and details of the way their
code already works). It also takes the 'decision' out of the client,
which seems preferable from a 'philosophical' pint of view, and means
that client libraries need not be affected.
I'm certainly not suggesting any official 'standardisation'. I'm merely
hoping to find a simple, practical, 'bottom-up' consensus that would
make switching between AMQP brokers easier for users.
If more brokers were to support some way of having nodes created on
demand purely based on broker side configuration (the details of which
would be broker specific), that would in my view be useful to anyone
looking to try out applications against different brokers.
As such, I'm keen to implement something like that in qpidd and would
also be keen to start talking to some of the other broker maintainers to
see if I could persuade others to do the same (if they have not already).
>> One other aspect of this that is important is how to determine if the node
>> should be a 'queue' or a 'topic', as the two most common node types. One
>> suggestion would be to have brokers recognise two specific capabilities for
>> these types. The 'queue' capability implies the ability to distribute
>> messages between consumers in some fashion, and to store messages when no
>> consumers are available. The 'topic' capability would distribute all
>> messages to all subscribers (i.e. non-competing consumers) and would drop
>> messages if there were no subscribers.
>>
>>
> Yes, we need to define what node-properties terms like "queue" and "topic"
> actually mean.
The two sentences above attempt to do that in a minimal way. To me they
capture the essential capabilities that people expect when thinking of a
queue or topic. I would of course be eager to here alternative
suggestions, whether it be entirely different mechanisms, or just
different capability names or descriptions. That's the purpose of the
original email really.
> Obviously there exist a number of sub-behaviours also (like
> the ability to support message priorities for instance).
Yes, but I wouldn't want to tie these all together. All the current 1.0
brokers support the basic distinction between 'queue' and 'topic'. I'd
like to get some consensus amongst the different communities around a
way of recognising that. Two simple capabilities seemed like a good way
to me, but I'm eager to hear other ideas.
(At present the most common approach is to use different conventions for
the name/address of the node, e.g. topic://my-topic or /queues/my-queue
etc).
> This definitely
> overlaps with management as we would want to have some commonality between
> names used in node properties and the names used for creation of nodes
> through the management spec.
I wouldn't want to tie consensus on a very simple thing to a larger more
comprehensive standardisation effort for management. Neither would I
want to get in your way there however. If you have different names or
descriptions that would align better with what you are doing, please
feel free to suggest them.
>> Any thoughts, comments, complaints, alternative suggestions etc? I'd like
>> to get agreement on something that is simple but useful for users and not
>> difficult for different brokers to implement to improve the chances of a
>> de-facto standard emerging.
[...]
> Where's [3]? :-)
Oops, forgot to add that or remove the reference. The point is simply
that sometimes getting a 'node not found' error is a good thing as it
highlights configuration errors simply and clearly rather than having
processes 'talk past' each other.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: AMQP 1.0 and creating nodes on the fly
Posted by Rob Godfrey <ro...@gmail.com>.
On 4 October 2013 13:23, Gordon Sim <gs...@redhat.com> wrote:
> A common convenience is to allow queues or topics to be created on demand,
> i.e. having them come into existence when a link is attached. This is
> useful where the messaging infrastructure is supposed to be a 'hidden' part
> of the system, and manual configuration is not desirable.
>
> The AMQP 1.0 specification has a 'dynamic' field defined for sources and
> targets used on attach, which if set causes the broker to create the node
> on demand. This works well for the intended case of 'temporary reply
> queues', but not where the application wants to choose the name[1].
>
> At present the qpid::messaging library over 1.0 treats a node name of '#'
> as a request for a broker assigned temporary queue. If the name is not '#'
> and the create option is specified, it will request a qpidd specific
> capability, create-on-demand[2].
>
> With ActiveMQ and ApolloMQ, when attaching to a node it is always created
> on demand. The configuration for it is determined by matching the name
> against a set of preconfigured rules (this part is similar to RabbitMQ
> 'policies').
>
> It would be great to have some convention that worked with different 1.0
> brokers.
>
> Ultimately if a common management protocol emerges, it could address this
> as well as many other things of course. However I think for this specific
> and limited case there is benefit in seeking consensus on something simple
> in the short term.
>
> Though I see some value in not creating nodes on demand in some cases[3],
> I like the ApolloMQ approach for those cases where the convenience of doing
> so outweighs disadvantages.
>
> I propose adding some mechanism for configuring qpidd to support the same
> behaviour for specific portions of the node namespace. This would be
> something like the RabbitMQ policies, with the ability to define a
> create-on-demand property for a given policy. Nodes matching the pttern for
> that policy would then be created on demand and would have any other
> settings in the policy applied to them.
>
>
So - there seem to be two different things here. One is the notion of an
AMQP property/capability for "create-on-demand" that AMQP "server"
containers might implement such that if they see a link attach to an
address which does not exist, and the given node property is specified,
then the server will (subject to any authorisation rules) create the node.
This idea seems like something we should propose and register and then
attempt to standardise in the OASIS AMQP specifications.
The second case of defining some namespace pattern within the broker
wherein any unrecognized names will lead to node creation seems like a
broker specific feature with no need (or requirement?) for standardisation.
Am I missing something here?
> One other aspect of this that is important is how to determine if the node
> should be a 'queue' or a 'topic', as the two most common node types. One
> suggestion would be to have brokers recognise two specific capabilities for
> these types. The 'queue' capability implies the ability to distribute
> messages between consumers in some fashion, and to store messages when no
> consumers are available. The 'topic' capability would distribute all
> messages to all subscribers (i.e. non-competing consumers) and would drop
> messages if there were no subscribers.
>
>
Yes, we need to define what node-properties terms like "queue" and "topic"
actually mean. Obviously there exist a number of sub-behaviours also (like
the ability to support message priorities for instance). This definitely
overlaps with management as we would want to have some commonality between
names used in node properties and the names used for creation of nodes
through the management spec.
> Any thoughts, comments, complaints, alternative suggestions etc? I'd like
> to get agreement on something that is simple but useful for users and not
> difficult for different brokers to implement to improve the chances of a
> de-facto standard emerging.
>
> --Gordon
>
> [1] The specification explicitly prohibits the name of the source/target
> being sent if the dynamic flag is set. The node name is specified as being
> assigned by the broker. There is also a field for passing desired
> properties of the node and/or communicating back the actual properties.
> Sending the dynamic node properties if the dynamic flag is not set is also
> prohibited.
>
> [2] However if particular node properties have been requested it will
> also send those, which strictly speaking is in violation of the
> specification. In any case, having the configuration specified by the first
> link to attach is error prone. The qpid::messaging implementation does have
> the ability to assert that the actual properties match the requested ones
> to highlight any errors early, but again that requires sending the node
> properties even when the dynamic flag is not set.
>
>
Where's [3]? :-)
Cheers,
Rob
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.**org<us...@qpid.apache.org>
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: AMQP 1.0 and creating nodes on the fly
Posted by Gordon Sim <gs...@redhat.com>.
On 10/08/2013 02:37 PM, Ted Ross wrote:
> For the record, Dispatch also uses this pattern to allocate a routable
> address that exists for the lifetime of the Receiver/Link.
By 'this pattern' here you mean the behaviour defined in the
specification around the 'dynamic' field in termini, right?
Just to clarify, I'm not suggesting any change to that. Its already
supported by the majority of brokers and serves a useful purpose well.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: AMQP 1.0 and creating nodes on the fly
Posted by Ted Ross <tr...@redhat.com>.
For the record, Dispatch also uses this pattern to allocate a routable
address that exists for the lifetime of the Receiver/Link. This is
useful as a way to establish a receiver for replies to requests. The
assigned address can be used in the reply-to field of a request message
and the responses will find their way to the receiver link.
-Ted
On 10/04/2013 07:23 AM, Gordon Sim wrote:
> A common convenience is to allow queues or topics to be created on
> demand, i.e. having them come into existence when a link is attached.
> This is useful where the messaging infrastructure is supposed to be a
> 'hidden' part of the system, and manual configuration is not desirable.
>
> The AMQP 1.0 specification has a 'dynamic' field defined for sources
> and targets used on attach, which if set causes the broker to create
> the node on demand. This works well for the intended case of
> 'temporary reply queues', but not where the application wants to
> choose the name[1].
>
> At present the qpid::messaging library over 1.0 treats a node name of
> '#' as a request for a broker assigned temporary queue. If the name is
> not '#' and the create option is specified, it will request a qpidd
> specific capability, create-on-demand[2].
>
> With ActiveMQ and ApolloMQ, when attaching to a node it is always
> created on demand. The configuration for it is determined by matching
> the name against a set of preconfigured rules (this part is similar to
> RabbitMQ 'policies').
>
> It would be great to have some convention that worked with different
> 1.0 brokers.
>
> Ultimately if a common management protocol emerges, it could address
> this as well as many other things of course. However I think for this
> specific and limited case there is benefit in seeking consensus on
> something simple in the short term.
>
> Though I see some value in not creating nodes on demand in some
> cases[3], I like the ApolloMQ approach for those cases where the
> convenience of doing so outweighs disadvantages.
>
> I propose adding some mechanism for configuring qpidd to support the
> same behaviour for specific portions of the node namespace. This would
> be something like the RabbitMQ policies, with the ability to define a
> create-on-demand property for a given policy. Nodes matching the
> pttern for that policy would then be created on demand and would have
> any other settings in the policy applied to them.
>
> One other aspect of this that is important is how to determine if the
> node should be a 'queue' or a 'topic', as the two most common node
> types. One suggestion would be to have brokers recognise two specific
> capabilities for these types. The 'queue' capability implies the
> ability to distribute messages between consumers in some fashion, and
> to store messages when no consumers are available. The 'topic'
> capability would distribute all messages to all subscribers (i.e.
> non-competing consumers) and would drop messages if there were no
> subscribers.
>
> Any thoughts, comments, complaints, alternative suggestions etc? I'd
> like to get agreement on something that is simple but useful for users
> and not difficult for different brokers to implement to improve the
> chances of a de-facto standard emerging.
>
> --Gordon
>
> [1] The specification explicitly prohibits the name of the
> source/target being sent if the dynamic flag is set. The node name is
> specified as being assigned by the broker. There is also a field for
> passing desired properties of the node and/or communicating back the
> actual properties. Sending the dynamic node properties if the dynamic
> flag is not set is also prohibited.
>
> [2] However if particular node properties have been requested it will
> also send those, which strictly speaking is in violation of the
> specification. In any case, having the configuration specified by the
> first link to attach is error prone. The qpid::messaging
> implementation does have the ability to assert that the actual
> properties match the requested ones to highlight any errors early, but
> again that requires sending the node properties even when the dynamic
> flag is not set.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org