JMS Temp Queue naming scheme

[Rajith Attapattu <ra...@gmail.com>]

Hi Qpidians,

Currently our temp queue naming is hard coded to "Temp_ " + uuid.
We could change this to a format that is configurable and captures more
useful information that can be utilized by,

a) management tools to provide more visibility
b) ACL modules.

I propose the following format
{host}.{port}.{user_prefix}.{uuid}.

The  above format is configurable with {uuid} being the only mandatory part.
The order of the tokens could also be changed.
The scheme can be set globally using a jvm argument or per connection using
connection URL.

Example.
{user_prefix}_{uuid}   = Temp_535fewfsgdsg  where user_prefix = "Temp"

I do have a patch ready for the above functionality.
However there are also some valid points to not making the temp queue naming
scheme configurable.

I open the floor for any discussion ion for us to come to an agreement on
this issue.
Once a decision is made I am quite happy to make the necessary changes.

Regards,

Rajith Attapattu
Red Hat
http://rajith.2rlabs.com/

Re: JMS Temp Queue naming scheme

[Aidan Skinner <ai...@apache.org>]

On Tue, Sep 30, 2008 at 10:46 PM, Rajith Attapattu <ra...@gmail.com>wrote:


> Currently our temp queue naming is hard coded to "Temp_ " + uuid.
> We could change this to a format that is configurable and captures more
> useful information that can be utilized by,
>

Isn't there a Jira for this already?

- Aidan
-- 
Apache Qpid - World Domination through Advanced Message Queueing
http://cwiki.apache.org/qpid
"Nine-tenths of wisdom consists in being wise in time." - Theodore Roosevelt

Re: JMS Temp Queue naming scheme

[Gordon Sim <gs...@redhat.com>]

Rajith Attapattu wrote:
> However there are also some valid points to not making the temp queue naming
> scheme configurable.

Could you elaborate on what those might be?

Re: JMS Temp Queue naming scheme

[Gordon Sim <gs...@redhat.com>]

Robert Godfrey wrote:
> The exchange to which temporary queues are bound should already be
> configurable... That I don't have a problem with.

Great, I didn't realise that! Is the binding key used also configurable?

If the routing details can be separately configured, the name of a 
temporary queue is probably only interesting in the clues it gives as to 
usage (e.g. making it easy to see that a queue that appears to be 
growing rapidly is a temp queue used as part of 'app-a' by 'user-b' 
_might_ be valuable).

Providing the scheme safely avoids name conflicts I don't see much 
_danger_ in it being configurable, though it may well be possible to 
have one scheme that conveys all the useful information would remove 
much of the value of such an option.

I guess I could envisage a scheme whereby default 'policies' would be 
applied to queues using rules with naming patterns, but equally a 
configurable mechanism for constructing the queues could just as well 
request that policy directly.

Re: JMS Temp Queue naming scheme

[Gordon Sim <gs...@redhat.com>]

Robert Godfrey wrote:
> 2008/10/1 Robert Godfrey <ro...@gmail.com>:
>> 2008/10/1 Gordon Sim <gs...@redhat.com>:
>>> Robert Godfrey wrote:
>>>> 2008/9/30 Rajith Attapattu <ra...@gmail.com>:
>>>>> Hi Qpidians,
>>>>>
>>>>> Currently our temp queue naming is hard coded to "Temp_ " + uuid.
>>>>> We could change this to a format that is configurable and captures more
>>>>> useful information that can be utilized by,
>>>> I think there are different point in here:
>>>>
>>>> 1) Should we change our temp queue naming to include more useful
>>>> information
>>>>
>>>> +1 from me
>>>>
>>>> 2) Should we add some sort of way for the client application to
>>>> configure the scheme for naming temporary queues created from JMS
>>>>
>>>> -1 From me (I have no problem with the low level API allowing any name
>>>> to be used)
>>> I tend to the opposite view! My 'theory' (vague as it is) is that it should
>>> be possible to use the JMS API but have flexibility in how those actions are
>>> 'mapped' into AMQP. (The name of the queue is interesting, but actually the
>>> exchange and binding details are in AMQP terms more valuable).
> 
> In general I agree... but I don't think that the *names* generated for
> temporary queues should be something that are relied upon...  In
> particular for uniqueness you've pretty much got to put a UUID in
> there - so you shouldn't be able to let the clients get around that...
> And what purpose is there in configuring the *name* of the queue,
> other than to add informational stuff.

Yes, I agree. Adding informational stuff may be useful, then again it 
may be that a single scheme can handle the informational needs of the 
vast majority of users.

>>>> My fundamental objection to configurability is that the client
>>>> applications shouldn't need to know how these queues are named and
>>>> that allowing them to configure it is to risk them building in
>>>> dependencies on behaviour we may not be able to maintain in the face
>>>> of changes to the underlying AMQP model.
>>> I agree that the clients themselves most likely don't care and they should
>>> be able to work without any further thought.
>>>
>>> There may be reasons for adjusting the underlying system configuration
>>> though, and allowing for that seems reasonable to me.
>>>
>>> Obviously any configuration or code that relies on a particular aspect of
>>> AMQP becomes dependent on that aspect and at risk from alteration or removal
>>> of that aspect by subsequent revisions to the specification.
>>>
>>> At present I can create my own AMQDestination instance and that is usable
>>> from JMS clients for publishing to or consuming from. I may construct it to
>>> use a particular AMQP 'routing scheme'. By doing so I am becoming dependent
>>> on some aspects of the AMQP protocol. There is certainly risk there and we
>>> should advise carefully, but I don't think we should prevent that.
>>>
> 
> I agree that we should give people that flexibility - but the
> Destination URL and the Connection URL are the places I would like to
> limit this to.  Having more byzantine options for configuring
> behaviour will make things unmaintainable and lead to weird
> configuration dependencies in people's installations that will only
> cause issues when they come to upgrade.  Limitting things to URLs
> whose format we can document in a single place and where we can
> clearly explain how they must change from one revision to another
> seems sensible to me. 

I agree, keeping things manageable and configurable through the URLs for 
connections or destinations seems valuable.

> As an aside I don't consider choosing the
> *name* of the generated temporary queues to be part of the routing
> scheme.

I agree.

>>> Allowing the strategy by which the AMQDestination constructed for a
>>> temporary queue is a further step along that path in my view.
>>>
>>> The alternative to configuration is to say either that a user cannot change
>>> how some of the more arbitrary decisions in the JMS client work or they have
>>> to write code to explicitly do what they want in AMQP terms (whether through
>>> Qpid extensions or another API). Moving this to configuration shields the
>>> application code from changes (which is still dependent on those features
>>> but should be easier to change for alternate deployments).
>>>
>>>> From a broker point of view we cannot depend on any naming scheme
>>>> since the names are client generated and thus non-Qpid clients may be
>>>> being used to connect to the broker (a use case for this functionality
>>>> as put to me involved being able to detect from the name that a
>>>> particular reply-to address related to a temporary queue).
>>>>  What is the use case for a client being able to configure the naming
>>>> scheme?
>>> As above, I think that where there are reasonable choices in how JMS is
>>> mapped to AMQP, making that mapping customisable is in general valuable.
>>>
>>> However as a motivating example, I might want a 'federation' where the
>>> relaying of messages between brokers in that federation is controlled by the
>>> exchange to which they are published.
>>>
>>> Some clients may need to use a 'service queue' that I then want to be
>>> offered by another client on a remote node of the federation. So I want the
>>> queue used to communicate with that 'service' to be configured to use an
>>> exchange specifically setup for inter-broker routing, and in addition have
>>> the clients that use it have temporary queues that will work across the
>>> federation also.
>>>
>>> By picking the exchange name and binding key used I can configure my system
>>> to work in this federated manner without changing my client.
>>>
> 
> But that requires you to have an intimate knowledge of exactly how
> your federation scheme (where your == Qpid C++ broker at this instance
> in time) works.

Yes, I suspect any scheme of 'federation' (as opposed to a distributed 
implementation of what attempts to appear as a single broker) probably 
requires some understanding of how inter-broker routing works and is 
configured.

> Surely the use of reply-to addresses should be transparent to the
> sending client and the broker should handle any on-the wire
> translation to handle travelling through federation links.  (Apologies
> if I am misunderstanding you here).

Rewriting the reply-to is certainly a nice idea (would have potential 
impact on signing of headers as and when that is looked at, but thats an 
aside).

> Obviously the "correct" answer is for AMQP to handle global
> addressing.  

Certainly any changes in this area to the protocol would have an impact 
on the optimal solution for the particular example I gave. Future 
improvements to the current implementation may do likewise.

> However setting "reply-to" on the JMS client should
> "just work" and you as an application programmer shouldn't need to
> worry about arcane details of the AMQP<->JMS mapping to get that
> working.

I agree. All I was saying is that should you wish to know about or alter 
the details of how the temporary queue is bound, that making that 
possible is not a bad thing providing it is implemented (and documented) 
in  maintainable, responsible way.

It sounds like we agree on that?

Re: JMS Temp Queue naming scheme

[Robert Godfrey <ro...@gmail.com>]

2008/10/1 Robert Godfrey <ro...@gmail.com>:
> 2008/10/1 Gordon Sim <gs...@redhat.com>:
>> Robert Godfrey wrote:
>>>
>>> 2008/9/30 Rajith Attapattu <ra...@gmail.com>:
>>>>
>>>> Hi Qpidians,
>>>>
>>>> Currently our temp queue naming is hard coded to "Temp_ " + uuid.
>>>> We could change this to a format that is configurable and captures more
>>>> useful information that can be utilized by,
>>>
>>> I think there are different point in here:
>>>
>>> 1) Should we change our temp queue naming to include more useful
>>> information
>>>
>>> +1 from me
>>>
>>> 2) Should we add some sort of way for the client application to
>>> configure the scheme for naming temporary queues created from JMS
>>>
>>> -1 From me (I have no problem with the low level API allowing any name
>>> to be used)
>>
>> I tend to the opposite view! My 'theory' (vague as it is) is that it should
>> be possible to use the JMS API but have flexibility in how those actions are
>> 'mapped' into AMQP. (The name of the queue is interesting, but actually the
>> exchange and binding details are in AMQP terms more valuable).

In general I agree... but I don't think that the *names* generated for
temporary queues should be something that are relied upon...  In
particular for uniqueness you've pretty much got to put a UUID in
there - so you shouldn't be able to let the clients get around that...
And what purpose is there in configuring the *name* of the queue,
other than to add informational stuff.

>>> My fundamental objection to configurability is that the client
>>> applications shouldn't need to know how these queues are named and
>>> that allowing them to configure it is to risk them building in
>>> dependencies on behaviour we may not be able to maintain in the face
>>> of changes to the underlying AMQP model.
>>
>> I agree that the clients themselves most likely don't care and they should
>> be able to work without any further thought.
>>
>> There may be reasons for adjusting the underlying system configuration
>> though, and allowing for that seems reasonable to me.
>>
>> Obviously any configuration or code that relies on a particular aspect of
>> AMQP becomes dependent on that aspect and at risk from alteration or removal
>> of that aspect by subsequent revisions to the specification.
>>
>> At present I can create my own AMQDestination instance and that is usable
>> from JMS clients for publishing to or consuming from. I may construct it to
>> use a particular AMQP 'routing scheme'. By doing so I am becoming dependent
>> on some aspects of the AMQP protocol. There is certainly risk there and we
>> should advise carefully, but I don't think we should prevent that.
>>

I agree that we should give people that flexibility - but the
Destination URL and the Connection URL are the places I would like to
limit this to.  Having more byzantine options for configuring
behaviour will make things unmaintainable and lead to weird
configuration dependencies in people's installations that will only
cause issues when they come to upgrade.  Limitting things to URLs
whose format we can document in a single place and where we can
clearly explain how they must change from one revision to another
seems sensible to me.  As an aside I don't consider choosing the
*name* of the generated temporary queues to be part of the routing
scheme.

>> Allowing the strategy by which the AMQDestination constructed for a
>> temporary queue is a further step along that path in my view.
>>
>> The alternative to configuration is to say either that a user cannot change
>> how some of the more arbitrary decisions in the JMS client work or they have
>> to write code to explicitly do what they want in AMQP terms (whether through
>> Qpid extensions or another API). Moving this to configuration shields the
>> application code from changes (which is still dependent on those features
>> but should be easier to change for alternate deployments).
>>
>>> From a broker point of view we cannot depend on any naming scheme
>>> since the names are client generated and thus non-Qpid clients may be
>>> being used to connect to the broker (a use case for this functionality
>>> as put to me involved being able to detect from the name that a
>>> particular reply-to address related to a temporary queue).
>>>  What is the use case for a client being able to configure the naming
>>> scheme?
>>
>> As above, I think that where there are reasonable choices in how JMS is
>> mapped to AMQP, making that mapping customisable is in general valuable.
>>
>> However as a motivating example, I might want a 'federation' where the
>> relaying of messages between brokers in that federation is controlled by the
>> exchange to which they are published.
>>
>> Some clients may need to use a 'service queue' that I then want to be
>> offered by another client on a remote node of the federation. So I want the
>> queue used to communicate with that 'service' to be configured to use an
>> exchange specifically setup for inter-broker routing, and in addition have
>> the clients that use it have temporary queues that will work across the
>> federation also.
>>
>> By picking the exchange name and binding key used I can configure my system
>> to work in this federated manner without changing my client.
>>

But that requires you to have an intimate knowledge of exactly how
your federation scheme (where your == Qpid C++ broker at this instance
in time) works.

Surely the use of reply-to addresses should be transparent to the
sending client and the broker should handle any on-the wire
translation to handle travelling through federation links.  (Apologies
if I am misunderstanding you here).

Obviously the "correct" answer is for AMQP to handle global
addressing.   However setting "reply-to" on the JMS client should
"just work" and you as an application programmer shouldn't need to
worry about arcane details of the AMQP<->JMS mapping to get that
working.

-- Rob

Re: JMS Temp Queue naming scheme

[Robert Godfrey <ro...@gmail.com>]

2008/10/1 Gordon Sim <gs...@redhat.com>:
> Robert Godfrey wrote:
>>
>> 2008/9/30 Rajith Attapattu <ra...@gmail.com>:
>>>
>>> Hi Qpidians,
>>>
>>> Currently our temp queue naming is hard coded to "Temp_ " + uuid.
>>> We could change this to a format that is configurable and captures more
>>> useful information that can be utilized by,
>>
>> I think there are different point in here:
>>
>> 1) Should we change our temp queue naming to include more useful
>> information
>>
>> +1 from me
>>
>> 2) Should we add some sort of way for the client application to
>> configure the scheme for naming temporary queues created from JMS
>>
>> -1 From me (I have no problem with the low level API allowing any name
>> to be used)
>
> I tend to the opposite view! My 'theory' (vague as it is) is that it should
> be possible to use the JMS API but have flexibility in how those actions are
> 'mapped' into AMQP. (The name of the queue is interesting, but actually the
> exchange and binding details are in AMQP terms more valuable).
>
>> My fundamental objection to configurability is that the client
>> applications shouldn't need to know how these queues are named and
>> that allowing them to configure it is to risk them building in
>> dependencies on behaviour we may not be able to maintain in the face
>> of changes to the underlying AMQP model.
>
> I agree that the clients themselves most likely don't care and they should
> be able to work without any further thought.
>
> There may be reasons for adjusting the underlying system configuration
> though, and allowing for that seems reasonable to me.
>
> Obviously any configuration or code that relies on a particular aspect of
> AMQP becomes dependent on that aspect and at risk from alteration or removal
> of that aspect by subsequent revisions to the specification.
>
> At present I can create my own AMQDestination instance and that is usable
> from JMS clients for publishing to or consuming from. I may construct it to
> use a particular AMQP 'routing scheme'. By doing so I am becoming dependent
> on some aspects of the AMQP protocol. There is certainly risk there and we
> should advise carefully, but I don't think we should prevent that.
>
> Allowing the strategy by which the AMQDestination constructed for a
> temporary queue is a further step along that path in my view.
>
> The alternative to configuration is to say either that a user cannot change
> how some of the more arbitrary decisions in the JMS client work or they have
> to write code to explicitly do what they want in AMQP terms (whether through
> Qpid extensions or another API). Moving this to configuration shields the
> application code from changes (which is still dependent on those features
> but should be easier to change for alternate deployments).
>
>> From a broker point of view we cannot depend on any naming scheme
>> since the names are client generated and thus non-Qpid clients may be
>> being used to connect to the broker (a use case for this functionality
>> as put to me involved being able to detect from the name that a
>> particular reply-to address related to a temporary queue).
>>  What is the use case for a client being able to configure the naming
>> scheme?
>
> As above, I think that where there are reasonable choices in how JMS is
> mapped to AMQP, making that mapping customisable is in general valuable.
>
> However as a motivating example, I might want a 'federation' where the
> relaying of messages between brokers in that federation is controlled by the
> exchange to which they are published.
>
> Some clients may need to use a 'service queue' that I then want to be
> offered by another client on a remote node of the federation. So I want the
> queue used to communicate with that 'service' to be configured to use an
> exchange specifically setup for inter-broker routing, and in addition have
> the clients that use it have temporary queues that will work across the
> federation also.
>
> By picking the exchange name and binding key used I can configure my system
> to work in this federated manner without changing my client.
>

The exchange to which temporary queues are bound should already be
configurable... That I don't have a problem with.

-- Rob

Re: JMS Temp Queue naming scheme

[Gordon Sim <gs...@redhat.com>]

Robert Godfrey wrote:
> 2008/9/30 Rajith Attapattu <ra...@gmail.com>:
>> Hi Qpidians,
>>
>> Currently our temp queue naming is hard coded to "Temp_ " + uuid.
>> We could change this to a format that is configurable and captures more
>> useful information that can be utilized by,
> 
> I think there are different point in here:
> 
> 1) Should we change our temp queue naming to include more useful information
> 
> +1 from me
> 
> 2) Should we add some sort of way for the client application to
> configure the scheme for naming temporary queues created from JMS
> 
> -1 From me (I have no problem with the low level API allowing any name
> to be used)

I tend to the opposite view! My 'theory' (vague as it is) is that it 
should be possible to use the JMS API but have flexibility in how those 
actions are 'mapped' into AMQP. (The name of the queue is interesting, 
but actually the exchange and binding details are in AMQP terms more 
valuable).

> My fundamental objection to configurability is that the client
> applications shouldn't need to know how these queues are named and
> that allowing them to configure it is to risk them building in
> dependencies on behaviour we may not be able to maintain in the face
> of changes to the underlying AMQP model.

I agree that the clients themselves most likely don't care and they 
should be able to work without any further thought.

There may be reasons for adjusting the underlying system configuration 
though, and allowing for that seems reasonable to me.

Obviously any configuration or code that relies on a particular aspect 
of AMQP becomes dependent on that aspect and at risk from alteration or 
removal of that aspect by subsequent revisions to the specification.

At present I can create my own AMQDestination instance and that is 
usable from JMS clients for publishing to or consuming from. I may 
construct it to use a particular AMQP 'routing scheme'. By doing so I am 
becoming dependent on some aspects of the AMQP protocol. There is 
certainly risk there and we should advise carefully, but I don't think 
we should prevent that.

Allowing the strategy by which the AMQDestination constructed for a 
temporary queue is a further step along that path in my view.

The alternative to configuration is to say either that a user cannot 
change how some of the more arbitrary decisions in the JMS client work 
or they have to write code to explicitly do what they want in AMQP terms 
(whether through Qpid extensions or another API). Moving this to 
configuration shields the application code from changes (which is still 
dependent on those features but should be easier to change for alternate 
deployments).

> From a broker point of view we cannot depend on any naming scheme
> since the names are client generated and thus non-Qpid clients may be
> being used to connect to the broker (a use case for this functionality
> as put to me involved being able to detect from the name that a
> particular reply-to address related to a temporary queue).
>  
> What is the use case for a client being able to configure the naming scheme?

As above, I think that where there are reasonable choices in how JMS is 
mapped to AMQP, making that mapping customisable is in general valuable.

However as a motivating example, I might want a 'federation' where the 
relaying of messages between brokers in that federation is controlled by 
the exchange to which they are published.

Some clients may need to use a 'service queue' that I then want to be 
offered by another client on a remote node of the federation. So I want 
the queue used to communicate with that 'service' to be configured to 
use an exchange specifically setup for inter-broker routing, and in 
addition have the clients that use it have temporary queues that will 
work across the federation also.

By picking the exchange name and binding key used I can configure my 
system to work in this federated manner without changing my client.

Re: JMS Temp Queue naming scheme

[Robert Godfrey <ro...@gmail.com>]

2008/9/30 Rajith Attapattu <ra...@gmail.com>:
> Hi Qpidians,
>
> Currently our temp queue naming is hard coded to "Temp_ " + uuid.
> We could change this to a format that is configurable and captures more
> useful information that can be utilized by,

I think there are different point in here:

1) Should we change our temp queue naming to include more useful information

+1 from me

2) Should we add some sort of way for the client application to
configure the scheme for naming temporary queues created from JMS

-1 From me (I have no problem with the low level API allowing any name
to be used)

My fundamental objection to configurability is that the client
applications shouldn't need to know how these queues are named and
that allowing them to configure it is to risk them building in
dependencies on behaviour we may not be able to maintain in the face
of changes to the underlying AMQP model.

>From a broker point of view we cannot depend on any naming scheme
since the names are client generated and thus non-Qpid clients may be
being used to connect to the broker (a use case for this functionality
as put to me involved being able to detect from the name that a
particular reply-to address related to a temporary queue).

What is the use case for a client being able to configure the naming scheme?

-- Rob

>
> a) management tools to provide more visibility
> b) ACL modules.
>
> I propose the following format
> {host}.{port}.{user_prefix}.{uuid}.
>
> The  above format is configurable with {uuid} being the only mandatory part.
> The order of the tokens could also be changed.
> The scheme can be set globally using a jvm argument or per connection using
> connection URL.
>
> Example.
> {user_prefix}_{uuid}   = Temp_535fewfsgdsg  where user_prefix = "Temp"
>
> I do have a patch ready for the above functionality.
> However there are also some valid points to not making the temp queue naming
> scheme configurable.
>
> I open the floor for any discussion ion for us to come to an agreement on
> this issue.
> Once a decision is made I am quite happy to make the necessary changes.
>
> Regards,
>
> Rajith Attapattu
> Red Hat
> http://rajith.2rlabs.com/
>

Re: JMS Temp Queue naming scheme

[Gordon Sim <gs...@redhat.com>]

Rajith Attapattu wrote:
> Hi Qpidians,
> 
> Currently our temp queue naming is hard coded to "Temp_ " + uuid.
> We could change this to a format that is configurable and captures more
> useful information that can be utilized by,
> 
> a) management tools to provide more visibility
> b) ACL modules.
> 
> I propose the following format
> {host}.{port}.{user_prefix}.{uuid}.
> 
> The  above format is configurable with {uuid} being the only mandatory part.
> The order of the tokens could also be changed.

As well as the name, allowing the exchange and binding key used to be 
configurable in some way would be useful.