You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Igor Natanzon <ig...@gmail.com> on 2019/01/25 15:16:20 UTC
Exclusive queues and configuration export
Java Broker 7.1, JMS Client 0.40, running on Linux
1. I am trying to find how to create a queue as an exclusive /
no-auto-delete. Is that feature available with latest versions of Java
broker? The queue description in the management console has a tag for
Exclusive but I found no way of actually setting it. My goal is setting up
an exclusive consumer on a particular queue (where message consumption
ordering is critical), to prevent accidental startup of another consumer
(its really more of an exclusive consumer tag, similar to max-consumers = 1
setting in Artemis). With custom applications I can achieve similar
functionality using JMSXGroupID, but with vendor applications that cannot
set the header value, I need a more application-agnostic approach.
2. I am configuring Qpid with a 3-node replicated cluster, which seems to
work really well, and queue / exchange configuration changes are nicely
propagated to other nodes in the cluster. However, user / group / acl
setups are local to a specific node. Is there a way to export all that
configuration from one node and import to another? I know I can do it with
ACL roles, but not with users and groups. I want to avoid tedious setup if
I want to add another node to the cluster.
Thanks!
Re: Exclusive queues and configuration export
Posted by Oleksandr Rudyy <or...@gmail.com>.
Hi Igor,
You can create new nodes by using your own initial configuration file [1].
The broker configuration is created from an initial configuration (some
sort of template) on first start-up.
If no initial configuration is provided, the default one [2] is used.
You can create your own initial configuration and start your broker as below
./bin/qpid-server -icp ./my-initial-configuration.json
You can create the initial configuration from an existing broker
configuration by removing all IDs and replacing the varying parts with
context variables
For example, if your node is configured as below
{
"id" : "0f39d567-cce8-4132-b467-b903dee01084",
"name" : "node1",
"type" : "BDB_HA",
"desiredState" : "ACTIVE",
"address" : "localhost:5000",
"groupName" : "ha",
"helperAddress" : "localhost:5000",
"permittedNodes" : [ "localhost:5000", "localhost:5001",
"localhost:5002", "localhost:5003" ],
"virtualHostInitialConfiguration" : "{}",
"lastUpdatedBy" : "guest",
"lastUpdatedTime" : 1548757160676,
"createdBy" : "guest",
"createdTime" : 1548757160352
}
you can convert your node configuration into initial configuration like
below
{
"name" : "${my_app.node_name}",
"type" : "BDB_HA",
"desiredState" : "ACTIVE",
"address" : "${my_app.node_address}",
"groupName" : "${my_app.group_name}",
"helperAddress" : "${my_app.node_helper_address}",
"permittedNodes" : "${my_app.permitted_nodes}"
}
you can specify all you need (users, acls, groups,etc) in your intial
configuration and start the broker by providing the values for variables
./bin/qpid-server -icp ./my-initial-configuration.json -prop
my_app.node_name=node1 -prop my_app.node_address=localhost:5000 -prop
my_app.node_helper_address=localhost:5000 -prop my_app.group_name=ha
-prop my_app.permitted_nodes='["localhost:5000","localhost:5001","localhost:5002"]'
or
export QPID_OPTS="-Dmy_app.node_name=node1
-Dmy_app.node_address=localhost:5000 ..."
./bin/qpid-server -icp ./my-initial-configuration.json
Though, you still need to modify all your nodes separately on
adding/changing users, groups, etc.
Another approach you can take is by writing your configuration scripts
which can call REST API and add/modify nodes as required.
Kind Regards,
Alex
[1]
http://qpid.apache.org/releases/qpid-broker-j-7.1.0/book/Java-Broker-Initial-Configuration.html
[2]
https://github.com/apache/qpid-broker-j/blob/master/broker-core/src/main/resources/initial-config.json
On Sat, 26 Jan 2019 at 10:48, Igor Natanzon <ig...@gmail.com> wrote:
> Thanks Alex. The rest api method to set exclusive consumer works perfectly!
>
> As far as group setup, I am thinking I can ause rest api to pull existing
> setup of groups and members, and push the setup to new nodes. Just need to
> write a little program to do that. Can't do it with user passwords but
> that's ok. I just want to avoid having to recreate group setup manually
> every time new nodes are deployed.
>
> On January 26, 2019 5:29:27 AM Oleksandr Rudyy <or...@gmail.com> wrote:
>
> > Hi Igor,
> >
> > The Qpid broker allows to create exclusive queues with one of below
> > exclusivity policies:
> > LINK; only one receiving link can consumer from the queue
> > SESSION; only consumers from one session can consume
> > CONNECTION; only consumers from one connection can consume
> > CONTAINER; only consumers from one container can consume
> > PRINCIPAL; only consumers with the specific username can consume
> > SHARED_SUBSCRIPTION; used by JMS 2.0 shared subscribers
> > NONE; non exclusive queue, the default policy.
> >
> > The exclusive queues mainly created from JMS clients. The explicit
> creation
> > of exclusive queue from Web UI has not been actually requested before.
> > That's why UI does not have an option to create an exclusive queue.
> >
> > You can still create exclusive queue using REST API. As you already
> > noticed, there is queue attributes "exclusive" which is used for
> specifying
> > queue exclusivity.
> > The policy you need is LINK. Thus, you need to set value of "exclusive"
> > attribute to "LINK" in queue creation request, for example,
> >
> > curl --user guest:guest -X PUT -d '{"durable":true,"exclusive":"LINK"}'
> > https://localhost:8443/api/v7.1/queue/default/default/exclusive-queue
> >
> > The above will create durable and exclusive queue on virtual host
> > "default". (Durable is true by default, you can skip specifying durable.)
> >
> > An attempt to attach second consumer to the queue above will result in
> > exception like the one below
> >
> > [1365383557:1] ->
> >
> Attach{name='qpid-jms:receiver:ID:21f3ce64-381e-4bc9-9610-91fc21ceb63e:1:1:2:exclusive-queue',
> > handle=2, role=RECEIVER, sndSettleMode=UNSETTLED, rcvSettleMode=FIRST,
> > source=Source{address='exclusive-queue', durable=NONE,
> > expiryPolicy=LINK_DETACH, timeout=0, dynamic=false,
> > dynamicNodeProperties=null, distributionMode=null, filter=null,
> > defaultOutcome=Modified{deliveryFailed=true, undeliverableHere=null,
> > messageAnnotations=null}, outcomes=[amqp:accepted:list,
> amqp:rejected:list,
> > amqp:released:list, amqp:modified:list], capabilities=[queue]},
> > target=Target{address='null', durable=NONE, expiryPolicy=SESSION_END,
> > timeout=0, dynamic=false, dynamicNodeProperties=null, capabilities=null},
> > unsettled=null, incompleteUnsettled=false, initialDeliveryCount=null,
> > maxMessageSize=null, offeredCapabilities=null, desiredCapabilities=null,
> > properties=null}
> > [1365383557:1] <-
> >
> Attach{name='qpid-jms:receiver:ID:21f3ce64-381e-4bc9-9610-91fc21ceb63e:1:1:2:exclusive-queue',
> > handle=2, role=SENDER, sndSettleMode=MIXED, rcvSettleMode=FIRST,
> > source=null, target=null, unsettled=null, incompleteUnsettled=false,
> > initialDeliveryCount=null, maxMessageSize=null, offeredCapabilities=null,
> > desiredCapabilities=null, properties=null}
> > [1365383557:1] <- Detach{handle=2, closed=true,
> > error=Error{condition=amqp:resource-locked, description='Cannot add an
> > exclusive consumer to the destination as there is an incompatible
> > exclusivity policy', info=null}}
> > 2019-01-26 10:18:58,544 [localhost:5672]] - WARN
> > AmqpResourceBuilder - Open of resource:(JmsConsumerInfo: {
> > ID:21f3ce64-381e-4bc9-9610-91fc21ceb63e:1:1:2, destination =
> > exclusive-queue }) failed: Cannot add an exclusive consumer to the
> > destination as there is an incompatible exclusivity policy [condition =
> > amqp:resource-locked]
> > [1365383557:1] -> Detach{handle=2, closed=true, error=null}
> > Caught exception, exiting.
> > javax.jms.JMSException: Cannot add an exclusive consumer to the
> destination
> > as there is an incompatible exclusivity policy [condition =
> > amqp:resource-locked]
> >
> >
> > As for specifying groups and users on BDB HA virtual host, the Qpid
> broker
> > does not allow you that. Potentially you can work around this limitation
> by
> > using LDAP authentication provider with groups stored on LDAP side.
> >
> > Kind Regards,
> > Alex
> >
> >
> >
> > On Fri, 25 Jan 2019 at 15:16, Igor Natanzon <ig...@gmail.com>
> wrote:
> >
> >> Java Broker 7.1, JMS Client 0.40, running on Linux
> >>
> >> 1. I am trying to find how to create a queue as an exclusive /
> >> no-auto-delete. Is that feature available with latest versions of Java
> >> broker? The queue description in the management console has a tag for
> >> Exclusive but I found no way of actually setting it. My goal is setting
> up
> >> an exclusive consumer on a particular queue (where message consumption
> >> ordering is critical), to prevent accidental startup of another consumer
> >> (its really more of an exclusive consumer tag, similar to max-consumers
> = 1
> >> setting in Artemis). With custom applications I can achieve similar
> >> functionality using JMSXGroupID, but with vendor applications that
> cannot
> >> set the header value, I need a more application-agnostic approach.
> >>
> >> 2. I am configuring Qpid with a 3-node replicated cluster, which seems
> to
> >> work really well, and queue / exchange configuration changes are nicely
> >> propagated to other nodes in the cluster. However, user / group / acl
> >> setups are local to a specific node. Is there a way to export all that
> >> configuration from one node and import to another? I know I can do it
> with
> >> ACL roles, but not with users and groups. I want to avoid tedious setup
> if
> >> I want to add another node to the cluster.
> >>
> >> Thanks!
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Exclusive queues and configuration export
Posted by Igor Natanzon <ig...@gmail.com>.
Thanks Alex. The rest api method to set exclusive consumer works perfectly!
As far as group setup, I am thinking I can ause rest api to pull existing
setup of groups and members, and push the setup to new nodes. Just need to
write a little program to do that. Can't do it with user passwords but
that's ok. I just want to avoid having to recreate group setup manually
every time new nodes are deployed.
On January 26, 2019 5:29:27 AM Oleksandr Rudyy <or...@gmail.com> wrote:
> Hi Igor,
>
> The Qpid broker allows to create exclusive queues with one of below
> exclusivity policies:
> LINK; only one receiving link can consumer from the queue
> SESSION; only consumers from one session can consume
> CONNECTION; only consumers from one connection can consume
> CONTAINER; only consumers from one container can consume
> PRINCIPAL; only consumers with the specific username can consume
> SHARED_SUBSCRIPTION; used by JMS 2.0 shared subscribers
> NONE; non exclusive queue, the default policy.
>
> The exclusive queues mainly created from JMS clients. The explicit creation
> of exclusive queue from Web UI has not been actually requested before.
> That's why UI does not have an option to create an exclusive queue.
>
> You can still create exclusive queue using REST API. As you already
> noticed, there is queue attributes "exclusive" which is used for specifying
> queue exclusivity.
> The policy you need is LINK. Thus, you need to set value of "exclusive"
> attribute to "LINK" in queue creation request, for example,
>
> curl --user guest:guest -X PUT -d '{"durable":true,"exclusive":"LINK"}'
> https://localhost:8443/api/v7.1/queue/default/default/exclusive-queue
>
> The above will create durable and exclusive queue on virtual host
> "default". (Durable is true by default, you can skip specifying durable.)
>
> An attempt to attach second consumer to the queue above will result in
> exception like the one below
>
> [1365383557:1] ->
> Attach{name='qpid-jms:receiver:ID:21f3ce64-381e-4bc9-9610-91fc21ceb63e:1:1:2:exclusive-queue',
> handle=2, role=RECEIVER, sndSettleMode=UNSETTLED, rcvSettleMode=FIRST,
> source=Source{address='exclusive-queue', durable=NONE,
> expiryPolicy=LINK_DETACH, timeout=0, dynamic=false,
> dynamicNodeProperties=null, distributionMode=null, filter=null,
> defaultOutcome=Modified{deliveryFailed=true, undeliverableHere=null,
> messageAnnotations=null}, outcomes=[amqp:accepted:list, amqp:rejected:list,
> amqp:released:list, amqp:modified:list], capabilities=[queue]},
> target=Target{address='null', durable=NONE, expiryPolicy=SESSION_END,
> timeout=0, dynamic=false, dynamicNodeProperties=null, capabilities=null},
> unsettled=null, incompleteUnsettled=false, initialDeliveryCount=null,
> maxMessageSize=null, offeredCapabilities=null, desiredCapabilities=null,
> properties=null}
> [1365383557:1] <-
> Attach{name='qpid-jms:receiver:ID:21f3ce64-381e-4bc9-9610-91fc21ceb63e:1:1:2:exclusive-queue',
> handle=2, role=SENDER, sndSettleMode=MIXED, rcvSettleMode=FIRST,
> source=null, target=null, unsettled=null, incompleteUnsettled=false,
> initialDeliveryCount=null, maxMessageSize=null, offeredCapabilities=null,
> desiredCapabilities=null, properties=null}
> [1365383557:1] <- Detach{handle=2, closed=true,
> error=Error{condition=amqp:resource-locked, description='Cannot add an
> exclusive consumer to the destination as there is an incompatible
> exclusivity policy', info=null}}
> 2019-01-26 10:18:58,544 [localhost:5672]] - WARN
> AmqpResourceBuilder - Open of resource:(JmsConsumerInfo: {
> ID:21f3ce64-381e-4bc9-9610-91fc21ceb63e:1:1:2, destination =
> exclusive-queue }) failed: Cannot add an exclusive consumer to the
> destination as there is an incompatible exclusivity policy [condition =
> amqp:resource-locked]
> [1365383557:1] -> Detach{handle=2, closed=true, error=null}
> Caught exception, exiting.
> javax.jms.JMSException: Cannot add an exclusive consumer to the destination
> as there is an incompatible exclusivity policy [condition =
> amqp:resource-locked]
>
>
> As for specifying groups and users on BDB HA virtual host, the Qpid broker
> does not allow you that. Potentially you can work around this limitation by
> using LDAP authentication provider with groups stored on LDAP side.
>
> Kind Regards,
> Alex
>
>
>
> On Fri, 25 Jan 2019 at 15:16, Igor Natanzon <ig...@gmail.com> wrote:
>
>> Java Broker 7.1, JMS Client 0.40, running on Linux
>>
>> 1. I am trying to find how to create a queue as an exclusive /
>> no-auto-delete. Is that feature available with latest versions of Java
>> broker? The queue description in the management console has a tag for
>> Exclusive but I found no way of actually setting it. My goal is setting up
>> an exclusive consumer on a particular queue (where message consumption
>> ordering is critical), to prevent accidental startup of another consumer
>> (its really more of an exclusive consumer tag, similar to max-consumers = 1
>> setting in Artemis). With custom applications I can achieve similar
>> functionality using JMSXGroupID, but with vendor applications that cannot
>> set the header value, I need a more application-agnostic approach.
>>
>> 2. I am configuring Qpid with a 3-node replicated cluster, which seems to
>> work really well, and queue / exchange configuration changes are nicely
>> propagated to other nodes in the cluster. However, user / group / acl
>> setups are local to a specific node. Is there a way to export all that
>> configuration from one node and import to another? I know I can do it with
>> ACL roles, but not with users and groups. I want to avoid tedious setup if
>> I want to add another node to the cluster.
>>
>> Thanks!
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Exclusive queues and configuration export
Posted by Oleksandr Rudyy <or...@gmail.com>.
Hi Igor,
The Qpid broker allows to create exclusive queues with one of below
exclusivity policies:
LINK; only one receiving link can consumer from the queue
SESSION; only consumers from one session can consume
CONNECTION; only consumers from one connection can consume
CONTAINER; only consumers from one container can consume
PRINCIPAL; only consumers with the specific username can consume
SHARED_SUBSCRIPTION; used by JMS 2.0 shared subscribers
NONE; non exclusive queue, the default policy.
The exclusive queues mainly created from JMS clients. The explicit creation
of exclusive queue from Web UI has not been actually requested before.
That's why UI does not have an option to create an exclusive queue.
You can still create exclusive queue using REST API. As you already
noticed, there is queue attributes "exclusive" which is used for specifying
queue exclusivity.
The policy you need is LINK. Thus, you need to set value of "exclusive"
attribute to "LINK" in queue creation request, for example,
curl --user guest:guest -X PUT -d '{"durable":true,"exclusive":"LINK"}'
https://localhost:8443/api/v7.1/queue/default/default/exclusive-queue
The above will create durable and exclusive queue on virtual host
"default". (Durable is true by default, you can skip specifying durable.)
An attempt to attach second consumer to the queue above will result in
exception like the one below
[1365383557:1] ->
Attach{name='qpid-jms:receiver:ID:21f3ce64-381e-4bc9-9610-91fc21ceb63e:1:1:2:exclusive-queue',
handle=2, role=RECEIVER, sndSettleMode=UNSETTLED, rcvSettleMode=FIRST,
source=Source{address='exclusive-queue', durable=NONE,
expiryPolicy=LINK_DETACH, timeout=0, dynamic=false,
dynamicNodeProperties=null, distributionMode=null, filter=null,
defaultOutcome=Modified{deliveryFailed=true, undeliverableHere=null,
messageAnnotations=null}, outcomes=[amqp:accepted:list, amqp:rejected:list,
amqp:released:list, amqp:modified:list], capabilities=[queue]},
target=Target{address='null', durable=NONE, expiryPolicy=SESSION_END,
timeout=0, dynamic=false, dynamicNodeProperties=null, capabilities=null},
unsettled=null, incompleteUnsettled=false, initialDeliveryCount=null,
maxMessageSize=null, offeredCapabilities=null, desiredCapabilities=null,
properties=null}
[1365383557:1] <-
Attach{name='qpid-jms:receiver:ID:21f3ce64-381e-4bc9-9610-91fc21ceb63e:1:1:2:exclusive-queue',
handle=2, role=SENDER, sndSettleMode=MIXED, rcvSettleMode=FIRST,
source=null, target=null, unsettled=null, incompleteUnsettled=false,
initialDeliveryCount=null, maxMessageSize=null, offeredCapabilities=null,
desiredCapabilities=null, properties=null}
[1365383557:1] <- Detach{handle=2, closed=true,
error=Error{condition=amqp:resource-locked, description='Cannot add an
exclusive consumer to the destination as there is an incompatible
exclusivity policy', info=null}}
2019-01-26 10:18:58,544 [localhost:5672]] - WARN
AmqpResourceBuilder - Open of resource:(JmsConsumerInfo: {
ID:21f3ce64-381e-4bc9-9610-91fc21ceb63e:1:1:2, destination =
exclusive-queue }) failed: Cannot add an exclusive consumer to the
destination as there is an incompatible exclusivity policy [condition =
amqp:resource-locked]
[1365383557:1] -> Detach{handle=2, closed=true, error=null}
Caught exception, exiting.
javax.jms.JMSException: Cannot add an exclusive consumer to the destination
as there is an incompatible exclusivity policy [condition =
amqp:resource-locked]
As for specifying groups and users on BDB HA virtual host, the Qpid broker
does not allow you that. Potentially you can work around this limitation by
using LDAP authentication provider with groups stored on LDAP side.
Kind Regards,
Alex
On Fri, 25 Jan 2019 at 15:16, Igor Natanzon <ig...@gmail.com> wrote:
> Java Broker 7.1, JMS Client 0.40, running on Linux
>
> 1. I am trying to find how to create a queue as an exclusive /
> no-auto-delete. Is that feature available with latest versions of Java
> broker? The queue description in the management console has a tag for
> Exclusive but I found no way of actually setting it. My goal is setting up
> an exclusive consumer on a particular queue (where message consumption
> ordering is critical), to prevent accidental startup of another consumer
> (its really more of an exclusive consumer tag, similar to max-consumers = 1
> setting in Artemis). With custom applications I can achieve similar
> functionality using JMSXGroupID, but with vendor applications that cannot
> set the header value, I need a more application-agnostic approach.
>
> 2. I am configuring Qpid with a 3-node replicated cluster, which seems to
> work really well, and queue / exchange configuration changes are nicely
> propagated to other nodes in the cluster. However, user / group / acl
> setups are local to a specific node. Is there a way to export all that
> configuration from one node and import to another? I know I can do it with
> ACL roles, but not with users and groups. I want to avoid tedious setup if
> I want to add another node to the cluster.
>
> Thanks!
>