You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by "Spencer.Doak" <Sp...@live.com> on 2014/09/18 01:56:34 UTC
QPID C++ - Dynamically Managing Broker
Hello,
I'm faced with a bit of an issue. For an application I'm working on, I'm in
need of several abilities:
1. Create, destroy, and clear queues on an exchange.
2. Create and destroy exchanges on a broker.
3. Fetch a list of all existing connections on a broker.
4. If possible, send a management message telling the broker to shutdown.
5. Verify the existence of a queue. (I have a workaround for this)
Based on what I've been reading, I believe QMF would be the way to go for
this. The issue is that I cannot seem to find much information about QMF
despite all of the digging I've been doing.
I have found:
https://qpid.apache.org/releases/qpid-0.28/qmf/cpp/api/index.html
and
http://qpid.apache.org/books/0.12/AMQP-Messaging-Broker-CPP-Book/pdf/AMQP-Messaging-Broker-CPP-Book.pdf
However these are somewhat vague in my opinion. In fact the book uses "???"
in locations where there should be links to more information.
Can somebody possibly supply an in-depth example/description of how to use
QMF for broker management in C++? I hate to be a bother, but I'm simply not
understanding how QMF is supposed to be used.
Thank you,
Spencer
--
View this message in context: http://qpid.2158936.n2.nabble.com/QPID-C-Dynamically-Managing-Broker-tp7613792.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: QPID C++ - Dynamically Managing Broker
Posted by "Spencer.Doak" <Sp...@live.com>.
Hey Gordon,
I retract my previous statement saying this did not work. Just now I
downloaded, compiled and ran a fresh copy of qmf_ctrl.cpp. Now it seems to
be functioning perfectly. I'm not sure why it wasn't working before. I must
have accidently modified the first copy of qmf_ctrl when reading through the
source.
For reference, I'm running the the QPID C++ Broker with QPID C++ Messaging
API 0.28. If I run into any issues with it malfunctioning, I'll post a reply
here.
Thank you,
Spencer
--
View this message in context: http://qpid.2158936.n2.nabble.com/QPID-C-Dynamically-Managing-Broker-tp7613792p7613921.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: QPID C++ - Dynamically Managing Broker
Posted by Gordon Sim <gs...@redhat.com>.
On 09/19/2014 06:21 PM, Spencer.Doak wrote:
> Sorry to be a bother, but I'm having two problems using the code you
> attached. The code is great, but when I go to create a queue, the queue is
> improperly named. Rather than using the name which is supplied, the queue
> created seems to be named using a UUID. The receiver also does not receiver
> a response from the broker when using the create command.
What version of client and broker are you using? I tested against trunk
admittedly, but most of this hasn't changed for a while and it works for me.
qmf_ctrl create queue my-queue
results on a queue named my-queue being created on the broker and the
program print 'ok'.
Can you turn logging on for the broker e.g. qpidd --log-enable trace+,
then run qmf_ctrl create queue my-queue, then send me the broker log?
> As a test, I wrote up the attached Python method to create a queue by name.
> It appears that the python method is able to create a queue with the correct
> name, but for some reason the C++ function is not able to.
>
> The receiver is not as big of an issue, though being able to receive a
> return value from the broker would of course be very useful.
>
> Would you be able to provide any clarification on why these errors occur and
> what can be done to fix them? Excuse my naivety.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: QPID C++ - Dynamically Managing Broker
Posted by "Spencer.Doak" <Sp...@live.com>.
Hey Gordon,
Sorry to be a bother, but I'm having two problems using the code you
attached. The code is great, but when I go to create a queue, the queue is
improperly named. Rather than using the name which is supplied, the queue
created seems to be named using a UUID. The receiver also does not receiver
a response from the broker when using the create command.
As a test, I wrote up the attached Python method to create a queue by name.
It appears that the python method is able to create a queue with the correct
name, but for some reason the C++ function is not able to.
The receiver is not as big of an issue, though being able to receive a
return value from the broker would of course be very useful.
Would you be able to provide any clarification on why these errors occur and
what can be done to fix them? Excuse my naivety.
Thank you,
Spencer
qmf_create_queue.py
<http://qpid.2158936.n2.nabble.com/file/n7613914/qmf_create_queue.py>
--
View this message in context: http://qpid.2158936.n2.nabble.com/QPID-C-Dynamically-Managing-Broker-tp7613792p7613914.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Add qmf shutdown command to the broker [was Re: QPID C++ -
Dynamically Managing Broker]
Posted by Alan Conway <ac...@redhat.com>.
On Wed, 2014-09-24 at 14:00 -0400, Chuck Rolke wrote:
> This makes sense as a useful command:
>
> 1. Must have ACL control.
> 2. Only do the 'most orderly shutdown' as if SIGINT or SIGTERM was received.
> 3. No RestartBroker switch.
>
+1 all of the above.
>
> ----- Original Message -----
> > From: "Alan Conway" <ac...@redhat.com>
> > To: users@qpid.apache.org
> > Cc: "Chuck Rolke" <cr...@redhat.com>, "Gordon Sim" <gs...@redhat.com>
> > Sent: Monday, September 22, 2014 4:10:12 PM
> > Subject: Add qmf shutdown command to the broker [was Re: QPID C++ - Dynamically Managing Broker]
> >
> > On Thu, 2014-09-18 at 15:12 -0700, Spencer.Doak wrote:
> > > Hey Gordon,
> > >
> > > Thank you very much! That should give me a great start on this task.
> > >
> > > As for the 'shutdown' command, that's actually exactly what I was thinking
> > > too. I'm thinking about running a receiver process on the broker machine.
> > > When it receives a message that says "shutdown" from an authenticated user,
> > > it will perform 'system("/sbin/service qpid-stop");' or whatever the
> > > relevant OS command is. In your opinion, is this a reasonable way to
> > > accomplish this task? Would there perhaps be a better way than creating a
> > > system call?
> >
> > Not presently. I've long thought we should have a qmf shutdown command
> > on the broker but never actually did anything about it. Mucking about
> > with extra processes is painful for such a basic task.
> >
> > There is a denial of service security concern, my feeling is that adding
> > a "shutdown" permission to the ACL rules would cover that.
> >
> > Anyone think this is a bad idea, or have a better idea?
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> > For additional commands, e-mail: users-help@qpid.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Add qmf shutdown command to the broker [was Re: QPID C++ -
Dynamically Managing Broker]
Posted by Chuck Rolke <cr...@redhat.com>.
This makes sense as a useful command:
1. Must have ACL control.
2. Only do the 'most orderly shutdown' as if SIGINT or SIGTERM was received.
3. No RestartBroker switch.
----- Original Message -----
> From: "Alan Conway" <ac...@redhat.com>
> To: users@qpid.apache.org
> Cc: "Chuck Rolke" <cr...@redhat.com>, "Gordon Sim" <gs...@redhat.com>
> Sent: Monday, September 22, 2014 4:10:12 PM
> Subject: Add qmf shutdown command to the broker [was Re: QPID C++ - Dynamically Managing Broker]
>
> On Thu, 2014-09-18 at 15:12 -0700, Spencer.Doak wrote:
> > Hey Gordon,
> >
> > Thank you very much! That should give me a great start on this task.
> >
> > As for the 'shutdown' command, that's actually exactly what I was thinking
> > too. I'm thinking about running a receiver process on the broker machine.
> > When it receives a message that says "shutdown" from an authenticated user,
> > it will perform 'system("/sbin/service qpid-stop");' or whatever the
> > relevant OS command is. In your opinion, is this a reasonable way to
> > accomplish this task? Would there perhaps be a better way than creating a
> > system call?
>
> Not presently. I've long thought we should have a qmf shutdown command
> on the broker but never actually did anything about it. Mucking about
> with extra processes is painful for such a basic task.
>
> There is a denial of service security concern, my feeling is that adding
> a "shutdown" permission to the ACL rules would cover that.
>
> Anyone think this is a bad idea, or have a better idea?
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Add qmf shutdown command to the broker [was Re: QPID C++ -
Dynamically Managing Broker]
Posted by Pavel Moravec <pm...@redhat.com>.
Kind regards,
Pavel
----- Original Message -----
> From: "Alan Conway" <ac...@redhat.com>
> To: users@qpid.apache.org
> Cc: "Chuck Rolke" <cr...@redhat.com>, "Gordon Sim" <gs...@redhat.com>
> Sent: Monday, September 22, 2014 10:10:12 PM
> Subject: Add qmf shutdown command to the broker [was Re: QPID C++ - Dynamically Managing Broker]
>
> On Thu, 2014-09-18 at 15:12 -0700, Spencer.Doak wrote:
> > Hey Gordon,
> >
> > Thank you very much! That should give me a great start on this task.
> >
> > As for the 'shutdown' command, that's actually exactly what I was thinking
> > too. I'm thinking about running a receiver process on the broker machine.
> > When it receives a message that says "shutdown" from an authenticated user,
> > it will perform 'system("/sbin/service qpid-stop");' or whatever the
> > relevant OS command is. In your opinion, is this a reasonable way to
> > accomplish this task? Would there perhaps be a better way than creating a
> > system call?
>
> Not presently. I've long thought we should have a qmf shutdown command
> on the broker but never actually did anything about it. Mucking about
> with extra processes is painful for such a basic task.
>
> There is a denial of service security concern, my feeling is that adding
> a "shutdown" permission to the ACL rules would cover that.
Note you can already delete anything from provisioning via QMF (until prevented by authentication and ACLs). This in my eyes is bigger security concern than possibility to shut down a broker. As deleting a durable queue, one loses all (durable) messages in.
Also, what is the difference between connecting to AMQP listening port, providing proper (stolen?) credentials and shutting the broker via QMF, and doing the same via ssh "killall qpidd"? (until you disable ssh from outside network where AMQP is allowed)
So I would be in favour of adding the shutdown QMF command and relevant ACL action.
>
> Anyone think this is a bad idea, or have a better idea?
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Add qmf shutdown command to the broker [was Re: QPID C++ -
Dynamically Managing Broker]
Posted by Alan Conway <ac...@redhat.com>.
On Wed, 2014-09-24 at 10:40 -0700, Spencer.Doak wrote:
> Hi,
>
> Well as you know, I'm definitely in support of this addition.
>
> I do not know how the exact details about the QMF listener, but I would
> imagine it could receive the shutdown message and do something like
> "std::exit(1);". Of course there might be some necessary clean-up before
> calling the exit command; however, I would guess that it's probably a pretty
> easy command to implement.
>
Yes pretty easy, not quite exit(1) more of a broker->shutdown() ;)
Biggest bit of work (for me) would be adding the ACL rule. Maybe
someone who knows the ACL stuff would like to take it?
I've JIRAed it, further suggestions can go on the JIRA:
https://issues.apache.org/jira/browse/QPID-6118
It's assigned to me for now but like I said if anyone who knows the ACL
stuff wants it, please be my guest!
Cheers,
Alan.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Add qmf shutdown command to the broker [was Re: QPID C++ -
Dynamically Managing Broker]
Posted by "Spencer.Doak" <Sp...@live.com>.
Hi,
Well as you know, I'm definitely in support of this addition.
I do not know how the exact details about the QMF listener, but I would
imagine it could receive the shutdown message and do something like
"std::exit(1);". Of course there might be some necessary clean-up before
calling the exit command; however, I would guess that it's probably a pretty
easy command to implement.
Best Regards,
Spencer
--
View this message in context: http://qpid.2158936.n2.nabble.com/QPID-C-Dynamically-Managing-Broker-tp7613792p7614174.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Add qmf shutdown command to the broker [was Re: QPID C++ -
Dynamically Managing Broker]
Posted by Alan Conway <ac...@redhat.com>.
On Wed, 2014-09-24 at 20:29 +0200, Jakub Scholz wrote:
> What do you expect the shutdown call to do in clustered brokers? Shutdown
> the whole cluster or just the active node?
Just the active node. I would think of it as being equivalent to a
remote kill -TERM. Useful if you don't have ssh or other admin access to
stop the service.
In a cluster it would be the same as a broker crashing or being killed
locally. For new HA under rgmanager, the broker would be restarted. I
don't see it as being useful in a cluster though, cluster suite already
has tools for remote management of clustered services.
>
> Personally, I don't really see the value in it. But at the same time - if
> it is properly secured with ACLs - I don't think its a big security issue.
>
> Regards
> Jakub
>
> On Mon, Sep 22, 2014 at 10:10 PM, Alan Conway <ac...@redhat.com> wrote:
>
> > On Thu, 2014-09-18 at 15:12 -0700, Spencer.Doak wrote:
> > > Hey Gordon,
> > >
> > > Thank you very much! That should give me a great start on this task.
> > >
> > > As for the 'shutdown' command, that's actually exactly what I was
> > thinking
> > > too. I'm thinking about running a receiver process on the broker machine.
> > > When it receives a message that says "shutdown" from an authenticated
> > user,
> > > it will perform 'system("/sbin/service qpid-stop");' or whatever the
> > > relevant OS command is. In your opinion, is this a reasonable way to
> > > accomplish this task? Would there perhaps be a better way than creating a
> > > system call?
> >
> > Not presently. I've long thought we should have a qmf shutdown command
> > on the broker but never actually did anything about it. Mucking about
> > with extra processes is painful for such a basic task.
> >
> > There is a denial of service security concern, my feeling is that adding
> > a "shutdown" permission to the ACL rules would cover that.
> >
> > Anyone think this is a bad idea, or have a better idea?
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> > For additional commands, e-mail: users-help@qpid.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Add qmf shutdown command to the broker [was Re: QPID C++ -
Dynamically Managing Broker]
Posted by Jakub Scholz <ja...@scholz.cz>.
What do you expect the shutdown call to do in clustered brokers? Shutdown
the whole cluster or just the active node?
Personally, I don't really see the value in it. But at the same time - if
it is properly secured with ACLs - I don't think its a big security issue.
Regards
Jakub
On Mon, Sep 22, 2014 at 10:10 PM, Alan Conway <ac...@redhat.com> wrote:
> On Thu, 2014-09-18 at 15:12 -0700, Spencer.Doak wrote:
> > Hey Gordon,
> >
> > Thank you very much! That should give me a great start on this task.
> >
> > As for the 'shutdown' command, that's actually exactly what I was
> thinking
> > too. I'm thinking about running a receiver process on the broker machine.
> > When it receives a message that says "shutdown" from an authenticated
> user,
> > it will perform 'system("/sbin/service qpid-stop");' or whatever the
> > relevant OS command is. In your opinion, is this a reasonable way to
> > accomplish this task? Would there perhaps be a better way than creating a
> > system call?
>
> Not presently. I've long thought we should have a qmf shutdown command
> on the broker but never actually did anything about it. Mucking about
> with extra processes is painful for such a basic task.
>
> There is a denial of service security concern, my feeling is that adding
> a "shutdown" permission to the ACL rules would cover that.
>
> Anyone think this is a bad idea, or have a better idea?
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Add qmf shutdown command to the broker [was Re: QPID C++ -
Dynamically Managing Broker]
Posted by Alan Conway <ac...@redhat.com>.
On Thu, 2014-09-18 at 15:12 -0700, Spencer.Doak wrote:
> Hey Gordon,
>
> Thank you very much! That should give me a great start on this task.
>
> As for the 'shutdown' command, that's actually exactly what I was thinking
> too. I'm thinking about running a receiver process on the broker machine.
> When it receives a message that says "shutdown" from an authenticated user,
> it will perform 'system("/sbin/service qpid-stop");' or whatever the
> relevant OS command is. In your opinion, is this a reasonable way to
> accomplish this task? Would there perhaps be a better way than creating a
> system call?
Not presently. I've long thought we should have a qmf shutdown command
on the broker but never actually did anything about it. Mucking about
with extra processes is painful for such a basic task.
There is a denial of service security concern, my feeling is that adding
a "shutdown" permission to the ACL rules would cover that.
Anyone think this is a bad idea, or have a better idea?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: QPID C++ - Dynamically Managing Broker
Posted by "Spencer.Doak" <Sp...@live.com>.
Hey Gordon,
Thank you very much! That should give me a great start on this task.
As for the 'shutdown' command, that's actually exactly what I was thinking
too. I'm thinking about running a receiver process on the broker machine.
When it receives a message that says "shutdown" from an authenticated user,
it will perform 'system("/sbin/service qpid-stop");' or whatever the
relevant OS command is. In your opinion, is this a reasonable way to
accomplish this task? Would there perhaps be a better way than creating a
system call?
Thank you in advance,
Spencer
--
View this message in context: http://qpid.2158936.n2.nabble.com/QPID-C-Dynamically-Managing-Broker-tp7613792p7613863.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: QPID C++ - Dynamically Managing Broker
Posted by Gordon Sim <gs...@redhat.com>.
On 09/18/2014 12:56 AM, Spencer.Doak wrote:
> Hello,
>
> I'm faced with a bit of an issue. For an application I'm working on, I'm in
> need of several abilities:
>
> 1. Create, destroy, and clear queues on an exchange.
> 2. Create and destroy exchanges on a broker.
> 3. Fetch a list of all existing connections on a broker.
> 4. If possible, send a management message telling the broker to shutdown.
> 5. Verify the existence of a queue. (I have a workaround for this)
>
> Based on what I've been reading, I believe QMF would be the way to go for
> this. The issue is that I cannot seem to find much information about QMF
> despite all of the digging I've been doing.
Yes, sorry about that, it is indeed not documented very well at all.
Fortunately, it's quite simple.
All you do is send specially structured map messages to the broker of
interest, and then process the replies that come back.
There are two categories of request. The first is a method request, the
two methods of most interest being 'create' and 'delete'. With these you
can create or delete queues, exchanges and bindings. The second category
is a query request. With these you get back the objects of a particular
type (e.g. connection, queue, exchange) or indeed a specific object by id.
All qmf request messages are sent to 'qmf.default.direct/broker '(i.e.
to the exchange named 'qmf.default.direct', with a routing key or
subject of 'broker'). The message should contain a reply-to address
through which responses will be sent.
There are two message properties that must also be set. These are
'x-amqp-0-10.app-id', which should always have the value 'qmf2' and
'qmf.opcode' which for method requests should always have the value
'_method_request' and for query requests should have the value
'_query_request'.
For a method requests the content is a map with three top-level entries.
The first is an entry with key _object_id whose value is a nested map
identifying the target of the command. For the commands considered here
(create and delete) the target is always the broker itself. Thus the
_object_id map contains a single value with key _object_name and value
org.apache.qpid.broker:broker:amqp-broker. The second is an entry with
key '_method_name' whose value is the method name (i.e. create or
delete). The third is an entry with key '_arguments' which is a nested
map represention the arguments to the method. For create and delete, the
name and type arguments are required. For create you often want an
additional 'properties' entry, whose value is a nested map containing
the properties of the object you want to create (e.g. for a queue
qpid.max_count or similar).
For a query request the content is a map with one entry named
'_schema_id' that usually contains a nested map with one entry,
'_class_name' containing the type of object to query, e.g. queue,
exchange etc. In addition there is an entry with key '_what' and value
'OBJECT'.
The expected response for a method request will have the qmf.opcode
property set to _method_response. For a query request it will be
_query_response.
I've attached a very rudimentary example that shows create, delete and
querying which I suspect is easier to follow then the textual
description of the structure. To use it specify one of create, delete or
list and a type (queue, exchange, binding etc). For create and delete
you also need a 'name'. The 'name' for a binding is a concatenation of
exchange name, queue name and key name, with '/' separating them. E.g.
qmf_ctrl create queue my-queue
qmf_ctrl create exchange my-exchange
qmf_ctrl create binding my-exchange/my-queue/my-key
qmf_ctrl create queue another-queue
qmf_ctrl list queue
qmf_ctrl create queue another-queue
qmf_ctrl list queue
qmf_ctrl list connection
Hopefully this is enough to get you started. If you have further
questions please don't hesitate to ask.
Regarding point 4. in your mail, there is at present no way to shutdown
the broker via qmf. You could perhaps create a special 'agent' program
to do that. I.e. it would run colocated with the broker, and listen on a
special address for a shutdown message, on receiving which it would send
a signal to the broker. You might want some sort of security around
that, either message based or acl controls on the address in question
for the broker.
--Gordon.