You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Pavel Moravec <pm...@redhat.com> on 2011/07/20 16:43:56 UTC
qpid-config slow
Hi all,
I realised on every qpid version & system I used a poor performance of qpid-config tool. It usually takes 2 to 3 seconds to execute a command, e.g.:
# time qpid-config add queue test
real 0m2.243s
user 0m0.161s
sys 0m0.027s
# time qpid-config del queue test
real 0m2.303s
user 0m0.208s
sys 0m0.037s
#
I wrote a simple C++ program to emulate some parts of qpid-config tool by connecting to the broker and sending proper QMF message(s). The program executes equivalent stuff almost instantly.
Why is qpid-config so slow? Is there some timeout waiting for (potentially negative) response? Does it make sense / would not it be usefull to speed it up?
Kind regards,
Pavel
Re: qpid-config slow
Posted by Ken Giusti <kg...@redhat.com>.
> Ken,
>
> QMFv2 solves this issue, can't we port the tools to QMFv2?
>
We're already part way there - the tools use the pure python qmf implementation, which supports v2 map data. There may be issues with using the tools against older v1-only brokers - I'd have to think about that.
But I'd really want to pinpoint the source of the perf issue first - my theory could be wrong.
> Carl.
>
>
> On 07/21/2011 01:47 PM, Ken Giusti wrote:
> > Alan's correct - I *believe* (not looking at the code in the
> > debugger ATM) that qpid-config is most likely downloading the broker
> > schema data on connection, and perhaps parsing that - all under
> > python.
> >
> > So I don't think bringing up the connection is the problem - it's
> > more like pulling in all the QMF-related data once we're connected,
> > but before we can issue commands.
> >
> > No doubt that whole approach could use a little performance
> > tuning....
> >
> > -K
> >
> > ----- Original Message -----
> >> On 07/20/2011 10:43 AM, Pavel Moravec wrote:
> >>> Hi all,
> >>> I realised on every qpid version& system I used a poor performance
> >>> of qpid-config tool. It usually takes 2 to 3 seconds to execute a
> >>> command, e.g.:
> >>>
> >>> # time qpid-config add queue test
> >>>
> >>> real 0m2.243s
> >>> user 0m0.161s
> >>> sys 0m0.027s
> >>> # time qpid-config del queue test
> >>>
> >>> real 0m2.303s
> >>> user 0m0.208s
> >>> sys 0m0.037s
> >>> #
> >>>
> >>> I wrote a simple C++ program to emulate some parts of qpid-config
> >>> tool by connecting to the broker and sending proper QMF
> >>> message(s).
> >>> The program executes equivalent stuff almost instantly.
> >>>
> >>> Why is qpid-config so slow? Is there some timeout waiting for
> >>> (potentially negative) response? Does it make sense / would not it
> >>> be usefull to speed it up?
> >>>
> >> It's an issue with QMF. Setting up the initial QMF connection is
> >> very
> >> slow. I'm
> >> not sure why. It most definitely would be good to speed it up, it
> >> annoys me no end.
> >>
> >> If you have a large number of operations to do, you can write your
> >> own
> >> client in
> >> python that sets up the QMF connection once, then uses it to do all
> >> the work.
> >> That's not particularly easy right now, qpid-config is not set up
> >> to
> >> be reused
> >> like that. It would be nice to have a little python API that
> >> mirrors
> >> the command
> >> line arguments to qpid-config. It would be even nicer to fix the
> >> real
> >> problem by
> >> getting the QMF init time down to something reasonable.
> >>
> >> ---------------------------------------------------------------------
> >> Apache Qpid - AMQP Messaging Implementation
> >> Project: http://qpid.apache.org
> >> Use/Interact: mailto:users-subscribe@qpid.apache.org
> > ---------------------------------------------------------------------
> > Apache Qpid - AMQP Messaging Implementation
> > Project: http://qpid.apache.org
> > Use/Interact: mailto:users-subscribe@qpid.apache.org
> >
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: qpid-config slow
Posted by Ken Giusti <kg...@redhat.com>.
> Ken,
>
> QMFv2 solves this issue, can't we port the tools to QMFv2?
>
We're already part way there - the tools use the pure python qmf implementation, which supports v2 map data. There may be issues with using the tools against older v1-only brokers - I'd have to think about that.
But I'd really want to pinpoint the source of the perf issue first - my theory could be wrong.
> Carl.
>
>
> On 07/21/2011 01:47 PM, Ken Giusti wrote:
> > Alan's correct - I *believe* (not looking at the code in the
> > debugger ATM) that qpid-config is most likely downloading the broker
> > schema data on connection, and perhaps parsing that - all under
> > python.
> >
> > So I don't think bringing up the connection is the problem - it's
> > more like pulling in all the QMF-related data once we're connected,
> > but before we can issue commands.
> >
> > No doubt that whole approach could use a little performance
> > tuning....
> >
> > -K
> >
> > ----- Original Message -----
> >> On 07/20/2011 10:43 AM, Pavel Moravec wrote:
> >>> Hi all,
> >>> I realised on every qpid version& system I used a poor performance
> >>> of qpid-config tool. It usually takes 2 to 3 seconds to execute a
> >>> command, e.g.:
> >>>
> >>> # time qpid-config add queue test
> >>>
> >>> real 0m2.243s
> >>> user 0m0.161s
> >>> sys 0m0.027s
> >>> # time qpid-config del queue test
> >>>
> >>> real 0m2.303s
> >>> user 0m0.208s
> >>> sys 0m0.037s
> >>> #
> >>>
> >>> I wrote a simple C++ program to emulate some parts of qpid-config
> >>> tool by connecting to the broker and sending proper QMF
> >>> message(s).
> >>> The program executes equivalent stuff almost instantly.
> >>>
> >>> Why is qpid-config so slow? Is there some timeout waiting for
> >>> (potentially negative) response? Does it make sense / would not it
> >>> be usefull to speed it up?
> >>>
> >> It's an issue with QMF. Setting up the initial QMF connection is
> >> very
> >> slow. I'm
> >> not sure why. It most definitely would be good to speed it up, it
> >> annoys me no end.
> >>
> >> If you have a large number of operations to do, you can write your
> >> own
> >> client in
> >> python that sets up the QMF connection once, then uses it to do all
> >> the work.
> >> That's not particularly easy right now, qpid-config is not set up
> >> to
> >> be reused
> >> like that. It would be nice to have a little python API that
> >> mirrors
> >> the command
> >> line arguments to qpid-config. It would be even nicer to fix the
> >> real
> >> problem by
> >> getting the QMF init time down to something reasonable.
> >>
> >> ---------------------------------------------------------------------
> >> Apache Qpid - AMQP Messaging Implementation
> >> Project: http://qpid.apache.org
> >> Use/Interact: mailto:users-subscribe@qpid.apache.org
> > ---------------------------------------------------------------------
> > Apache Qpid - AMQP Messaging Implementation
> > Project: http://qpid.apache.org
> > Use/Interact: mailto:users-subscribe@qpid.apache.org
> >
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: qpid-config slow
Posted by Carl Trieloff <cc...@redhat.com>.
Ken,
QMFv2 solves this issue, can't we port the tools to QMFv2?
Carl.
On 07/21/2011 01:47 PM, Ken Giusti wrote:
> Alan's correct - I *believe* (not looking at the code in the debugger ATM) that qpid-config is most likely downloading the broker schema data on connection, and perhaps parsing that - all under python.
>
> So I don't think bringing up the connection is the problem - it's more like pulling in all the QMF-related data once we're connected, but before we can issue commands.
>
> No doubt that whole approach could use a little performance tuning....
>
> -K
>
> ----- Original Message -----
>> On 07/20/2011 10:43 AM, Pavel Moravec wrote:
>>> Hi all,
>>> I realised on every qpid version& system I used a poor performance
>>> of qpid-config tool. It usually takes 2 to 3 seconds to execute a
>>> command, e.g.:
>>>
>>> # time qpid-config add queue test
>>>
>>> real 0m2.243s
>>> user 0m0.161s
>>> sys 0m0.027s
>>> # time qpid-config del queue test
>>>
>>> real 0m2.303s
>>> user 0m0.208s
>>> sys 0m0.037s
>>> #
>>>
>>> I wrote a simple C++ program to emulate some parts of qpid-config
>>> tool by connecting to the broker and sending proper QMF message(s).
>>> The program executes equivalent stuff almost instantly.
>>>
>>> Why is qpid-config so slow? Is there some timeout waiting for
>>> (potentially negative) response? Does it make sense / would not it
>>> be usefull to speed it up?
>>>
>> It's an issue with QMF. Setting up the initial QMF connection is very
>> slow. I'm
>> not sure why. It most definitely would be good to speed it up, it
>> annoys me no end.
>>
>> If you have a large number of operations to do, you can write your own
>> client in
>> python that sets up the QMF connection once, then uses it to do all
>> the work.
>> That's not particularly easy right now, qpid-config is not set up to
>> be reused
>> like that. It would be nice to have a little python API that mirrors
>> the command
>> line arguments to qpid-config. It would be even nicer to fix the real
>> problem by
>> getting the QMF init time down to something reasonable.
>>
>> ---------------------------------------------------------------------
>> Apache Qpid - AMQP Messaging Implementation
>> Project: http://qpid.apache.org
>> Use/Interact: mailto:users-subscribe@qpid.apache.org
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: qpid-config slow
Posted by Carl Trieloff <cc...@redhat.com>.
Ken,
QMFv2 solves this issue, can't we port the tools to QMFv2?
Carl.
On 07/21/2011 01:47 PM, Ken Giusti wrote:
> Alan's correct - I *believe* (not looking at the code in the debugger ATM) that qpid-config is most likely downloading the broker schema data on connection, and perhaps parsing that - all under python.
>
> So I don't think bringing up the connection is the problem - it's more like pulling in all the QMF-related data once we're connected, but before we can issue commands.
>
> No doubt that whole approach could use a little performance tuning....
>
> -K
>
> ----- Original Message -----
>> On 07/20/2011 10:43 AM, Pavel Moravec wrote:
>>> Hi all,
>>> I realised on every qpid version& system I used a poor performance
>>> of qpid-config tool. It usually takes 2 to 3 seconds to execute a
>>> command, e.g.:
>>>
>>> # time qpid-config add queue test
>>>
>>> real 0m2.243s
>>> user 0m0.161s
>>> sys 0m0.027s
>>> # time qpid-config del queue test
>>>
>>> real 0m2.303s
>>> user 0m0.208s
>>> sys 0m0.037s
>>> #
>>>
>>> I wrote a simple C++ program to emulate some parts of qpid-config
>>> tool by connecting to the broker and sending proper QMF message(s).
>>> The program executes equivalent stuff almost instantly.
>>>
>>> Why is qpid-config so slow? Is there some timeout waiting for
>>> (potentially negative) response? Does it make sense / would not it
>>> be usefull to speed it up?
>>>
>> It's an issue with QMF. Setting up the initial QMF connection is very
>> slow. I'm
>> not sure why. It most definitely would be good to speed it up, it
>> annoys me no end.
>>
>> If you have a large number of operations to do, you can write your own
>> client in
>> python that sets up the QMF connection once, then uses it to do all
>> the work.
>> That's not particularly easy right now, qpid-config is not set up to
>> be reused
>> like that. It would be nice to have a little python API that mirrors
>> the command
>> line arguments to qpid-config. It would be even nicer to fix the real
>> problem by
>> getting the QMF init time down to something reasonable.
>>
>> ---------------------------------------------------------------------
>> Apache Qpid - AMQP Messaging Implementation
>> Project: http://qpid.apache.org
>> Use/Interact: mailto:users-subscribe@qpid.apache.org
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: qpid-config slow
Posted by Ken Giusti <kg...@redhat.com>.
Alan's correct - I *believe* (not looking at the code in the debugger ATM) that qpid-config is most likely downloading the broker schema data on connection, and perhaps parsing that - all under python.
So I don't think bringing up the connection is the problem - it's more like pulling in all the QMF-related data once we're connected, but before we can issue commands.
No doubt that whole approach could use a little performance tuning....
-K
----- Original Message -----
> On 07/20/2011 10:43 AM, Pavel Moravec wrote:
> > Hi all,
> > I realised on every qpid version& system I used a poor performance
> > of qpid-config tool. It usually takes 2 to 3 seconds to execute a
> > command, e.g.:
> >
> > # time qpid-config add queue test
> >
> > real 0m2.243s
> > user 0m0.161s
> > sys 0m0.027s
> > # time qpid-config del queue test
> >
> > real 0m2.303s
> > user 0m0.208s
> > sys 0m0.037s
> > #
> >
> > I wrote a simple C++ program to emulate some parts of qpid-config
> > tool by connecting to the broker and sending proper QMF message(s).
> > The program executes equivalent stuff almost instantly.
> >
> > Why is qpid-config so slow? Is there some timeout waiting for
> > (potentially negative) response? Does it make sense / would not it
> > be usefull to speed it up?
> >
>
> It's an issue with QMF. Setting up the initial QMF connection is very
> slow. I'm
> not sure why. It most definitely would be good to speed it up, it
> annoys me no end.
>
> If you have a large number of operations to do, you can write your own
> client in
> python that sets up the QMF connection once, then uses it to do all
> the work.
> That's not particularly easy right now, qpid-config is not set up to
> be reused
> like that. It would be nice to have a little python API that mirrors
> the command
> line arguments to qpid-config. It would be even nicer to fix the real
> problem by
> getting the QMF init time down to something reasonable.
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: qpid-config slow
Posted by Alan Conway <ac...@redhat.com>.
On 07/20/2011 10:43 AM, Pavel Moravec wrote:
> Hi all,
> I realised on every qpid version& system I used a poor performance of qpid-config tool. It usually takes 2 to 3 seconds to execute a command, e.g.:
>
> # time qpid-config add queue test
>
> real 0m2.243s
> user 0m0.161s
> sys 0m0.027s
> # time qpid-config del queue test
>
> real 0m2.303s
> user 0m0.208s
> sys 0m0.037s
> #
>
> I wrote a simple C++ program to emulate some parts of qpid-config tool by connecting to the broker and sending proper QMF message(s). The program executes equivalent stuff almost instantly.
>
> Why is qpid-config so slow? Is there some timeout waiting for (potentially negative) response? Does it make sense / would not it be usefull to speed it up?
>
It's an issue with QMF. Setting up the initial QMF connection is very slow. I'm
not sure why. It most definitely would be good to speed it up, it annoys me no end.
If you have a large number of operations to do, you can write your own client in
python that sets up the QMF connection once, then uses it to do all the work.
That's not particularly easy right now, qpid-config is not set up to be reused
like that. It would be nice to have a little python API that mirrors the command
line arguments to qpid-config. It would be even nicer to fix the real problem by
getting the QMF init time down to something reasonable.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: qpid-config slow
Posted by fadams <fr...@blueyonder.co.uk>.
My guess is that it's simply due to the interpreted nature of python.
I've been writing a Java port of qpid-config that uses QMF2 and that
similarly takes a little time to give results, but that's pretty much down
to the JVM startup. I wrote code to listen out for QMF Events and that
responds to Events really quickly - and my Event handler is doing the same
sort of thing as qpid-config - b queues, so I'd be willing to bet that it is
just due to the time taken to fire up the python interpreter.
Regards,
Fraser
Pavel Moravec wrote:
>
> Hi all,
> I realised on every qpid version & system I used a poor performance of
> qpid-config tool. It usually takes 2 to 3 seconds to execute a command,
> e.g.:
>
> Kind regards,
> Pavel
>
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/qpid-config-slow-tp6602945p6603440.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: qpid-config slow
Posted by NimbusParc <ni...@yahoo.in>.
Upgrade your qpid-tools.( Download from Official Website
<http://www.apache.org/dyn/closer.cgi/qpid/0.24/qpid-tools-0.24.tar.gz> .)
Also specify the broker while using the qpid-tools.
if you specify the * -a * option it will take less than a second. Otherwise,
it will take time and will be slow.
(i.e). To create a queue...* $ qpid-config add queue my_queue -a
[broker-address] *
RajendraNPTL wrote
> Ya.. it is too slow.. For me it is taking 7 sec nearly..
>
> real 0m6.933s
> user 0m0.319s
> sys 0m0.028s
--
View this message in context: http://qpid.2158936.n2.nabble.com/qpid-config-slow-tp6602945p7598205.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org