You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Fraser Adams <fr...@blueyonder.co.uk> on 2017/04/24 16:43:26 UTC
Does anyone recall a significant performance gain between qpid 0.22
and 0.24?
Hi All,
For context I'm playing around with writing a Dockerised build system to
build Dockerised qpid components. I've been focussing on qpid cpp and
seem to have ironed out most of the quirks between versions.
As I was trying out building some older versions I noticed that 0.22 and
below seem to be significantly slower than 0.24 and above. 0.24 to trunk
seem to be consistent with each other and 0.8 to 0.22 seem to be
consistent with each, other but I'm seeing a big leap (something like an
order of magnitude improvement) between 0.22 and 0.24.
The test is a pretty simple producer/consumer using largish messages
(50KB) using the qpid::messaging C++ API - identical test for each version.
One thing I did notice was that when running top for 0.22 and below
qpidd was only registering ~42% but for 0.24 upwards it was registering
~150%, but the producer and consumer clients are single threaded. At
face value this hints at qpidd doing more blocking in 0.22 and below,
but it'd be good to get some thoughts from people who may have better
insight. I'm using ring queues so I don't believe it's anything to do
with flow control behaviours.
It might be something to do with the build system, but the way I build
0.22 and 0.24 is the same, moreover for very old versions (0.12 and
below) I found the CMake build unreliable so I fell back to automake but
the performance of those earlier versions is still roughly consistent
with 0.14 to 0.22 which are built using CMake, if I was seeing different
figures between the CMake versions and automake versions I'd be more
suspicious it was build system related, but that big difference in CPU
usage between 0.22 and 0.24 definitely makes me suspicious that 0.24
onwards has had some improvements with respect to blocking behaviour.
Kind of weird and a bit archaeological, but I'd be grateful for any
thoughts.
Frase
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Does anyone recall a significant performance gain between qpid
0.22 and 0.24?
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
As an update to this it *appears* to be a difference in the client
runtime (messaging/client) more than the broker, that is to say I ran a
test where I was using 0.24 qpidd but the client libraries from 0.22 and
I was seeing the lower performance (and lower qpidd CPU utilisation I
mentioned below). I subsequently tried using 0.22 qpidd but the client
libraries from 0.24 and I saw the higher performance (and 150% qpidd
CPU utilisation) that I had observed below. The actual produce/consumer
application code is the same I'm just building with different versions
of the client libraries.
F.
On 24/04/17 22:59, Fraser Adams wrote:
> Hi Gordon,
>
> It's a pretty simple test with AMQP 0.10 non-persisted messages.
>
> The only obvious thing I noticed was the difference in CPU utilisation
> noted from top with the higher throughput versions reporting higher
> CPU utilisations. Both producer and consumer are single threaded and
> the application code is identical it's just the qpid versions that
> have varied.
>
>
> On 24/04/17 21:08, Gordon Sim wrote:
>> On 24/04/17 17:43, Fraser Adams wrote:
>>> Hi All,
>>>
>>> For context I'm playing around with writing a Dockerised build
>>> system to
>>> build Dockerised qpid components. I've been focussing on qpid cpp and
>>> seem to have ironed out most of the quirks between versions.
>>>
>>> As I was trying out building some older versions I noticed that 0.22
>>> and
>>> below seem to be significantly slower than 0.24 and above. 0.24 to
>>> trunk
>>> seem to be consistent with each other and 0.8 to 0.22 seem to be
>>> consistent with each, other but I'm seeing a big leap (something
>>> like an
>>> order of magnitude improvement) between 0.22 and 0.24.
>>>
>>> The test is a pretty simple producer/consumer using largish messages
>>> (50KB) using the qpid::messaging C++ API - identical test for each
>>> version.
>>
>> Is it using AMQP 0-10 or 1.0? Are the messages durable (and persisted)?
>>
>>> One thing I did notice was that when running top for 0.22 and below
>>> qpidd was only registering ~42% but for 0.24 upwards it was registering
>>> ~150%, but the producer and consumer clients are single threaded. At
>>> face value this hints at qpidd doing more blocking in 0.22 and below,
>>> but it'd be good to get some thoughts from people who may have better
>>> insight. I'm using ring queues so I don't believe it's anything to do
>>> with flow control behaviours.
>>>
>>> It might be something to do with the build system, but the way I build
>>> 0.22 and 0.24 is the same, moreover for very old versions (0.12 and
>>> below) I found the CMake build unreliable so I fell back to automake
>>> but
>>> the performance of those earlier versions is still roughly consistent
>>> with 0.14 to 0.22 which are built using CMake, if I was seeing
>>> different
>>> figures between the CMake versions and automake versions I'd be more
>>> suspicious it was build system related, but that big difference in CPU
>>> usage between 0.22 and 0.24 definitely makes me suspicious that 0.24
>>> onwards has had some improvements with respect to blocking behaviour.
>>>
>>>
>>> Kind of weird and a bit archaeological, but I'd be grateful for any
>>> thoughts.
>>
>> The only things I current recall are some of the commits for the AMQP
>> 1.0 path in https://issues.apache.org/jira/browse/QPID-6021, however
>> from what I can tell 0.24 was released before any of them were made
>> so unlikely to be the issue.
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Does anyone recall a significant performance gain between qpid
0.22 and 0.24?
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
Hi Gordon,
It's a pretty simple test with AMQP 0.10 non-persisted messages.
The only obvious thing I noticed was the difference in CPU utilisation
noted from top with the higher throughput versions reporting higher CPU
utilisations. Both producer and consumer are single threaded and the
application code is identical it's just the qpid versions that have varied.
On 24/04/17 21:08, Gordon Sim wrote:
> On 24/04/17 17:43, Fraser Adams wrote:
>> Hi All,
>>
>> For context I'm playing around with writing a Dockerised build system to
>> build Dockerised qpid components. I've been focussing on qpid cpp and
>> seem to have ironed out most of the quirks between versions.
>>
>> As I was trying out building some older versions I noticed that 0.22 and
>> below seem to be significantly slower than 0.24 and above. 0.24 to trunk
>> seem to be consistent with each other and 0.8 to 0.22 seem to be
>> consistent with each, other but I'm seeing a big leap (something like an
>> order of magnitude improvement) between 0.22 and 0.24.
>>
>> The test is a pretty simple producer/consumer using largish messages
>> (50KB) using the qpid::messaging C++ API - identical test for each
>> version.
>
> Is it using AMQP 0-10 or 1.0? Are the messages durable (and persisted)?
>
>> One thing I did notice was that when running top for 0.22 and below
>> qpidd was only registering ~42% but for 0.24 upwards it was registering
>> ~150%, but the producer and consumer clients are single threaded. At
>> face value this hints at qpidd doing more blocking in 0.22 and below,
>> but it'd be good to get some thoughts from people who may have better
>> insight. I'm using ring queues so I don't believe it's anything to do
>> with flow control behaviours.
>>
>> It might be something to do with the build system, but the way I build
>> 0.22 and 0.24 is the same, moreover for very old versions (0.12 and
>> below) I found the CMake build unreliable so I fell back to automake but
>> the performance of those earlier versions is still roughly consistent
>> with 0.14 to 0.22 which are built using CMake, if I was seeing different
>> figures between the CMake versions and automake versions I'd be more
>> suspicious it was build system related, but that big difference in CPU
>> usage between 0.22 and 0.24 definitely makes me suspicious that 0.24
>> onwards has had some improvements with respect to blocking behaviour.
>>
>>
>> Kind of weird and a bit archaeological, but I'd be grateful for any
>> thoughts.
>
> The only things I current recall are some of the commits for the AMQP
> 1.0 path in https://issues.apache.org/jira/browse/QPID-6021, however
> from what I can tell 0.24 was released before any of them were made so
> unlikely to be the issue.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Does anyone recall a significant performance gain between qpid
0.22 and 0.24?
Posted by Gordon Sim <gs...@redhat.com>.
On 24/04/17 17:43, Fraser Adams wrote:
> Hi All,
>
> For context I'm playing around with writing a Dockerised build system to
> build Dockerised qpid components. I've been focussing on qpid cpp and
> seem to have ironed out most of the quirks between versions.
>
> As I was trying out building some older versions I noticed that 0.22 and
> below seem to be significantly slower than 0.24 and above. 0.24 to trunk
> seem to be consistent with each other and 0.8 to 0.22 seem to be
> consistent with each, other but I'm seeing a big leap (something like an
> order of magnitude improvement) between 0.22 and 0.24.
>
> The test is a pretty simple producer/consumer using largish messages
> (50KB) using the qpid::messaging C++ API - identical test for each version.
Is it using AMQP 0-10 or 1.0? Are the messages durable (and persisted)?
> One thing I did notice was that when running top for 0.22 and below
> qpidd was only registering ~42% but for 0.24 upwards it was registering
> ~150%, but the producer and consumer clients are single threaded. At
> face value this hints at qpidd doing more blocking in 0.22 and below,
> but it'd be good to get some thoughts from people who may have better
> insight. I'm using ring queues so I don't believe it's anything to do
> with flow control behaviours.
>
> It might be something to do with the build system, but the way I build
> 0.22 and 0.24 is the same, moreover for very old versions (0.12 and
> below) I found the CMake build unreliable so I fell back to automake but
> the performance of those earlier versions is still roughly consistent
> with 0.14 to 0.22 which are built using CMake, if I was seeing different
> figures between the CMake versions and automake versions I'd be more
> suspicious it was build system related, but that big difference in CPU
> usage between 0.22 and 0.24 definitely makes me suspicious that 0.24
> onwards has had some improvements with respect to blocking behaviour.
>
>
> Kind of weird and a bit archaeological, but I'd be grateful for any
> thoughts.
The only things I current recall are some of the commits for the AMQP
1.0 path in https://issues.apache.org/jira/browse/QPID-6021, however
from what I can tell 0.24 was released before any of them were made so
unlikely to be the issue.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org