You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Andrew Wright <at...@mac.com> on 2010/04/07 21:36:06 UTC
Re: Clustering overhead
On 31 Mar 2010, at 14:30, Andrew Wright wrote:
> On 31 Mar 2010, at 14:02, Alan Conway wrote:
>
>> On 03/30/2010 05:38 PM, Andrew Wright wrote:
>>> Hi all,
>>>
>>> I've recently run some tests to try and see how much overhead
>>> clustering
>>> brings. In short - I saw roughly a 50% reduction in message
>>> throughput
>>> when clients ran against a 2 node cluster vs a standalone broker.
>>>
>>
>> I get about a 25% reduction running against a 4 node cluster.
>> Try --worker-threads=4.
<snip>
Just a quick follow-up - setting worker-threads=4 noticeably improved
throughput on a single queue in both the standalone and clustered
scenarios. It reduced total throughput somewhat when loading 8 queues
simultaneously (which you'd expect). The ratio of standalone:clustered
throughput remained at about 2:1. Will let know if we get any further
interesting results.
Cheers,
Andrew
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Clustering overhead
Posted by Carl Trieloff <cc...@redhat.com>.
On 04/09/2010 01:39 PM, Andrew Wright wrote:
>
> On 8 Apr 2010, at 13:16, Carl Trieloff wrote:
>
>> On 04/07/2010 03:36 PM, Andrew Wright wrote:
>>>
>>> On 31 Mar 2010, at 14:30, Andrew Wright wrote:
>>>
>>>> On 31 Mar 2010, at 14:02, Alan Conway wrote:
>>>>
>>>>> On 03/30/2010 05:38 PM, Andrew Wright wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> I've recently run some tests to try and see how much overhead
>>>>>> clustering
>>>>>> brings. In short - I saw roughly a 50% reduction in message
>>>>>> throughput
>>>>>> when clients ran against a 2 node cluster vs a standalone broker.
>>>>>>
>>>>>
>>>>> I get about a 25% reduction running against a 4 node cluster.
>>>>> Try --worker-threads=4.
>>> <snip>
>>>
>>> Just a quick follow-up - setting worker-threads=4 noticeably
>>> improved throughput on a single queue in both the standalone and
>>> clustered scenarios. It reduced total throughput somewhat when
>>> loading 8 queues simultaneously (which you'd expect). The ratio of
>>> standalone:clustered throughput remained at about 2:1. Will let know
>>> if we get any further interesting results.
>>>
>>>
>>
>> I know this is odd to ask, but what hardware are you running on
>> brand. As machines that have SMI's are known to reduce throughput in
>> clustered configurations. My employer has worked with some
>> manufactures to provide bios's for certain models for reduced SMIs to
>> resolve this issue.
>>
>> Carl.
>>
>
> Interesting. This particular test was on HP DL380 G5s. We tend to have
> a mix of G5 and G6s kicking around. I see there is some literature on
> the redhat site re: tuning these models for realtime. Will talk it
> over with our unix team.
>
Go to http://www.redhat.com/mrg/ and link through to the certified RT
page and pull the updated HP BIOS from HP for MRG for G5 or the
instructions for BIOS setting for G6. That should resolve it for you for
Qpid. If you need assistance, you can also call you HP or Red Hat
account manager
Carl.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Clustering overhead
Posted by Andrew Wright <at...@mac.com>.
On 8 Apr 2010, at 13:16, Carl Trieloff wrote:
> On 04/07/2010 03:36 PM, Andrew Wright wrote:
>>
>> On 31 Mar 2010, at 14:30, Andrew Wright wrote:
>>
>>> On 31 Mar 2010, at 14:02, Alan Conway wrote:
>>>
>>>> On 03/30/2010 05:38 PM, Andrew Wright wrote:
>>>>> Hi all,
>>>>>
>>>>> I've recently run some tests to try and see how much overhead
>>>>> clustering
>>>>> brings. In short - I saw roughly a 50% reduction in message
>>>>> throughput
>>>>> when clients ran against a 2 node cluster vs a standalone broker.
>>>>>
>>>>
>>>> I get about a 25% reduction running against a 4 node cluster.
>>>> Try --worker-threads=4.
>> <snip>
>>
>> Just a quick follow-up - setting worker-threads=4 noticeably
>> improved throughput on a single queue in both the standalone and
>> clustered scenarios. It reduced total throughput somewhat when
>> loading 8 queues simultaneously (which you'd expect). The ratio of
>> standalone:clustered throughput remained at about 2:1. Will let
>> know if we get any further interesting results.
>>
>>
>
> I know this is odd to ask, but what hardware are you running on
> brand. As machines that have SMI's are known to reduce throughput in
> clustered configurations. My employer has worked with some
> manufactures to provide bios's for certain models for reduced SMIs
> to resolve this issue.
>
> Carl.
>
Interesting. This particular test was on HP DL380 G5s. We tend to have
a mix of G5 and G6s kicking around. I see there is some literature on
the redhat site re: tuning these models for realtime. Will talk it
over with our unix team.
Cheers,
Andrew
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Clustering overhead
Posted by Carl Trieloff <cc...@redhat.com>.
On 04/07/2010 03:36 PM, Andrew Wright wrote:
>
> On 31 Mar 2010, at 14:30, Andrew Wright wrote:
>
>> On 31 Mar 2010, at 14:02, Alan Conway wrote:
>>
>>> On 03/30/2010 05:38 PM, Andrew Wright wrote:
>>>> Hi all,
>>>>
>>>> I've recently run some tests to try and see how much overhead
>>>> clustering
>>>> brings. In short - I saw roughly a 50% reduction in message throughput
>>>> when clients ran against a 2 node cluster vs a standalone broker.
>>>>
>>>
>>> I get about a 25% reduction running against a 4 node cluster.
>>> Try --worker-threads=4.
> <snip>
>
> Just a quick follow-up - setting worker-threads=4 noticeably improved
> throughput on a single queue in both the standalone and clustered
> scenarios. It reduced total throughput somewhat when loading 8 queues
> simultaneously (which you'd expect). The ratio of standalone:clustered
> throughput remained at about 2:1. Will let know if we get any further
> interesting results.
>
>
I know this is odd to ask, but what hardware are you running on brand.
As machines that have SMI's are known to reduce throughput in clustered
configurations. My employer has worked with some manufactures to provide
bios's for certain models for reduced SMIs to resolve this issue.
Carl.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org