You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Gordon Sim <gs...@redhat.com> on 2014/04/10 12:26:53 UTC
dispatch router perf tests (was Re: [VOTE] Dispatch Router release
0.2)
On 04/10/2014 10:53 AM, Michael Goulish wrote:
> Also did 40 million-message load-test
> 3 separate boxes:
> * 1 box for 40 senders
> * 1 box for 2-router network
> * 1 box for 40 receivers.
> * messages are 117 bytes each
>
> results:
>
> * 182,000 messages per second, round-trip, i.e. 364,000 transfers per second.
> * memory on routers not growing
> * the 2 routers used 34% of total available CPU on a 16-core box.
> * throughput on each ethernet interface was 21 MB/sec
This is interesting... what did you use for these tests? Existing
checked in clients or some custom benchmarking tool?
Were the senders and receivers paired? I.e. did each sender send to one
specific receiver? Or did each sender broadcast to all receivers? How
did you measure the throughput?
One of the key experiments I would like to see is a measurement of how
overall throughput can scale out when adding extra routers. So e.g. for
a broadcast pattern, with publishers publishing at some fixed rate, see
how many publishers and receivers can be supported at that rate by one
router, then add in a second router and see how many more publishers and
receivers the combined network can then handle. (A similar test could be
done for unicast).
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: dispatch router perf tests (was Re: [VOTE] Dispatch Router release
0.2)
Posted by Ted Ross <tr...@redhat.com>.
On 04/10/2014 06:26 AM, Gordon Sim wrote:
> On 04/10/2014 10:53 AM, Michael Goulish wrote:
>> Also did 40 million-message load-test
>> 3 separate boxes:
>> * 1 box for 40 senders
>> * 1 box for 2-router network
>> * 1 box for 40 receivers.
>> * messages are 117 bytes each
>>
>> results:
>>
>> * 182,000 messages per second, round-trip, i.e. 364,000 transfers
>> per second.
>> * memory on routers not growing
>> * the 2 routers used 34% of total available CPU on a 16-core box.
>> * throughput on each ethernet interface was 21 MB/sec
This is a promising early result. There are two major areas for
performance tuning that I have in mind. The first is converting
Dispatch to use the new event API in Proton Engine. This should reduce
CPU utilization per transfer by eliminating an oft-repeated linear
search through all the available links. The second is improving the
lock architecture around the routing data structures. Currently there is
a single router lock which has too-large of a scope. Fixing this should
increase the utilization of available CPU.
>
> This is interesting... what did you use for these tests? Existing
> checked in clients or some custom benchmarking tool?
>
> Were the senders and receivers paired? I.e. did each sender send to
> one specific receiver? Or did each sender broadcast to all receivers?
> How did you measure the throughput?
>
> One of the key experiments I would like to see is a measurement of how
> overall throughput can scale out when adding extra routers. So e.g.
> for a broadcast pattern, with publishers publishing at some fixed
> rate, see how many publishers and receivers can be supported at that
> rate by one router, then add in a second router and see how many more
> publishers and receivers the combined network can then handle. (A
> similar test could be done for unicast).
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: dispatch router perf tests (was Re: [VOTE] Dispatch Router
release 0.2)
Posted by Michael Goulish <mg...@redhat.com>.
----- Original Message -----
> On 04/10/2014 10:53 AM, Michael Goulish wrote:
> > Also did 40 million-message load-test
> > 3 separate boxes:
> > * 1 box for 40 senders
> > * 1 box for 2-router network
> > * 1 box for 40 receivers.
> > * messages are 117 bytes each
> >
> > results:
> >
> > * 182,000 messages per second, round-trip, i.e. 364,000 transfers per
> > second.
> > * memory on routers not growing
> > * the 2 routers used 34% of total available CPU on a 16-core box.
> > * throughput on each ethernet interface was 21 MB/sec
>
> This is interesting... what did you use for these tests? Existing
> checked in clients or some custom benchmarking tool?
I used a sender and a receiver client that I wrote in C++.
>
> Were the senders and receivers paired? I.e. did each sender send to one
> specific receiver? Or did each sender broadcast to all receivers? How
> did you measure the throughput?
Well, I *thought* that they were paired, but in fact he senders
were broadcasting such that each sender's message went to 1 receiver on
each router.
>
> One of the key experiments I would like to see is a measurement of how
> overall throughput can scale out when adding extra routers. So e.g. for
> a broadcast pattern, with publishers publishing at some fixed rate, see
> how many publishers and receivers can be supported at that rate by one
> router, then add in a second router and see how many more publishers and
> receivers the combined network can then handle. (A similar test could be
> done for unicast).
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org