Re: c++: performance of new vs. old API

[Alan Conway <ac...@redhat.com>]

On 04/22/2010 02:20 PM, Rafael Schloming wrote:
> Alan Conway wrote:
>> I'm happy to report that after appropriate tuning of the tests, the
>> new API seems to perform as well as the old for basic thruput/latency
>> tests. Detailed results attached.
>>
>> The main change since my previous mail on the subject was to disable
>> setting sequence number and timestamp properties and compare results
>> with a reliable receiver - all fair changes since they are equivalent
>> to what perftest does.
>>
>> There is one mystery with the new api: an unreliable sender is
>> _slower_ than reliable sender. As far as I can see by inspection and
>> profiling, an unreliable sender does strictly less work than reliable
>> one so this puzzles me. If anyone has a theory I'd love to hear it!
>
> Have you isolated that it is actually the sender that is slower, or is
> it that overall throughput is slower when unreliably sending?

Yes, by sending to a queue with no subscriber.

> If it is the latter it doesn't seem entirely implausible since overall
> throughput is generally gated by consumption,
>which means a more
> efficient producer (i.e. the unreliable one) might actually result in
> slower overall throughput because it simply ends up filling the queue
> more quickly.
>

I don't follow. Filling the queue faster could certainly lead to higher 
latencies but I don't see how it could lower throughput in principal. In 
practice you might have contention or other interactions within the broker that 
would lower througput.

Also doing isolated send & isolated receive experiments suggests it is actually 
the _sender_ that is the bottleneck. In a send/receive experiment the receiver 
obviously can't be faster than the sender. However if you just send to a queue 
and then just receive from that queue the receiver is quite a bit faster.

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org

Re: c++: performance of new vs. old API

[Rafael Schloming <ra...@redhat.com>]

Alan Conway wrote:
> On 04/22/2010 02:20 PM, Rafael Schloming wrote:
>> Alan Conway wrote:
>>> I'm happy to report that after appropriate tuning of the tests, the
>>> new API seems to perform as well as the old for basic thruput/latency
>>> tests. Detailed results attached.
>>>
>>> The main change since my previous mail on the subject was to disable
>>> setting sequence number and timestamp properties and compare results
>>> with a reliable receiver - all fair changes since they are equivalent
>>> to what perftest does.
>>>
>>> There is one mystery with the new api: an unreliable sender is
>>> _slower_ than reliable sender. As far as I can see by inspection and
>>> profiling, an unreliable sender does strictly less work than reliable
>>> one so this puzzles me. If anyone has a theory I'd love to hear it!
>>
>> Have you isolated that it is actually the sender that is slower, or is
>> it that overall throughput is slower when unreliably sending?
> 
> Yes, by sending to a queue with no subscriber.
> 
>> If it is the latter it doesn't seem entirely implausible since overall
>> throughput is generally gated by consumption,
>> which means a more
>> efficient producer (i.e. the unreliable one) might actually result in
>> slower overall throughput because it simply ends up filling the queue
>> more quickly.
>>
> 
> I don't follow. Filling the queue faster could certainly lead to higher 
> latencies but I don't see how it could lower throughput in principal. In 
> practice you might have contention or other interactions within the 
> broker that would lower througput.

This is a fairly common thing in my experience. I presume the cost of 
memory management is just more expensive for larger queues, e.g. if the 
broker growth pushes other stuff to swap, or if the larger queue results 
in worse use of the cache, then throughput will suffer significantly.

> Also doing isolated send & isolated receive experiments suggests it is 
> actually the _sender_ that is the bottleneck. In a send/receive 
> experiment the receiver obviously can't be faster than the sender. 
> However if you just send to a queue and then just receive from that 
> queue the receiver is quite a bit faster.

That's interesting, from what I remember this has always been the other 
way around.

--Rafael


---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org