Axis performance

[Douglas Bitting <Do...@agile.com>]

Just FYI, here is a report that doesn't paint Axis in a very good light, performance-wise.  Like any benchmark, one has to take it with a grain of
salt since the benchmark methodology may have nothing to do with how one uses the software in question.  So, FWIW...

http://www.extreme.indiana.edu/xgws/papers/soap-hpdc2002/soap-hpdc2002.pdf

Cheers,
--Doug

Doug Bitting
Agile Software
1 Almaden Blvd
San Jose, CA  95113
(408) 999-7120

Re: Axis performance

[Scott Nichol <sn...@scottnichol.com>]

> Delayed ACK
> withholds the ACK for a packet for typically 200 ms when either the
> packet is not a full MAC frame or the preceeding packet is not a full
> frame, although the delay is cancelled if the server has data to send
to
> the client, in which case the ACK is sent with the data.

Correction: the ACK is withheld when the packet is not a full MAC frame
*AND* the preceeding packet was not a full frame.

Scott Nichol

Re: Axis performance

[Scott Nichol <sn...@scottnichol.com>]

> Regarding nagling, HTTP1.0 is so aggressively sub-optimal that it is
always
> tempting to turn it off, yet TCP_NODELAY on long haul links is both
> anti-social and often counter-productive. Maybe clients should decide
> dynamically whether to delay or not depending on whether or not the
endpoint
> is local subnet or not -or even on the experience of the latest call.
The
> indiana paper used HTTP1.1

I ran some tests and looked at some traces, and I never saw the typical
Nagle delay.  This has nothing to do with HTTP 1.1, however; the
behavior is the same for HTTP 1.0.  The key is the buffer size specified
for the BufferedOutputStream.  With an 8k buffer, multiple writes to the
underlying stream occur only when headers+payload > 8192.  When this
happens, either the headers are > 1460 (Ethernet MSS), in which case
there would be no delayed ACK on the first write, since a full segment
would be sent, or the payload is > 1460, in which case more than a
segment of output is queued behind the headers, which causes the TCP/IP
stacks I tested to abandon the Nagle wait and send the next (full)
segment before an ACK is received for the first segment.

Scott Nichol

Re: Axis performance

[Steve Loughran <st...@iseran.com>]

----- Original Message -----
From: "Kenneth Chiu" <ch...@cs.indiana.edu>
To: <ax...@xml.apache.org>
Sent: Thursday, October 17, 2002 2:08 PM
Subject: Re: Axis performance


> On Tue, 15 Oct 2002, Steve Loughran wrote:
> > Interesting to see that FP to ascii conversion takes so long; one of my
> > projects is very FP intensive. But this is an XML parser issue, not SOAP
> > stack per-se. Maybe I should look at using a JNI to msxml bridge rather
than
> > xerces/J for the old xml parser. The proposal to use a binary
representation
> > of FP is interesting; there are the ISO standard representations after
all,
> > but it does go against the fundamentals of both XML and XML Schema.
>
> A possible compromise which might be acceptable for
> scientific computing is to send the double as text, but
> include a hint which can be used to speed up the FP
> conversion.  We haven't investigated this yet, though.

What is the big bottlenect with FP conversion? Does the runtime multiply the
mantissa by ten every time a new digit comes in, then has to exponentiate it
with the exponential digits?

Maybe the trick would be send it as hex encoded network byte order doubles.
Use Float.intBitsToFloat() or Double.longBitsToDouble() to convert from the
bits to the number and back.

Actually, maybe just sending the stuff as long and int would suffice, though
that would be dangerously confusing to some people.

-steve

Re: Axis performance

[Kenneth Chiu <ch...@cs.indiana.edu>]

On Tue, 15 Oct 2002, Steve Loughran wrote:
> Interesting to see that FP to ascii conversion takes so long; one of my
> projects is very FP intensive. But this is an XML parser issue, not SOAP
> stack per-se. Maybe I should look at using a JNI to msxml bridge rather than
> xerces/J for the old xml parser. The proposal to use a binary representation
> of FP is interesting; there are the ISO standard representations after all,
> but it does go against the fundamentals of both XML and XML Schema.

A possible compromise which might be acceptable for
scientific computing is to send the double as text, but
include a hint which can be used to speed up the FP
conversion.  We haven't investigated this yet, though.

> ----- Original Message -----
> From: "Scott Nichol" <sn...@scottnichol.com>
> To: <ax...@xml.apache.org>
> Sent: Tuesday, October 15, 2002 9:03 AM
> Subject: Re: Axis performance
> 
> 
> > Something this paper does not touch upon that has been much discussed on
> > the Apache SOAP lists is disabling the Nagle algorithm, i.e. calling
> > setTcpNoDelay(true), on the client socket.  Basically, with the Nagle
> > algorithm enabled (the default), the client TCP/IP stack sends each data
> > packet after receiving the ACK for the previous packet.  This interacts
> > with the implementation of delayed ACK on the server.  Delayed ACK
> > withholds the ACK for a packet for typically 200 ms when either the
> > packet is not a full MAC frame or the preceeding packet is not a full
> > frame, although the delay is cancelled if the server has data to send to
> > the client, in which case the ACK is sent with the data.
> >
> > It had been my assumption that Nagling would have no effect on Apache
> > SOAP since it uses a buffered output stream, the size of which can be
> > set to be larger than a MAC frame.  My initial tests with a Linux server
> > seemed to confirm this, but I got different results running with a Win2k
> > server.  It turns out that in the Red Hat 7.3 build I was using, delayed
> > ACK is disabled.
> >
> > I had assumed using buffered output would suffice since with a buffer
> > size greater than the frame size, the buffering mechanism would always
> > send full frames until the final flush.  This is my recollection of how
> > it works in C.  This is *not* how it works in the Java libraries.  When
> > the data for a write operation will not fit in the current buffer, the
> > current buffer (which is *not* full) is written to the underlying
> > stream.  For Apache SOAP, the HTTP headers are written by the buffered
> > output stream in one write operation, then the data is written in
> > another.  This causes 2 or more writes to the underlying stream whenever
> > the buffer is not large enough to hold all the headers and data at once.
> > The first write is just the HTTP headers and is typically not a full
> > Ethernet frame.  The Nagle algorithm and delayed ACK combine to give an
> > unnecessary and unwanted 200 ms delay.
> 
> hmm.
> 
> >
> > I do not have an Axis setup to confirm this affects Axis, but based on
> > my research with Apache SOAP, I am confident that it does.
> >
> > Scott Nichol
> >
> 
> 

Re: Axis performance

[Steve Loughran <st...@iseran.com>]

I do like the indiana work on the perf of XML based communications
protocols. Its good to have people looking at this rigorously, even with the
bias of scientific grid computing (hence the concerns with floats, rather
than say date/time conversion)

Regarding nagling, HTTP1.0 is so aggressively sub-optimal that it is always
tempting to turn it off, yet TCP_NODELAY on long haul links is both
anti-social and often counter-productive. Maybe clients should decide
dynamically whether to delay or not depending on whether or not the endpoint
is local subnet or not -or even on the experience of the latest call. The
indiana paper used HTTP1.1

Interesting to see that FP to ascii conversion takes so long; one of my
projects is very FP intensive. But this is an XML parser issue, not SOAP
stack per-se. Maybe I should look at using a JNI to msxml bridge rather than
xerces/J for the old xml parser. The proposal to use a binary representation
of FP is interesting; there are the ISO standard representations after all,
but it does go against the fundamentals of both XML and XML Schema.



----- Original Message -----
From: "Scott Nichol" <sn...@scottnichol.com>
To: <ax...@xml.apache.org>
Sent: Tuesday, October 15, 2002 9:03 AM
Subject: Re: Axis performance


> Something this paper does not touch upon that has been much discussed on
> the Apache SOAP lists is disabling the Nagle algorithm, i.e. calling
> setTcpNoDelay(true), on the client socket.  Basically, with the Nagle
> algorithm enabled (the default), the client TCP/IP stack sends each data
> packet after receiving the ACK for the previous packet.  This interacts
> with the implementation of delayed ACK on the server.  Delayed ACK
> withholds the ACK for a packet for typically 200 ms when either the
> packet is not a full MAC frame or the preceeding packet is not a full
> frame, although the delay is cancelled if the server has data to send to
> the client, in which case the ACK is sent with the data.
>
> It had been my assumption that Nagling would have no effect on Apache
> SOAP since it uses a buffered output stream, the size of which can be
> set to be larger than a MAC frame.  My initial tests with a Linux server
> seemed to confirm this, but I got different results running with a Win2k
> server.  It turns out that in the Red Hat 7.3 build I was using, delayed
> ACK is disabled.
>
> I had assumed using buffered output would suffice since with a buffer
> size greater than the frame size, the buffering mechanism would always
> send full frames until the final flush.  This is my recollection of how
> it works in C.  This is *not* how it works in the Java libraries.  When
> the data for a write operation will not fit in the current buffer, the
> current buffer (which is *not* full) is written to the underlying
> stream.  For Apache SOAP, the HTTP headers are written by the buffered
> output stream in one write operation, then the data is written in
> another.  This causes 2 or more writes to the underlying stream whenever
> the buffer is not large enough to hold all the headers and data at once.
> The first write is just the HTTP headers and is typically not a full
> Ethernet frame.  The Nagle algorithm and delayed ACK combine to give an
> unnecessary and unwanted 200 ms delay.

hmm.

>
> I do not have an Axis setup to confirm this affects Axis, but based on
> my research with Apache SOAP, I am confident that it does.
>
> Scott Nichol
>

Re: Axis performance

[Scott Nichol <sn...@scottnichol.com>]

Something this paper does not touch upon that has been much discussed on
the Apache SOAP lists is disabling the Nagle algorithm, i.e. calling
setTcpNoDelay(true), on the client socket.  Basically, with the Nagle
algorithm enabled (the default), the client TCP/IP stack sends each data
packet after receiving the ACK for the previous packet.  This interacts
with the implementation of delayed ACK on the server.  Delayed ACK
withholds the ACK for a packet for typically 200 ms when either the
packet is not a full MAC frame or the preceeding packet is not a full
frame, although the delay is cancelled if the server has data to send to
the client, in which case the ACK is sent with the data.

It had been my assumption that Nagling would have no effect on Apache
SOAP since it uses a buffered output stream, the size of which can be
set to be larger than a MAC frame.  My initial tests with a Linux server
seemed to confirm this, but I got different results running with a Win2k
server.  It turns out that in the Red Hat 7.3 build I was using, delayed
ACK is disabled.

I had assumed using buffered output would suffice since with a buffer
size greater than the frame size, the buffering mechanism would always
send full frames until the final flush.  This is my recollection of how
it works in C.  This is *not* how it works in the Java libraries.  When
the data for a write operation will not fit in the current buffer, the
current buffer (which is *not* full) is written to the underlying
stream.  For Apache SOAP, the HTTP headers are written by the buffered
output stream in one write operation, then the data is written in
another.  This causes 2 or more writes to the underlying stream whenever
the buffer is not large enough to hold all the headers and data at once.
The first write is just the HTTP headers and is typically not a full
Ethernet frame.  The Nagle algorithm and delayed ACK combine to give an
unnecessary and unwanted 200 ms delay.

I do not have an Axis setup to confirm this affects Axis, but based on
my research with Apache SOAP, I am confident that it does.

Scott Nichol

----- Original Message -----
From: "Douglas Bitting" <Do...@agile.com>
To: <ax...@xml.apache.org>
Sent: Tuesday, October 15, 2002 11:02 AM
Subject: Axis performance


> Just FYI, here is a report that doesn't paint Axis in a very good
light, performance-wise.  Like any benchmark, one has to take it with a
grain of
> salt since the benchmark methodology may have nothing to do with how
one uses the software in question.  So, FWIW...
>
>
http://www.extreme.indiana.edu/xgws/papers/soap-hpdc2002/soap-hpdc2002.p
df
>
> Cheers,
> --Doug
>
> Doug Bitting
> Agile Software
> 1 Almaden Blvd
> San Jose, CA  95113
> (408) 999-7120
>
>
>