You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@axis.apache.org by Martin Jericho <ma...@radiocity.com.au> on 2002/09/25 05:09:33 UTC
Performance problems with RPC messages over 20k
I was doing some benchmarking to test how much it would impact on
performance to break up a single, large request into several smaller ones.
I was expecting of course that for a fixed volume of data, dividing it into
more separate messages would increase the overheads and make things slower.
What I found was quite surprising. It seems that once a message gets bigger
than 20kb, the response time increases at a rate much greater than the
linear relationship one would expect. I did some tests with a bean
containing an array of other beans. The size of the message with no array
elements is 1571 bytes, and each array element is 772 bytes.
The times recorded are from calling the service method on the client until
receiving the response back from the server (the response is just a string).
The results were as follows:
Number of calls, Number of Array Items, Total Response Time in Seconds
0001, 1000, 20.7
0002, 0500, 13.6
0004, 0250, 9.9
0005, 0200, 9.7
0010, 0100, 7.6
0020, 0050, 7.2
0040, 0025, 6.8
0050, 0020, 6.9
0100, 0010, 7.3
0200, 0005, 9.4
0250, 0004, 10.5
0500, 0002, 15.4
1000, 0001, 25.6
So the most efficent way to send my 1000 beans was in 40 separate messages
each containing 25 beans, each of about 20kb in size.
Does anyone know an explanation for this? It seems to me that there must be
something in axis which has been very poorly implemented to cause this
blowout in performace.
Re: Performance problems with RPC messages over 20k
Posted by Dennis Sosnoski <dm...@sosnoski.com>.
WJCarpenter wrote:
>>the times). These figures are from Sun JRE 1.3.1 on Linux, running on a
>> PIIIm with 256MB RAM. I used "-Xmx64M -Xms64M" options on the Java
>>command line to avoid a lot of threshing as the heap grew; running with
>>
>>
>I am curious if you measured heap use and if 64 MB is enough? I haven't
>done any testing of this sort with Axis, but in Apache SOAP I routinely
>use 256 MB and it is often worth it. Anyhow, it would be interesting
>to hear about memory figures for Axis, too.
>
>
From a quick look with "-verbose:gc" on the client JVM the 320KB
messages are showing several partial garbage collections for each
request/response round trip, with a total of about 12MB collected. In a
run of 11 round trips I had one full garbage collection after the 10th
round trip, which collected about 58MB. Judging from this it looks like
the total trash generated on the client side is about 18MB for each
round trip of my 320KB message.
That's pretty high, but consistent with what I've seen of the code.
There's a lot of short-lived object creation. A lot of it looks tied to
the JAX-RPC interface, and that's going to be difficult to change.
From looking at these figures it doesn't look like adding more memory
is going to help on the client side, unless you're sending really huge
(multi-MB) messages - in which case you should probably not be using
SOAP. :-) If you run with the default JVM settings you start with only
2MB, which is definitely not enough, but setting "-Xms32M" or "-Xms64M"
should be more than enough for any practical client applications. For
the server more memory is definitely going to be useful, especially for
real world applications with multiple overlapping requests. Just how
much depends on the message size and rate, as well as other demands on
the server - it's probably good to start with at least "-Xmx64M -Xms64M"
and try going up from there to see if it helps your particular environment.
- Dennis
Dennis M. Sosnoski
Enterprise Java, XML, and Web Services Support
http://www.sosnoski.com
Re: Performance problems with RPC messages over 20k
Posted by WJCarpenter <bi...@carpenter.org>.
> the times). These figures are from Sun JRE 1.3.1 on Linux, running on a
> PIIIm with 256MB RAM. I used "-Xmx64M -Xms64M" options on the Java
> command line to avoid a lot of threshing as the heap grew; running with
I am curious if you measured heap use and if 64 MB is enough? I haven't
done any testing of this sort with Axis, but in Apache SOAP I routinely
use 256 MB and it is often worth it. Anyhow, it would be interesting
to hear about memory figures for Axis, too.
Re: Performance problems with RPC messages over 20k
Posted by Dennis Sosnoski <dm...@sosnoski.com>.
I investigated this further and found that there definitely is a problem
in 1.0 with large messages using RPC encoding. With my particular test
data it started showing up at the 320KB message size and got
exponentially worse with larger sizes. I think I've tracked this down,
and have entered a bug report and fix against the offending code. In my
tests the fix keeps performance stable at least into the 1.3MB range.
That done, I figured I should correct my earlier, overly- (or at least
prematurely-) optimistic, statement about Axis performance with large
messages. :-)
- Dennis
Dennis M. Sosnoski
Enterprise Java, XML, and Web Services Support
http://www.sosnoski.com
Dennis Sosnoski wrote:
> In my own tests (running client and server on a single system) I found
> Axis performance went up at first as I increased the message size,
> then basically leveled off. Here's what my raw results look like:
>
> Message size Roundtrip Time (ms.)
> 10KB 107
> 20KB 162
> 40KB 289
> 80KB 491
> 160KB 981
> 320KB 2000
>
> Martin Jericho wrote:
>
>> I was doing some benchmarking to test how much it would impact on
>> performance to break up a single, large request into several smaller
>> ones.
>> I was expecting of course that for a fixed volume of data, dividing
>> it into
>> more separate messages would increase the overheads and make things
>> slower.
>>
>> What I found was quite surprising. It seems that once a message gets
>> bigger
>> than 20kb, the response time increases at a rate much greater than the
>> linear relationship one would expect. I did some tests with a bean
>> containing an array of other beans. The size of the message with no
>> array
>> elements is 1571 bytes, and each array element is 772 bytes.
>>
>> The times recorded are from calling the service method on the client
>> until
>> receiving the response back from the server (the response is just a
>> string).
>>
>> The results were as follows:
>>
>> Number of calls, Number of Array Items, Total Response Time in
>> Seconds
>> 0001, 1000, 20.7
>> 0002, 0500, 13.6
>> 0004, 0250, 9.9
>> 0005, 0200, 9.7
>> 0010, 0100, 7.6
>> 0020, 0050, 7.2
>> 0040, 0025, 6.8
>> 0050, 0020, 6.9
>> 0100, 0010, 7.3
>> 0200, 0005, 9.4
>> 0250, 0004, 10.5
>> 0500, 0002, 15.4
>> 1000, 0001, 25.6
>>
>> So the most efficent way to send my 1000 beans was in 40 separate
>> messages
>> each containing 25 beans, each of about 20kb in size.
>>
>> Does anyone know an explanation for this? It seems to me that there
>> must be
>> something in axis which has been very poorly implemented to cause this
>> blowout in performace.
>>
>>
>>
>>
>
Re: Performance problems with RPC messages over 20k
Posted by Dennis Sosnoski <dm...@sosnoski.com>.
Martin,
I noticed this email a while ago and wanted to look into it. I see by
your recent email that you're now getting away from using Axis, but
thought it might be of interest to other people on the list anyway.
Assuming you were using separate client and server systems for this
test, I suspect you'd see a similar curve using any SOAP implementation.
If you consider how this works, when you send all the data as a single
message you have a completely linear process - the request is generated
as text on the client, then sent to the server, then converted back into
objects on the server, and finally processed by your server code. The
response then goes through the same series of steps getting back to the
client. When you break your data up into several requests you allow
several of these steps to be executed in parallel. In particular, your
client can be working on one request while an earlier request is being
transmitted to the server, the server is working on an earlier request
or response, and a still earlier response is being transmitted back to
the client.
If you ran your tests with client and server on the same system I
wouldn't expect to see the kind of results you found. Let me know if
this is the case, perhaps there are some unusual aspects to your data
that account for the differences.
Seeing this did make me curious about Axis performance, though. In my
own tests (running client and server on a single system) I found Axis
performance went up at first as I increased the message size, then
basically leveled off. Here's what my raw results look like:
Message size Roundtrip Time (ms.)
10KB 107
20KB 162
40KB 289
80KB 491
160KB 981
320KB 2000
Message sizes are the actual character count for the request and
response, times are the average over 11 requests and responses,
excluding the first request and response (to avoid bringing in class
loading overhead and such - this is basically a constant added to all
the times). These figures are from Sun JRE 1.3.1 on Linux, running on a
PIIIm with 256MB RAM. I used "-Xmx64M -Xms64M" options on the Java
command line to avoid a lot of threshing as the heap grew; running with
the default settings will add more overhead to the handling time of
larger messages initially, until the JVM gets enough memory to run
efficiently.
My data consists of an object graph with variable numbers of objects.
There are a lot of links between objects, so this might not be typical
of what you'd see working with flatter data structures. My actual
service processing just reverses the order of elements in arrays, so it
doesn't contribute anything significant to the overall time.
- Dennis
Dennis M. Sosnoski
Enterprise Java, XML, and Web Services Support
http://www.sosnoski.com
Martin Jericho wrote:
>I was doing some benchmarking to test how much it would impact on
>performance to break up a single, large request into several smaller ones.
>I was expecting of course that for a fixed volume of data, dividing it into
>more separate messages would increase the overheads and make things slower.
>
>What I found was quite surprising. It seems that once a message gets bigger
>than 20kb, the response time increases at a rate much greater than the
>linear relationship one would expect. I did some tests with a bean
>containing an array of other beans. The size of the message with no array
>elements is 1571 bytes, and each array element is 772 bytes.
>
>The times recorded are from calling the service method on the client until
>receiving the response back from the server (the response is just a string).
>
>The results were as follows:
>
>Number of calls, Number of Array Items, Total Response Time in Seconds
>0001, 1000, 20.7
>0002, 0500, 13.6
>0004, 0250, 9.9
>0005, 0200, 9.7
>0010, 0100, 7.6
>0020, 0050, 7.2
>0040, 0025, 6.8
>0050, 0020, 6.9
>0100, 0010, 7.3
>0200, 0005, 9.4
>0250, 0004, 10.5
>0500, 0002, 15.4
>1000, 0001, 25.6
>
>So the most efficent way to send my 1000 beans was in 40 separate messages
>each containing 25 beans, each of about 20kb in size.
>
>Does anyone know an explanation for this? It seems to me that there must be
>something in axis which has been very poorly implemented to cause this
>blowout in performace.
>
>
>
>