You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@axis.apache.org by Philipp Perner <ph...@healthgate.at> on 2006/12/12 19:56:27 UTC
MTOM Benchmarks
Hi,
Does anyone have benchmarks of the current releases, when trying to use
SwA with MTOM?
Many people wrote, that they transfer files with up to 100MB.
But what are the performances of these web-services? And what
configuration, web-server and hardware do they use?
To start this topic, I would like to post some figures I came up using
my webService:
My Web-Service receives a datahandler which is written to harddisc.
After successfull writing it sends a string "success" back to the client.
Both, client and server, are localhost.
Web-Server: Apache Tomcat 5.0.28
Version of Axis: Apache Axis2 1.1
Used Modules: Apache Rampart 1.1
Databinding: ADB
Attachment Type: Base64Binary
Hardware: Intel P4 3.2 GHz with 1028MB Ram
Java Key Stores: service.jks and client.jks from rampart samples
Here are the benchmarks:
no rampart items:
size 10 kb: 188ms
size 2753kb: 15.183 sec
size 14214 kb: 81.001 sec
rampart items: timestamp signature
size 10 kb: 242ms
size 2753kb: 13.594 sec
size 14214 kb: 108.829 sec
rampart items: timestamp signature encrypt
size 10 kb: 297 ms
size 2753kb: 19,484 sec
size 14214 kb: 173,579 sec
I would appreciate if someone could send some benchmarks, or at least
comment on my figures.
Are these usual transaction times? Which times occur when sending files
up to 100MB?
cheers,
Philipp
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: MTOM Benchmarks
Posted by Thilina Gunarathne <cs...@gmail.com>.
Hi all,
Wow.. Interesting...
I did some tests some time back with the objective of testing the
effectiveness of Axis2 attachment file caching.. IIRC I was able to
upload a 800 MB file in about 35 mins time using the same machine as
server and the client.. Cannot remember the machine configuration I
used.
IMHO the tests needs be done in both ways with/without file caching.
In the latter case we'll not be able to test with large attachments.
>* Client sends "Hello" --> Server returns attachment
+1.. I would like to add the reverse of this too... Client uploads a file..
>* Attachment echo
+1
>* On-the-fly generation (instead of file reading)
We need to make sure that we write out whatever we generate as an when
they are generated, If not we might run in to out of memory situations
with larger sizes.
>* On-the-fly process (instead of file writing)
>* Stress testing (multiple concurrent clients)
+1
Keep us informed. More than willing to hear the results.. :)
~Thilina
On 12/13/06, Rodrigo Ruiz <rr...@gridsystems.com> wrote:
> Hi Philipp,
>
> I am afraid I have no results to contribute at this moment, but I will :-)
>
> By TCP/IP stack and File I/O implementations I was referring to the
> internal implementation of these subsystems in each OS.
>
> Typically, Linux and Solaris have more robust and performant
> implementations of TCP/IP than Windows 2000 Pro or XP, as the latter are
> more workstation-oriented while the former are more server-oriented.
> With a few rare exceptions, specifying the OS you are giving enough
> information about the TCP/IP implementation.
>
> The File I/O subsystem is different. The performance profile will depend
> on the filesystem and the Operating System. In Windows you may find
> FAT32 and NTFS filesystems, in Linux you can have a lot of
> possibilities, and other OSs have their own propietary filesystems. All
> of them have different performance/robustness characteristics.
>
> Axis2 has a wiki, and I suppose we could use it, at least for setting up
> a starting point and/or for the first results. Of course, if more people
> join us, and we collect lots of results, a wiki will not be the correct
> tool. You can find a link to the wiki in the Axis2 home page left menu.
>
>
> Back to the benchmark, I think there are a few more scenarios to take
> into account (this is, for a general benchmark):
>
> * Test the impact of other specifications:
> * WS-SecureConversation
> * WS-ReliableMessage
>
> * Client sends "Hello" --> Server returns attachment
> * Attachment echo
> * On-the-fly generation (instead of file reading)
> * On-the-fly process (instead of file writing)
> * Stress testing (multiple concurrent clients)
>
> Does any of these scenarios make sense to you?
>
> Cheers,
> Rodrigo
>
> --
> -------------------------------------------------------------------
> GRID SYSTEMS, S.A. Rodrigo Ruiz
> Parc Bit - Edificio 17 Research Coordinator
> 07121 Palma de Mallorca rruiz@gridsystems.com
> Baleares - Spain Tel: +34 971 435 085
> http://www.gridsystems.com/ Fax: +34 971 435 082
> -------------------------------------------------------------------
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>
--
Thilina Gunarathne
WSO2, Inc.; http://www.wso2.com/
Home page: http://webservices.apache.org/~thilina/
Blog: http://thilinag.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: MTOM Benchmarks
Posted by Rodrigo Ruiz <rr...@gridsystems.com>.
Hi Philipp,
I am afraid I have no results to contribute at this moment, but I will :-)
By TCP/IP stack and File I/O implementations I was referring to the
internal implementation of these subsystems in each OS.
Typically, Linux and Solaris have more robust and performant
implementations of TCP/IP than Windows 2000 Pro or XP, as the latter are
more workstation-oriented while the former are more server-oriented.
With a few rare exceptions, specifying the OS you are giving enough
information about the TCP/IP implementation.
The File I/O subsystem is different. The performance profile will depend
on the filesystem and the Operating System. In Windows you may find
FAT32 and NTFS filesystems, in Linux you can have a lot of
possibilities, and other OSs have their own propietary filesystems. All
of them have different performance/robustness characteristics.
Axis2 has a wiki, and I suppose we could use it, at least for setting up
a starting point and/or for the first results. Of course, if more people
join us, and we collect lots of results, a wiki will not be the correct
tool. You can find a link to the wiki in the Axis2 home page left menu.
Back to the benchmark, I think there are a few more scenarios to take
into account (this is, for a general benchmark):
* Test the impact of other specifications:
* WS-SecureConversation
* WS-ReliableMessage
* Client sends "Hello" --> Server returns attachment
* Attachment echo
* On-the-fly generation (instead of file reading)
* On-the-fly process (instead of file writing)
* Stress testing (multiple concurrent clients)
Does any of these scenarios make sense to you?
Cheers,
Rodrigo
--
-------------------------------------------------------------------
GRID SYSTEMS, S.A. Rodrigo Ruiz
Parc Bit - Edificio 17 Research Coordinator
07121 Palma de Mallorca rruiz@gridsystems.com
Baleares - Spain Tel: +34 971 435 085
http://www.gridsystems.com/ Fax: +34 971 435 082
-------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: MTOM Benchmarks
Posted by Philipp Perner <ph...@healthgate.at>.
and i use caching
Philipp Perner schrieb:
> you are right.
>
> I missed this significant property.
> I'm just working on benchmarks concerning outflow security of the
> client including:
>
> -) optimizeParts
> -) encryptionSymAlgorithm
> -) encryptionKeyTransportAlgorithm
>
> like provided here [0]
>
> cheers, Philipp
>
> [0] http://ws.apache.org/axis2/modules/rampart/1_0/security-module.html
>
>
>
> Paul Fremantle schrieb:
>> Philipp
>>
>> One more question. Are you using the ability of Axis2 to stream the
>> file directly to disk?
>>
>> <parameter name="cacheAttachments" locked="false">true</parameter>
>> <parameter name="attachmentDIR" locked="false">/temp</parameter>
>> <parameter name="sizeThreshold" locked="false">4000</parameter>
>>
>> Paul
>>
>> On 12/13/06, Philipp Perner <ph...@healthgate.at> wrote:
>>> Hi Rodrigo,
>>>
>>> Thanks for your quick and very useful reply. I would appreciate, if
>>> somebody else could join the discussion too.
>>> Now I would like to comment on your comments. :)
>>>
>>> Rodrigo Ruiz wrote:
>>> > * I think the benchmark is basically OK. I suppose you are measuring
>>> > total round-trip times, aren't you?
>>> I think so. What I understand as total round-trip time is from
>>> executing
>>> the stub until getting the response.
>>> > * You don't mention anything about mean times. Although a single
>>> > execution may be enough for large attachments, for small ones it
>>> would
>>> > be useful to repeat the invocation several times, and obtain means
>>> and
>>> > deviations.
>>> I tried every file size with 4 different file types: executable, jpeg,
>>> zip, xml
>>> From these roundtrips I calculated the mean time - this is the one
>>> displayed
>>> The deviation of the roundtrips can be neglected.
>>> > * Personally, my opinion is that having both, client and server, on
>>> > the same host is not the most typical scenario, but I think it still
>>> > must be included in any benchmark, as it has its own particularities.
>>> > I will come back to this later.
>>> For development purposes I don't have the possibilities to test in a
>>> distributed area. But in near future I can provide data of "real"
>>> remote
>>> invocation.
>>> > * In order to ensure that the results can be compared, sooner or
>>> later
>>> > a common source code should be used. This one smells to a project ;-)
>>> You are absolutely right. I can provide my wsdl file, to give other
>>> people the chance to test it.
>>> > * You have not included information about your disk in your
>>> > configuration. As you are writing the attachment to disk, its
>>> > characteristics are relevant (IDE / SATA, rpm, and the like).
>>> > Depending on the OS, the filesystem should be also specified.
>>> Ok, you are right. Here is some data:
>>>
>>> Chipset Intel 945G
>>> CPU Name Intel Pentium 4
>>> CPU MHz 3200
>>> No. of processors 1
>>> System Bus 533 MHz FSB
>>> Memory 1024 MB DDR2 SD-RAM
>>> HDD SATA-II 80GB 7200 rpm
>>> Filesystem NTFS
>>> LAN Broadcom BCM5751, 1000 Mbit/s
>>> OS Windows2000 SP4
>>>
>>> > * TCP/IP stack implementations are totally different between OSs, and
>>> > the same can be said about disk I/O. It also influence the default
>>> JVM
>>> > settings.
>>> TCP/IP Stack implementations?
>>> Concerning disc I/O:
>>> write speed average: 22,37 MB/s
>>> read speed average: 1342,09 MB/s
>>> > * The JVM version is another important field to include in the
>>> > configuration. And also its configuration flags (GC policies, etc.)
>>> The axis server runs on j2sdk1.4.2 whereas my client application , i.e.
>>> a second tomcat web application for upload, runs on jdk1.5.0_08
>>> Due to Java Heap Space errors, I started the axis2 client with the java
>>> option -Xms128M -Xmx1024M
>>>
>>> > * Just for curiosity, do you really have 1028MB, or is it an
>>> errata? :-D
>>> of course not ;) 1024
>>> > * The file sizes look a bit arbitrary. Maybe it would be better to
>>> use
>>> > "nicer" sizes ;-)
>>> I took sizes of files I found on my computer in the types I mentioned
>>> above. And so these sizes "occurred"
>>> > * The times you have obtained show a rather poor performance. I am
>>> > just guessing now but, does your client get the attachment data
>>> from a
>>> > file?
>>> You are right i'm reading and writing from the same hard disc.
>>> > Reading and writing on the same disk may be the cause for these long
>>> > times. If this is the case, you could try to generate the data on the
>>> > fly and see how the figures change. You are using rather small sizes,
>>> > so you could use in-memory byte arrays with random data.
>>> That's a good idea
>>> > Hey, this seems to me a very good subject to work on, specially now
>>> > that SPEC has cancelled its web service benchmark project
>>> (AppPlatform).
>>> That was my intention. Do you have some benchmarks I can compare to
>>> mine?
>>> What do you mean with publishing in a wiki? Is there already existing a
>>> wiki we can publish our results?
>>>
>>> thanks for your very useful comments,
>>>
>>> cheers Philipp
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
>>> For additional commands, e-mail: axis-user-help@ws.apache.org
>>>
>>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: MTOM Benchmarks
Posted by Philipp Perner <ph...@healthgate.at>.
you are right.
I missed this significant property.
I'm just working on benchmarks concerning outflow security of the client including:
-) optimizeParts
-) encryptionSymAlgorithm
-) encryptionKeyTransportAlgorithm
like provided here [0]
cheers,
Philipp
[0] http://ws.apache.org/axis2/modules/rampart/1_0/security-module.html
Paul Fremantle schrieb:
> Philipp
>
> One more question. Are you using the ability of Axis2 to stream the
> file directly to disk?
>
> <parameter name="cacheAttachments" locked="false">true</parameter>
> <parameter name="attachmentDIR" locked="false">/temp</parameter>
> <parameter name="sizeThreshold" locked="false">4000</parameter>
>
> Paul
>
> On 12/13/06, Philipp Perner <ph...@healthgate.at> wrote:
>> Hi Rodrigo,
>>
>> Thanks for your quick and very useful reply. I would appreciate, if
>> somebody else could join the discussion too.
>> Now I would like to comment on your comments. :)
>>
>> Rodrigo Ruiz wrote:
>> > * I think the benchmark is basically OK. I suppose you are measuring
>> > total round-trip times, aren't you?
>> I think so. What I understand as total round-trip time is from executing
>> the stub until getting the response.
>> > * You don't mention anything about mean times. Although a single
>> > execution may be enough for large attachments, for small ones it would
>> > be useful to repeat the invocation several times, and obtain means and
>> > deviations.
>> I tried every file size with 4 different file types: executable, jpeg,
>> zip, xml
>> From these roundtrips I calculated the mean time - this is the one
>> displayed
>> The deviation of the roundtrips can be neglected.
>> > * Personally, my opinion is that having both, client and server, on
>> > the same host is not the most typical scenario, but I think it still
>> > must be included in any benchmark, as it has its own particularities.
>> > I will come back to this later.
>> For development purposes I don't have the possibilities to test in a
>> distributed area. But in near future I can provide data of "real" remote
>> invocation.
>> > * In order to ensure that the results can be compared, sooner or later
>> > a common source code should be used. This one smells to a project ;-)
>> You are absolutely right. I can provide my wsdl file, to give other
>> people the chance to test it.
>> > * You have not included information about your disk in your
>> > configuration. As you are writing the attachment to disk, its
>> > characteristics are relevant (IDE / SATA, rpm, and the like).
>> > Depending on the OS, the filesystem should be also specified.
>> Ok, you are right. Here is some data:
>>
>> Chipset Intel 945G
>> CPU Name Intel Pentium 4
>> CPU MHz 3200
>> No. of processors 1
>> System Bus 533 MHz FSB
>> Memory 1024 MB DDR2 SD-RAM
>> HDD SATA-II 80GB 7200 rpm
>> Filesystem NTFS
>> LAN Broadcom BCM5751, 1000 Mbit/s
>> OS Windows2000 SP4
>>
>> > * TCP/IP stack implementations are totally different between OSs, and
>> > the same can be said about disk I/O. It also influence the default JVM
>> > settings.
>> TCP/IP Stack implementations?
>> Concerning disc I/O:
>> write speed average: 22,37 MB/s
>> read speed average: 1342,09 MB/s
>> > * The JVM version is another important field to include in the
>> > configuration. And also its configuration flags (GC policies, etc.)
>> The axis server runs on j2sdk1.4.2 whereas my client application , i.e.
>> a second tomcat web application for upload, runs on jdk1.5.0_08
>> Due to Java Heap Space errors, I started the axis2 client with the java
>> option -Xms128M -Xmx1024M
>>
>> > * Just for curiosity, do you really have 1028MB, or is it an
>> errata? :-D
>> of course not ;) 1024
>> > * The file sizes look a bit arbitrary. Maybe it would be better to use
>> > "nicer" sizes ;-)
>> I took sizes of files I found on my computer in the types I mentioned
>> above. And so these sizes "occurred"
>> > * The times you have obtained show a rather poor performance. I am
>> > just guessing now but, does your client get the attachment data from a
>> > file?
>> You are right i'm reading and writing from the same hard disc.
>> > Reading and writing on the same disk may be the cause for these long
>> > times. If this is the case, you could try to generate the data on the
>> > fly and see how the figures change. You are using rather small sizes,
>> > so you could use in-memory byte arrays with random data.
>> That's a good idea
>> > Hey, this seems to me a very good subject to work on, specially now
>> > that SPEC has cancelled its web service benchmark project
>> (AppPlatform).
>> That was my intention. Do you have some benchmarks I can compare to
>> mine?
>> What do you mean with publishing in a wiki? Is there already existing a
>> wiki we can publish our results?
>>
>> thanks for your very useful comments,
>>
>> cheers Philipp
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
>> For additional commands, e-mail: axis-user-help@ws.apache.org
>>
>>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: MTOM Benchmarks
Posted by Paul Fremantle <pz...@gmail.com>.
Philipp
One more question. Are you using the ability of Axis2 to stream the
file directly to disk?
<parameter name="cacheAttachments" locked="false">true</parameter>
<parameter name="attachmentDIR" locked="false">/temp</parameter>
<parameter name="sizeThreshold" locked="false">4000</parameter>
Paul
On 12/13/06, Philipp Perner <ph...@healthgate.at> wrote:
> Hi Rodrigo,
>
> Thanks for your quick and very useful reply. I would appreciate, if
> somebody else could join the discussion too.
> Now I would like to comment on your comments. :)
>
> Rodrigo Ruiz wrote:
> > * I think the benchmark is basically OK. I suppose you are measuring
> > total round-trip times, aren't you?
> I think so. What I understand as total round-trip time is from executing
> the stub until getting the response.
> > * You don't mention anything about mean times. Although a single
> > execution may be enough for large attachments, for small ones it would
> > be useful to repeat the invocation several times, and obtain means and
> > deviations.
> I tried every file size with 4 different file types: executable, jpeg,
> zip, xml
> From these roundtrips I calculated the mean time - this is the one
> displayed
> The deviation of the roundtrips can be neglected.
> > * Personally, my opinion is that having both, client and server, on
> > the same host is not the most typical scenario, but I think it still
> > must be included in any benchmark, as it has its own particularities.
> > I will come back to this later.
> For development purposes I don't have the possibilities to test in a
> distributed area. But in near future I can provide data of "real" remote
> invocation.
> > * In order to ensure that the results can be compared, sooner or later
> > a common source code should be used. This one smells to a project ;-)
> You are absolutely right. I can provide my wsdl file, to give other
> people the chance to test it.
> > * You have not included information about your disk in your
> > configuration. As you are writing the attachment to disk, its
> > characteristics are relevant (IDE / SATA, rpm, and the like).
> > Depending on the OS, the filesystem should be also specified.
> Ok, you are right. Here is some data:
>
> Chipset Intel 945G
> CPU Name Intel Pentium 4
> CPU MHz 3200
> No. of processors 1
> System Bus 533 MHz FSB
> Memory 1024 MB DDR2 SD-RAM
> HDD SATA-II 80GB 7200 rpm
> Filesystem NTFS
> LAN Broadcom BCM5751, 1000 Mbit/s
> OS Windows2000 SP4
>
> > * TCP/IP stack implementations are totally different between OSs, and
> > the same can be said about disk I/O. It also influence the default JVM
> > settings.
> TCP/IP Stack implementations?
> Concerning disc I/O:
> write speed average: 22,37 MB/s
> read speed average: 1342,09 MB/s
> > * The JVM version is another important field to include in the
> > configuration. And also its configuration flags (GC policies, etc.)
> The axis server runs on j2sdk1.4.2 whereas my client application , i.e.
> a second tomcat web application for upload, runs on jdk1.5.0_08
> Due to Java Heap Space errors, I started the axis2 client with the java
> option -Xms128M -Xmx1024M
>
> > * Just for curiosity, do you really have 1028MB, or is it an errata? :-D
> of course not ;) 1024
> > * The file sizes look a bit arbitrary. Maybe it would be better to use
> > "nicer" sizes ;-)
> I took sizes of files I found on my computer in the types I mentioned
> above. And so these sizes "occurred"
> > * The times you have obtained show a rather poor performance. I am
> > just guessing now but, does your client get the attachment data from a
> > file?
> You are right i'm reading and writing from the same hard disc.
> > Reading and writing on the same disk may be the cause for these long
> > times. If this is the case, you could try to generate the data on the
> > fly and see how the figures change. You are using rather small sizes,
> > so you could use in-memory byte arrays with random data.
> That's a good idea
> > Hey, this seems to me a very good subject to work on, specially now
> > that SPEC has cancelled its web service benchmark project (AppPlatform).
> That was my intention. Do you have some benchmarks I can compare to mine?
> What do you mean with publishing in a wiki? Is there already existing a
> wiki we can publish our results?
>
> thanks for your very useful comments,
>
> cheers Philipp
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>
--
Paul Fremantle
VP/Technology, WSO2 and OASIS WS-RX TC Co-chair
http://bloglines.com/blog/paulfremantle
paul@wso2.com
"Oxygenating the Web Service Platform", www.wso2.com
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: MTOM Benchmarks
Posted by Philipp Perner <ph...@healthgate.at>.
Hi Rodrigo,
Thanks for your quick and very useful reply. I would appreciate, if
somebody else could join the discussion too.
Now I would like to comment on your comments. :)
Rodrigo Ruiz wrote:
> * I think the benchmark is basically OK. I suppose you are measuring
> total round-trip times, aren't you?
I think so. What I understand as total round-trip time is from executing
the stub until getting the response.
> * You don't mention anything about mean times. Although a single
> execution may be enough for large attachments, for small ones it would
> be useful to repeat the invocation several times, and obtain means and
> deviations.
I tried every file size with 4 different file types: executable, jpeg,
zip, xml
From these roundtrips I calculated the mean time - this is the one
displayed
The deviation of the roundtrips can be neglected.
> * Personally, my opinion is that having both, client and server, on
> the same host is not the most typical scenario, but I think it still
> must be included in any benchmark, as it has its own particularities.
> I will come back to this later.
For development purposes I don't have the possibilities to test in a
distributed area. But in near future I can provide data of "real" remote
invocation.
> * In order to ensure that the results can be compared, sooner or later
> a common source code should be used. This one smells to a project ;-)
You are absolutely right. I can provide my wsdl file, to give other
people the chance to test it.
> * You have not included information about your disk in your
> configuration. As you are writing the attachment to disk, its
> characteristics are relevant (IDE / SATA, rpm, and the like).
> Depending on the OS, the filesystem should be also specified.
Ok, you are right. Here is some data:
Chipset Intel 945G
CPU Name Intel Pentium 4
CPU MHz 3200
No. of processors 1
System Bus 533 MHz FSB
Memory 1024 MB DDR2 SD-RAM
HDD SATA-II 80GB 7200 rpm
Filesystem NTFS
LAN Broadcom BCM5751, 1000 Mbit/s
OS Windows2000 SP4
> * TCP/IP stack implementations are totally different between OSs, and
> the same can be said about disk I/O. It also influence the default JVM
> settings.
TCP/IP Stack implementations?
Concerning disc I/O:
write speed average: 22,37 MB/s
read speed average: 1342,09 MB/s
> * The JVM version is another important field to include in the
> configuration. And also its configuration flags (GC policies, etc.)
The axis server runs on j2sdk1.4.2 whereas my client application , i.e.
a second tomcat web application for upload, runs on jdk1.5.0_08
Due to Java Heap Space errors, I started the axis2 client with the java
option -Xms128M -Xmx1024M
> * Just for curiosity, do you really have 1028MB, or is it an errata? :-D
of course not ;) 1024
> * The file sizes look a bit arbitrary. Maybe it would be better to use
> "nicer" sizes ;-)
I took sizes of files I found on my computer in the types I mentioned
above. And so these sizes "occurred"
> * The times you have obtained show a rather poor performance. I am
> just guessing now but, does your client get the attachment data from a
> file?
You are right i'm reading and writing from the same hard disc.
> Reading and writing on the same disk may be the cause for these long
> times. If this is the case, you could try to generate the data on the
> fly and see how the figures change. You are using rather small sizes,
> so you could use in-memory byte arrays with random data.
That's a good idea
> Hey, this seems to me a very good subject to work on, specially now
> that SPEC has cancelled its web service benchmark project (AppPlatform).
That was my intention. Do you have some benchmarks I can compare to mine?
What do you mean with publishing in a wiki? Is there already existing a
wiki we can publish our results?
thanks for your very useful comments,
cheers Philipp
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: MTOM Benchmarks
Posted by Rodrigo Ruiz <rr...@gridsystems.com>.
Hi Philipp,
I think some benchmarks on data transfers would be very useful to all of
us. Maybe we could put the results into the wiki, so everybody can read
them, and contribute. Or being more ambitious, somebody could even
consider to start an independent project, in order to compare different
SOAP stack implementations.
I have some comments about the data you provide :-)
* I think the benchmark is basically OK. I suppose you are measuring
total round-trip times, aren't you?
* You don't mention anything about mean times. Although a single
execution may be enough for large attachments, for small ones it would
be useful to repeat the invocation several times, and obtain means and
deviations.
* Personally, my opinion is that having both, client and server, on the
same host is not the most typical scenario, but I think it still must be
included in any benchmark, as it has its own particularities. I will
come back to this later.
* In order to ensure that the results can be compared, sooner or later a
common source code should be used. This one smells to a project ;-)
* You have not included information about your disk in your
configuration. As you are writing the attachment to disk, its
characteristics are relevant (IDE / SATA, rpm, and the like). Depending
on the OS, the filesystem should be also specified.
* The Operating System is also missing. TCP/IP stack implementations are
totally different between OSs, and the same can be said about disk I/O.
It also influence the default JVM settings.
* The JVM version is another important field to include in the
configuration. And also its configuration flags (GC policies, etc.)
* Just for curiosity, do you really have 1028MB, or is it an errata? :-D
* The file sizes look a bit arbitrary. Maybe it would be better to use
"nicer" sizes ;-)
* The times you have obtained show a rather poor performance. I am just
guessing now but, does your client get the attachment data from a file?
Reading and writing on the same disk may be the cause for these long
times. If this is the case, you could try to generate the data on the
fly and see how the figures change. You are using rather small sizes, so
you could use in-memory byte arrays with random data.
Hey, this seems to me a very good subject to work on, specially now that
SPEC has cancelled its web service benchmark project (AppPlatform).
Best regards,
Rodrigo Ruiz
Philipp Perner wrote:
> Hi,
>
> Does anyone have benchmarks of the current releases, when trying to use
> SwA with MTOM?
> Many people wrote, that they transfer files with up to 100MB.
> But what are the performances of these web-services? And what
> configuration, web-server and hardware do they use?
>
> To start this topic, I would like to post some figures I came up using
> my webService:
> My Web-Service receives a datahandler which is written to harddisc.
> After successfull writing it sends a string "success" back to the client.
> Both, client and server, are localhost.
>
> Web-Server: Apache Tomcat 5.0.28
> Version of Axis: Apache Axis2 1.1
> Used Modules: Apache Rampart 1.1
> Databinding: ADB
> Attachment Type: Base64Binary
> Hardware: Intel P4 3.2 GHz with 1028MB Ram
> Java Key Stores: service.jks and client.jks from rampart samples
>
> Here are the benchmarks:
> no rampart items:
> size 10 kb: 188ms
> size 2753kb: 15.183 sec
> size 14214 kb: 81.001 sec
>
> rampart items: timestamp signature
> size 10 kb: 242ms
> size 2753kb: 13.594 sec
> size 14214 kb: 108.829 sec
>
> rampart items: timestamp signature encrypt
> size 10 kb: 297 ms
> size 2753kb: 19,484 sec
> size 14214 kb: 173,579 sec
>
> I would appreciate if someone could send some benchmarks, or at least
> comment on my figures.
> Are these usual transaction times? Which times occur when sending files
> up to 100MB?
>
> cheers,
>
> Philipp
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org