You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@axis.apache.org by Betsy Frey <BF...@SERENA.com> on 2007/03/19 16:05:31 UTC
[AXIS2] MTOM performance
I've just made some measurements, transferring files using MTOM in Axis2
on tomcat. My client and servlet are on the same machine. Attachments
are cached. For files of 1mb or more, the transfer rate,
client-to-servlet, is about 3.7mb per minute. In contrast, a file
system copy transfers at about 2.5gb per minute: 675 times faster than
Axis2.
Why is MTOM performance is so sluggish?
Thanks for any assistance,
Betsy
**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
RE: [AXIS2] MTOM performance
Posted by Betsy Frey <BF...@SERENA.com>.
Thilina - thanks for this information.
Betsy
-----Original Message-----
From: Thilina Gunarathne [mailto:csethil@gmail.com]
Sent: Friday, March 23, 2007 2:18 AM
To: axis-user@ws.apache.org
Subject: Re: [AXIS2] MTOM performance
I did a quick test against the snapshots and the results I got were in
the range of ~6.5 MB/s, which translates in to 375 to 400 MB/Min..
My environments is Ubuntu Linux 6.10
java version "1.5.0_10"
file system: ext3
Apache Tomcat 5.5.20
FileCacheThreshold 4 bytes
The program I used was the SoapWithAttachments sample of Axis2.
Thanks,
Thilina
On 3/20/07, Betsy Frey <BF...@serena.com> wrote:
> Thanks, Thilina. The latest Axiom jars make a 3x difference. I'm now
> seeing about 10mb/minute.
> Are more optimizations planned?
> Betsy
>
> -----Original Message-----
> From: Thilina Gunarathne [mailto:csethil@gmail.com]
> Sent: Tuesday, March 20, 2007 4:16 AM
> To: axis-user@ws.apache.org
> Subject: Re: [AXIS2] MTOM performance
>
> Betsy,
> > > copying it. I continue to see it grow by about 3.7mb/minute.
> I did some testing after the 1.0 release and the numbers I got were
> ~20 mb/min... MAy be something went wrong from 1.0 to 1.1...
>
> It would be really great if you can let us know the transfer rates you
> got when using the latest axiom jars [1]..
>
> ~Thilina
>
ttp://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axio
> m-api-SNAPSHOT.jar
>
http://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axi
> om-dom-SNAPSHOT.jar
>
http://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axi
> om-impl-SNAPSHOT.jar
>
>
> > On 3/19/07, Betsy Frey <BF...@serena.com> wrote:
> > > Thanks, Rodirigo, but the problem is not my disks.
> > > I changed the code to generate bytes on the fly, instead of from a
> > > source file. I changed the reference from localhost to 127.0.0.1.
> I am
> > > just looking at the file in the attachment directory; not moving
or
> > > copying it. I continue to see it grow by about 3.7mb/minute.
> > > Separately, I have confirmed that the transfer is in binary in a
> MIME
> > > section; MTOM is enabled.
> > > So the question remains: why is MTOM performance so sluggish, and
> is
> > > there anything to be done about it?
> > > Thanks.
> > > Betsy
> > >
> > > -----Original Message-----
> > > From: Rodrigo Ruiz [mailto:rruiz@gridsystems.com]
> > > Sent: Monday, March 19, 2007 8:57 AM
> > > To: axis-user@ws.apache.org
> > > Subject: Re: [AXIS2] MTOM performance
> > >
> > > Several reasons :-)
> > >
> > > First, a file system copy can make use of low-level OS mechanisms
to
> > > speed up transfers, and even in some cases, delegate the job to
the
> disk
> > > manager chipset. You will never get these speeds with a web
service,
> as
> > > it will never have direct access to the client file system.
> > >
> > > Said this, if you think about what the client and server do,
leaving
> > > aside the overhead from XML transfer and processing, you will
notice
> > > that having the client and server on the same machine is not a
good
> > > testcase.
> > >
> > > On one side, the client must read the file from disk. If the
client
> is
> > > correctly configured and programmed, the client will send the file
> > > contents to the server at the same time it is reading it
> (concurrently),
> > > so to avoid putting the whole file in memory.
> > >
> > > On the other side, the server will write the incoming bytes to a
> file
> > > while it is receiving the client request. Finally, once the whole
> file
> > > is received in the server side, your service will usually move the
> file
> > > to its final destination. If you copy the file instead of moving
it,
> the
> > > service will perform even worse ;-)
> > >
> > > Now, if you join both sides, you will see the system must read and
> write
> > > the same file at different locations of the disk. This will make
> your
> > > test waste a lot of time just in disk head seeks.
> > >
> > > I would propose you to make sure that you follow these guides:
> > >
> > > * Make sure reads and writes are, at least, from/to different
> (physical)
> > > disks. If you have only one disk per machine, but you have several
> > > machines, try to put client and server in different hosts. Take
into
> > > account that a fast ethernet bandwidth is far better than 3.7MB
per
> > > minute ;-)
> > >
> > > * If you want to do some tests for only one of the sides, another
> > > approach could be to create mockups for the file readers and
> writers.
> > > For example, if you use a DataHandler in the client side, you can
> > > implement a fake DataSource that generates random bytes on the
fly,
> > > instead of reading them from a file. Such a DataSource will allow
> you to
> > > test your server side performance without the client interfering
in
> your
> > > disk accesses.
> > >
> > > * Always use the localhost address (127.0.0.1) to contact the
> server. In
> > > many environments you will get different (and worse) results if
you
> use
> > > the machine public IP address.
> > >
> > > * Make sure you do your best to "move" the file in the server side
> to
> > > its final location instead of copying it. The performance is
totally
> > > different.
> > >
> > > Hope this helps you,
> > > Rodrigo Ruiz
> > >
> > > Betsy Frey wrote:
> > > > I've just made some measurements, transferring files using MTOM
in
> > > Axis2
> > > > on tomcat. My client and servlet are on the same machine.
> > > Attachments
> > > > are cached. For files of 1mb or more, the transfer rate,
> > > > client-to-servlet, is about 3.7mb per minute. In contrast, a
file
> > > > system copy transfers at about 2.5gb per minute: 675 times
faster
> > > than
> > > > Axis2.
> > > > Why is MTOM performance is so sluggish?
> > > > Thanks for any assistance,
> > > > Betsy
> > > >
> > > >
> > > >
> > > >
> **********************************************************************
> > > > This email and any files transmitted with it are confidential
and
> > > > intended solely for the use of the individual or entity to whom
> they
> > > > are addressed. Any unauthorized review, use, disclosure or
> > > distribution is prohibited. If you are not the intended recipient,
> > > please contact the sender by reply e-mail and destroy all copies
of
> the
> > > original message.
> > > >
> > > >
> > > >
> ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > > > For additional commands, e-mail: axis-user-help@ws.apache.org
> > > >
> > >
> > > --
> > >
-------------------------------------------------------------------
> > > GRID SYSTEMS, S.A. Rodrigo Ruiz
> > > Parc Bit - Edificio 17 Research Coordinator
> > > 07121 Palma de Mallorca rruiz@gridsystems.com
> > > Baleares - Spain Tel: +34 971 435 085
> > > http://www.gridsystems.com/ Fax: +34 971 435 082
> > >
-------------------------------------------------------------------
> > >
> > >
> ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > > For additional commands, e-mail: axis-user-help@ws.apache.org
> > >
> > >
> > >
> ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > > For additional commands, e-mail: axis-user-help@ws.apache.org
> > >
> > >
> >
> >
> > --
> > Thilina Gunarathne - http://www.wso2.com -
> http://thilinag.blogspot.com
> >
>
>
> --
> Thilina Gunarathne - http://www.wso2.com -
> http://thilinag.blogspot.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>
>
> **********************************************************************
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient,
please contact the sender by reply e-mail and destroy all copies of the
original message.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>
--
Thilina Gunarathne - http://www.wso2.com -
http://thilinag.blogspot.com
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: [AXIS2] MTOM performance
Posted by Thilina Gunarathne <cs...@gmail.com>.
I did a quick test against the snapshots and the results I got were in
the range of ~6.5 MB/s, which translates in to 375 to 400 MB/Min..
My environments is Ubuntu Linux 6.10
java version "1.5.0_10"
file system: ext3
Apache Tomcat 5.5.20
FileCacheThreshold 4 bytes
The program I used was the SoapWithAttachments sample of Axis2.
Thanks,
Thilina
On 3/20/07, Betsy Frey <BF...@serena.com> wrote:
> Thanks, Thilina. The latest Axiom jars make a 3x difference. I'm now
> seeing about 10mb/minute.
> Are more optimizations planned?
> Betsy
>
> -----Original Message-----
> From: Thilina Gunarathne [mailto:csethil@gmail.com]
> Sent: Tuesday, March 20, 2007 4:16 AM
> To: axis-user@ws.apache.org
> Subject: Re: [AXIS2] MTOM performance
>
> Betsy,
> > > copying it. I continue to see it grow by about 3.7mb/minute.
> I did some testing after the 1.0 release and the numbers I got were
> ~20 mb/min... MAy be something went wrong from 1.0 to 1.1...
>
> It would be really great if you can let us know the transfer rates you
> got when using the latest axiom jars [1]..
>
> ~Thilina
> ttp://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axio
> m-api-SNAPSHOT.jar
> http://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axi
> om-dom-SNAPSHOT.jar
> http://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axi
> om-impl-SNAPSHOT.jar
>
>
> > On 3/19/07, Betsy Frey <BF...@serena.com> wrote:
> > > Thanks, Rodirigo, but the problem is not my disks.
> > > I changed the code to generate bytes on the fly, instead of from a
> > > source file. I changed the reference from localhost to 127.0.0.1.
> I am
> > > just looking at the file in the attachment directory; not moving or
> > > copying it. I continue to see it grow by about 3.7mb/minute.
> > > Separately, I have confirmed that the transfer is in binary in a
> MIME
> > > section; MTOM is enabled.
> > > So the question remains: why is MTOM performance so sluggish, and
> is
> > > there anything to be done about it?
> > > Thanks.
> > > Betsy
> > >
> > > -----Original Message-----
> > > From: Rodrigo Ruiz [mailto:rruiz@gridsystems.com]
> > > Sent: Monday, March 19, 2007 8:57 AM
> > > To: axis-user@ws.apache.org
> > > Subject: Re: [AXIS2] MTOM performance
> > >
> > > Several reasons :-)
> > >
> > > First, a file system copy can make use of low-level OS mechanisms to
> > > speed up transfers, and even in some cases, delegate the job to the
> disk
> > > manager chipset. You will never get these speeds with a web service,
> as
> > > it will never have direct access to the client file system.
> > >
> > > Said this, if you think about what the client and server do, leaving
> > > aside the overhead from XML transfer and processing, you will notice
> > > that having the client and server on the same machine is not a good
> > > testcase.
> > >
> > > On one side, the client must read the file from disk. If the client
> is
> > > correctly configured and programmed, the client will send the file
> > > contents to the server at the same time it is reading it
> (concurrently),
> > > so to avoid putting the whole file in memory.
> > >
> > > On the other side, the server will write the incoming bytes to a
> file
> > > while it is receiving the client request. Finally, once the whole
> file
> > > is received in the server side, your service will usually move the
> file
> > > to its final destination. If you copy the file instead of moving it,
> the
> > > service will perform even worse ;-)
> > >
> > > Now, if you join both sides, you will see the system must read and
> write
> > > the same file at different locations of the disk. This will make
> your
> > > test waste a lot of time just in disk head seeks.
> > >
> > > I would propose you to make sure that you follow these guides:
> > >
> > > * Make sure reads and writes are, at least, from/to different
> (physical)
> > > disks. If you have only one disk per machine, but you have several
> > > machines, try to put client and server in different hosts. Take into
> > > account that a fast ethernet bandwidth is far better than 3.7MB per
> > > minute ;-)
> > >
> > > * If you want to do some tests for only one of the sides, another
> > > approach could be to create mockups for the file readers and
> writers.
> > > For example, if you use a DataHandler in the client side, you can
> > > implement a fake DataSource that generates random bytes on the fly,
> > > instead of reading them from a file. Such a DataSource will allow
> you to
> > > test your server side performance without the client interfering in
> your
> > > disk accesses.
> > >
> > > * Always use the localhost address (127.0.0.1) to contact the
> server. In
> > > many environments you will get different (and worse) results if you
> use
> > > the machine public IP address.
> > >
> > > * Make sure you do your best to "move" the file in the server side
> to
> > > its final location instead of copying it. The performance is totally
> > > different.
> > >
> > > Hope this helps you,
> > > Rodrigo Ruiz
> > >
> > > Betsy Frey wrote:
> > > > I've just made some measurements, transferring files using MTOM in
> > > Axis2
> > > > on tomcat. My client and servlet are on the same machine.
> > > Attachments
> > > > are cached. For files of 1mb or more, the transfer rate,
> > > > client-to-servlet, is about 3.7mb per minute. In contrast, a file
> > > > system copy transfers at about 2.5gb per minute: 675 times faster
> > > than
> > > > Axis2.
> > > > Why is MTOM performance is so sluggish?
> > > > Thanks for any assistance,
> > > > Betsy
> > > >
> > > >
> > > >
> > > >
> **********************************************************************
> > > > This email and any files transmitted with it are confidential and
> > > > intended solely for the use of the individual or entity to whom
> they
> > > > are addressed. Any unauthorized review, use, disclosure or
> > > distribution is prohibited. If you are not the intended recipient,
> > > please contact the sender by reply e-mail and destroy all copies of
> the
> > > original message.
> > > >
> > > >
> > > >
> ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > > > For additional commands, e-mail: axis-user-help@ws.apache.org
> > > >
> > >
> > > --
> > > -------------------------------------------------------------------
> > > GRID SYSTEMS, S.A. Rodrigo Ruiz
> > > Parc Bit - Edificio 17 Research Coordinator
> > > 07121 Palma de Mallorca rruiz@gridsystems.com
> > > Baleares - Spain Tel: +34 971 435 085
> > > http://www.gridsystems.com/ Fax: +34 971 435 082
> > > -------------------------------------------------------------------
> > >
> > >
> ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > > For additional commands, e-mail: axis-user-help@ws.apache.org
> > >
> > >
> > >
> ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > > For additional commands, e-mail: axis-user-help@ws.apache.org
> > >
> > >
> >
> >
> > --
> > Thilina Gunarathne - http://www.wso2.com -
> http://thilinag.blogspot.com
> >
>
>
> --
> Thilina Gunarathne - http://www.wso2.com -
> http://thilinag.blogspot.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>
>
> **********************************************************************
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>
--
Thilina Gunarathne - http://www.wso2.com - http://thilinag.blogspot.com
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
RE: [AXIS2] MTOM performance
Posted by Betsy Frey <BF...@SERENA.com>.
Thanks, Thilina. The latest Axiom jars make a 3x difference. I'm now
seeing about 10mb/minute.
Are more optimizations planned?
Betsy
-----Original Message-----
From: Thilina Gunarathne [mailto:csethil@gmail.com]
Sent: Tuesday, March 20, 2007 4:16 AM
To: axis-user@ws.apache.org
Subject: Re: [AXIS2] MTOM performance
Betsy,
> > copying it. I continue to see it grow by about 3.7mb/minute.
I did some testing after the 1.0 release and the numbers I got were
~20 mb/min... MAy be something went wrong from 1.0 to 1.1...
It would be really great if you can let us know the transfer rates you
got when using the latest axiom jars [1]..
~Thilina
ttp://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axio
m-api-SNAPSHOT.jar
http://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axi
om-dom-SNAPSHOT.jar
http://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axi
om-impl-SNAPSHOT.jar
> On 3/19/07, Betsy Frey <BF...@serena.com> wrote:
> > Thanks, Rodirigo, but the problem is not my disks.
> > I changed the code to generate bytes on the fly, instead of from a
> > source file. I changed the reference from localhost to 127.0.0.1.
I am
> > just looking at the file in the attachment directory; not moving or
> > copying it. I continue to see it grow by about 3.7mb/minute.
> > Separately, I have confirmed that the transfer is in binary in a
MIME
> > section; MTOM is enabled.
> > So the question remains: why is MTOM performance so sluggish, and
is
> > there anything to be done about it?
> > Thanks.
> > Betsy
> >
> > -----Original Message-----
> > From: Rodrigo Ruiz [mailto:rruiz@gridsystems.com]
> > Sent: Monday, March 19, 2007 8:57 AM
> > To: axis-user@ws.apache.org
> > Subject: Re: [AXIS2] MTOM performance
> >
> > Several reasons :-)
> >
> > First, a file system copy can make use of low-level OS mechanisms to
> > speed up transfers, and even in some cases, delegate the job to the
disk
> > manager chipset. You will never get these speeds with a web service,
as
> > it will never have direct access to the client file system.
> >
> > Said this, if you think about what the client and server do, leaving
> > aside the overhead from XML transfer and processing, you will notice
> > that having the client and server on the same machine is not a good
> > testcase.
> >
> > On one side, the client must read the file from disk. If the client
is
> > correctly configured and programmed, the client will send the file
> > contents to the server at the same time it is reading it
(concurrently),
> > so to avoid putting the whole file in memory.
> >
> > On the other side, the server will write the incoming bytes to a
file
> > while it is receiving the client request. Finally, once the whole
file
> > is received in the server side, your service will usually move the
file
> > to its final destination. If you copy the file instead of moving it,
the
> > service will perform even worse ;-)
> >
> > Now, if you join both sides, you will see the system must read and
write
> > the same file at different locations of the disk. This will make
your
> > test waste a lot of time just in disk head seeks.
> >
> > I would propose you to make sure that you follow these guides:
> >
> > * Make sure reads and writes are, at least, from/to different
(physical)
> > disks. If you have only one disk per machine, but you have several
> > machines, try to put client and server in different hosts. Take into
> > account that a fast ethernet bandwidth is far better than 3.7MB per
> > minute ;-)
> >
> > * If you want to do some tests for only one of the sides, another
> > approach could be to create mockups for the file readers and
writers.
> > For example, if you use a DataHandler in the client side, you can
> > implement a fake DataSource that generates random bytes on the fly,
> > instead of reading them from a file. Such a DataSource will allow
you to
> > test your server side performance without the client interfering in
your
> > disk accesses.
> >
> > * Always use the localhost address (127.0.0.1) to contact the
server. In
> > many environments you will get different (and worse) results if you
use
> > the machine public IP address.
> >
> > * Make sure you do your best to "move" the file in the server side
to
> > its final location instead of copying it. The performance is totally
> > different.
> >
> > Hope this helps you,
> > Rodrigo Ruiz
> >
> > Betsy Frey wrote:
> > > I've just made some measurements, transferring files using MTOM in
> > Axis2
> > > on tomcat. My client and servlet are on the same machine.
> > Attachments
> > > are cached. For files of 1mb or more, the transfer rate,
> > > client-to-servlet, is about 3.7mb per minute. In contrast, a file
> > > system copy transfers at about 2.5gb per minute: 675 times faster
> > than
> > > Axis2.
> > > Why is MTOM performance is so sluggish?
> > > Thanks for any assistance,
> > > Betsy
> > >
> > >
> > >
> > >
**********************************************************************
> > > This email and any files transmitted with it are confidential and
> > > intended solely for the use of the individual or entity to whom
they
> > > are addressed. Any unauthorized review, use, disclosure or
> > distribution is prohibited. If you are not the intended recipient,
> > please contact the sender by reply e-mail and destroy all copies of
the
> > original message.
> > >
> > >
> > >
---------------------------------------------------------------------
> > > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > > For additional commands, e-mail: axis-user-help@ws.apache.org
> > >
> >
> > --
> > -------------------------------------------------------------------
> > GRID SYSTEMS, S.A. Rodrigo Ruiz
> > Parc Bit - Edificio 17 Research Coordinator
> > 07121 Palma de Mallorca rruiz@gridsystems.com
> > Baleares - Spain Tel: +34 971 435 085
> > http://www.gridsystems.com/ Fax: +34 971 435 082
> > -------------------------------------------------------------------
> >
> >
---------------------------------------------------------------------
> > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > For additional commands, e-mail: axis-user-help@ws.apache.org
> >
> >
> >
---------------------------------------------------------------------
> > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > For additional commands, e-mail: axis-user-help@ws.apache.org
> >
> >
>
>
> --
> Thilina Gunarathne - http://www.wso2.com -
http://thilinag.blogspot.com
>
--
Thilina Gunarathne - http://www.wso2.com -
http://thilinag.blogspot.com
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: [AXIS2] MTOM performance
Posted by Thilina Gunarathne <cs...@gmail.com>.
Betsy,
> > copying it. I continue to see it grow by about 3.7mb/minute.
I did some testing after the 1.0 release and the numbers I got were
~20 mb/min... MAy be something went wrong from 1.0 to 1.1...
It would be really great if you can let us know the transfer rates you
got when using the latest axiom jars [1]..
~Thilina
ttp://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axiom-api-SNAPSHOT.jar
http://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axiom-dom-SNAPSHOT.jar
http://people.apache.org/repository/org.apache.ws.commons.axiom/jars/axiom-impl-SNAPSHOT.jar
> On 3/19/07, Betsy Frey <BF...@serena.com> wrote:
> > Thanks, Rodirigo, but the problem is not my disks.
> > I changed the code to generate bytes on the fly, instead of from a
> > source file. I changed the reference from localhost to 127.0.0.1. I am
> > just looking at the file in the attachment directory; not moving or
> > copying it. I continue to see it grow by about 3.7mb/minute.
> > Separately, I have confirmed that the transfer is in binary in a MIME
> > section; MTOM is enabled.
> > So the question remains: why is MTOM performance so sluggish, and is
> > there anything to be done about it?
> > Thanks.
> > Betsy
> >
> > -----Original Message-----
> > From: Rodrigo Ruiz [mailto:rruiz@gridsystems.com]
> > Sent: Monday, March 19, 2007 8:57 AM
> > To: axis-user@ws.apache.org
> > Subject: Re: [AXIS2] MTOM performance
> >
> > Several reasons :-)
> >
> > First, a file system copy can make use of low-level OS mechanisms to
> > speed up transfers, and even in some cases, delegate the job to the disk
> > manager chipset. You will never get these speeds with a web service, as
> > it will never have direct access to the client file system.
> >
> > Said this, if you think about what the client and server do, leaving
> > aside the overhead from XML transfer and processing, you will notice
> > that having the client and server on the same machine is not a good
> > testcase.
> >
> > On one side, the client must read the file from disk. If the client is
> > correctly configured and programmed, the client will send the file
> > contents to the server at the same time it is reading it (concurrently),
> > so to avoid putting the whole file in memory.
> >
> > On the other side, the server will write the incoming bytes to a file
> > while it is receiving the client request. Finally, once the whole file
> > is received in the server side, your service will usually move the file
> > to its final destination. If you copy the file instead of moving it, the
> > service will perform even worse ;-)
> >
> > Now, if you join both sides, you will see the system must read and write
> > the same file at different locations of the disk. This will make your
> > test waste a lot of time just in disk head seeks.
> >
> > I would propose you to make sure that you follow these guides:
> >
> > * Make sure reads and writes are, at least, from/to different (physical)
> > disks. If you have only one disk per machine, but you have several
> > machines, try to put client and server in different hosts. Take into
> > account that a fast ethernet bandwidth is far better than 3.7MB per
> > minute ;-)
> >
> > * If you want to do some tests for only one of the sides, another
> > approach could be to create mockups for the file readers and writers.
> > For example, if you use a DataHandler in the client side, you can
> > implement a fake DataSource that generates random bytes on the fly,
> > instead of reading them from a file. Such a DataSource will allow you to
> > test your server side performance without the client interfering in your
> > disk accesses.
> >
> > * Always use the localhost address (127.0.0.1) to contact the server. In
> > many environments you will get different (and worse) results if you use
> > the machine public IP address.
> >
> > * Make sure you do your best to "move" the file in the server side to
> > its final location instead of copying it. The performance is totally
> > different.
> >
> > Hope this helps you,
> > Rodrigo Ruiz
> >
> > Betsy Frey wrote:
> > > I've just made some measurements, transferring files using MTOM in
> > Axis2
> > > on tomcat. My client and servlet are on the same machine.
> > Attachments
> > > are cached. For files of 1mb or more, the transfer rate,
> > > client-to-servlet, is about 3.7mb per minute. In contrast, a file
> > > system copy transfers at about 2.5gb per minute: 675 times faster
> > than
> > > Axis2.
> > > Why is MTOM performance is so sluggish?
> > > Thanks for any assistance,
> > > Betsy
> > >
> > >
> > >
> > > **********************************************************************
> > > This email and any files transmitted with it are confidential and
> > > intended solely for the use of the individual or entity to whom they
> > > are addressed. Any unauthorized review, use, disclosure or
> > distribution is prohibited. If you are not the intended recipient,
> > please contact the sender by reply e-mail and destroy all copies of the
> > original message.
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > > For additional commands, e-mail: axis-user-help@ws.apache.org
> > >
> >
> > --
> > -------------------------------------------------------------------
> > GRID SYSTEMS, S.A. Rodrigo Ruiz
> > Parc Bit - Edificio 17 Research Coordinator
> > 07121 Palma de Mallorca rruiz@gridsystems.com
> > Baleares - Spain Tel: +34 971 435 085
> > http://www.gridsystems.com/ Fax: +34 971 435 082
> > -------------------------------------------------------------------
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > For additional commands, e-mail: axis-user-help@ws.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > For additional commands, e-mail: axis-user-help@ws.apache.org
> >
> >
>
>
> --
> Thilina Gunarathne - http://www.wso2.com - http://thilinag.blogspot.com
>
--
Thilina Gunarathne - http://www.wso2.com - http://thilinag.blogspot.com
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: [AXIS2] MTOM performance
Posted by Thilina Gunarathne <cs...@gmail.com>.
Please use the latest Axiom jars... I fixed some major performance
issues in the file caching lately...
~Thilina
On 3/19/07, Betsy Frey <BF...@serena.com> wrote:
> Thanks, Rodirigo, but the problem is not my disks.
> I changed the code to generate bytes on the fly, instead of from a
> source file. I changed the reference from localhost to 127.0.0.1. I am
> just looking at the file in the attachment directory; not moving or
> copying it. I continue to see it grow by about 3.7mb/minute.
> Separately, I have confirmed that the transfer is in binary in a MIME
> section; MTOM is enabled.
> So the question remains: why is MTOM performance so sluggish, and is
> there anything to be done about it?
> Thanks.
> Betsy
>
> -----Original Message-----
> From: Rodrigo Ruiz [mailto:rruiz@gridsystems.com]
> Sent: Monday, March 19, 2007 8:57 AM
> To: axis-user@ws.apache.org
> Subject: Re: [AXIS2] MTOM performance
>
> Several reasons :-)
>
> First, a file system copy can make use of low-level OS mechanisms to
> speed up transfers, and even in some cases, delegate the job to the disk
> manager chipset. You will never get these speeds with a web service, as
> it will never have direct access to the client file system.
>
> Said this, if you think about what the client and server do, leaving
> aside the overhead from XML transfer and processing, you will notice
> that having the client and server on the same machine is not a good
> testcase.
>
> On one side, the client must read the file from disk. If the client is
> correctly configured and programmed, the client will send the file
> contents to the server at the same time it is reading it (concurrently),
> so to avoid putting the whole file in memory.
>
> On the other side, the server will write the incoming bytes to a file
> while it is receiving the client request. Finally, once the whole file
> is received in the server side, your service will usually move the file
> to its final destination. If you copy the file instead of moving it, the
> service will perform even worse ;-)
>
> Now, if you join both sides, you will see the system must read and write
> the same file at different locations of the disk. This will make your
> test waste a lot of time just in disk head seeks.
>
> I would propose you to make sure that you follow these guides:
>
> * Make sure reads and writes are, at least, from/to different (physical)
> disks. If you have only one disk per machine, but you have several
> machines, try to put client and server in different hosts. Take into
> account that a fast ethernet bandwidth is far better than 3.7MB per
> minute ;-)
>
> * If you want to do some tests for only one of the sides, another
> approach could be to create mockups for the file readers and writers.
> For example, if you use a DataHandler in the client side, you can
> implement a fake DataSource that generates random bytes on the fly,
> instead of reading them from a file. Such a DataSource will allow you to
> test your server side performance without the client interfering in your
> disk accesses.
>
> * Always use the localhost address (127.0.0.1) to contact the server. In
> many environments you will get different (and worse) results if you use
> the machine public IP address.
>
> * Make sure you do your best to "move" the file in the server side to
> its final location instead of copying it. The performance is totally
> different.
>
> Hope this helps you,
> Rodrigo Ruiz
>
> Betsy Frey wrote:
> > I've just made some measurements, transferring files using MTOM in
> Axis2
> > on tomcat. My client and servlet are on the same machine.
> Attachments
> > are cached. For files of 1mb or more, the transfer rate,
> > client-to-servlet, is about 3.7mb per minute. In contrast, a file
> > system copy transfers at about 2.5gb per minute: 675 times faster
> than
> > Axis2.
> > Why is MTOM performance is so sluggish?
> > Thanks for any assistance,
> > Betsy
> >
> >
> >
> > **********************************************************************
> > This email and any files transmitted with it are confidential and
> > intended solely for the use of the individual or entity to whom they
> > are addressed. Any unauthorized review, use, disclosure or
> distribution is prohibited. If you are not the intended recipient,
> please contact the sender by reply e-mail and destroy all copies of the
> original message.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > For additional commands, e-mail: axis-user-help@ws.apache.org
> >
>
> --
> -------------------------------------------------------------------
> GRID SYSTEMS, S.A. Rodrigo Ruiz
> Parc Bit - Edificio 17 Research Coordinator
> 07121 Palma de Mallorca rruiz@gridsystems.com
> Baleares - Spain Tel: +34 971 435 085
> http://www.gridsystems.com/ Fax: +34 971 435 082
> -------------------------------------------------------------------
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>
--
Thilina Gunarathne - http://www.wso2.com - http://thilinag.blogspot.com
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
RE: [AXIS2] MTOM performance
Posted by Betsy Frey <BF...@SERENA.com>.
Thanks, Rodirigo, but the problem is not my disks.
I changed the code to generate bytes on the fly, instead of from a
source file. I changed the reference from localhost to 127.0.0.1. I am
just looking at the file in the attachment directory; not moving or
copying it. I continue to see it grow by about 3.7mb/minute.
Separately, I have confirmed that the transfer is in binary in a MIME
section; MTOM is enabled.
So the question remains: why is MTOM performance so sluggish, and is
there anything to be done about it?
Thanks.
Betsy
-----Original Message-----
From: Rodrigo Ruiz [mailto:rruiz@gridsystems.com]
Sent: Monday, March 19, 2007 8:57 AM
To: axis-user@ws.apache.org
Subject: Re: [AXIS2] MTOM performance
Several reasons :-)
First, a file system copy can make use of low-level OS mechanisms to
speed up transfers, and even in some cases, delegate the job to the disk
manager chipset. You will never get these speeds with a web service, as
it will never have direct access to the client file system.
Said this, if you think about what the client and server do, leaving
aside the overhead from XML transfer and processing, you will notice
that having the client and server on the same machine is not a good
testcase.
On one side, the client must read the file from disk. If the client is
correctly configured and programmed, the client will send the file
contents to the server at the same time it is reading it (concurrently),
so to avoid putting the whole file in memory.
On the other side, the server will write the incoming bytes to a file
while it is receiving the client request. Finally, once the whole file
is received in the server side, your service will usually move the file
to its final destination. If you copy the file instead of moving it, the
service will perform even worse ;-)
Now, if you join both sides, you will see the system must read and write
the same file at different locations of the disk. This will make your
test waste a lot of time just in disk head seeks.
I would propose you to make sure that you follow these guides:
* Make sure reads and writes are, at least, from/to different (physical)
disks. If you have only one disk per machine, but you have several
machines, try to put client and server in different hosts. Take into
account that a fast ethernet bandwidth is far better than 3.7MB per
minute ;-)
* If you want to do some tests for only one of the sides, another
approach could be to create mockups for the file readers and writers.
For example, if you use a DataHandler in the client side, you can
implement a fake DataSource that generates random bytes on the fly,
instead of reading them from a file. Such a DataSource will allow you to
test your server side performance without the client interfering in your
disk accesses.
* Always use the localhost address (127.0.0.1) to contact the server. In
many environments you will get different (and worse) results if you use
the machine public IP address.
* Make sure you do your best to "move" the file in the server side to
its final location instead of copying it. The performance is totally
different.
Hope this helps you,
Rodrigo Ruiz
Betsy Frey wrote:
> I've just made some measurements, transferring files using MTOM in
Axis2
> on tomcat. My client and servlet are on the same machine.
Attachments
> are cached. For files of 1mb or more, the transfer rate,
> client-to-servlet, is about 3.7mb per minute. In contrast, a file
> system copy transfers at about 2.5gb per minute: 675 times faster
than
> Axis2.
> Why is MTOM performance is so sluggish?
> Thanks for any assistance,
> Betsy
>
>
>
> **********************************************************************
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient,
please contact the sender by reply e-mail and destroy all copies of the
original message.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
--
-------------------------------------------------------------------
GRID SYSTEMS, S.A. Rodrigo Ruiz
Parc Bit - Edificio 17 Research Coordinator
07121 Palma de Mallorca rruiz@gridsystems.com
Baleares - Spain Tel: +34 971 435 085
http://www.gridsystems.com/ Fax: +34 971 435 082
-------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org
Re: [AXIS2] MTOM performance
Posted by Rodrigo Ruiz <rr...@gridsystems.com>.
Several reasons :-)
First, a file system copy can make use of low-level OS mechanisms to
speed up transfers, and even in some cases, delegate the job to the disk
manager chipset. You will never get these speeds with a web service, as
it will never have direct access to the client file system.
Said this, if you think about what the client and server do, leaving
aside the overhead from XML transfer and processing, you will notice
that having the client and server on the same machine is not a good
testcase.
On one side, the client must read the file from disk. If the client is
correctly configured and programmed, the client will send the file
contents to the server at the same time it is reading it (concurrently),
so to avoid putting the whole file in memory.
On the other side, the server will write the incoming bytes to a file
while it is receiving the client request. Finally, once the whole file
is received in the server side, your service will usually move the file
to its final destination. If you copy the file instead of moving it, the
service will perform even worse ;-)
Now, if you join both sides, you will see the system must read and write
the same file at different locations of the disk. This will make your
test waste a lot of time just in disk head seeks.
I would propose you to make sure that you follow these guides:
* Make sure reads and writes are, at least, from/to different (physical)
disks. If you have only one disk per machine, but you have several
machines, try to put client and server in different hosts. Take into
account that a fast ethernet bandwidth is far better than 3.7MB per
minute ;-)
* If you want to do some tests for only one of the sides, another
approach could be to create mockups for the file readers and writers.
For example, if you use a DataHandler in the client side, you can
implement a fake DataSource that generates random bytes on the fly,
instead of reading them from a file. Such a DataSource will allow you to
test your server side performance without the client interfering in your
disk accesses.
* Always use the localhost address (127.0.0.1) to contact the server. In
many environments you will get different (and worse) results if you use
the machine public IP address.
* Make sure you do your best to "move" the file in the server side to
its final location instead of copying it. The performance is totally
different.
Hope this helps you,
Rodrigo Ruiz
Betsy Frey wrote:
> I've just made some measurements, transferring files using MTOM in Axis2
> on tomcat. My client and servlet are on the same machine. Attachments
> are cached. For files of 1mb or more, the transfer rate,
> client-to-servlet, is about 3.7mb per minute. In contrast, a file
> system copy transfers at about 2.5gb per minute: 675 times faster than
> Axis2.
> Why is MTOM performance is so sluggish?
> Thanks for any assistance,
> Betsy
>
>
>
> **********************************************************************
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
--
-------------------------------------------------------------------
GRID SYSTEMS, S.A. Rodrigo Ruiz
Parc Bit - Edificio 17 Research Coordinator
07121 Palma de Mallorca rruiz@gridsystems.com
Baleares - Spain Tel: +34 971 435 085
http://www.gridsystems.com/ Fax: +34 971 435 082
-------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org