You are viewing a plain text version of this content. The canonical link for it is here.
Posted to soap-user@ws.apache.org by Anthony LaMarca <ag...@yahoo.com> on 2001/10/19 18:31:54 UTC

Terrible soap performance...Help!

Ack! I'm stuck, and hoping one you wonderful people can bail me out.

I'm running soap 2.2 and I'm getting terrible performance running the server on
a windows 2000 box. I've run soap on both tomcat and jetty and I get the same
thing: about 200ms for a roundtrip call. My service is trivial, returns a fixed
string. Its not startup costs; if I do the calls over and over again they're
all about 200ms. If I move the server over to a (much slower) linux box, the
call time drops to around 20ms. All of the machines are connected to the same
100MB/s hub, so there shouldn't be any network latency issues. The client is
also a win2k box.

Any idea why win2k would be screwed up but linux (redhat 7.2) would be ok?
My only thought is that our dns setup might be wacky, our sys-admin is kind of
clueless. But I'm invoking the services with ip addresses rather than
hostnames, so I'm not sure dns has anything to do with it.

how can I fix this? 

Thank you thank you thank you!!!!!!!

-Anthony

__________________________________________________
Do You Yahoo!?
Make a great connection at Yahoo! Personals.
http://personals.yahoo.com

Re: Terrible soap performance...Help!

Posted by Heitzso <he...@home.com>.
Your performance matches what I've seen on Win2K with sun 1.3.1, 
1.3.1_01 and
ibm's lastest 1.3.0.   I don't believe you're going to get the overhead 
down from
there on a Win2K box until we jump to the next architecture of apache soap.

I've seen a wide range of times on Linux when I timed with different JVMs.
However, I don't recall seeing times drop down to the level you're 
reporting.  
What JVM/OS/version are you running on Linux?

Heitzso


Anthony LaMarca wrote:

>Ack! I'm stuck, and hoping one you wonderful people can bail me out.
>
>I'm running soap 2.2 and I'm getting terrible performance running the server on
>a windows 2000 box. I've run soap on both tomcat and jetty and I get the same
>thing: about 200ms for a roundtrip call. My service is trivial, returns a fixed
>string. Its not startup costs; if I do the calls over and over again they're
>all about 200ms. If I move the server over to a (much slower) linux box, the
>call time drops to around 20ms. All of the machines are connected to the same
>100MB/s hub, so there shouldn't be any network latency issues. The client is
>also a win2k box.
>
>Any idea why win2k would be screwed up but linux (redhat 7.2) would be ok?
>My only thought is that our dns setup might be wacky, our sys-admin is kind of
>clueless. But I'm invoking the services with ip addresses rather than
>hostnames, so I'm not sure dns has anything to do with it.
>
>how can I fix this? 
>
>Thank you thank you thank you!!!!!!!
>
>-Anthony
>
>__________________________________________________
>Do You Yahoo!?
>Make a great connection at Yahoo! Personals.
>http://personals.yahoo.com
>




Re: Terrible soap performance...Help!

Posted by Heitzso <he...@home.com>.
Your performance matches what I've seen on Win2K with sun 1.3.1, 
1.3.1_01 and
ibm's lastest 1.3.0.   I don't believe you're going to get the overhead 
down from
there on a Win2K box until we jump to the next architecture of apache soap.

I've seen a wide range of times on Linux when I timed with different JVMs.
However, I don't recall seeing times drop down to the level you're 
reporting.  
What JVM/OS/version are you running on Linux?

Heitzso


Anthony LaMarca wrote:

>Ack! I'm stuck, and hoping one you wonderful people can bail me out.
>
>I'm running soap 2.2 and I'm getting terrible performance running the server on
>a windows 2000 box. I've run soap on both tomcat and jetty and I get the same
>thing: about 200ms for a roundtrip call. My service is trivial, returns a fixed
>string. Its not startup costs; if I do the calls over and over again they're
>all about 200ms. If I move the server over to a (much slower) linux box, the
>call time drops to around 20ms. All of the machines are connected to the same
>100MB/s hub, so there shouldn't be any network latency issues. The client is
>also a win2k box.
>
>Any idea why win2k would be screwed up but linux (redhat 7.2) would be ok?
>My only thought is that our dns setup might be wacky, our sys-admin is kind of
>clueless. But I'm invoking the services with ip addresses rather than
>hostnames, so I'm not sure dns has anything to do with it.
>
>how can I fix this? 
>
>Thank you thank you thank you!!!!!!!
>
>-Anthony
>
>__________________________________________________
>Do You Yahoo!?
>Make a great connection at Yahoo! Personals.
>http://personals.yahoo.com
>




RE: Terrible soap performance...Help!

Posted by Barnaby James <ba...@adobe.com>.
Hmm. I figured as much. I guess you need to look at the source and see what
happens inbetween. Which XML parser is it using? (With JAXP this is
sometimes not that easy to figure out!) That could be slow. Or maybe the
JavaBean introspection stuff is not cached. When I was trying to improve the
performence of the product I was working on, I wrote a "lightweight"
profiling class. It was just a singleton class and you could time something
(just using the Date class) and then put it in a bucket with a name (like
"parseDocument") -- it would calculate stats on the timings like
min/max/average etc. Then you run a web whacker on it sending lots of
requests to the server and after you print out the stats. This gives you a
way of seeing how long things take on the server when it is under load.
Profilers (in java especially) don't work very well for this type of thing
since they bring the server to a crawl so anything that is a contention
bottleneck won't really show up.

It might be worth posting your findings to soap-dev -- probably someone
there would know more about what the fixed request overhead is and why it
might be so large. If I get some free cycles, I might take a look at it
myself.

Barnaby

-----Original Message-----
From: Anthony LaMarca [mailto:aglamarca@yahoo.com]
Sent: Friday, October 19, 2001 11:24 AM
To: soap-user@xml.apache.org
Subject: RE: Terrible soap performance...Help!



Barnaby,

Whoo, you're barking up the right tree. A simple servlet is fast (~5ms) from
win2k to win2k. So the problem seems to be in the soap stuff. So, any
thoughts
as to where to look? Could soap be using TCP slow-start? Any idea what soap
2.2
would be doing for 200ms on win2k that it would do on linux?

-Anthony

> Yeah, but it goes through all of the SOAP stuff to do it! I was suggesting
> doing it as a servlet to determine if it is the servlet container /
network
> issues, or something specific to the SOAP layer. For example, to return 3,
> it has to generate an XML document (amongst other things).


__________________________________________________
Do You Yahoo!?
Make a great connection at Yahoo! Personals.
http://personals.yahoo.com


RE: Terrible soap performance...Help!

Posted by Barnaby James <ba...@adobe.com>.
Hmm. I figured as much. I guess you need to look at the source and see what
happens inbetween. Which XML parser is it using? (With JAXP this is
sometimes not that easy to figure out!) That could be slow. Or maybe the
JavaBean introspection stuff is not cached. When I was trying to improve the
performence of the product I was working on, I wrote a "lightweight"
profiling class. It was just a singleton class and you could time something
(just using the Date class) and then put it in a bucket with a name (like
"parseDocument") -- it would calculate stats on the timings like
min/max/average etc. Then you run a web whacker on it sending lots of
requests to the server and after you print out the stats. This gives you a
way of seeing how long things take on the server when it is under load.
Profilers (in java especially) don't work very well for this type of thing
since they bring the server to a crawl so anything that is a contention
bottleneck won't really show up.

It might be worth posting your findings to soap-dev -- probably someone
there would know more about what the fixed request overhead is and why it
might be so large. If I get some free cycles, I might take a look at it
myself.

Barnaby

-----Original Message-----
From: Anthony LaMarca [mailto:aglamarca@yahoo.com]
Sent: Friday, October 19, 2001 11:24 AM
To: soap-user@xml.apache.org
Subject: RE: Terrible soap performance...Help!



Barnaby,

Whoo, you're barking up the right tree. A simple servlet is fast (~5ms) from
win2k to win2k. So the problem seems to be in the soap stuff. So, any
thoughts
as to where to look? Could soap be using TCP slow-start? Any idea what soap
2.2
would be doing for 200ms on win2k that it would do on linux?

-Anthony

> Yeah, but it goes through all of the SOAP stuff to do it! I was suggesting
> doing it as a servlet to determine if it is the servlet container /
network
> issues, or something specific to the SOAP layer. For example, to return 3,
> it has to generate an XML document (amongst other things).


__________________________________________________
Do You Yahoo!?
Make a great connection at Yahoo! Personals.
http://personals.yahoo.com


RE: Terrible soap performance...Help!

Posted by Anthony LaMarca <ag...@yahoo.com>.
Barnaby,

Whoo, you're barking up the right tree. A simple servlet is fast (~5ms) from
win2k to win2k. So the problem seems to be in the soap stuff. So, any thoughts
as to where to look? Could soap be using TCP slow-start? Any idea what soap 2.2
would be doing for 200ms on win2k that it would do on linux?

-Anthony

> Yeah, but it goes through all of the SOAP stuff to do it! I was suggesting
> doing it as a servlet to determine if it is the servlet container / network
> issues, or something specific to the SOAP layer. For example, to return 3,
> it has to generate an XML document (amongst other things).


__________________________________________________
Do You Yahoo!?
Make a great connection at Yahoo! Personals.
http://personals.yahoo.com

RE: Terrible soap performance...Help!

Posted by Anthony LaMarca <ag...@yahoo.com>.
Barnaby,

Whoo, you're barking up the right tree. A simple servlet is fast (~5ms) from
win2k to win2k. So the problem seems to be in the soap stuff. So, any thoughts
as to where to look? Could soap be using TCP slow-start? Any idea what soap 2.2
would be doing for 200ms on win2k that it would do on linux?

-Anthony

> Yeah, but it goes through all of the SOAP stuff to do it! I was suggesting
> doing it as a servlet to determine if it is the servlet container / network
> issues, or something specific to the SOAP layer. For example, to return 3,
> it has to generate an XML document (amongst other things).


__________________________________________________
Do You Yahoo!?
Make a great connection at Yahoo! Personals.
http://personals.yahoo.com

RE: Terrible soap performance...Help!

Posted by Barnaby James <ba...@adobe.com>.
> Barnaby:  My soap service is already the equivelant of 'Hello World'
(returns
> '3' rather than 'Hellp World')

Yeah, but it goes through all of the SOAP stuff to do it! I was suggesting
doing it as a servlet to determine if it is the servlet container / network
issues, or something specific to the SOAP layer. For example, to return 3,
it has to generate an XML document (amongst other things).

>> socket.setTcpNoDelay(true))

> Any idea where the outgoing socket to do this to would be? (What piece of
code?
> I assume I should get the source version of soap and monkey with it. Or
should
> I be messing with the tomcat source instead?)

I'm pretty sure you would have to monkey with the Tomcat code to do this. In
3.2.3 I think the factory for sockets is
org.apache.tomcat.net.ServerSocketFactory. It's probably some place
different if you are using Catalina.

Barnaby


RE: Terrible soap performance...Help!

Posted by Barnaby James <ba...@adobe.com>.
> Barnaby:  My soap service is already the equivelant of 'Hello World'
(returns
> '3' rather than 'Hellp World')

Yeah, but it goes through all of the SOAP stuff to do it! I was suggesting
doing it as a servlet to determine if it is the servlet container / network
issues, or something specific to the SOAP layer. For example, to return 3,
it has to generate an XML document (amongst other things).

>> socket.setTcpNoDelay(true))

> Any idea where the outgoing socket to do this to would be? (What piece of
code?
> I assume I should get the source version of soap and monkey with it. Or
should
> I be messing with the tomcat source instead?)

I'm pretty sure you would have to monkey with the Tomcat code to do this. In
3.2.3 I think the factory for sockets is
org.apache.tomcat.net.ServerSocketFactory. It's probably some place
different if you are using Catalina.

Barnaby


RE: Terrible soap performance...Help!

Posted by Anthony LaMarca <ag...@yahoo.com>.
Thanks for th great responses.

Barnaby:  My soap service is already the equivelant of 'Hello World' (returns
'3' rather than 'Hellp World')

Heitzso:

I fibbed a bit, I just re-checked the linux and the round trip call time varied
between 31ms and 47ms. Still pretty respecable. Running Java 1.3.0_02 w/
hotspot, tomcat 3.2.3 and soap 2.2

Glen (or anybody):

> socket.setTcpNoDelay(true))

Any idea where the outgoing socket to do this to would be? (What piece of code?
I assume I should get the source version of soap and monkey with it. Or should
I be messing with the tomcat source instead?)

Thanks!

-Anthony


__________________________________________________
Do You Yahoo!?
Make a great connection at Yahoo! Personals.
http://personals.yahoo.com

RE: Terrible soap performance...Help!

Posted by Anthony LaMarca <ag...@yahoo.com>.
Thanks for th great responses.

Barnaby:  My soap service is already the equivelant of 'Hello World' (returns
'3' rather than 'Hellp World')

Heitzso:

I fibbed a bit, I just re-checked the linux and the round trip call time varied
between 31ms and 47ms. Still pretty respecable. Running Java 1.3.0_02 w/
hotspot, tomcat 3.2.3 and soap 2.2

Glen (or anybody):

> socket.setTcpNoDelay(true))

Any idea where the outgoing socket to do this to would be? (What piece of code?
I assume I should get the source version of soap and monkey with it. Or should
I be messing with the tomcat source instead?)

Thanks!

-Anthony


__________________________________________________
Do You Yahoo!?
Make a great connection at Yahoo! Personals.
http://personals.yahoo.com

RE: Terrible soap performance...Help!

Posted by Barnaby James <ba...@adobe.com>.
You need to isolate whether it is the servlet container or SOAP. Try making
a really trivial Hello World Servlet and see what the performence is with
that. If that is OK, then it's something in SOAP -- maybe there is a fixed
overhead for every SOAP call regardless of it's simplicity. For example,
parsing XML documents is not exactly cheap!

200 ms sounds pretty bad to me. I did alot of perfomence testing of Servlet
containers in a previous job (NT and Solaris) and while Tomcat 3.2.3 is
pretty poor in performence compared to something like Caucho Resin (which
rocks!) I don't think it is that bad. Anyone have any performence statistics
(anecdotal or otherwise) on the performence of Tomcat 4.0 vs Tomcat 3.2.3.

I'd be interested to hear what you come up with.

Barnaby

-----Original Message-----
From: Anthony LaMarca [mailto:aglamarca@yahoo.com]
Sent: Friday, October 19, 2001 9:32 AM
To: soap-user@xml.apache.org
Subject: Terrible soap performance...Help!



Ack! I'm stuck, and hoping one you wonderful people can bail me out.

I'm running soap 2.2 and I'm getting terrible performance running the server
on
a windows 2000 box. I've run soap on both tomcat and jetty and I get the
same
thing: about 200ms for a roundtrip call. My service is trivial, returns a
fixed
string. Its not startup costs; if I do the calls over and over again they're
all about 200ms. If I move the server over to a (much slower) linux box, the
call time drops to around 20ms. All of the machines are connected to the
same
100MB/s hub, so there shouldn't be any network latency issues. The client is
also a win2k box.

Any idea why win2k would be screwed up but linux (redhat 7.2) would be ok?
My only thought is that our dns setup might be wacky, our sys-admin is kind
of
clueless. But I'm invoking the services with ip addresses rather than
hostnames, so I'm not sure dns has anything to do with it.

how can I fix this?

Thank you thank you thank you!!!!!!!

-Anthony

__________________________________________________
Do You Yahoo!?
Make a great connection at Yahoo! Personals.
http://personals.yahoo.com


RE: Terrible soap performance...Help!

Posted by Barnaby James <ba...@adobe.com>.
You need to isolate whether it is the servlet container or SOAP. Try making
a really trivial Hello World Servlet and see what the performence is with
that. If that is OK, then it's something in SOAP -- maybe there is a fixed
overhead for every SOAP call regardless of it's simplicity. For example,
parsing XML documents is not exactly cheap!

200 ms sounds pretty bad to me. I did alot of perfomence testing of Servlet
containers in a previous job (NT and Solaris) and while Tomcat 3.2.3 is
pretty poor in performence compared to something like Caucho Resin (which
rocks!) I don't think it is that bad. Anyone have any performence statistics
(anecdotal or otherwise) on the performence of Tomcat 4.0 vs Tomcat 3.2.3.

I'd be interested to hear what you come up with.

Barnaby

-----Original Message-----
From: Anthony LaMarca [mailto:aglamarca@yahoo.com]
Sent: Friday, October 19, 2001 9:32 AM
To: soap-user@xml.apache.org
Subject: Terrible soap performance...Help!



Ack! I'm stuck, and hoping one you wonderful people can bail me out.

I'm running soap 2.2 and I'm getting terrible performance running the server
on
a windows 2000 box. I've run soap on both tomcat and jetty and I get the
same
thing: about 200ms for a roundtrip call. My service is trivial, returns a
fixed
string. Its not startup costs; if I do the calls over and over again they're
all about 200ms. If I move the server over to a (much slower) linux box, the
call time drops to around 20ms. All of the machines are connected to the
same
100MB/s hub, so there shouldn't be any network latency issues. The client is
also a win2k box.

Any idea why win2k would be screwed up but linux (redhat 7.2) would be ok?
My only thought is that our dns setup might be wacky, our sys-admin is kind
of
clueless. But I'm invoking the services with ip addresses rather than
hostnames, so I'm not sure dns has anything to do with it.

how can I fix this?

Thank you thank you thank you!!!!!!!

-Anthony

__________________________________________________
Do You Yahoo!?
Make a great connection at Yahoo! Personals.
http://personals.yahoo.com