You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cxf.apache.org by rouble <r....@gmail.com> on 2012/09/11 19:08:14 UTC
CXF Performance (Requests being throttled)
CXF Gurus,
We are using CXF for a bunch of web services on our enterprise grade
servers. We were noticing that our throughput of requests was peaking
at an average of 42 transactions/second (a transaction here is 1 SOAP
request/response combo).
To investigate whether the bottleneck was in the system or in the web
service, we added a simple API (getStartTime) that just returns a
string without communicating with the rest of the system. The API has
no parameters, and just returns a small string (timestamp) - so the
size of the message is very small.
We used SOAPUI's load test feature to run multiple threads against the
server that just call getStartTime. However, once again, we can only
get a max throughput of 42 transactions per second - no matter how
many client threads we have. The system CPU is about 50% idle during
the test and we do not seem to be memory bound either.
We're running CXF 2.4.3 in Tomcat 7.0. Tomcat's config had
acceptCount="100", maxThreads="150" - which should not cause any
throttling to kick in. I set both those to 300 but that did not change
our throughput. I checked iptables - there is no throttling going on
there either. Since I have checked everything else it seems like the
bottleneck is in CXF. Is there something in CXF that throttles
requests? If so, is it configurable?
I know 42 is the answer to life the universe and everything but does
anyone have other thoughts on how to increase our system throughput?
tia,
rouble
Re: CXF Performance (Requests being throttled)
Posted by rouble <r....@gmail.com>.
Ignore my last response.
After some more profiling we discovered the following things affect
performance: logging + schema validation + custom validation +
authentication + ssl.
Without any of the above we get 1932 transactions per second. Here is
a breakdown of where we lose performance:
with ssl (tomcat configuration):
1463.41 transactions per second
24.28% decline in performance
with basic authentication (tomcat configuration):
64.725 transactions per second
96.65% decline in performance
with logging (log4j - two log statements per API call):
453.72 transactions per second
76.52% decline in performance
with schema validation enabled:
59.975 transactions per second
96.90% decline in performance
with custom validation event handler:
1929.315 transactions per second
0.17% decline in performance
The heavy hitters (that cause over 96% performance decline) are tomcat
basic authentication and schema validation enabled. We are looking to
see how we can disable these. I just wanted to share these findings
with the alias. These are easily reproducible on our setup. Let me
know if anyone has any thoughts on these numbers.
tia,
rouble
> On Wed, Sep 12, 2012 at 2:46 PM, Daniel Kulp <dk...@apache.org> wrote:
>>
>>
>> For simple things on my machine, I get upwards of 5K or more per sec. Do you have any HTTP filters or anything that may be causing an issue?
>>
>> I would start with something simple like CXF's wsdl_first example, try running that, and see what you get. Then start adding in some of your configs and such.
>>
>>
>> Dan
>>
>>
>>
>>
>> On Sep 11, 2012, at 1:08 PM, rouble <r....@gmail.com> wrote:
>>
>>> CXF Gurus,
>>>
>>> We are using CXF for a bunch of web services on our enterprise grade
>>> servers. We were noticing that our throughput of requests was peaking
>>> at an average of 42 transactions/second (a transaction here is 1 SOAP
>>> request/response combo).
>>>
>>> To investigate whether the bottleneck was in the system or in the web
>>> service, we added a simple API (getStartTime) that just returns a
>>> string without communicating with the rest of the system. The API has
>>> no parameters, and just returns a small string (timestamp) - so the
>>> size of the message is very small.
>>>
>>> We used SOAPUI's load test feature to run multiple threads against the
>>> server that just call getStartTime. However, once again, we can only
>>> get a max throughput of 42 transactions per second - no matter how
>>> many client threads we have. The system CPU is about 50% idle during
>>> the test and we do not seem to be memory bound either.
>>>
>>> We're running CXF 2.4.3 in Tomcat 7.0. Tomcat's config had
>>> acceptCount="100", maxThreads="150" - which should not cause any
>>> throttling to kick in. I set both those to 300 but that did not change
>>> our throughput. I checked iptables - there is no throttling going on
>>> there either. Since I have checked everything else it seems like the
>>> bottleneck is in CXF. Is there something in CXF that throttles
>>> requests? If so, is it configurable?
>>>
>>> I know 42 is the answer to life the universe and everything but does
>>> anyone have other thoughts on how to increase our system throughput?
>>>
>>> tia,
>>> rouble
>>
>> --
>> Daniel Kulp
>> dkulp@apache.org - http://dankulp.com/blog
>> Talend Community Coder - http://coders.talend.com
>>
>
>
>
> --
> this message is composed of 100% recycled ascii
Re: CXF Performance (Requests being throttled)
Posted by Pranab Mehta <ro...@gmail.com>.
After some more profiling we discovered the following things affect
performance: logging + schema validation + custom validation +
authentication + ssl.
Without any of the above we get 1932 transactions per second.
with ssl
1463.41
24.28%
with authentication
64.725 96.65%
with logging
453.72
76.52%
with schema validation
59.86 60.09 59.975
96.90%
with custom validation
1911.22 1947.41 1929.315
0.17%
On Wed, Sep 12, 2012 at 2:46 PM, Daniel Kulp <dk...@apache.org> wrote:
>
>
> For simple things on my machine, I get upwards of 5K or more per sec. Do you have any HTTP filters or anything that may be causing an issue?
>
> I would start with something simple like CXF's wsdl_first example, try running that, and see what you get. Then start adding in some of your configs and such.
>
>
> Dan
>
>
>
>
> On Sep 11, 2012, at 1:08 PM, rouble <r....@gmail.com> wrote:
>
>> CXF Gurus,
>>
>> We are using CXF for a bunch of web services on our enterprise grade
>> servers. We were noticing that our throughput of requests was peaking
>> at an average of 42 transactions/second (a transaction here is 1 SOAP
>> request/response combo).
>>
>> To investigate whether the bottleneck was in the system or in the web
>> service, we added a simple API (getStartTime) that just returns a
>> string without communicating with the rest of the system. The API has
>> no parameters, and just returns a small string (timestamp) - so the
>> size of the message is very small.
>>
>> We used SOAPUI's load test feature to run multiple threads against the
>> server that just call getStartTime. However, once again, we can only
>> get a max throughput of 42 transactions per second - no matter how
>> many client threads we have. The system CPU is about 50% idle during
>> the test and we do not seem to be memory bound either.
>>
>> We're running CXF 2.4.3 in Tomcat 7.0. Tomcat's config had
>> acceptCount="100", maxThreads="150" - which should not cause any
>> throttling to kick in. I set both those to 300 but that did not change
>> our throughput. I checked iptables - there is no throttling going on
>> there either. Since I have checked everything else it seems like the
>> bottleneck is in CXF. Is there something in CXF that throttles
>> requests? If so, is it configurable?
>>
>> I know 42 is the answer to life the universe and everything but does
>> anyone have other thoughts on how to increase our system throughput?
>>
>> tia,
>> rouble
>
> --
> Daniel Kulp
> dkulp@apache.org - http://dankulp.com/blog
> Talend Community Coder - http://coders.talend.com
>
--
this message is composed of 100% recycled ascii
Re: CXF Performance (Requests being throttled)
Posted by Daniel Kulp <dk...@apache.org>.
For simple things on my machine, I get upwards of 5K or more per sec. Do you have any HTTP filters or anything that may be causing an issue?
I would start with something simple like CXF's wsdl_first example, try running that, and see what you get. Then start adding in some of your configs and such.
Dan
On Sep 11, 2012, at 1:08 PM, rouble <r....@gmail.com> wrote:
> CXF Gurus,
>
> We are using CXF for a bunch of web services on our enterprise grade
> servers. We were noticing that our throughput of requests was peaking
> at an average of 42 transactions/second (a transaction here is 1 SOAP
> request/response combo).
>
> To investigate whether the bottleneck was in the system or in the web
> service, we added a simple API (getStartTime) that just returns a
> string without communicating with the rest of the system. The API has
> no parameters, and just returns a small string (timestamp) - so the
> size of the message is very small.
>
> We used SOAPUI's load test feature to run multiple threads against the
> server that just call getStartTime. However, once again, we can only
> get a max throughput of 42 transactions per second - no matter how
> many client threads we have. The system CPU is about 50% idle during
> the test and we do not seem to be memory bound either.
>
> We're running CXF 2.4.3 in Tomcat 7.0. Tomcat's config had
> acceptCount="100", maxThreads="150" - which should not cause any
> throttling to kick in. I set both those to 300 but that did not change
> our throughput. I checked iptables - there is no throttling going on
> there either. Since I have checked everything else it seems like the
> bottleneck is in CXF. Is there something in CXF that throttles
> requests? If so, is it configurable?
>
> I know 42 is the answer to life the universe and everything but does
> anyone have other thoughts on how to increase our system throughput?
>
> tia,
> rouble
--
Daniel Kulp
dkulp@apache.org - http://dankulp.com/blog
Talend Community Coder - http://coders.talend.com
RE: CXF Performance (Requests being throttled)
Posted by Andrei Shakirin <as...@talend.com>.
Hi Rouble,
42 transactions per seconds is definitely not acceptable result even for CXF 2.4.
You can see some benchmarks here http://www.liquid-reality.de/pages/viewpage.action?pageId=5865562.
It is quite difficult to analyze the problem without code and configuration of your service.
Could you provide the sources and configuration?
Regards,
Andrei.
-----Original Message-----
From: rouble@gmail.com [mailto:rouble@gmail.com] On Behalf Of rouble
Sent: Dienstag, 11. September 2012 19:08
To: users@cxf.apache.org
Subject: CXF Performance (Requests being throttled)
CXF Gurus,
We are using CXF for a bunch of web services on our enterprise grade servers. We were noticing that our throughput of requests was peaking at an average of 42 transactions/second (a transaction here is 1 SOAP request/response combo).
To investigate whether the bottleneck was in the system or in the web service, we added a simple API (getStartTime) that just returns a string without communicating with the rest of the system. The API has no parameters, and just returns a small string (timestamp) - so the size of the message is very small.
We used SOAPUI's load test feature to run multiple threads against the server that just call getStartTime. However, once again, we can only get a max throughput of 42 transactions per second - no matter how many client threads we have. The system CPU is about 50% idle during the test and we do not seem to be memory bound either.
We're running CXF 2.4.3 in Tomcat 7.0. Tomcat's config had acceptCount="100", maxThreads="150" - which should not cause any throttling to kick in. I set both those to 300 but that did not change our throughput. I checked iptables - there is no throttling going on there either. Since I have checked everything else it seems like the bottleneck is in CXF. Is there something in CXF that throttles requests? If so, is it configurable?
I know 42 is the answer to life the universe and everything but does anyone have other thoughts on how to increase our system throughput?
tia,
rouble