You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cxf.apache.org by user239 <us...@gmail.com> on 2014/12/03 10:28:11 UTC
Async transport performance using cxf-rt-transports-http-hc
I was trying to see if there are any advantages of using the async transport
over the regular sync http transport. So I wrote a small test program that
makes 2000 concurrent requests. The server simply waits in a non-blocking
fashion for 1-2 seconds and returns the response.
I profiled it in Netbeans and the results were a bit surprising. Both
transports completed all the requests in the same time (30-35 seconds) and
used the same number of threads (around 30). But the async transport used
200 MB of memory, while the sync one didn't exceed 50 MB according to the VM
Telemetry Overview.
So I have a couple of questions:
1. Does this look like a memory issue with the async transport? Or is this
by design?
2. In what scenario does the async transport clearly outperform the sync
one? Looks like in the program I wrote it was the same and even consumed
much more memory. Or maybe I need to tweak some server/client settings?
I was thinking about using the async transport in a new project for better
scalability, but after running these tests I'm not sure there are any
benefits.
My client code basically looks like this (I'm using CXF 3.0.2 and JDK 6):
MyService service = new MyService();
final int N = 2000;
final Date startTime = new Date();
final AtomicInteger runningRequestCount = new AtomicInteger(N);
final IMyService client = service.getBasicHttpBindingIMyService();
for (long i = 0; i < N; i++) {
client.TestMethodAsync(i, new AsyncHandler<TestMethodResponse>() {
@Override
public void handleResponse(Response<TestMethodResponse>
response) {
try {
Result result = response.get().getTestMethodResult();
// print some information from the "result"
if (runningRequestCount.decrementAndGet() == 0) {
Date endTime = new Date();
Long elapsedTime = endTime.getTime() -
startTime.getTime();
System.out.println("All completed! Took " +
elapsedTime + " milliseconds");
System.exit(0);
}
} catch (InterruptedException ex) {
Logger.getLogger(MyProgram.class.getName()).log(Level.SEVERE, null, ex);
} catch (ExecutionException ex) {
Logger.getLogger(MyProgram.class.getName()).log(Level.SEVERE, null, ex);
}
}
});
}
--
View this message in context: http://cxf.547215.n5.nabble.com/Async-transport-performance-using-cxf-rt-transports-http-hc-tp5751832.html
Sent from the cxf-user mailing list archive at Nabble.com.
Re: Async transport performance using cxf-rt-transports-http-hc
Posted by Aki Yoshida <el...@gmail.com>.
do you have cxf-rt-transports-http-hc in your dependency?
I also saw a few difficulties in getting it work nicely when I was
pushing some high traffic a while ago.
I could get a higher throughput with the async-option but experienced
that the connection handling was sensitively influenced by the load.
I wanted to look into it sometime.
2014-12-03 10:28 GMT+01:00 user239 <us...@gmail.com>:
> I was trying to see if there are any advantages of using the async transport
> over the regular sync http transport. So I wrote a small test program that
> makes 2000 concurrent requests. The server simply waits in a non-blocking
> fashion for 1-2 seconds and returns the response.
>
> I profiled it in Netbeans and the results were a bit surprising. Both
> transports completed all the requests in the same time (30-35 seconds) and
> used the same number of threads (around 30). But the async transport used
> 200 MB of memory, while the sync one didn't exceed 50 MB according to the VM
> Telemetry Overview.
>
> So I have a couple of questions:
> 1. Does this look like a memory issue with the async transport? Or is this
> by design?
> 2. In what scenario does the async transport clearly outperform the sync
> one? Looks like in the program I wrote it was the same and even consumed
> much more memory. Or maybe I need to tweak some server/client settings?
>
> I was thinking about using the async transport in a new project for better
> scalability, but after running these tests I'm not sure there are any
> benefits.
>
> My client code basically looks like this (I'm using CXF 3.0.2 and JDK 6):
>
> MyService service = new MyService();
>
> final int N = 2000;
>
> final Date startTime = new Date();
>
> final AtomicInteger runningRequestCount = new AtomicInteger(N);
>
> final IMyService client = service.getBasicHttpBindingIMyService();
>
> for (long i = 0; i < N; i++) {
> client.TestMethodAsync(i, new AsyncHandler<TestMethodResponse>() {
> @Override
> public void handleResponse(Response<TestMethodResponse>
> response) {
> try {
> Result result = response.get().getTestMethodResult();
>
> // print some information from the "result"
>
> if (runningRequestCount.decrementAndGet() == 0) {
> Date endTime = new Date();
>
> Long elapsedTime = endTime.getTime() -
> startTime.getTime();
>
> System.out.println("All completed! Took " +
> elapsedTime + " milliseconds");
>
> System.exit(0);
> }
> } catch (InterruptedException ex) {
>
> Logger.getLogger(MyProgram.class.getName()).log(Level.SEVERE, null, ex);
> } catch (ExecutionException ex) {
>
> Logger.getLogger(MyProgram.class.getName()).log(Level.SEVERE, null, ex);
> }
> }
> });
> }
>
>
>
> --
> View this message in context: http://cxf.547215.n5.nabble.com/Async-transport-performance-using-cxf-rt-transports-http-hc-tp5751832.html
> Sent from the cxf-user mailing list archive at Nabble.com.
RE: Async transport performance using cxf-rt-transports-http-hc
Posted by Andrei Shakirin <as...@talend.com>.
Hi,
> -----Original Message-----
> From: user239 [mailto:user239@gmail.com]
> Sent: Mittwoch, 3. Dezember 2014 21:17
> To: users@cxf.apache.org
> Subject: Re: Async transport performance using cxf-rt-transports-http-hc
>
> Thanks, Sergey, yes, I'm actually using callbacks here, but looks like they don't
> require the async transport. The regular one (cxf-rt-transports-http) also works
> fine and shows similar (or even better) performance.
>
> I debugged it further and noticed that the regular (non-async) transport always
> creates 24 workqueue threads to execute the requests. I modified the test so
> that the server now has a 5-second delay and the client only sends
> 75 concurrent requests. Somehow the non-async transport managed to initiate
> all 75 requests at the same time using only 24 threads! And they all completed
> in 5.7 seconds. How is that possible? Does it actually somehow use NIO under
> the hood?
Very interesting, I expect that async transport is slower for singly/low number of requests, but scale better.
If number parallel requests are essentially high as number of available threads async transport should benefit.
Yes, if cxf-rt-transports-http-hc module is found on the classpath, CXF will use NIO channel.
Regards,
Andrei.
>
> I also saw this
> <http://cxf.547215.n5.nabble.com/Async-http-client-experiments-
> td5711683.html>
> threadby Daniel Kulp where he got 4x performance improvement by using the
> async transport (from 35 seconds down to 9 seconds). But in my experiments
> they are almost the same. Or maybe the regular transport became more
> efficient since 2012?
>
>
>
> --
> View this message in context: http://cxf.547215.n5.nabble.com/Async-
> transport-performance-using-cxf-rt-transports-http-hc-
> tp5751832p5751876.html
> Sent from the cxf-user mailing list archive at Nabble.com.
Re: Async transport performance using cxf-rt-transports-http-hc
Posted by user239 <us...@gmail.com>.
Thanks, Sergey, yes, I'm actually using callbacks here, but looks like they
don't require the async transport. The regular one (cxf-rt-transports-http)
also works fine and shows similar (or even better) performance.
I debugged it further and noticed that the regular (non-async) transport
always creates 24 workqueue threads to execute the requests. I modified the
test so that the server now has a 5-second delay and the client only sends
75 concurrent requests. Somehow the non-async transport managed to initiate
all 75 requests at the same time using only 24 threads! And they all
completed in 5.7 seconds. How is that possible? Does it actually somehow use
NIO under the hood?
I also saw this
<http://cxf.547215.n5.nabble.com/Async-http-client-experiments-td5711683.html>
threadby Daniel Kulp where he got 4x performance improvement by using the
async transport (from 35 seconds down to 9 seconds). But in my experiments
they are almost the same. Or maybe the regular transport became more
efficient since 2012?
--
View this message in context: http://cxf.547215.n5.nabble.com/Async-transport-performance-using-cxf-rt-transports-http-hc-tp5751832p5751876.html
Sent from the cxf-user mailing list archive at Nabble.com.
Re: Async transport performance using cxf-rt-transports-http-hc
Posted by Sergey Beryozkin <sb...@gmail.com>.
I think the primary benefit of working with the async transport is that
it supports an asynchronous style of client programming, where Future,
Invocation callbacks are used.
Though it is interesting indeed how it can be optimized, given that it
can be used to do the sync invocations
Cheers, Sergey
On 03/12/14 09:28, user239 wrote:
> I was trying to see if there are any advantages of using the async transport
> over the regular sync http transport. So I wrote a small test program that
> makes 2000 concurrent requests. The server simply waits in a non-blocking
> fashion for 1-2 seconds and returns the response.
>
> I profiled it in Netbeans and the results were a bit surprising. Both
> transports completed all the requests in the same time (30-35 seconds) and
> used the same number of threads (around 30). But the async transport used
> 200 MB of memory, while the sync one didn't exceed 50 MB according to the VM
> Telemetry Overview.
>
> So I have a couple of questions:
> 1. Does this look like a memory issue with the async transport? Or is this
> by design?
> 2. In what scenario does the async transport clearly outperform the sync
> one? Looks like in the program I wrote it was the same and even consumed
> much more memory. Or maybe I need to tweak some server/client settings?
>
> I was thinking about using the async transport in a new project for better
> scalability, but after running these tests I'm not sure there are any
> benefits.
>
> My client code basically looks like this (I'm using CXF 3.0.2 and JDK 6):
>
> MyService service = new MyService();
>
> final int N = 2000;
>
> final Date startTime = new Date();
>
> final AtomicInteger runningRequestCount = new AtomicInteger(N);
>
> final IMyService client = service.getBasicHttpBindingIMyService();
>
> for (long i = 0; i < N; i++) {
> client.TestMethodAsync(i, new AsyncHandler<TestMethodResponse>() {
> @Override
> public void handleResponse(Response<TestMethodResponse>
> response) {
> try {
> Result result = response.get().getTestMethodResult();
>
> // print some information from the "result"
>
> if (runningRequestCount.decrementAndGet() == 0) {
> Date endTime = new Date();
>
> Long elapsedTime = endTime.getTime() -
> startTime.getTime();
>
> System.out.println("All completed! Took " +
> elapsedTime + " milliseconds");
>
> System.exit(0);
> }
> } catch (InterruptedException ex) {
>
> Logger.getLogger(MyProgram.class.getName()).log(Level.SEVERE, null, ex);
> } catch (ExecutionException ex) {
>
> Logger.getLogger(MyProgram.class.getName()).log(Level.SEVERE, null, ex);
> }
> }
> });
> }
>
>
>
> --
> View this message in context: http://cxf.547215.n5.nabble.com/Async-transport-performance-using-cxf-rt-transports-http-hc-tp5751832.html
> Sent from the cxf-user mailing list archive at Nabble.com.
>
Re: Async transport performance using cxf-rt-transports-http-hc
Posted by Christian Schneider <ch...@die-schneider.net>.
Ok .. then I missunderstood. I thought you were using cxf aync http on
the server side.
As far as I know the async http transport is mainly useful at the server
side. There it can keep the thread count low.
On the client side there are normally never more than about 100 threads
for typical use cases.
Christian
On 03.12.2014 21:01, user239 wrote:
> Christian, my server code is really simple, just waiting asynchronously for
> 1-2 seconds, so I don't expect it to be a bottleneck. Even when I send just
> 100 request, it's the same result: both transports show similar times.
>
> (written in C#)
> static Random rnd = new Random();
>
> public async Task<Result> TestMethod(long id)
> {
> var result = new Result { Id = id };
>
> // set some properties on "result"
>
> await Task.Delay(rnd.Next(1000, 2000));
>
> return result;
> }
>
>
>
> --
> View this message in context: http://cxf.547215.n5.nabble.com/Async-transport-performance-using-cxf-rt-transports-http-hc-tp5751832p5751875.html
> Sent from the cxf-user mailing list archive at Nabble.com.
--
Christian Schneider
http://www.liquid-reality.de
Open Source Architect
http://www.talend.com
Re: Async transport performance using cxf-rt-transports-http-hc
Posted by user239 <us...@gmail.com>.
After doing some more research, I think at this point I should reformulate my
question... What I really want to understand is this:
How does the standard NON-async transport (cxf-rt-transports-http) work on
the client for Async methods with callbacks? Looks like it doesn't just
queue outstanding requests (when there are more than 24 of them). It's much
smarter than that. Because if I have 50 requests or more, all of them are
sent to the server at the same and come back at the same time, even though
there are only 24 active threads.
What magic does it do? Especially for sending the requests. I could look at
the code if I knew where to start... I think understanding how it works will
help me understand the difference between "transports-http" and
"transports-http-hc" and will make it more clear when to use which.
--
View this message in context: http://cxf.547215.n5.nabble.com/Async-transport-performance-using-cxf-rt-transports-http-hc-tp5751832p5751949.html
Sent from the cxf-user mailing list archive at Nabble.com.
Re: Async transport performance using cxf-rt-transports-http-hc
Posted by Daniel Kulp <dk...@apache.org>.
I really don’t know anything about C#, but are you sure the server can accept 1000 simultaneous requests and have all 1000 dispatched into the TestMethod? This sounds more like a server side bottleneck.
For example, I know with our Jetty based standalone services, the max would be 250ish (255 maximum threads, but some of the threads are used for the acceptors). For the older versions of jetty, the max would have been 16.
Dan
> On Dec 3, 2014, at 3:01 PM, user239 <us...@gmail.com> wrote:
>
> Christian, my server code is really simple, just waiting asynchronously for
> 1-2 seconds, so I don't expect it to be a bottleneck. Even when I send just
> 100 request, it's the same result: both transports show similar times.
>
> (written in C#)
> static Random rnd = new Random();
>
> public async Task<Result> TestMethod(long id)
> {
> var result = new Result { Id = id };
>
> // set some properties on "result"
>
> await Task.Delay(rnd.Next(1000, 2000));
>
> return result;
> }
>
>
>
> --
> View this message in context: http://cxf.547215.n5.nabble.com/Async-transport-performance-using-cxf-rt-transports-http-hc-tp5751832p5751875.html
> Sent from the cxf-user mailing list archive at Nabble.com.
--
Daniel Kulp
dkulp@apache.org - http://dankulp.com/blog
Talend Community Coder - http://coders.talend.com
Re: Async transport performance using cxf-rt-transports-http-hc
Posted by user239 <us...@gmail.com>.
Christian, my server code is really simple, just waiting asynchronously for
1-2 seconds, so I don't expect it to be a bottleneck. Even when I send just
100 request, it's the same result: both transports show similar times.
(written in C#)
static Random rnd = new Random();
public async Task<Result> TestMethod(long id)
{
var result = new Result { Id = id };
// set some properties on "result"
await Task.Delay(rnd.Next(1000, 2000));
return result;
}
--
View this message in context: http://cxf.547215.n5.nabble.com/Async-transport-performance-using-cxf-rt-transports-http-hc-tp5751832p5751875.html
Sent from the cxf-user mailing list archive at Nabble.com.
Re: Async transport performance using cxf-rt-transports-http-hc
Posted by Christian Schneider <ch...@die-schneider.net>.
Howw does your code on the server side look like?
Christian
On 03.12.2014 10:28, user239 wrote:
> I was trying to see if there are any advantages of using the async transport
> over the regular sync http transport. So I wrote a small test program that
> makes 2000 concurrent requests. The server simply waits in a non-blocking
> fashion for 1-2 seconds and returns the response.
>
> I profiled it in Netbeans and the results were a bit surprising. Both
> transports completed all the requests in the same time (30-35 seconds) and
> used the same number of threads (around 30). But the async transport used
> 200 MB of memory, while the sync one didn't exceed 50 MB according to the VM
> Telemetry Overview.
>
> So I have a couple of questions:
> 1. Does this look like a memory issue with the async transport? Or is this
> by design?
> 2. In what scenario does the async transport clearly outperform the sync
> one? Looks like in the program I wrote it was the same and even consumed
> much more memory. Or maybe I need to tweak some server/client settings?
>
> I was thinking about using the async transport in a new project for better
> scalability, but after running these tests I'm not sure there are any
> benefits.
>
> My client code basically looks like this (I'm using CXF 3.0.2 and JDK 6):
>
> MyService service = new MyService();
>
> final int N = 2000;
>
> final Date startTime = new Date();
>
> final AtomicInteger runningRequestCount = new AtomicInteger(N);
>
> final IMyService client = service.getBasicHttpBindingIMyService();
>
> for (long i = 0; i < N; i++) {
> client.TestMethodAsync(i, new AsyncHandler<TestMethodResponse>() {
> @Override
> public void handleResponse(Response<TestMethodResponse>
> response) {
> try {
> Result result = response.get().getTestMethodResult();
>
> // print some information from the "result"
>
> if (runningRequestCount.decrementAndGet() == 0) {
> Date endTime = new Date();
>
> Long elapsedTime = endTime.getTime() -
> startTime.getTime();
>
> System.out.println("All completed! Took " +
> elapsedTime + " milliseconds");
>
> System.exit(0);
> }
> } catch (InterruptedException ex) {
>
> Logger.getLogger(MyProgram.class.getName()).log(Level.SEVERE, null, ex);
> } catch (ExecutionException ex) {
>
> Logger.getLogger(MyProgram.class.getName()).log(Level.SEVERE, null, ex);
> }
> }
> });
> }
>
>
>
> --
> View this message in context: http://cxf.547215.n5.nabble.com/Async-transport-performance-using-cxf-rt-transports-http-hc-tp5751832.html
> Sent from the cxf-user mailing list archive at Nabble.com.
--
Christian Schneider
http://www.liquid-reality.de
Open Source Architect
http://www.talend.com