You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@synapse.apache.org by Oleg Kalnichevski <ol...@apache.org> on 2007/03/23 10:38:16 UTC
HttpCore stability and performance was Re: Fwd: [jira] Resolved:
(HTTPCORE-60) Transport appears to be hanging because an unchecked
exception caused the I/O dispatch thread to terminate
On Fri, 2007-03-23 at 08:51 +0000, ant elder wrote:
> I've tried on several JDKs now and _always_ get similar intermittent
> I/O related errors. I can use JMeter directly against Axis2-1.1.1
> without any problems at all, so this does look like some issue with
> the NIO transport. Be really good to hear from other Windows users to
> see if this is just my specific environment or a more general problem
> problem.
>
> To recreate:
>
> 1) build Synapse server sample by running 'ant' in the samples
> \axis2Server\src\SimpleStockQuoteService directory
> 2) start the sample service by running samples\axis2Server
> \axis2server.bat
> 3) get the Synapse config (either 8 or 501) from
> http://people.apache.org/~antelder/temp/, put in repository\conf
> \sample and start syanps: bin\synapse.bat -sample=8
> 4) get the JMeter config test1.jmx from
> http://people.apache.org/~antelder/temp/, start Jmeter and File ->
> Open and point to the test1.jmx file
> 5) JMeter Run -> Start and after not to long IO errors should appear
> in the Syanpse console
>
> ...ant
>
Anthony, Asankha and all
It _suspect_ Windows specific implementation of NIO pipes is the
culprit. By itself HttpCore can take a fair amount of load. I ran a few
tests just to give you a rough indication as to where HttpCore stands in
term of performance and stability. Full disclaimer: I run Linux, not
Windows.
The NIO transport can process 200000 HTTP GET requests over 500
simultaneous connections delivering raw throughput of 1883 messages per
second. This is not stellar but okayish enough for a start
======================================================================
Server Software: Jakarta-HttpComponents-NIO/1.1
Server Hostname: localhost
Server Port: 8080
Document Path: /index.html
Document Length: 12926 bytes
Concurrency Level: 500
Time taken for tests: 106.191969 seconds
Complete requests: 200000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 200000
Total transferred: -1674510608 bytes
HTML transferred: -1708134128 bytes
Requests per second: 1883.38 [#/sec] (mean)
Time per request: 265.480 [ms] (mean)
Time per request: 0.531 [ms] (mean, across all concurrent
requests)
Transfer rate: -15399.13 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 44.8 0 3136
Processing: 215 262 26.4 250 418
Waiting: 6 131 74.5 130 410
Total: 215 264 53.0 250 3383
Percentage of the requests served within a certain time (ms)
50% 250
66% 286
75% 289
80% 291
90% 297
95% 300
98% 311
99% 326
100% 3383 (longest request)
======================================================================
For comparison HttpCore transport based on classic I/O performs twice as
fast. I never was a big fan of NIO and these numbers go to show why. NIO
starts paying off only when dealing with several thousands of
simultaneous connections, but I am digressing.
======================================================================
Server Software: Jakarta-HttpComponents/1.1
Server Hostname: localhost
Server Port: 8080
Document Path: /index.html
Document Length: 12926 bytes
Concurrency Level: 500
Time taken for tests: 56.582312 seconds
Complete requests: 200000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 200000
Total transferred: -1676941116 bytes
HTML transferred: -1709741444 bytes
Requests per second: 3534.67 [#/sec] (mean)
Time per request: 141.456 [ms] (mean)
Time per request: 0.283 [ms] (mean, across all concurrent
requests)
Transfer rate: -28942.56 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 29 829.7 0 45069
Processing: 0 93 632.8 43 31867
Waiting: 0 88 632.3 40 31867
Total: 0 123 1363.5 43 56477
Percentage of the requests served within a certain time (ms)
50% 43
66% 90
75% 111
80% 121
90% 157
95% 199
98% 275
99% 356
100% 56477 (longest request)
======================================================================
So, bottom line, it can well be NIO pipes on Windows simply do not scale
well. Maybe. So, at some point you might look at an option of replacing
NIO pipes with some custom code based on shared buffers provided in
HttpCore NIO extensions.
Cheers
Oleg
> ---------- Forwarded message ----------
> From: Asankha C. Perera <as...@wso2.com>
> Date: Mar 22, 2007 4:58 PM
> Subject: Re: [jira] Resolved: (HTTPCORE-60) Transport appears to be
> hanging because an unchecked exception caused the I/O dispatch thread
> to terminate
> To: HttpComponents Project <ht...@jakarta.apache.org>
>
> Oleg/Ant
>
> I am guessing this is something to do with Windows or the JDK you
> use.. But I am unable to test this week, so will try to my best to try
> this sometime next week. As I said, on Linux I have run the system
> through thousands of messages and multiple threads concurrently and
> have fixed all the issues I came across.
>
> So Oleg, I do not see this as a blocker for the HttpCore release - but
> I will use your latest snapshots in Synapse to check on this in future
> if it occurs again
>
> thanks
> asankha
>
> Oleg Kalnichevski (JIRA) wrote:
> > [ https://issues.apache.org/jira/browse/HTTPCORE-60?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> > ]
> >
> > Oleg Kalnichevski resolved HTTPCORE-60.
> > ---------------------------------------
> >
> > Resolution: Fixed
> >
> > Anthony
> > It turned out ClosedChannelException is a checked I/O exception so it cannot kill the I/O dispatch thread. So, apparently I was wrong in my initial assertion about the cause of the Synapse I/O transport lockup. I tweaked HttpCore code a little and changed the IOSessionImpl to catch all ChannelClosedException-s thrown by the underlying byte channel just in case.
> >
> >
> > Please review the changes and let me know if it is okay to proceed with the release
> >
> > Oleg
> >
> >
> > > Transport appears to be hanging because an unchecked exception caused the I/O dispatch thread to terminate
> > > ----------------------------------------------------------------------------------------------------------
> > >
> > >
> > > Key: HTTPCORE-60
> > > URL: https://issues.apache.org/jira/browse/HTTPCORE-60
> > >
> > > Project: HttpComponents Core
> > > Issue Type: Bug
> > > Affects Versions: 4.0-alpha4
> > > Reporter: ant elder
> > > Assigned To: Oleg Kalnichevski
> > > Fix For: 4.0-alpha4
> > >
> > >
> > >
> > > See discussion on synapse-dev mailing list: http://www.nabble.com/Intermittent-IO-Errors-using-Synapse-tf3439957.html
> > >
> > > The transport appears to be hanging because an unchecked exception
> > > caused the I/O dispatch thread to terminate. I believe there are several
> > > different types of problems (at least two) that we are seeing here.
> > >
> > > [I/O reactor worker thread 5] ERROR ServerHandler - I/O Error : null
> > >
> > > > java.nio.channels.ClosedChannelException
> > > > at
> > > > sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:112)
> > > > at
> > > > sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:139)
> > > >
> > > >
> >
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> httpcomponents-dev-unsubscribe@jakarta.apache.org For additional
> commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: synapse-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: synapse-dev-help@ws.apache.org