You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by "Oleg Kalnichevski (JIRA)" <ji...@apache.org> on 2007/09/30 13:17:50 UTC
[jira] Resolved: (HTTPCORE-118) Some strange issue under a heavy
stress test
[ https://issues.apache.org/jira/browse/HTTPCORE-118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Oleg Kalnichevski resolved HTTPCORE-118.
----------------------------------------
Resolution: Invalid
Hi Halo
I do not think this is a bug in HttpCore but rather a problem with your test case.
Consider this:
(1) In your test you open thousands of connections in a tight loop and immediately close then after having executed just one request
(2) You are using HTTP/1.1 protocol, but do not include 'Connection: close' header to explicitly inform the server you do not intend to keep the connection alive.
(3) The server assumes all these thousands of connections should be kept alive. Each open connection takes up some memory of the heap.
(4) If you start a significant number of worker threads the server eventually gets overwhelmed with connection requests and runs out of memory
While your test case does not violate the HTTP specification, I would consider it a pretty gross abuse of the protocol.
Please consider re-running your tests with 'Connection: close' header included
String ReuestHead = "POST /TerminalLocationNotification HTTP/1.1\r\nHOST: LOCALHOST\r\nConnection: close\r\nContent-Length: ";
or using another tool to stress test HttpCore that supports persistent connections such as Apache benchmark
All stress tests I have conducted myself seem to suggest HttpCore can take a significant amount of load
=====================================================================
With connection keep-alive option
=====================================================================
oleg@okhost:~$ ab -c 200 -n 20000 -k http://localhost:8080/index.html
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
...
Server Software: Jakarta-HttpComponents-NIO/1.1
Server Hostname: localhost
Server Port: 8080
Document Path: /index.html
Document Length: 12926 bytes
Concurrency Level: 200
Time taken for tests: 8.510525 seconds
Complete requests: 20000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 20000
Total transferred: 261880000 bytes
HTML transferred: 258520000 bytes
Requests per second: 2350.03 [#/sec] (mean)
Time per request: 85.105 [ms] (mean)
Time per request: 0.426 [ms] (mean, across all concurrent requests)
Transfer rate: 30050.09 [Kbytes/sec] received
=====================================================================
Without connection keep-alive option
=====================================================================
oleg@okhost:~$ ab -c 200 -n 20000 http://localhost:8080/index.html
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
...
Server Software: Jakarta-HttpComponents-NIO/1.1
Server Hostname: localhost
Server Port: 8080
Document Path: /index.html
Document Length: 12926 bytes
Concurrency Level: 200
Time taken for tests: 15.643071 seconds
Complete requests: 20000
Failed requests: 0
Write errors: 0
Total transferred: 261491490 bytes
HTML transferred: 258610482 bytes
Requests per second: 1278.52 [#/sec] (mean)
Time per request: 156.431 [ms] (mean)
Time per request: 0.782 [ms] (mean, across all concurrent requests)
Transfer rate: 16324.29 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 90 624.5 0 9001
Processing: 0 56 46.6 52 1321
Waiting: 0 55 46.5 50 1321
Total: 0 147 632.1 53 9066
> Some strange issue under a heavy stress test
> ---------------------------------------------
>
> Key: HTTPCORE-118
> URL: https://issues.apache.org/jira/browse/HTTPCORE-118
> Project: HttpComponents Core
> Issue Type: Bug
> Components: Examples, HttpCore, HttpCore NIO
> Affects Versions: 4.0-alpha5
> Environment: Windows xp(sp2), JDK 1.5
> Reporter: Halo chen
> Priority: Critical
> Fix For: 4.0-alpha5
>
> Attachments: a.txt, SendToIA.java
>
>
> Thanks very much for your contributions.
> I use your httpcore to develop a embed web application, very well, I think.
> But, when i do some stress test, i find a strange issue, please see my test case:
> public class SendToIA implements Runnable {
> String[] args;
> SendToIA( String[] args )
> {
> this.args = args;
> }
> public void run()
> {
> try {
> FileInputStream fis = new FileInputStream("a.txt");
> int len = fis.available();
> byte[] buf = new byte[len];
> fis.read(buf);
> fis.close();
> String ReuestHead = "POST /TerminalLocationNotification HTTP/1.1\r\nHOST: LOCALHOST\r\nContent-Length: ";
> String ReuestLen = String.valueOf(len) + "\r\n\r\n";
> String RequestBody = new String(buf);
> String httpRequest = ReuestHead + ReuestLen + RequestBody;
> String host = "localhost";
> int port = 8086;
> if( args.length == 1 )
> {
> host = args[0];
> }
> else if( args.length == 2 )
> {
> host = args[0];
> port = Integer.parseInt( args[ 1 ] );
> }
>
> System.out.println(new java.util.Date());
> for (int i = 0; i < 20000; i++) {
> Socket connection = new Socket( host, port );
> connection.setReuseAddress( true );
> PrintWriter out = new PrintWriter(connection.getOutputStream(), true);
> out.print(httpRequest);
> out.flush();
> connection.close();
> System.out.print("\b\rcont:"+i);
> }
> System.out.println(new java.util.Date());
> } catch (SecurityException e) {
> e.printStackTrace();
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> public static void main(String[] args) throws IOException {
> int threadcount = 1;
> if( args.length == 3 )
> {
> threadcount = Integer.parseInt( args[ 2 ] );
> }
> Thread[] t = new Thread[ threadcount ];
> for( int i = 0 ; i < threadcount ; i ++ )
> {
> new Thread( new SendToIA( args ) ).start();
> }
> for( int i = 0 ; i < threadcount ; i ++ )
> {
> try{t[i].join();}catch( Exception e ){}
> }
> }
> }
> Unfortunately, the application was crashed.
> The first , I think my application has some mistake. so I run the example( org.apache.http.examples.nio.NHttpServer) in the HttpCore binary package. Oh,No!
> The same issue was occured!
> Why? I found the memory usage of the application reach 170M!!, and then I add the JVM parameter -Xmx256M.Okay, the application can running continue. but after a long time, the memory usage of the application reach 256M and then the same thing was occured! So , I think that the HttpCore has some memory leak.
> Can you give me some advices to resolve this issue? or I have some mistakes?
> details of error occured:
> ...
> Exception in thread "I/O reactor worker thread 2" java.lang.OutOfMemoryError: Java heap space
> ....
> java.lang.NullPointerException
> at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.verifyWorkers(AbstractMultiworkerIOReactor.java:136)
> at org.apache.http.impl.nio.reactor.DefaultListeningIOReactor.execute(DefaultListeningIOReactor.java:111)
> ....
> Exception in thread "I/O reactor worker thread 8" java.lang.OutOfMemoryError: Java heap space
> Exception in thread "Thread-6" java.lang.OutOfMemoryError: Java heap space
> Sorry , I'm chinese , my english is bad, may be i haven't explain the issue clearly. but I need your help.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org