You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by Wolfgang Hoschek <wh...@lbl.gov> on 2003/08/14 01:14:05 UTC
Performance HTTPClient vs. ab
I am planning on sending and receiving MANY (tens of tousands) small
SOAP-like serial messages over the same persistent HTTP 1.1 connection,
using a single client and thread (nothing fancy there). A High
performance, low latency http library would be ideal.
HTTPClient performance seems devastating wrt. ab (apache benchmark):
4300 requests/sec vs. 20 requests/sec, i.e. two orders of magnitute
difference, no matter what options I try with HTTPClient.
Source code to reproduce and full data is attached.
If anyone is interested, please let me know what I'm doing wrong,
reproduce the effect and/or post working code that performs well in your
experience.
ab -k -n 10000 http://localhost:8080/discofish/HelloWorldServlet > /tmp/out
Requests per second: 4313.48 [#/sec] (mean)
(For more data see full output in attachment)
[doggy /home/oliver/g5/users/hoschek] time disco-java.sh
gov.lbl.dsd.discofish.client.HTTPBug 100
http://localhost:8080/discofish/HelloWorldServlet > /tmp/out2
real 0m4.876s
user 0m0.550s
sys 0m0.060s
--> Requests per second: 20.0
(For more data see full output in attachment)
Similar performance gap on static HTML pages (no need to use a servlet).
Environment: httpclient-2.0rc1, jdk-1.4.2, tomcat-4.1.27 (server.xml out
of the box), 2GHz Pentium IV, Redhat 8.0.
Thanks,
Wolfgang.
Re: Performance HTTPClient vs. ab
Posted by Ortwin Glück <or...@nose.ch>.
Wolfgang,
please try using the SimpleHttpConnectionManager instead of
MultiThreadedHttpConnectionManager.
Odi
Wolfgang Hoschek wrote:
> I am planning on sending and receiving MANY (tens of tousands) small
> SOAP-like serial messages over the same persistent HTTP 1.1 connection,
> using a single client and thread (nothing fancy there). A High
> performance, low latency http library would be ideal.
>
> HTTPClient performance seems devastating wrt. ab (apache benchmark):
> 4300 requests/sec vs. 20 requests/sec, i.e. two orders of magnitute
> difference, no matter what options I try with HTTPClient.
>
> Source code to reproduce and full data is attached.
> If anyone is interested, please let me know what I'm doing wrong,
> reproduce the effect and/or post working code that performs well in your
> experience.
>
> ab -k -n 10000 http://localhost:8080/discofish/HelloWorldServlet > /tmp/out
>
> Requests per second: 4313.48 [#/sec] (mean)
>
> (For more data see full output in attachment)
>
>
> [doggy /home/oliver/g5/users/hoschek] time disco-java.sh
> gov.lbl.dsd.discofish.client.HTTPBug 100
> http://localhost:8080/discofish/HelloWorldServlet > /tmp/out2
>
> real 0m4.876s
> user 0m0.550s
> sys 0m0.060s
>
> --> Requests per second: 20.0
>
> (For more data see full output in attachment)
>
> Similar performance gap on static HTML pages (no need to use a servlet).
>
> Environment: httpclient-2.0rc1, jdk-1.4.2, tomcat-4.1.27 (server.xml out
> of the box), 2GHz Pentium IV, Redhat 8.0.
>
> Thanks,
> Wolfgang.
>
>
> ------------------------------------------------------------------------
>
> This is ApacheBench, Version 2.0.40-dev <$Revision: 1.116 $> apache-2.0
> Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
> Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/
>
> Benchmarking localhost (be patient)
>
>
> Server Software: Apache
> Server Hostname: localhost
> Server Port: 8080
>
> Document Path: /discofish/HelloWorldServlet
> Document Length: 8 bytes
>
> Concurrency Level: 1
> Time taken for tests: 2.210253 seconds
> Complete requests: 10000
> Failed requests: 0
> Write errors: 0
> Keep-Alive requests: 9900
> Total transferred: 1579500 bytes
> HTML transferred: 80000 bytes
> Requests per second: 4524.37 [#/sec] (mean)
> Time per request: 0.221 [ms] (mean)
> Time per request: 0.221 [ms] (mean, across all concurrent requests)
> Transfer rate: 697.66 [Kbytes/sec] received
>
> Connection Times (ms)
> min mean[+/-sd] median max
> Connect: 0 0 0.0 0 0
> Processing: 0 11 7.6 11 37
> Waiting: 0 0 0.5 0 17
> Total: 0 11 7.6 11 37
>
> Percentage of the requests served within a certain time (ms)
> 50% 11
> 66% 14
> 75% 16
> 80% 17
> 90% 19
> 95% 26
> 98% 32
> 99% 35
> 100% 37 (longest request)
>
>
> ------------------------------------------------------------------------
>
>
> ......................................................................
> ..............................goodbye
>
>
> ------------------------------------------------------------------------
>
> package gov.lbl.dsd.discofish.client;
>
> import java.io.IOException;
>
> import org.apache.commons.httpclient.Header;
> import org.apache.commons.httpclient.HttpClient;
> import org.apache.commons.httpclient.HttpRecoverableException;
> import org.apache.commons.httpclient.MultiThreadedHttpConnectionManager;
> import org.apache.commons.httpclient.methods.PostMethod;
>
>
> public class HTTPBug {
>
> protected HttpClient client;
> protected String url;
>
> public static void main(String[] args) throws Exception {
> int k=0;
>
> int size = 1000;
> if (args.length > k) size = Integer.parseInt(args[k++]);
>
> //String url = "http://localhost:8080/discofish/HelloWorldServlet";
> String url = "http://localhost:8080/discofish/index.html";
> //String url = "http://itg.lbl.gov/~hoschek/discofish/disco-grep-usage.txt";
> if (args.length > k) url = args[k++];
>
> MultiThreadedHttpConnectionManager connMgr = new MultiThreadedHttpConnectionManager();
> //connMgr.setMaxConnectionsPerHost(200);
> //connMgr.setMaxTotalConnections(200);
> HttpClient client = new HttpClient(connMgr);
> //HttpClient client = new HttpClient();
>
> for (int i=0; i<size; i++) {
> PostMethod method = new PostMethod(url);
> method.setRequestHeader("Content-type", "text/xml; charset=ISO-8859-1");
> method.setRequestHeader("Accept", "application/soap+xml, application/dime, multipart/related, text/*");
> method.setRequestHeader("SOAPAction", "");
> //method.setRequestContentLength(PostMethod.CONTENT_LENGTH_CHUNKED);
> //method.setRequestBody(XMLUtil.msg2SoapString(new Receive(new TransactionID(), 1, 1, System.currentTimeMillis() + 10000, "single")));
> method.setRequestBody("hello");
> //method.setRequestContentLength(5);
> int statusCode = -1;
>
> /*Header[] headers;
> headers = method.getRequestHeaders();
> for (int j=0; j < headers.length; j++) {
> System.out.print(headers[j].toString());
> }
>
> String requestBody = method.getRequestBodyAsString();
> System.out.println(requestBody);
> System.out.println();*/
>
>
> try {
> // execute the method.
> statusCode = client.executeMethod(method);
> } catch (HttpRecoverableException e) {
> System.out.println("A recoverable exception occurred, retrying. " + e.getMessage());
> } catch (IOException e) {
> System.out.println("Failed to download file.");
> e.printStackTrace();
> System.exit(-1);
> }
> // Check that we didn't run out of retries.
> if (statusCode == -1) {
> System.out.println("Failed to recover from exception.");
> System.exit(-2);
> }
>
> //String responseBody = method.getResponseBodyAsString();
>
> /*headers = method.getResponseHeaders();
> for (int j=0; j < headers.length; j++) {
> System.out.print(headers[j].toString());
> }
>
> System.out.println(responseBody);*/
> if (i % 70 == 0) System.out.println();
> System.out.print('.');
>
> method.releaseConnection();
> //method.recycle();
> }
>
> System.out.println("goodbye");
> }
>
> }
>
>
> ------------------------------------------------------------------------
>
> package gov.lbl.dsd.discofish.server;
>
> import java.io.ByteArrayOutputStream;
> import java.io.IOException;
> import java.io.PrintWriter;
>
> import javax.servlet.ServletException;
> import javax.servlet.http.HttpServlet;
> import javax.servlet.http.HttpServletRequest;
> import javax.servlet.http.HttpServletResponse;
>
> public class HelloWorldServlet extends HttpServlet {
>
> public void doPost(HttpServletRequest request, HttpServletResponse response) throws IOException, ServletException {
> doGet(request, response);
> }
>
> public void doGet(HttpServletRequest request, HttpServletResponse response)
> throws IOException, ServletException {
>
> response.setContentType("text/xml");
>
> ByteArrayOutputStream bytes = new ByteArrayOutputStream(1000);
> PrintWriter out = new PrintWriter(bytes, true); // true forces flushing
>
> out.println("</HTML>");
>
> // Set the content length to the size of the buffer
> response.setContentLength(bytes.size());
>
> // Send the buffer
> bytes.writeTo(response.getOutputStream());
>
> /*out.flush();
> out.close();*/
> //response.flushBuffer();
> }
>
> }
>
>
>
> ------------------------------------------------------------------------
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-httpclient-dev-help@jakarta.apache.org
--
_________________________________________________________________
NOSE applied intelligence ag
ortwin glück [www] http://www.nose.ch
software engineer [email] ortwin.glueck@nose.ch
hardturmstrasse 171 [pgp key] 0x81CF3416
8005 zurich [office] +41-1-277 57 35
switzerland [fax] +41-1-277 57 12