You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by 世涛 <st...@alibaba-inc.com> on 2010/12/28 09:33:41 UTC
HttpClient4.0.3 performance problem, much slower than HttpClient3.1?
Hi all:
I'm considering to update HttpClient3.1 to HttpClient4.0.3 in my new project, and I must be sure that system speed will not be slower.
So I did some tests, but I found that Httpclient4's performance is not as good as HttpClient3.1.
In my test, I use two Linux servers(A and B, both 8-core CPU, 1.5G memory) in the same LAN.
There is Apache2.2 installed on Server-B, and Server-B acts as a static web server.
And Server-A runs test programs based on HttpClient, the test is to finish 1000 requests to Server-B sequentially.
Surprisingly, the time that HttpClient4.0.3 used is almost twice as HttpClient3.1.
My test-code is very simple, just the basic classes and methods.
Anybody has ever done the similar test before? Is the result normal or I did some thing wrong? 3x!
------
Best regards,
stone
________________________________
This email (including any attachments) is confidential and may be legally privileged. If you received this email in error, please delete it immediately and do not copy it or use it for any purpose or disclose its contents to any other person. Thank you.
本电邮(包括任何附件)可能含有机密资料并受法律保护。如您不是正确的收件人，请您立即删除本邮件。请不要将本电邮进行复制并用作任何其他用途、或透露本邮件之内容。谢谢。
Re: HttpClient4.0.3 performance problem, much slower than
HttpClient3.1?
Posted by Oleg Kalnichevski <ol...@apache.org>.
...
> Thanks very much for your quick reply!
>
> I just did the benchmark test from your email, in the benchmark, the
> number of requests was 10000, but the result seems not satisfying.
> The result data is below:
>
> [zhangshitao@inc-130-120 http-benchmark]$ ./httpclient4.sh
> http://10.20.130.119/100k.txt 10000
> Server Software: Apache/2.2.15 (Unix) mod_ssl/2.2.15 OpenSSL/0.9.8o
>
> Document URI: http://10.20.130.119/100k.txt
> Document Length: 100000 bytes
>
> Time taken for tests: 30.528 seconds
> Complete requests: 10000
> Failed requests: 0
> Content transferred: 1000000000 bytes
> Requests per second: 327.56815 [#/sec] (mean)
> Time per request: 3.0528 [ms] (mean)
> [zhangshitao@inc-130-120 http-benchmark]$ ./httpclient3.sh
> http://10.20.130.119/100k.txt 10000
> Server Software: Apache/2.2.15 (Unix) mod_ssl/2.2.15 OpenSSL/0.9.8o
>
> Document URI: http://10.20.130.119/100k.txt
> Document Length: 100000 bytes
>
> Time taken for tests: 25.333 seconds
> Complete requests: 10000
> Failed requests: 0
> Content transferred: 1000000000 bytes
> Requests per second: 394.74203 [#/sec] (mean)
> Time per request: 2.5333 [ms] (mean)
>
> --
> ASC-北京研发部-应用工程部
> 张世涛
> 分机号：73354
> 旺旺：stonecheung2010
>
>
If I execute the benchmark with the same parameters I get different
results. HttpCore / HttpClient 4.1 are comfortably faster than any other
HTTP agent in the benchmark including HttpClient 3.1
Disclaimer: I ran the tests on a lousy single core notebook against
localhost.
I do not have access to similarly sided servers like those you have used
in your tests, so I cannot verify your numbers. However, knowing HC 3.1
code quite well I simply do not see how possibly it can be faster than
HC 4.0
If you have any theories please do let me know.
Oleg
---
oleg@ubuntu:~/src/apache.org/httpcomponents/httpclient/httpclient-benchmark$ mvn test -Dhc.benchmark.n-requests=10000 -Dhc.benchmark.content-len=100000
...
[INFO] [exec:java {execution: test-run}]
=================================
HTTP agent: Apache HttpClient 3.1
---------------------------------
10000 GET requests
---------------------------------
Document URI: http://localhost:56323/rnd?c=100000
Document Length: 100000 bytes
Concurrency level: 100
Time taken for tests: 18.57 seconds
Complete requests: 9999
Failed requests: 1
Content transferred: 999900000 bytes
Requests per second: 538.4491 [#/sec] (mean)
Time per request: 1.8571857 [ms] (mean)
---------------------------------
10000 POST requests
---------------------------------
Document URI: http://localhost:56323/echo
Document Length: 146544 bytes
Concurrency level: 100
Time taken for tests: 19.22 seconds
Complete requests: 10000
Failed requests: 0
Content transferred: 1489821168 bytes
Requests per second: 520.2914 [#/sec] (mean)
Time per request: 1.922 [ms] (mean)
---------------------------------
=================================
HTTP agent: JRE HTTP 1.6.0_10
---------------------------------
10000 GET requests
---------------------------------
Document URI: http://localhost:56323/rnd?c=100000
Document Length: 100000 bytes
Concurrency level: 100
Time taken for tests: 15.221 seconds
Complete requests: 10000
Failed requests: 0
Content transferred: 1000000000 bytes
Requests per second: 656.98706 [#/sec] (mean)
Time per request: 1.5221 [ms] (mean)
---------------------------------
10000 POST requests
---------------------------------
Document URI: http://localhost:56323/echo
Document Length: 145256 bytes
Concurrency level: 100
Time taken for tests: 15.579 seconds
Complete requests: 10000
Failed requests: 0
Content transferred: 1461957416 bytes
Requests per second: 641.8897 [#/sec] (mean)
Time per request: 1.5579 [ms] (mean)
---------------------------------
=================================
HTTP agent: Apache HttpCore 4 (ver: 4.1)
---------------------------------
10000 GET requests
---------------------------------
Document URI: http://localhost:56323/rnd?c=100000
Document Length: 100000 bytes
Concurrency level: 100
Time taken for tests: 11.061 seconds
Complete requests: 10000
Failed requests: 0
Content transferred: 1000000000 bytes
Requests per second: 904.0774 [#/sec] (mean)
Time per request: 1.1061 [ms] (mean)
---------------------------------
10000 POST requests
---------------------------------
Document URI: http://localhost:56323/echo
Document Length: 146704 bytes
Concurrency level: 100
Time taken for tests: 13.893 seconds
Complete requests: 10000
Failed requests: 0
Content transferred: 1489995840 bytes
Requests per second: 719.787 [#/sec] (mean)
Time per request: 1.3893 [ms] (mean)
---------------------------------
=================================
HTTP agent: Apache HttpClient 4 (ver: 4.1-beta2-SNAPSHOT)
---------------------------------
10000 GET requests
---------------------------------
Document URI: http://localhost:56323/rnd?c=100000
Document Length: 100000 bytes
Concurrency level: 100
Time taken for tests: 11.844 seconds
Complete requests: 10000
Failed requests: 0
Content transferred: 1000000000 bytes
Requests per second: 844.3094 [#/sec] (mean)
Time per request: 1.1844 [ms] (mean)
---------------------------------
10000 POST requests
---------------------------------
Document URI: http://localhost:56323/echo
Document Length: 146704 bytes
Concurrency level: 100
Time taken for tests: 14.31 seconds
Complete requests: 10000
Failed requests: 0
Content transferred: 1499028464 bytes
Requests per second: 698.812 [#/sec] (mean)
Time per request: 1.431 [ms] (mean)
---------------------------------
=================================
HTTP agent: Jetty 7.2.0.v20101020
---------------------------------
10000 GET requests
---------------------------------
Document URI: http://localhost:56323/rnd?c=100000
Document Length: 100000 bytes
Concurrency level: 100
Time taken for tests: 18.613 seconds
Complete requests: 10000
Failed requests: 0
Content transferred: 1000000000 bytes
Requests per second: 537.25885 [#/sec] (mean)
Time per request: 1.8613 [ms] (mean)
---------------------------------
10000 POST requests
---------------------------------
Document URI: http://localhost:56323/echo
Document Length: 196608 bytes
Concurrency level: 100
Time taken for tests: 24.55 seconds
Complete requests: 10000
Failed requests: 0
Content transferred: 1844425632 bytes
Requests per second: 407.332 [#/sec] (mean)
Time per request: 2.455 [ms] (mean)
---------------------------------
=================================
HTTP agent: Ning Async HTTP client 1.4.0
---------------------------------
10000 GET requests
---------------------------------
Document URI: http://localhost:56323/rnd?c=100000
Document Length: 100000 bytes
Concurrency level: 100
Time taken for tests: 17.817 seconds
Complete requests: 10000
Failed requests: 0
Content transferred: 1000000000 bytes
Requests per second: 561.2617 [#/sec] (mean)
Time per request: 1.7817 [ms] (mean)
---------------------------------
10000 POST requests
---------------------------------
Document URI: http://localhost:56323/echo
Document Length: 146424 bytes
Concurrency level: 100
Time taken for tests: 26.434 seconds
Complete requests: 10000
Failed requests: 0
Content transferred: 1461602176 bytes
Requests per second: 378.30066 [#/sec] (mean)
Time per request: 2.6434 [ms] (mean)
---------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: HttpClient4.0.3 performance problem, much slower than HttpClient3.1?
Posted by 世涛 <st...@alibaba-inc.com>.
于 2010-12-28 17:03, Oleg Kalnichevski 写道:
> On Tue, 2010-12-28 at 16:33 +0800, 世涛 wrote:
>> Hi all:
>>
>> I'm considering to update HttpClient3.1 to HttpClient4.0.3 in my new project, and I must be sure that system speed will not be slower.
>> So I did some tests, but I found that Httpclient4's performance is not as good as HttpClient3.1.
>>
>> In my test, I use two Linux servers(A and B, both 8-core CPU, 1.5G memory) in the same LAN.
>> There is Apache2.2 installed on Server-B, and Server-B acts as a static web server.
>> And Server-A runs test programs based on HttpClient, the test is to finish 1000 requests to Server-B sequentially.
>>
>> Surprisingly, the time that HttpClient4.0.3 used is almost twice as HttpClient3.1.
>>
>> My test-code is very simple, just the basic classes and methods.
>>
>> Anybody has ever done the similar test before? Is the result normal or I did some thing wrong? 3x!
>>
>> ------
>> Best regards,
>> stone
>>
> (1) 1000 requests is simply not enough to draw any meaningful conclusion
> about performance of an HTTP client.
>
> (2) Make sure to turn off the stale connection check, which has a major
> performance impact on small requests
>
> In my tests HttpClient 4.x turns out to be 30% to 200% faster than
> HttpClient 3.1 depending on the number of concurrent execution threads
>
> http://wiki.apache.org/HttpComponents/HttpClient3vsHttpClient4vsHttpCore
>
> You can find the source code of the benchmark here. Feel free to run it
> against your servers to see if you get similar results
>
> http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/httpclient-benchmark/
>
> Oleg
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
Thanks very much for your quick reply!
I just did the benchmark test from your email, in the benchmark, the
number of requests was 10000, but the result seems not satisfying.
The result data is below:
[zhangshitao@inc-130-120 http-benchmark]$ ./httpclient4.sh
http://10.20.130.119/100k.txt 10000
Server Software: Apache/2.2.15 (Unix) mod_ssl/2.2.15 OpenSSL/0.9.8o
Document URI: http://10.20.130.119/100k.txt
Document Length: 100000 bytes
Time taken for tests: 30.528 seconds
Complete requests: 10000
Failed requests: 0
Content transferred: 1000000000 bytes
Requests per second: 327.56815 [#/sec] (mean)
Time per request: 3.0528 [ms] (mean)
[zhangshitao@inc-130-120 http-benchmark]$ ./httpclient3.sh
http://10.20.130.119/100k.txt 10000
Server Software: Apache/2.2.15 (Unix) mod_ssl/2.2.15 OpenSSL/0.9.8o
Document URI: http://10.20.130.119/100k.txt
Document Length: 100000 bytes
Time taken for tests: 25.333 seconds
Complete requests: 10000
Failed requests: 0
Content transferred: 1000000000 bytes
Requests per second: 394.74203 [#/sec] (mean)
Time per request: 2.5333 [ms] (mean)
--
ASC-北京研发部-应用工程部
张世涛
分机号：73354
旺旺：stonecheung2010
________________________________
This email (including any attachments) is confidential and may be legally privileged. If you received this email in error, please delete it immediately and do not copy it or use it for any purpose or disclose its contents to any other person. Thank you.
本电邮(包括任何附件)可能含有机密资料并受法律保护。如您不是正确的收件人，请您立即删除本邮件。请不要将本电邮进行复制并用作任何其他用途、或透露本邮件之内容。谢谢。
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: HttpClient4.0.3 performance problem, much slower than
HttpClient3.1?
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, 2010-12-28 at 16:33 +0800, 世涛 wrote:
> Hi all:
>
> I'm considering to update HttpClient3.1 to HttpClient4.0.3 in my new project, and I must be sure that system speed will not be slower.
> So I did some tests, but I found that Httpclient4's performance is not as good as HttpClient3.1.
>
> In my test, I use two Linux servers(A and B, both 8-core CPU, 1.5G memory) in the same LAN.
> There is Apache2.2 installed on Server-B, and Server-B acts as a static web server.
> And Server-A runs test programs based on HttpClient, the test is to finish 1000 requests to Server-B sequentially.
>
> Surprisingly, the time that HttpClient4.0.3 used is almost twice as HttpClient3.1.
>
> My test-code is very simple, just the basic classes and methods.
>
> Anybody has ever done the similar test before? Is the result normal or I did some thing wrong? 3x!
>
> ------
> Best regards,
> stone
>
(1) 1000 requests is simply not enough to draw any meaningful conclusion
about performance of an HTTP client.
(2) Make sure to turn off the stale connection check, which has a major
performance impact on small requests
In my tests HttpClient 4.x turns out to be 30% to 200% faster than
HttpClient 3.1 depending on the number of concurrent execution threads
http://wiki.apache.org/HttpComponents/HttpClient3vsHttpClient4vsHttpCore
You can find the source code of the benchmark here. Feel free to run it
against your servers to see if you get similar results
http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/httpclient-benchmark/
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org