You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by "Oleksandr Alesinskyy (JIRA)" <ji...@apache.org> on 2010/03/06 23:40:27 UTC
[jira] Created: (HTTPCLIENT-925) HttpClient 4.0.1 is approx. 3 time
slower then older 3.1 when works against www.google.com
HttpClient 4.0.1 is approx. 3 time slower then older 3.1 when works against www.google.com
------------------------------------------------------------------------------------------
Key: HTTPCLIENT-925
URL: https://issues.apache.org/jira/browse/HTTPCLIENT-925
Project: HttpComponents HttpClient
Issue Type: Improvement
Reporter: Oleksandr Alesinskyy
Priority: Minor
I have tried benchmarks from httpclient-benchmark - results are some what funny - while for most tried sites 4.0.1 is marginally faster then 3.1, for a couple of sites it is significantly slower, the most noticeable for www.google.com, test were repeated multiple times, here are sample output from 4.0.1 and 3..1
*4.0.1*
Server Software: gws
Document URI: http://www.google.com
Document Length: 8459 bytes
Time taken for tests: 3.952 seconds
Complete requests: 10
Failed requests: 0
Content transferred: 84524 bytes
Requests per second: 2.5303645 [#/sec] (mean)
Time per request: 395.2 [ms] (mean)
*3.1*
Server Software: gws
Document URI: http://www.google.com
Document Length: 6847 bytes
Time taken for tests: 1.354 seconds
Complete requests: 10
Failed requests: 0
Content transferred: 68488 bytes
Requests per second: 7.3855247 [#/sec] (mean)
Time per request: 135.4 [ms] (mean)
It is interesting, why.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-925) HttpClient 4.0.1 is approx. 3
time slower then older 3.1 when works against www.google.com
Posted by "Oleksandr Alesinskyy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12922088#action_12922088 ]
Oleksandr Alesinskyy commented on HTTPCLIENT-925:
-------------------------------------------------
For me it really does not matter now - I have dropped idea to migrate to 4.x altogether. Anyhow it may be worth mentioning somewhere that such migration may cause unexpected issues.
> HttpClient 4.0.1 is approx. 3 time slower then older 3.1 when works against www.google.com
> ------------------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-925
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-925
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Reporter: Oleksandr Alesinskyy
> Priority: Minor
> Attachments: google.zip
>
>
> I have tried benchmarks from httpclient-benchmark - results are some what funny - while for most tried sites 4.0.1 is marginally faster then 3.1, for a couple of sites it is significantly slower, the most noticeable for www.google.com, test were repeated multiple times, here are sample output from 4.0.1 and 3..1
> *4.0.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 8459 bytes
> Time taken for tests: 3.952 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 84524 bytes
> Requests per second: 2.5303645 [#/sec] (mean)
> Time per request: 395.2 [ms] (mean)
> *3.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 6847 bytes
> Time taken for tests: 1.354 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 68488 bytes
> Requests per second: 7.3855247 [#/sec] (mean)
> Time per request: 135.4 [ms] (mean)
> It is interesting, why.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-925) HttpClient 4.0.1 is approx. 3
time slower then older 3.1 when works against www.google.com
Posted by "Oleksandr Alesinskyy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842423#action_12842423 ]
Oleksandr Alesinskyy commented on HTTPCLIENT-925:
-------------------------------------------------
It is a part of the equation - but not all of it. Namely, after changing of the user agent in the 4.0.1 benchmark
DefaultHttpClient httpclient = new DefaultHttpClient(params);
// Reset user agent to the 3.1 value
HttpProtocolParams.setUserAgent(params,
"Jakarta Commons-HttpClient");
Response sizes become (almost) identical but the performance difference persists (tests have been executed practically at the same time):
*3.1*
Server Software: gws
Document URI: http://www.google.com
Document Length: 6861 bytes
Time taken for tests: 9.711 seconds
Complete requests: 100
Failed requests: 0
Content transferred: 686462 bytes
Requests per second: 10.2976 [#/sec] (mean)
Time per request: 97.11 [ms] (mean)
*4.0.1*
Server Software: gws
Document URI: http://www.google.com
Document Length: 6865 bytes
Time taken for tests: 34.789 seconds
Complete requests: 100
Failed requests: 0
Content transferred: 686482 bytes
Requests per second: 2.8744717 [#/sec] (mean)
Time per request: 347.89 [ms] (mean)
*run.bat*
for /L %%i in (1,1,10) do (
java -cp .;* org/apache/http/client/benchmark/TestHttpClient4 http://www.google.com 100 2>&1 1>>4_
java -cp .;* org/apache/http/client/benchmark/TestHttpClient3 http://www.google.com 100 2>&1 1>>3_
)
> HttpClient 4.0.1 is approx. 3 time slower then older 3.1 when works against www.google.com
> ------------------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-925
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-925
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Reporter: Oleksandr Alesinskyy
> Priority: Minor
> Attachments: google.zip
>
>
> I have tried benchmarks from httpclient-benchmark - results are some what funny - while for most tried sites 4.0.1 is marginally faster then 3.1, for a couple of sites it is significantly slower, the most noticeable for www.google.com, test were repeated multiple times, here are sample output from 4.0.1 and 3..1
> *4.0.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 8459 bytes
> Time taken for tests: 3.952 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 84524 bytes
> Requests per second: 2.5303645 [#/sec] (mean)
> Time per request: 395.2 [ms] (mean)
> *3.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 6847 bytes
> Time taken for tests: 1.354 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 68488 bytes
> Requests per second: 7.3855247 [#/sec] (mean)
> Time per request: 135.4 [ms] (mean)
> It is interesting, why.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-925) HttpClient 4.0.1 is approx. 3
time slower then older 3.1 when works against www.google.com
Posted by "Oleksandr Alesinskyy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842383#action_12842383 ]
Oleksandr Alesinskyy commented on HTTPCLIENT-925:
-------------------------------------------------
(1) "Performance benchmarks against public web sites are utterly pointless for many reasons. For one, they cannot be reliable as most of the time you end up measuring performance of your internet link anyhow and not that of the HTTP stack. "
They are absolutely pointless for the absolute values, but are not so much pointless for side-by-side comparisons (when compared stacks are run at the same time.
In this case I have run each test many times, looping a pair (TestHttpClient3, TestHttpClient4). Results were consistent.. So some real difference likely to exists.
(2) "10 requests statistically are not enough "
I agree that 10 requests are statistically insignificant, but I have clearly pointed that it is "a sample output". Behavior with 100 and 1000 requests is the same.
(3) That's true - and not only for the google, and it is the most interesting part - what is so different in requests created by 4.0.1 and 3.1.
> HttpClient 4.0.1 is approx. 3 time slower then older 3.1 when works against www.google.com
> ------------------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-925
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-925
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Reporter: Oleksandr Alesinskyy
> Priority: Minor
>
> I have tried benchmarks from httpclient-benchmark - results are some what funny - while for most tried sites 4.0.1 is marginally faster then 3.1, for a couple of sites it is significantly slower, the most noticeable for www.google.com, test were repeated multiple times, here are sample output from 4.0.1 and 3..1
> *4.0.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 8459 bytes
> Time taken for tests: 3.952 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 84524 bytes
> Requests per second: 2.5303645 [#/sec] (mean)
> Time per request: 395.2 [ms] (mean)
> *3.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 6847 bytes
> Time taken for tests: 1.354 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 68488 bytes
> Requests per second: 7.3855247 [#/sec] (mean)
> Time per request: 135.4 [ms] (mean)
> It is interesting, why.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Updated: (HTTPCLIENT-925) HttpClient 4.0.1 is approx. 3 time
slower then older 3.1 when works against www.google.com
Posted by "Oleksandr Alesinskyy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Oleksandr Alesinskyy updated HTTPCLIENT-925:
--------------------------------------------
Attachment: google.zip
As Sebb suggested, I tried both benchmarks against "local" Google, in my case google.de. Results for both versions are almost identical (4.0.1. is slightly slower but it is explainable as google generates longer responses for requests sent over 4.0.1).
But for google.com difference persists. That means that ways to "wrap" requests in 3.1 and 4.0.1 are not semantically identical and this fact better to be reflected in the documentation.
See results in the attached zip, filenames are self-explanatory.
> HttpClient 4.0.1 is approx. 3 time slower then older 3.1 when works against www.google.com
> ------------------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-925
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-925
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Reporter: Oleksandr Alesinskyy
> Priority: Minor
> Attachments: google.zip
>
>
> I have tried benchmarks from httpclient-benchmark - results are some what funny - while for most tried sites 4.0.1 is marginally faster then 3.1, for a couple of sites it is significantly slower, the most noticeable for www.google.com, test were repeated multiple times, here are sample output from 4.0.1 and 3..1
> *4.0.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 8459 bytes
> Time taken for tests: 3.952 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 84524 bytes
> Requests per second: 2.5303645 [#/sec] (mean)
> Time per request: 395.2 [ms] (mean)
> *3.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 6847 bytes
> Time taken for tests: 1.354 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 68488 bytes
> Requests per second: 7.3855247 [#/sec] (mean)
> Time per request: 135.4 [ms] (mean)
> It is interesting, why.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-925) HttpClient 4.0.1 is approx. 3
time slower then older 3.1 when works against www.google.com
Posted by "Oleksandr Alesinskyy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842396#action_12842396 ]
Oleksandr Alesinskyy commented on HTTPCLIENT-925:
-------------------------------------------------
All that you have written very well may be true - but it should be identical for both versions. but it is not.
> HttpClient 4.0.1 is approx. 3 time slower then older 3.1 when works against www.google.com
> ------------------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-925
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-925
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Reporter: Oleksandr Alesinskyy
> Priority: Minor
>
> I have tried benchmarks from httpclient-benchmark - results are some what funny - while for most tried sites 4.0.1 is marginally faster then 3.1, for a couple of sites it is significantly slower, the most noticeable for www.google.com, test were repeated multiple times, here are sample output from 4.0.1 and 3..1
> *4.0.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 8459 bytes
> Time taken for tests: 3.952 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 84524 bytes
> Requests per second: 2.5303645 [#/sec] (mean)
> Time per request: 395.2 [ms] (mean)
> *3.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 6847 bytes
> Time taken for tests: 1.354 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 68488 bytes
> Requests per second: 7.3855247 [#/sec] (mean)
> Time per request: 135.4 [ms] (mean)
> It is interesting, why.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-925) HttpClient 4.0.1 is approx. 3
time slower then older 3.1 when works against www.google.com
Posted by "Sebb (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12922077#action_12922077 ]
Sebb commented on HTTPCLIENT-925:
---------------------------------
I suggest you use a protocol analyzer such as WireShark.
You can use this to determine the elapsed time between sending the request to GWS and getting the responses, which is outside HttpClient's control.
> HttpClient 4.0.1 is approx. 3 time slower then older 3.1 when works against www.google.com
> ------------------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-925
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-925
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Reporter: Oleksandr Alesinskyy
> Priority: Minor
> Attachments: google.zip
>
>
> I have tried benchmarks from httpclient-benchmark - results are some what funny - while for most tried sites 4.0.1 is marginally faster then 3.1, for a couple of sites it is significantly slower, the most noticeable for www.google.com, test were repeated multiple times, here are sample output from 4.0.1 and 3..1
> *4.0.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 8459 bytes
> Time taken for tests: 3.952 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 84524 bytes
> Requests per second: 2.5303645 [#/sec] (mean)
> Time per request: 395.2 [ms] (mean)
> *3.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 6847 bytes
> Time taken for tests: 1.354 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 68488 bytes
> Requests per second: 7.3855247 [#/sec] (mean)
> Time per request: 135.4 [ms] (mean)
> It is interesting, why.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-925) HttpClient 4.0.1 is approx. 3
time slower then older 3.1 when works against www.google.com
Posted by "Oleksandr Alesinskyy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842416#action_12842416 ]
Oleksandr Alesinskyy commented on HTTPCLIENT-925:
-------------------------------------------------
That is the point - why the same request send via 3.0.1 and 4.0.1 renders different results?
As HTTP client positions itself as merely transport library (http://hc.apache.org/httpcomponents-client/tutorial/html/preface.html#d4e30) it is only naturally to anticipate that the same requests would bring the same response regardless of the version used. If it is not so it is only fair to provide some kind of warning in the documentation (in README. changelog or whatever).
> HttpClient 4.0.1 is approx. 3 time slower then older 3.1 when works against www.google.com
> ------------------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-925
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-925
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Reporter: Oleksandr Alesinskyy
> Priority: Minor
> Attachments: google.zip
>
>
> I have tried benchmarks from httpclient-benchmark - results are some what funny - while for most tried sites 4.0.1 is marginally faster then 3.1, for a couple of sites it is significantly slower, the most noticeable for www.google.com, test were repeated multiple times, here are sample output from 4.0.1 and 3..1
> *4.0.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 8459 bytes
> Time taken for tests: 3.952 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 84524 bytes
> Requests per second: 2.5303645 [#/sec] (mean)
> Time per request: 395.2 [ms] (mean)
> *3.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 6847 bytes
> Time taken for tests: 1.354 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 68488 bytes
> Requests per second: 7.3855247 [#/sec] (mean)
> Time per request: 135.4 [ms] (mean)
> It is interesting, why.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-925) HttpClient 4.0.1 is approx. 3
time slower then older 3.1 when works against www.google.com
Posted by "Sebb (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842398#action_12842398 ]
Sebb commented on HTTPCLIENT-925:
---------------------------------
The comparisons between the google results are invalid because of the different result sizes.
Scaling the results by the size factor is not valid - clearly the server is responding differently.
> HttpClient 4.0.1 is approx. 3 time slower then older 3.1 when works against www.google.com
> ------------------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-925
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-925
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Reporter: Oleksandr Alesinskyy
> Priority: Minor
>
> I have tried benchmarks from httpclient-benchmark - results are some what funny - while for most tried sites 4.0.1 is marginally faster then 3.1, for a couple of sites it is significantly slower, the most noticeable for www.google.com, test were repeated multiple times, here are sample output from 4.0.1 and 3..1
> *4.0.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 8459 bytes
> Time taken for tests: 3.952 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 84524 bytes
> Requests per second: 2.5303645 [#/sec] (mean)
> Time per request: 395.2 [ms] (mean)
> *3.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 6847 bytes
> Time taken for tests: 1.354 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 68488 bytes
> Requests per second: 7.3855247 [#/sec] (mean)
> Time per request: 135.4 [ms] (mean)
> It is interesting, why.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-925) HttpClient 4.0.1 is approx. 3
time slower then older 3.1 when works against www.google.com
Posted by "Sebb (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842338#action_12842338 ]
Sebb commented on HTTPCLIENT-925:
---------------------------------
Why are the response documents different lengths?
Could it be due to redirection?
Perhaps try with your local google site instead.
> HttpClient 4.0.1 is approx. 3 time slower then older 3.1 when works against www.google.com
> ------------------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-925
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-925
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Reporter: Oleksandr Alesinskyy
> Priority: Minor
>
> I have tried benchmarks from httpclient-benchmark - results are some what funny - while for most tried sites 4.0.1 is marginally faster then 3.1, for a couple of sites it is significantly slower, the most noticeable for www.google.com, test were repeated multiple times, here are sample output from 4.0.1 and 3..1
> *4.0.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 8459 bytes
> Time taken for tests: 3.952 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 84524 bytes
> Requests per second: 2.5303645 [#/sec] (mean)
> Time per request: 395.2 [ms] (mean)
> *3.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 6847 bytes
> Time taken for tests: 1.354 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 68488 bytes
> Requests per second: 7.3855247 [#/sec] (mean)
> Time per request: 135.4 [ms] (mean)
> It is interesting, why.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Resolved: (HTTPCLIENT-925) HttpClient 4.0.1 is approx. 3
time slower then older 3.1 when works against www.google.com
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Oleg Kalnichevski resolved HTTPCLIENT-925.
------------------------------------------
Resolution: Not A Problem
Oleksandr,
(1) Performance benchmarks against public web sites are utterly pointless for many reasons. For one, they cannot be reliable as most of the time you end up measuring performance of your internet link anyhow and not that of the HTTP stack.
(2) 10 requests statistically are not enough and cannot be considered representative. You have got to be measuring performance by executing over several thousand requests over a significant time span (at least half a minute).
(3) It looks like google responds differently to requests issued by different versions of HttpClient, which also makes the results inaccurate and unrepresentative.
I just recently changed the benchmarks to use an embedded HTTP server as a target server. You can see the updated results here:
http://wiki.apache.org/HttpComponents/HttpClient3vsHttpClient4vsHttpCore
Oleg
> HttpClient 4.0.1 is approx. 3 time slower then older 3.1 when works against www.google.com
> ------------------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-925
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-925
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Reporter: Oleksandr Alesinskyy
> Priority: Minor
>
> I have tried benchmarks from httpclient-benchmark - results are some what funny - while for most tried sites 4.0.1 is marginally faster then 3.1, for a couple of sites it is significantly slower, the most noticeable for www.google.com, test were repeated multiple times, here are sample output from 4.0.1 and 3..1
> *4.0.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 8459 bytes
> Time taken for tests: 3.952 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 84524 bytes
> Requests per second: 2.5303645 [#/sec] (mean)
> Time per request: 395.2 [ms] (mean)
> *3.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 6847 bytes
> Time taken for tests: 1.354 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 68488 bytes
> Requests per second: 7.3855247 [#/sec] (mean)
> Time per request: 135.4 [ms] (mean)
> It is interesting, why.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-925) HttpClient 4.0.1 is approx. 3
time slower then older 3.1 when works against www.google.com
Posted by "Oleksandr Alesinskyy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12922051#action_12922051 ]
Oleksandr Alesinskyy commented on HTTPCLIENT-925:
-------------------------------------------------
Not listening to what?
You said that it is due to USER-AGENT.
Then this header was set to the same value as in the version 3.1 - which reduces response sizes to the level of 3.1 but processing was still much slower.
While not the bug of HttpClient client as such this effect definitely deserves to be mentioned in the readme or the migration (3.x->4.x) guide.
> HttpClient 4.0.1 is approx. 3 time slower then older 3.1 when works against www.google.com
> ------------------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-925
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-925
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Reporter: Oleksandr Alesinskyy
> Priority: Minor
> Attachments: google.zip
>
>
> I have tried benchmarks from httpclient-benchmark - results are some what funny - while for most tried sites 4.0.1 is marginally faster then 3.1, for a couple of sites it is significantly slower, the most noticeable for www.google.com, test were repeated multiple times, here are sample output from 4.0.1 and 3..1
> *4.0.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 8459 bytes
> Time taken for tests: 3.952 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 84524 bytes
> Requests per second: 2.5303645 [#/sec] (mean)
> Time per request: 395.2 [ms] (mean)
> *3.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 6847 bytes
> Time taken for tests: 1.354 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 68488 bytes
> Requests per second: 7.3855247 [#/sec] (mean)
> Time per request: 135.4 [ms] (mean)
> It is interesting, why.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-925) HttpClient 4.0.1 is approx. 3
time slower then older 3.1 when works against www.google.com
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842391#action_12842391 ]
Oleg Kalnichevski commented on HTTPCLIENT-925:
----------------------------------------------
> They are absolutely pointless for the absolute values, but are not so much pointless for side-by-side comparisons
> (when compared stacks are run at the same time.
>
> In this case I have run each test many times, looping a pair (TestHttpClient3, TestHttpClient4). Results were consistent..
> So some real difference likely to exists.
Even if you disregard such variable factors as network traffic and load on the target system by other agents, the simple fact that you are executing thousand requests against a high profile site in a tight loop from the same IP is quite likely to trigger some kind of DoS defence mechanism further rendering the results inaccurate.
> and it is the most interesting part - what is so different in requests created by 4.0.1 and 3.1.
User-Agent header
Oleg
> HttpClient 4.0.1 is approx. 3 time slower then older 3.1 when works against www.google.com
> ------------------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-925
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-925
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Reporter: Oleksandr Alesinskyy
> Priority: Minor
>
> I have tried benchmarks from httpclient-benchmark - results are some what funny - while for most tried sites 4.0.1 is marginally faster then 3.1, for a couple of sites it is significantly slower, the most noticeable for www.google.com, test were repeated multiple times, here are sample output from 4.0.1 and 3..1
> *4.0.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 8459 bytes
> Time taken for tests: 3.952 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 84524 bytes
> Requests per second: 2.5303645 [#/sec] (mean)
> Time per request: 395.2 [ms] (mean)
> *3.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 6847 bytes
> Time taken for tests: 1.354 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 68488 bytes
> Requests per second: 7.3855247 [#/sec] (mean)
> Time per request: 135.4 [ms] (mean)
> It is interesting, why.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-925) HttpClient 4.0.1 is approx. 3
time slower then older 3.1 when works against www.google.com
Posted by "Sebb (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842402#action_12842402 ]
Sebb commented on HTTPCLIENT-925:
---------------------------------
This is getting nowhere.
The comparisons between 3.1 and 4.0.1 mean nothing unless they return the same page size.
> HttpClient 4.0.1 is approx. 3 time slower then older 3.1 when works against www.google.com
> ------------------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-925
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-925
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Reporter: Oleksandr Alesinskyy
> Priority: Minor
> Attachments: google.zip
>
>
> I have tried benchmarks from httpclient-benchmark - results are some what funny - while for most tried sites 4.0.1 is marginally faster then 3.1, for a couple of sites it is significantly slower, the most noticeable for www.google.com, test were repeated multiple times, here are sample output from 4.0.1 and 3..1
> *4.0.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 8459 bytes
> Time taken for tests: 3.952 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 84524 bytes
> Requests per second: 2.5303645 [#/sec] (mean)
> Time per request: 395.2 [ms] (mean)
> *3.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 6847 bytes
> Time taken for tests: 1.354 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 68488 bytes
> Requests per second: 7.3855247 [#/sec] (mean)
> Time per request: 135.4 [ms] (mean)
> It is interesting, why.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-925) HttpClient 4.0.1 is approx. 3
time slower then older 3.1 when works against www.google.com
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842424#action_12842424 ]
Oleg Kalnichevski commented on HTTPCLIENT-925:
----------------------------------------------
What is the point of all this if you are not listening?
Oleg
> HttpClient 4.0.1 is approx. 3 time slower then older 3.1 when works against www.google.com
> ------------------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-925
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-925
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Reporter: Oleksandr Alesinskyy
> Priority: Minor
> Attachments: google.zip
>
>
> I have tried benchmarks from httpclient-benchmark - results are some what funny - while for most tried sites 4.0.1 is marginally faster then 3.1, for a couple of sites it is significantly slower, the most noticeable for www.google.com, test were repeated multiple times, here are sample output from 4.0.1 and 3..1
> *4.0.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 8459 bytes
> Time taken for tests: 3.952 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 84524 bytes
> Requests per second: 2.5303645 [#/sec] (mean)
> Time per request: 395.2 [ms] (mean)
> *3.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 6847 bytes
> Time taken for tests: 1.354 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 68488 bytes
> Requests per second: 7.3855247 [#/sec] (mean)
> Time per request: 135.4 [ms] (mean)
> It is interesting, why.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
[jira] Commented: (HTTPCLIENT-925) HttpClient 4.0.1 is approx. 3
time slower then older 3.1 when works against www.google.com
Posted by "Oleg Kalnichevski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842420#action_12842420 ]
Oleg Kalnichevski commented on HTTPCLIENT-925:
----------------------------------------------
Sweet Gezuz! Because of the USER-AGENT header
Oleg
> HttpClient 4.0.1 is approx. 3 time slower then older 3.1 when works against www.google.com
> ------------------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-925
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-925
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Reporter: Oleksandr Alesinskyy
> Priority: Minor
> Attachments: google.zip
>
>
> I have tried benchmarks from httpclient-benchmark - results are some what funny - while for most tried sites 4.0.1 is marginally faster then 3.1, for a couple of sites it is significantly slower, the most noticeable for www.google.com, test were repeated multiple times, here are sample output from 4.0.1 and 3..1
> *4.0.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 8459 bytes
> Time taken for tests: 3.952 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 84524 bytes
> Requests per second: 2.5303645 [#/sec] (mean)
> Time per request: 395.2 [ms] (mean)
> *3.1*
> Server Software: gws
> Document URI: http://www.google.com
> Document Length: 6847 bytes
> Time taken for tests: 1.354 seconds
> Complete requests: 10
> Failed requests: 0
> Content transferred: 68488 bytes
> Requests per second: 7.3855247 [#/sec] (mean)
> Time per request: 135.4 [ms] (mean)
> It is interesting, why.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org