You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jmeter.apache.org by Philippe Mouawad <ph...@gmail.com> on 2016/01/30 00:31:00 UTC
Important Regression in nightly build compared to 2.13 or r1715087
Hello,
I made a real load test today using nightly build and faced an important
issue.
Here are the details:
- Test uses 1000 Threads on 1 Instance
- It uses "Download Embedded Resources"
- Socket Timeout is set to 10s
There is no overloading of the machine, no impacting GC
Very rapidly, I start getting a lot of errors:
- Non HTTP response code: org.apache.http.conn.ConnectTimeoutException
message:Non HTTP response message: Connect to www.foo.com:80 timed out
Rate of error varies between 15% and 30%.
Note that if I navigate on the application, I don't face the errors.
I ran the same test using exactly the same configuration:
- Same machine
- Same JVM version and tuning
- Same user.properties
- Same hc.parameters
But jmeter r1715087
And error rate is 0.30%.
Note the target server has a load balancer that returns a keep-alive
duration set to 2 (2 seconds).
This issue is a blocker one for the release of next version.
I compared code with revision 1715087 and I don't see many changes in
HTTPHC4Impl that would explain this regression.
I commented out some suspects , retried but I get same results.
I also upgraded to HttpClient 4.5.2 and uncommented the code expected to be
added, same results.
So for now I tend to suspect an issue in HttpClient/Core.
--
Regards
Philippe M.
Re: Important Regression in nightly build compared to 2.13 or r1715087
Posted by Philippe Mouawad <ph...@gmail.com>.
Upgrade to 4.5.2 and fix commited yesterday evening
On Sunday, January 31, 2016, Philippe Mouawad <ph...@gmail.com>
wrote:
> Thanks Vladimir.
>
> @Felix, with http-client-4.5.2 and this line uncommented:
> // FIXME when 4.5.2 is released, see
> https://issues.apache.org/jira/browse/HTTPCORE-397
> ((AbstractHttpClient)
> httpClient).setReuseStrategy(DefaultClientConnectionReuseStrategy.INSTANCE);
>
> I don't reproduce the Connect Timeout issue due to keep alive.
> Note this is the version I am testing and that face the issue. So there is
> another problem.
>
> Regards
> Philippe
>
> On Sun, Jan 31, 2016 at 8:58 PM, Vladimir Sitnikov <
> sitnikov.vladimir@gmail.com
> <javascript:_e(%7B%7D,'cvml','sitnikov.vladimir@gmail.com');>> wrote:
>
>> JFR confirms the issue is not related with GC/locking/excessive
>> processing.
>>
>> I wonder if tcp dump/wireshark would help to analyze the difference.
>>
>> Findings so far:
>> 1)
>> org.apache.jmeter.protocol.http.parser.LagartoBasedHtmlParser#getEmbeddedResourceURLs
>> is converting response bytes to String (see [1]).
>> It is better to use SampleResult#getResponseDataAsString since the
>> latter would cache the string, thus eliminate repeated utf-8 decoding.
>>
>> 2) In 3.0
>> org.apache.jmeter.protocol.http.control.HC4CookieHandler#getCookiesForUrl
>> is a little bit more involved since cookieSpec.match delegates to
>> org.apache.http.impl.cookie.DefaultCookieSpec.match(Cookie,
>> CookieOrigin) -> CookieSpecBase.match ->
>> PublicSuffixDomainFilter.match ->...
>> In 2.14 it is much simpler.
>> In 3.0, DefaultCookieSpec.match accounts for 3% of all cpu time.
>>
>> I do not see how those issues could result in "connect timeout" though.
>>
>> [1]:
>> https://github.com/apache/jmeter/blob/9a1b9a02c7593553cb6047bddcfbccb79ec72fc9/src/protocol/http/org/apache/jmeter/protocol/http/parser/LagartoBasedHtmlParser.java#L210
>>
>> Vladimir
>>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: Important Regression in nightly build compared to 2.13 or r1715087
Posted by Philippe Mouawad <ph...@gmail.com>.
Thanks Vladimir.
@Felix, with http-client-4.5.2 and this line uncommented:
// FIXME when 4.5.2 is released, see
https://issues.apache.org/jira/browse/HTTPCORE-397
((AbstractHttpClient)
httpClient).setReuseStrategy(DefaultClientConnectionReuseStrategy.INSTANCE);
I don't reproduce the Connect Timeout issue due to keep alive.
Note this is the version I am testing and that face the issue. So there is
another problem.
Regards
Philippe
On Sun, Jan 31, 2016 at 8:58 PM, Vladimir Sitnikov <
sitnikov.vladimir@gmail.com> wrote:
> JFR confirms the issue is not related with GC/locking/excessive processing.
>
> I wonder if tcp dump/wireshark would help to analyze the difference.
>
> Findings so far:
> 1)
> org.apache.jmeter.protocol.http.parser.LagartoBasedHtmlParser#getEmbeddedResourceURLs
> is converting response bytes to String (see [1]).
> It is better to use SampleResult#getResponseDataAsString since the
> latter would cache the string, thus eliminate repeated utf-8 decoding.
>
> 2) In 3.0
> org.apache.jmeter.protocol.http.control.HC4CookieHandler#getCookiesForUrl
> is a little bit more involved since cookieSpec.match delegates to
> org.apache.http.impl.cookie.DefaultCookieSpec.match(Cookie,
> CookieOrigin) -> CookieSpecBase.match ->
> PublicSuffixDomainFilter.match ->...
> In 2.14 it is much simpler.
> In 3.0, DefaultCookieSpec.match accounts for 3% of all cpu time.
>
> I do not see how those issues could result in "connect timeout" though.
>
> [1]:
> https://github.com/apache/jmeter/blob/9a1b9a02c7593553cb6047bddcfbccb79ec72fc9/src/protocol/http/org/apache/jmeter/protocol/http/parser/LagartoBasedHtmlParser.java#L210
>
> Vladimir
>
--
Cordialement.
Philippe Mouawad.
Re: Important Regression in nightly build compared to 2.13 or r1715087
Posted by Vladimir Sitnikov <si...@gmail.com>.
JFR confirms the issue is not related with GC/locking/excessive processing.
I wonder if tcp dump/wireshark would help to analyze the difference.
Findings so far:
1) org.apache.jmeter.protocol.http.parser.LagartoBasedHtmlParser#getEmbeddedResourceURLs
is converting response bytes to String (see [1]).
It is better to use SampleResult#getResponseDataAsString since the
latter would cache the string, thus eliminate repeated utf-8 decoding.
2) In 3.0 org.apache.jmeter.protocol.http.control.HC4CookieHandler#getCookiesForUrl
is a little bit more involved since cookieSpec.match delegates to
org.apache.http.impl.cookie.DefaultCookieSpec.match(Cookie,
CookieOrigin) -> CookieSpecBase.match ->
PublicSuffixDomainFilter.match ->...
In 2.14 it is much simpler.
In 3.0, DefaultCookieSpec.match accounts for 3% of all cpu time.
I do not see how those issues could result in "connect timeout" though.
[1]: https://github.com/apache/jmeter/blob/9a1b9a02c7593553cb6047bddcfbccb79ec72fc9/src/protocol/http/org/apache/jmeter/protocol/http/parser/LagartoBasedHtmlParser.java#L210
Vladimir
Re: Important Regression in nightly build compared to 2.13 or r1715087
Posted by Philippe Mouawad <ph...@gmail.com>.
Hi Felix,
I also thought that at the begining and tried to apply some patches on code.
Find them attached at :
https://bz.apache.org/bugzilla/show_bug.cgi?id=58950
Note also I tried using 4.5.2 and uncommented this line in :
- HTTPHC4Impl:
((AbstractHttpClient)
httpClient).setReuseStrategy(DefaultClientConnectionReuseStrategy.INSTANCE);
Regards
On Sun, Jan 31, 2016 at 6:20 PM, Felix Schumacher <
felix.schumacher@internetallee.de> wrote:
> Am 31.01.2016 um 18:07 schrieb Felix Schumacher:
>
>> Am 31.01.2016 um 17:52 schrieb Philippe Mouawad:
>>
>>> On Sun, Jan 31, 2016 at 5:45 PM, Felix Schumacher <
>>> felix.schumacher@internetallee.de> wrote:
>>>
>>> Am 31.01.2016 um 17:41 schrieb Philippe Mouawad:
>>>>
>>>> On Sun, Jan 31, 2016 at 5:30 PM, Felix Schumacher <
>>>>> felix.schumacher@internetallee.de> wrote:
>>>>>
>>>>> Am 30.01.2016 um 00:31 schrieb Philippe Mouawad:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>>> I made a real load test today using nightly build and faced an
>>>>>>> important
>>>>>>> issue.
>>>>>>>
>>>>>>> Here are the details:
>>>>>>> - Test uses 1000 Threads on 1 Instance
>>>>>>> - It uses "Download Embedded Resources"
>>>>>>> - Socket Timeout is set to 10s
>>>>>>>
>>>>>>> No time between the requests?
>>>>>>>
>>>>>> Variable Time between requests. It is supposed to reproduce user
>>>>> website
>>>>> activity.
>>>>> The timer is not fixed which can harden comparison but I always
>>>>> reproduce
>>>>> the difference of behaviour between version for all runs.
>>>>>
>>>>> So how many requests per second do you simulate (roughly)?
>>>>
>>>> 20 to 30 per second
>>>
>>> I tried to load test the ROOT webapp of tomcat 8 (without a timed delay)
>>>> and did not get any Exceptions.
>>>>
>>>> Does it mean there is no network ? only locally ?
>>>>
>>> I am in ideal conditions in term of load testing as I am on a dedicated
>>> machine.
>>>
>> OK. I think I see your problem (even if I don't see the exceptions).
>>
>> I test a locally running tomcat 8 with 1000 threads running for 1000
>> times one http sampler, which has a gaussian timer that was configured with
>> 30000.0 and 300 milliseconds.
>>
>> Now, if I run it with httpclient4 I get errors after a short period of
>> time (about 5 to 10 seconds). No errors, when run with httpclient3.
>>
>
> If I let the test run with one round, only, there is no error. The same is
> true, when I disable keep alive. So it seems to be a change in keep alive
> handling.
>
> Regards,
> Felix
>
>>
>>
>> Regards,
>> Felix
>> ||
>>
>>> Regards,
>>>> Felix
>>>>
>>>>
>>>>
>>>> Is your server capable of serving thousand requests simultaneously?
>>>>>
>>>>>> Yes, no problem on this side.
>>>>>>
>>>>> Number of requests per second is not high at all (50 samples included
>>>>> Transaction Controller which encapsulate Http Requests ).
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>>> Felix
>>>>>>
>>>>>> There is no overloading of the machine, no impacting GC
>>>>>>
>>>>>>> Very rapidly, I start getting a lot of errors:
>>>>>>>
>>>>>>>
>>>>>>> - Non HTTP response code:
>>>>>>> org.apache.http.conn.ConnectTimeoutException
>>>>>>> message:Non HTTP response message: Connect towww.foo.com:80
>>>>>>> timed
>>>>>>> out
>>>>>>>
>>>>>>>
>>>>>>> Rate of error varies between 15% and 30%.
>>>>>>> Note that if I navigate on the application, I don't face the errors.
>>>>>>>
>>>>>>> I ran the same test using exactly the same configuration:
>>>>>>>
>>>>>>> - Same machine
>>>>>>> - Same JVM version and tuning
>>>>>>> - Same user.properties
>>>>>>> - Same hc.parameters
>>>>>>>
>>>>>>> But jmeter r1715087
>>>>>>> And error rate is 0.30%.
>>>>>>>
>>>>>>>
>>>>>>> Note the target server has a load balancer that returns a keep-alive
>>>>>>> duration set to 2 (2 seconds).
>>>>>>>
>>>>>>> This issue is a blocker one for the release of next version.
>>>>>>> I compared code with revision 1715087 and I don't see many changes in
>>>>>>> HTTPHC4Impl that would explain this regression.
>>>>>>> I commented out some suspects , retried but I get same results.
>>>>>>>
>>>>>>>
>>>>>>> I also upgraded to HttpClient 4.5.2 and uncommented the code
>>>>>>> expected to
>>>>>>> be
>>>>>>> added, same results.
>>>>>>>
>>>>>>> So for now I tend to suspect an issue in HttpClient/Core.
>>>>>>>
>>>>>>>
>>>>>>>
>>
>
--
Cordialement.
Philippe Mouawad.
Re: Important Regression in nightly build compared to 2.13 or
r1715087
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 31.01.2016 um 18:07 schrieb Felix Schumacher:
> Am 31.01.2016 um 17:52 schrieb Philippe Mouawad:
>> On Sun, Jan 31, 2016 at 5:45 PM, Felix Schumacher <
>> felix.schumacher@internetallee.de> wrote:
>>
>>> Am 31.01.2016 um 17:41 schrieb Philippe Mouawad:
>>>
>>>> On Sun, Jan 31, 2016 at 5:30 PM, Felix Schumacher <
>>>> felix.schumacher@internetallee.de> wrote:
>>>>
>>>> Am 30.01.2016 um 00:31 schrieb Philippe Mouawad:
>>>>> Hello,
>>>>>> I made a real load test today using nightly build and faced an important
>>>>>> issue.
>>>>>>
>>>>>> Here are the details:
>>>>>> - Test uses 1000 Threads on 1 Instance
>>>>>> - It uses "Download Embedded Resources"
>>>>>> - Socket Timeout is set to 10s
>>>>>>
>>>>>> No time between the requests?
>>>> Variable Time between requests. It is supposed to reproduce user website
>>>> activity.
>>>> The timer is not fixed which can harden comparison but I always reproduce
>>>> the difference of behaviour between version for all runs.
>>>>
>>> So how many requests per second do you simulate (roughly)?
>>>
>> 20 to 30 per second
>>
>>> I tried to load test the ROOT webapp of tomcat 8 (without a timed delay)
>>> and did not get any Exceptions.
>>>
>>> Does it mean there is no network ? only locally ?
>> I am in ideal conditions in term of load testing as I am on a dedicated
>> machine.
> OK. I think I see your problem (even if I don't see the exceptions).
>
> I test a locally running tomcat 8 with 1000 threads running for 1000
> times one http sampler, which has a gaussian timer that was configured
> with 30000.0 and 300 milliseconds.
>
> Now, if I run it with httpclient4 I get errors after a short period of
> time (about 5 to 10 seconds). No errors, when run with httpclient3.
If I let the test run with one round, only, there is no error. The same
is true, when I disable keep alive. So it seems to be a change in keep
alive handling.
Regards,
Felix
>
>
> Regards,
> Felix
> ||
>>> Regards,
>>> Felix
>>>
>>>
>>>
>>>> Is your server capable of serving thousand requests simultaneously?
>>>>> Yes, no problem on this side.
>>>> Number of requests per second is not high at all (50 samples included
>>>> Transaction Controller which encapsulate Http Requests ).
>>>>
>>>>
>>>> Regards,
>>>>> Felix
>>>>>
>>>>> There is no overloading of the machine, no impacting GC
>>>>>> Very rapidly, I start getting a lot of errors:
>>>>>>
>>>>>>
>>>>>> - Non HTTP response code:
>>>>>> org.apache.http.conn.ConnectTimeoutException
>>>>>> message:Non HTTP response message: Connect towww.foo.com:80 timed
>>>>>> out
>>>>>>
>>>>>>
>>>>>> Rate of error varies between 15% and 30%.
>>>>>> Note that if I navigate on the application, I don't face the errors.
>>>>>>
>>>>>> I ran the same test using exactly the same configuration:
>>>>>>
>>>>>> - Same machine
>>>>>> - Same JVM version and tuning
>>>>>> - Same user.properties
>>>>>> - Same hc.parameters
>>>>>>
>>>>>> But jmeter r1715087
>>>>>> And error rate is 0.30%.
>>>>>>
>>>>>>
>>>>>> Note the target server has a load balancer that returns a keep-alive
>>>>>> duration set to 2 (2 seconds).
>>>>>>
>>>>>> This issue is a blocker one for the release of next version.
>>>>>> I compared code with revision 1715087 and I don't see many changes in
>>>>>> HTTPHC4Impl that would explain this regression.
>>>>>> I commented out some suspects , retried but I get same results.
>>>>>>
>>>>>>
>>>>>> I also upgraded to HttpClient 4.5.2 and uncommented the code expected to
>>>>>> be
>>>>>> added, same results.
>>>>>>
>>>>>> So for now I tend to suspect an issue in HttpClient/Core.
>>>>>>
>>>>>>
>
Re: Important Regression in nightly build compared to 2.13 or
r1715087
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 31.01.2016 um 17:52 schrieb Philippe Mouawad:
> On Sun, Jan 31, 2016 at 5:45 PM, Felix Schumacher <
> felix.schumacher@internetallee.de> wrote:
>
>> Am 31.01.2016 um 17:41 schrieb Philippe Mouawad:
>>
>>> On Sun, Jan 31, 2016 at 5:30 PM, Felix Schumacher <
>>> felix.schumacher@internetallee.de> wrote:
>>>
>>> Am 30.01.2016 um 00:31 schrieb Philippe Mouawad:
>>>> Hello,
>>>>> I made a real load test today using nightly build and faced an important
>>>>> issue.
>>>>>
>>>>> Here are the details:
>>>>> - Test uses 1000 Threads on 1 Instance
>>>>> - It uses "Download Embedded Resources"
>>>>> - Socket Timeout is set to 10s
>>>>>
>>>>> No time between the requests?
>>> Variable Time between requests. It is supposed to reproduce user website
>>> activity.
>>> The timer is not fixed which can harden comparison but I always reproduce
>>> the difference of behaviour between version for all runs.
>>>
>> So how many requests per second do you simulate (roughly)?
>>
> 20 to 30 per second
>
>> I tried to load test the ROOT webapp of tomcat 8 (without a timed delay)
>> and did not get any Exceptions.
>>
>> Does it mean there is no network ? only locally ?
> I am in ideal conditions in term of load testing as I am on a dedicated
> machine.
OK. I think I see your problem (even if I don't see the exceptions).
I test a locally running tomcat 8 with 1000 threads running for 1000
times one http sampler, which has a gaussian timer that was configured
with 30000.0 and 300 milliseconds.
Now, if I run it with httpclient4 I get errors after a short period of
time (about 5 to 10 seconds). No errors, when run with httpclient3.
Regards,
Felix
||
>> Regards,
>> Felix
>>
>>
>>
>>> Is your server capable of serving thousand requests simultaneously?
>>>> Yes, no problem on this side.
>>> Number of requests per second is not high at all (50 samples included
>>> Transaction Controller which encapsulate Http Requests ).
>>>
>>>
>>> Regards,
>>>> Felix
>>>>
>>>> There is no overloading of the machine, no impacting GC
>>>>> Very rapidly, I start getting a lot of errors:
>>>>>
>>>>>
>>>>> - Non HTTP response code:
>>>>> org.apache.http.conn.ConnectTimeoutException
>>>>> message:Non HTTP response message: Connect to www.foo.com:80 timed
>>>>> out
>>>>>
>>>>>
>>>>> Rate of error varies between 15% and 30%.
>>>>> Note that if I navigate on the application, I don't face the errors.
>>>>>
>>>>> I ran the same test using exactly the same configuration:
>>>>>
>>>>> - Same machine
>>>>> - Same JVM version and tuning
>>>>> - Same user.properties
>>>>> - Same hc.parameters
>>>>>
>>>>> But jmeter r1715087
>>>>> And error rate is 0.30%.
>>>>>
>>>>>
>>>>> Note the target server has a load balancer that returns a keep-alive
>>>>> duration set to 2 (2 seconds).
>>>>>
>>>>> This issue is a blocker one for the release of next version.
>>>>> I compared code with revision 1715087 and I don't see many changes in
>>>>> HTTPHC4Impl that would explain this regression.
>>>>> I commented out some suspects , retried but I get same results.
>>>>>
>>>>>
>>>>> I also upgraded to HttpClient 4.5.2 and uncommented the code expected to
>>>>> be
>>>>> added, same results.
>>>>>
>>>>> So for now I tend to suspect an issue in HttpClient/Core.
>>>>>
>>>>>
>
Re: Important Regression in nightly build compared to 2.13 or r1715087
Posted by Philippe Mouawad <ph...@gmail.com>.
On Sun, Jan 31, 2016 at 5:45 PM, Felix Schumacher <
felix.schumacher@internetallee.de> wrote:
> Am 31.01.2016 um 17:41 schrieb Philippe Mouawad:
>
>> On Sun, Jan 31, 2016 at 5:30 PM, Felix Schumacher <
>> felix.schumacher@internetallee.de> wrote:
>>
>> Am 30.01.2016 um 00:31 schrieb Philippe Mouawad:
>>>
>>> Hello,
>>>> I made a real load test today using nightly build and faced an important
>>>> issue.
>>>>
>>>> Here are the details:
>>>> - Test uses 1000 Threads on 1 Instance
>>>> - It uses "Download Embedded Resources"
>>>> - Socket Timeout is set to 10s
>>>>
>>>> No time between the requests?
>>>
>> Variable Time between requests. It is supposed to reproduce user website
>> activity.
>> The timer is not fixed which can harden comparison but I always reproduce
>> the difference of behaviour between version for all runs.
>>
> So how many requests per second do you simulate (roughly)?
>
20 to 30 per second
>
> I tried to load test the ROOT webapp of tomcat 8 (without a timed delay)
> and did not get any Exceptions.
>
> Does it mean there is no network ? only locally ?
I am in ideal conditions in term of load testing as I am on a dedicated
machine.
> Regards,
> Felix
>
>
>
>>
>> Is your server capable of serving thousand requests simultaneously?
>>>
>>> Yes, no problem on this side.
>> Number of requests per second is not high at all (50 samples included
>> Transaction Controller which encapsulate Http Requests ).
>>
>>
>> Regards,
>>> Felix
>>>
>>> There is no overloading of the machine, no impacting GC
>>>>
>>>> Very rapidly, I start getting a lot of errors:
>>>>
>>>>
>>>> - Non HTTP response code:
>>>> org.apache.http.conn.ConnectTimeoutException
>>>> message:Non HTTP response message: Connect to www.foo.com:80 timed
>>>> out
>>>>
>>>>
>>>> Rate of error varies between 15% and 30%.
>>>> Note that if I navigate on the application, I don't face the errors.
>>>>
>>>> I ran the same test using exactly the same configuration:
>>>>
>>>> - Same machine
>>>> - Same JVM version and tuning
>>>> - Same user.properties
>>>> - Same hc.parameters
>>>>
>>>> But jmeter r1715087
>>>> And error rate is 0.30%.
>>>>
>>>>
>>>> Note the target server has a load balancer that returns a keep-alive
>>>> duration set to 2 (2 seconds).
>>>>
>>>> This issue is a blocker one for the release of next version.
>>>> I compared code with revision 1715087 and I don't see many changes in
>>>> HTTPHC4Impl that would explain this regression.
>>>> I commented out some suspects , retried but I get same results.
>>>>
>>>>
>>>> I also upgraded to HttpClient 4.5.2 and uncommented the code expected to
>>>> be
>>>> added, same results.
>>>>
>>>> So for now I tend to suspect an issue in HttpClient/Core.
>>>>
>>>>
>>>
>>
>
--
Cordialement.
Philippe Mouawad.
Re: Important Regression in nightly build compared to 2.13 or
r1715087
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 31.01.2016 um 17:41 schrieb Philippe Mouawad:
> On Sun, Jan 31, 2016 at 5:30 PM, Felix Schumacher <
> felix.schumacher@internetallee.de> wrote:
>
>> Am 30.01.2016 um 00:31 schrieb Philippe Mouawad:
>>
>>> Hello,
>>> I made a real load test today using nightly build and faced an important
>>> issue.
>>>
>>> Here are the details:
>>> - Test uses 1000 Threads on 1 Instance
>>> - It uses "Download Embedded Resources"
>>> - Socket Timeout is set to 10s
>>>
>> No time between the requests?
> Variable Time between requests. It is supposed to reproduce user website
> activity.
> The timer is not fixed which can harden comparison but I always reproduce
> the difference of behaviour between version for all runs.
So how many requests per second do you simulate (roughly)?
I tried to load test the ROOT webapp of tomcat 8 (without a timed delay)
and did not get any Exceptions.
Regards,
Felix
>
>
>> Is your server capable of serving thousand requests simultaneously?
>>
> Yes, no problem on this side.
> Number of requests per second is not high at all (50 samples included
> Transaction Controller which encapsulate Http Requests ).
>
>
>> Regards,
>> Felix
>>
>>> There is no overloading of the machine, no impacting GC
>>>
>>> Very rapidly, I start getting a lot of errors:
>>>
>>>
>>> - Non HTTP response code: org.apache.http.conn.ConnectTimeoutException
>>> message:Non HTTP response message: Connect to www.foo.com:80 timed
>>> out
>>>
>>>
>>> Rate of error varies between 15% and 30%.
>>> Note that if I navigate on the application, I don't face the errors.
>>>
>>> I ran the same test using exactly the same configuration:
>>>
>>> - Same machine
>>> - Same JVM version and tuning
>>> - Same user.properties
>>> - Same hc.parameters
>>>
>>> But jmeter r1715087
>>> And error rate is 0.30%.
>>>
>>>
>>> Note the target server has a load balancer that returns a keep-alive
>>> duration set to 2 (2 seconds).
>>>
>>> This issue is a blocker one for the release of next version.
>>> I compared code with revision 1715087 and I don't see many changes in
>>> HTTPHC4Impl that would explain this regression.
>>> I commented out some suspects , retried but I get same results.
>>>
>>>
>>> I also upgraded to HttpClient 4.5.2 and uncommented the code expected to
>>> be
>>> added, same results.
>>>
>>> So for now I tend to suspect an issue in HttpClient/Core.
>>>
>>
>
Re: Important Regression in nightly build compared to 2.13 or r1715087
Posted by Philippe Mouawad <ph...@gmail.com>.
On Sun, Jan 31, 2016 at 5:30 PM, Felix Schumacher <
felix.schumacher@internetallee.de> wrote:
> Am 30.01.2016 um 00:31 schrieb Philippe Mouawad:
>
>> Hello,
>> I made a real load test today using nightly build and faced an important
>> issue.
>>
>> Here are the details:
>> - Test uses 1000 Threads on 1 Instance
>> - It uses "Download Embedded Resources"
>> - Socket Timeout is set to 10s
>>
> No time between the requests?
Variable Time between requests. It is supposed to reproduce user website
activity.
The timer is not fixed which can harden comparison but I always reproduce
the difference of behaviour between version for all runs.
> Is your server capable of serving thousand requests simultaneously?
>
Yes, no problem on this side.
Number of requests per second is not high at all (50 samples included
Transaction Controller which encapsulate Http Requests ).
>
> Regards,
> Felix
>
>>
>> There is no overloading of the machine, no impacting GC
>>
>> Very rapidly, I start getting a lot of errors:
>>
>>
>> - Non HTTP response code: org.apache.http.conn.ConnectTimeoutException
>> message:Non HTTP response message: Connect to www.foo.com:80 timed
>> out
>>
>>
>> Rate of error varies between 15% and 30%.
>> Note that if I navigate on the application, I don't face the errors.
>>
>> I ran the same test using exactly the same configuration:
>>
>> - Same machine
>> - Same JVM version and tuning
>> - Same user.properties
>> - Same hc.parameters
>>
>> But jmeter r1715087
>> And error rate is 0.30%.
>>
>>
>> Note the target server has a load balancer that returns a keep-alive
>> duration set to 2 (2 seconds).
>>
>> This issue is a blocker one for the release of next version.
>> I compared code with revision 1715087 and I don't see many changes in
>> HTTPHC4Impl that would explain this regression.
>> I commented out some suspects , retried but I get same results.
>>
>>
>> I also upgraded to HttpClient 4.5.2 and uncommented the code expected to
>> be
>> added, same results.
>>
>> So for now I tend to suspect an issue in HttpClient/Core.
>>
>
>
--
Cordialement.
Philippe Mouawad.
Re: Important Regression in nightly build compared to 2.13 or
r1715087
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 30.01.2016 um 00:31 schrieb Philippe Mouawad:
> Hello,
> I made a real load test today using nightly build and faced an important
> issue.
>
> Here are the details:
> - Test uses 1000 Threads on 1 Instance
> - It uses "Download Embedded Resources"
> - Socket Timeout is set to 10s
No time between the requests? Is your server capable of serving thousand
requests simultaneously?
Regards,
Felix
>
> There is no overloading of the machine, no impacting GC
>
> Very rapidly, I start getting a lot of errors:
>
>
> - Non HTTP response code: org.apache.http.conn.ConnectTimeoutException
> message:Non HTTP response message: Connect to www.foo.com:80 timed out
>
>
> Rate of error varies between 15% and 30%.
> Note that if I navigate on the application, I don't face the errors.
>
> I ran the same test using exactly the same configuration:
>
> - Same machine
> - Same JVM version and tuning
> - Same user.properties
> - Same hc.parameters
>
> But jmeter r1715087
> And error rate is 0.30%.
>
>
> Note the target server has a load balancer that returns a keep-alive
> duration set to 2 (2 seconds).
>
> This issue is a blocker one for the release of next version.
> I compared code with revision 1715087 and I don't see many changes in
> HTTPHC4Impl that would explain this regression.
> I commented out some suspects , retried but I get same results.
>
>
> I also upgraded to HttpClient 4.5.2 and uncommented the code expected to be
> added, same results.
>
> So for now I tend to suspect an issue in HttpClient/Core.
Re: Important Regression in nightly build compared to 2.13 or r1715087
Posted by Philippe Mouawad <ph...@gmail.com>.
Hi sebb,
Do you have some idea in mind ? What information do you want ?
For info, I made a test with same "working" jmeter version rr1715087 and
only changed httpclient/core/mime to 4.5.2/4.4.4/4.5.1 and I reproduce the
issue.
Regards
On Sat, Jan 30, 2016 at 1:40 AM, sebb <se...@gmail.com> wrote:
> On 30 January 2016 at 00:13, Philippe Mouawad
> <ph...@gmail.com> wrote:
> > Well,
> > it's a Connect Timeout (connect Timeout being set to 10 s), I cannot
> tell
> > as Timeout occurs on an embedded resources.
> > But it seems issues is related to Embedded resources download.
>
> So add some debug then to show the actual times.
>
> > Regards
> >
> > On Sat, Jan 30, 2016 at 1:00 AM, sebb <se...@gmail.com> wrote:
> >
> >> On 29 January 2016 at 23:47, Philippe Mouawad
> >> <ph...@gmail.com> wrote:
> >> > On Saturday, January 30, 2016, sebb <se...@gmail.com> wrote:
> >> >
> >> >> On 29 January 2016 at 23:31, Philippe Mouawad
> >> >> <philippe.mouawad@gmail.com <javascript:;>> wrote:
> >> >> > Hello,
> >> >> > I made a real load test today using nightly build and faced an
> >> important
> >> >> > issue.
> >> >> >
> >> >> > Here are the details:
> >> >> > - Test uses 1000 Threads on 1 Instance
> >> >> > - It uses "Download Embedded Resources"
> >> >> > - Socket Timeout is set to 10s
> >> >> >
> >> >> > There is no overloading of the machine, no impacting GC
> >> >> >
> >> >> > Very rapidly, I start getting a lot of errors:
> >> >> >
> >> >> >
> >> >> > - Non HTTP response code:
> >> org.apache.http.conn.ConnectTimeoutException
> >> >> > message:Non HTTP response message: Connect to www.foo.com:80
> timed
> >> >> out
> >> >>
> >> >> What is the elapsed time till the timeout occurs?
> >> >
> >> >
> >> > In the initial test, it happens after 2 minutes at around 300.
> threads.
> >> >
> >> > with a test at 300 threads and a very rapid rampup (5 seconds, with a
> >> > startup delay of 10 seconds) to reproduce it happens after 16 seconds
> >> >
> >>
> >> I meant - how long between the start of a sample until the timeout?
> >> Does it vary much between samples?
> >>
> >> >>
> >> >> >
> >> >> > Rate of error varies between 15% and 30%.
> >> >> > Note that if I navigate on the application, I don't face the
> errors.
> >> >> >
> >> >> > I ran the same test using exactly the same configuration:
> >> >> >
> >> >> > - Same machine
> >> >> > - Same JVM version and tuning
> >> >> > - Same user.properties
> >> >> > - Same hc.parameters
> >> >> >
> >> >> > But jmeter r1715087
> >> >> > And error rate is 0.30%.
> >> >> >
> >> >> >
> >> >> > Note the target server has a load balancer that returns a
> keep-alive
> >> >> > duration set to 2 (2 seconds).
> >> >> >
> >> >> > This issue is a blocker one for the release of next version.
> >> >> > I compared code with revision 1715087 and I don't see many changes
> in
> >> >> > HTTPHC4Impl that would explain this regression.
> >> >> > I commented out some suspects , retried but I get same results.
> >> >> >
> >> >> >
> >> >> > I also upgraded to HttpClient 4.5.2 and uncommented the code
> expected
> >> to
> >> >> be
> >> >> > added, same results.
> >> >> >
> >> >> > So for now I tend to suspect an issue in HttpClient/Core.
> >> >> > --
> >> >> > Regards
> >> >> > Philippe M.
> >> >>
> >> >
> >> >
> >> > --
> >> > Cordialement.
> >> > Philippe Mouawad.
> >>
> >
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>
--
Cordialement.
Philippe Mouawad.
Re: Important Regression in nightly build compared to 2.13 or r1715087
Posted by sebb <se...@gmail.com>.
On 30 January 2016 at 00:13, Philippe Mouawad
<ph...@gmail.com> wrote:
> Well,
> it's a Connect Timeout (connect Timeout being set to 10 s), I cannot tell
> as Timeout occurs on an embedded resources.
> But it seems issues is related to Embedded resources download.
So add some debug then to show the actual times.
> Regards
>
> On Sat, Jan 30, 2016 at 1:00 AM, sebb <se...@gmail.com> wrote:
>
>> On 29 January 2016 at 23:47, Philippe Mouawad
>> <ph...@gmail.com> wrote:
>> > On Saturday, January 30, 2016, sebb <se...@gmail.com> wrote:
>> >
>> >> On 29 January 2016 at 23:31, Philippe Mouawad
>> >> <philippe.mouawad@gmail.com <javascript:;>> wrote:
>> >> > Hello,
>> >> > I made a real load test today using nightly build and faced an
>> important
>> >> > issue.
>> >> >
>> >> > Here are the details:
>> >> > - Test uses 1000 Threads on 1 Instance
>> >> > - It uses "Download Embedded Resources"
>> >> > - Socket Timeout is set to 10s
>> >> >
>> >> > There is no overloading of the machine, no impacting GC
>> >> >
>> >> > Very rapidly, I start getting a lot of errors:
>> >> >
>> >> >
>> >> > - Non HTTP response code:
>> org.apache.http.conn.ConnectTimeoutException
>> >> > message:Non HTTP response message: Connect to www.foo.com:80 timed
>> >> out
>> >>
>> >> What is the elapsed time till the timeout occurs?
>> >
>> >
>> > In the initial test, it happens after 2 minutes at around 300. threads.
>> >
>> > with a test at 300 threads and a very rapid rampup (5 seconds, with a
>> > startup delay of 10 seconds) to reproduce it happens after 16 seconds
>> >
>>
>> I meant - how long between the start of a sample until the timeout?
>> Does it vary much between samples?
>>
>> >>
>> >> >
>> >> > Rate of error varies between 15% and 30%.
>> >> > Note that if I navigate on the application, I don't face the errors.
>> >> >
>> >> > I ran the same test using exactly the same configuration:
>> >> >
>> >> > - Same machine
>> >> > - Same JVM version and tuning
>> >> > - Same user.properties
>> >> > - Same hc.parameters
>> >> >
>> >> > But jmeter r1715087
>> >> > And error rate is 0.30%.
>> >> >
>> >> >
>> >> > Note the target server has a load balancer that returns a keep-alive
>> >> > duration set to 2 (2 seconds).
>> >> >
>> >> > This issue is a blocker one for the release of next version.
>> >> > I compared code with revision 1715087 and I don't see many changes in
>> >> > HTTPHC4Impl that would explain this regression.
>> >> > I commented out some suspects , retried but I get same results.
>> >> >
>> >> >
>> >> > I also upgraded to HttpClient 4.5.2 and uncommented the code expected
>> to
>> >> be
>> >> > added, same results.
>> >> >
>> >> > So for now I tend to suspect an issue in HttpClient/Core.
>> >> > --
>> >> > Regards
>> >> > Philippe M.
>> >>
>> >
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
Re: Important Regression in nightly build compared to 2.13 or r1715087
Posted by Philippe Mouawad <ph...@gmail.com>.
Well,
it's a Connect Timeout (connect Timeout being set to 10 s), I cannot tell
as Timeout occurs on an embedded resources.
But it seems issues is related to Embedded resources download.
Regards
On Sat, Jan 30, 2016 at 1:00 AM, sebb <se...@gmail.com> wrote:
> On 29 January 2016 at 23:47, Philippe Mouawad
> <ph...@gmail.com> wrote:
> > On Saturday, January 30, 2016, sebb <se...@gmail.com> wrote:
> >
> >> On 29 January 2016 at 23:31, Philippe Mouawad
> >> <philippe.mouawad@gmail.com <javascript:;>> wrote:
> >> > Hello,
> >> > I made a real load test today using nightly build and faced an
> important
> >> > issue.
> >> >
> >> > Here are the details:
> >> > - Test uses 1000 Threads on 1 Instance
> >> > - It uses "Download Embedded Resources"
> >> > - Socket Timeout is set to 10s
> >> >
> >> > There is no overloading of the machine, no impacting GC
> >> >
> >> > Very rapidly, I start getting a lot of errors:
> >> >
> >> >
> >> > - Non HTTP response code:
> org.apache.http.conn.ConnectTimeoutException
> >> > message:Non HTTP response message: Connect to www.foo.com:80 timed
> >> out
> >>
> >> What is the elapsed time till the timeout occurs?
> >
> >
> > In the initial test, it happens after 2 minutes at around 300. threads.
> >
> > with a test at 300 threads and a very rapid rampup (5 seconds, with a
> > startup delay of 10 seconds) to reproduce it happens after 16 seconds
> >
>
> I meant - how long between the start of a sample until the timeout?
> Does it vary much between samples?
>
> >>
> >> >
> >> > Rate of error varies between 15% and 30%.
> >> > Note that if I navigate on the application, I don't face the errors.
> >> >
> >> > I ran the same test using exactly the same configuration:
> >> >
> >> > - Same machine
> >> > - Same JVM version and tuning
> >> > - Same user.properties
> >> > - Same hc.parameters
> >> >
> >> > But jmeter r1715087
> >> > And error rate is 0.30%.
> >> >
> >> >
> >> > Note the target server has a load balancer that returns a keep-alive
> >> > duration set to 2 (2 seconds).
> >> >
> >> > This issue is a blocker one for the release of next version.
> >> > I compared code with revision 1715087 and I don't see many changes in
> >> > HTTPHC4Impl that would explain this regression.
> >> > I commented out some suspects , retried but I get same results.
> >> >
> >> >
> >> > I also upgraded to HttpClient 4.5.2 and uncommented the code expected
> to
> >> be
> >> > added, same results.
> >> >
> >> > So for now I tend to suspect an issue in HttpClient/Core.
> >> > --
> >> > Regards
> >> > Philippe M.
> >>
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>
--
Cordialement.
Philippe Mouawad.
Re: Important Regression in nightly build compared to 2.13 or r1715087
Posted by sebb <se...@gmail.com>.
On 29 January 2016 at 23:47, Philippe Mouawad
<ph...@gmail.com> wrote:
> On Saturday, January 30, 2016, sebb <se...@gmail.com> wrote:
>
>> On 29 January 2016 at 23:31, Philippe Mouawad
>> <philippe.mouawad@gmail.com <javascript:;>> wrote:
>> > Hello,
>> > I made a real load test today using nightly build and faced an important
>> > issue.
>> >
>> > Here are the details:
>> > - Test uses 1000 Threads on 1 Instance
>> > - It uses "Download Embedded Resources"
>> > - Socket Timeout is set to 10s
>> >
>> > There is no overloading of the machine, no impacting GC
>> >
>> > Very rapidly, I start getting a lot of errors:
>> >
>> >
>> > - Non HTTP response code: org.apache.http.conn.ConnectTimeoutException
>> > message:Non HTTP response message: Connect to www.foo.com:80 timed
>> out
>>
>> What is the elapsed time till the timeout occurs?
>
>
> In the initial test, it happens after 2 minutes at around 300. threads.
>
> with a test at 300 threads and a very rapid rampup (5 seconds, with a
> startup delay of 10 seconds) to reproduce it happens after 16 seconds
>
I meant - how long between the start of a sample until the timeout?
Does it vary much between samples?
>>
>> >
>> > Rate of error varies between 15% and 30%.
>> > Note that if I navigate on the application, I don't face the errors.
>> >
>> > I ran the same test using exactly the same configuration:
>> >
>> > - Same machine
>> > - Same JVM version and tuning
>> > - Same user.properties
>> > - Same hc.parameters
>> >
>> > But jmeter r1715087
>> > And error rate is 0.30%.
>> >
>> >
>> > Note the target server has a load balancer that returns a keep-alive
>> > duration set to 2 (2 seconds).
>> >
>> > This issue is a blocker one for the release of next version.
>> > I compared code with revision 1715087 and I don't see many changes in
>> > HTTPHC4Impl that would explain this regression.
>> > I commented out some suspects , retried but I get same results.
>> >
>> >
>> > I also upgraded to HttpClient 4.5.2 and uncommented the code expected to
>> be
>> > added, same results.
>> >
>> > So for now I tend to suspect an issue in HttpClient/Core.
>> > --
>> > Regards
>> > Philippe M.
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
Re: Important Regression in nightly build compared to 2.13 or r1715087
Posted by Philippe Mouawad <ph...@gmail.com>.
On Saturday, January 30, 2016, sebb <se...@gmail.com> wrote:
> On 29 January 2016 at 23:31, Philippe Mouawad
> <philippe.mouawad@gmail.com <javascript:;>> wrote:
> > Hello,
> > I made a real load test today using nightly build and faced an important
> > issue.
> >
> > Here are the details:
> > - Test uses 1000 Threads on 1 Instance
> > - It uses "Download Embedded Resources"
> > - Socket Timeout is set to 10s
> >
> > There is no overloading of the machine, no impacting GC
> >
> > Very rapidly, I start getting a lot of errors:
> >
> >
> > - Non HTTP response code: org.apache.http.conn.ConnectTimeoutException
> > message:Non HTTP response message: Connect to www.foo.com:80 timed
> out
>
> What is the elapsed time till the timeout occurs?
In the initial test, it happens after 2 minutes at around 300. threads.
with a test at 300 threads and a very rapid rampup (5 seconds, with a
startup delay of 10 seconds) to reproduce it happens after 16 seconds
>
> >
> > Rate of error varies between 15% and 30%.
> > Note that if I navigate on the application, I don't face the errors.
> >
> > I ran the same test using exactly the same configuration:
> >
> > - Same machine
> > - Same JVM version and tuning
> > - Same user.properties
> > - Same hc.parameters
> >
> > But jmeter r1715087
> > And error rate is 0.30%.
> >
> >
> > Note the target server has a load balancer that returns a keep-alive
> > duration set to 2 (2 seconds).
> >
> > This issue is a blocker one for the release of next version.
> > I compared code with revision 1715087 and I don't see many changes in
> > HTTPHC4Impl that would explain this regression.
> > I commented out some suspects , retried but I get same results.
> >
> >
> > I also upgraded to HttpClient 4.5.2 and uncommented the code expected to
> be
> > added, same results.
> >
> > So for now I tend to suspect an issue in HttpClient/Core.
> > --
> > Regards
> > Philippe M.
>
--
Cordialement.
Philippe Mouawad.
Re: Important Regression in nightly build compared to 2.13 or r1715087
Posted by sebb <se...@gmail.com>.
On 29 January 2016 at 23:31, Philippe Mouawad
<ph...@gmail.com> wrote:
> Hello,
> I made a real load test today using nightly build and faced an important
> issue.
>
> Here are the details:
> - Test uses 1000 Threads on 1 Instance
> - It uses "Download Embedded Resources"
> - Socket Timeout is set to 10s
>
> There is no overloading of the machine, no impacting GC
>
> Very rapidly, I start getting a lot of errors:
>
>
> - Non HTTP response code: org.apache.http.conn.ConnectTimeoutException
> message:Non HTTP response message: Connect to www.foo.com:80 timed out
What is the elapsed time till the timeout occurs?
>
> Rate of error varies between 15% and 30%.
> Note that if I navigate on the application, I don't face the errors.
>
> I ran the same test using exactly the same configuration:
>
> - Same machine
> - Same JVM version and tuning
> - Same user.properties
> - Same hc.parameters
>
> But jmeter r1715087
> And error rate is 0.30%.
>
>
> Note the target server has a load balancer that returns a keep-alive
> duration set to 2 (2 seconds).
>
> This issue is a blocker one for the release of next version.
> I compared code with revision 1715087 and I don't see many changes in
> HTTPHC4Impl that would explain this regression.
> I commented out some suspects , retried but I get same results.
>
>
> I also upgraded to HttpClient 4.5.2 and uncommented the code expected to be
> added, same results.
>
> So for now I tend to suspect an issue in HttpClient/Core.
> --
> Regards
> Philippe M.