You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jmeter.apache.org by Philippe Mouawad <ph...@gmail.com> on 2018/02/24 18:06:19 UTC
Bug 58807
Hello,
While working on Migration to last HC4.5 APIs and testing Client
Certificate (see another question on how to include tests for it), I
confirmed that the bug reported by Rainer was still here.
This bug has the following impact:
- Whenever you set "https.use.cached.ssl.context=false " which is
required for Client Certificate Testing, you end up :
- Replaying SSL Handshake on every sample while we shouldn't
- Losing KeepAlive
- Another thing he tackles is that fact that within Thread Group
iteration we reuse objects we shouldn't
IMO, we need to tackle it, looking at Rainer patch I agree with almost all
his proposals except for what I describe below.
Let's see use most common cases :
- 0/ We are testing HTTP, Rainer is correct saying that when we switch
to next iteration and are using KeepAlive we use the same connection while
we shouldn't , this would be PROBLEM A below
- 1/ We are testing HTTPS only without client certificate, then
https.use.cached.ssl.context=true is ok within 1 thread group iteration.
What would be unrealistic is the fact that SSL Handshake will not occur for
next iteration. Let's call this case PROBLEM A.
- 2/ We are testing HTTPS only WITH client certificate, then since
https.use.cached.ssl.context=false is needed we have a problem PROBLEM B.
*ASSUMPTION:*
First, I think that we should consider that Each Thread Group Iteration is
a new user, indeed, If one wants to make the same user loop, he can easily
add a Loop Controller as child of Thread Group and it's ok.
*So let's zoom to PROBLEM B first:*
- If we make the assumption above, part of the patch of Rainer can be
reused:
- Have a thread local hold the fact that reset occured, but Maybe
using JMeterVariables would be better particularly if one day we
decide to
switch to different model 1 VU == 1 Thread.
- What should be changed in Rainer patch is the SSL reset, indeed for
HC4.5 we need to close idle and expired connections to really reset SSL
State
- What I don't get about Rainer patch is wether we should do the
close on all HTTPCLIENT Instances of VUs or not. I think we should but It
would be great if another brain or more would confirm
*Let's zoom to PROBLEM A:*
In summary, whenever we make a new Thread Group iteration, we must start
with new connections and not reuse existing ones.
So Rainer patch looks ok to me.
But:
1/ I would personally set default to
-
reuse.http.connections=false
2/ I would not introduce reuse.http.client
3/ I would not as a consequence reset HttpClient as it is not needed
to reset state if I am not wrong and it would
introduce "initialization cost" in requests timing
What's your thoughts ?
--
Regards
Philippe M.
Re: Bug 58807
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello,
Any feedback on this ?
Thanks
On Wed, Feb 28, 2018 at 10:41 PM, Philippe Mouawad <
philippe.mouawad@gmail.com> wrote:
> Hello,
> Anybody had a chance to look at committed code regarding this bug ?
>
> Thanks
>
> On Sun, Feb 25, 2018 at 7:21 PM, Philippe Mouawad <
> philippe.mouawad@gmail.com> wrote:
>
>> Hello,
>> After further thinking,
>>
>> https.use.cached.ssl.context should be false by default and maybe instead
>> of having 2 properties we should only have 1 property:
>>
>> - httpclient.reset_state_on_thread_group_iteration
>> - It true we reset SSL Context and close connection on ThreadGroup
>> iteration
>> - If false we allow reuse of both
>>
>> And we would deprecate https.use.cached.ssl.context
>>
>> Thoughts ?
>>
>> Regards
>>
>> On Sat, Feb 24, 2018 at 10:48 PM, Philippe Mouawad <
>> philippe.mouawad@gmail.com> wrote:
>>
>>> Hello,
>>> I attached a patch to:
>>> - https://bz.apache.org/bugzilla/show_bug.cgi?id=58807
>>>
>>> Hope somebody can have a look.
>>>
>>> Regards
>>>
>>> On Sat, Feb 24, 2018 at 7:06 PM, Philippe Mouawad <
>>> philippe.mouawad@gmail.com> wrote:
>>>
>>>> Hello,
>>>> While working on Migration to last HC4.5 APIs and testing Client
>>>> Certificate (see another question on how to include tests for it), I
>>>> confirmed that the bug reported by Rainer was still here.
>>>>
>>>> This bug has the following impact:
>>>>
>>>> - Whenever you set "https.use.cached.ssl.context=false " which is
>>>> required for Client Certificate Testing, you end up :
>>>> - Replaying SSL Handshake on every sample while we shouldn't
>>>> - Losing KeepAlive
>>>> - Another thing he tackles is that fact that within Thread Group
>>>> iteration we reuse objects we shouldn't
>>>>
>>>> IMO, we need to tackle it, looking at Rainer patch I agree with almost
>>>> all his proposals except for what I describe below.
>>>>
>>>>
>>>> Let's see use most common cases :
>>>>
>>>> - 0/ We are testing HTTP, Rainer is correct saying that when we
>>>> switch to next iteration and are using KeepAlive we use the same connection
>>>> while we shouldn't , this would be PROBLEM A below
>>>> - 1/ We are testing HTTPS only without client certificate, then
>>>> https.use.cached.ssl.context=true is ok within 1 thread group
>>>> iteration. What would be unrealistic is the fact that SSL Handshake will
>>>> not occur for next iteration. Let's call this case PROBLEM A.
>>>> - 2/ We are testing HTTPS only WITH client certificate, then since
>>>> https.use.cached.ssl.context=false is needed we have a problem
>>>> PROBLEM B.
>>>>
>>>> *ASSUMPTION:*
>>>>
>>>> First, I think that we should consider that Each Thread Group Iteration
>>>> is a new user, indeed, If one wants to make the same user loop, he can
>>>> easily add a Loop Controller as child of Thread Group and it's ok.
>>>>
>>>> *So let's zoom to PROBLEM B first:*
>>>>
>>>> - If we make the assumption above, part of the patch of Rainer can
>>>> be reused:
>>>> - Have a thread local hold the fact that reset occured, but
>>>> Maybe using JMeterVariables would be better particularly if one day we
>>>> decide to switch to different model 1 VU == 1 Thread.
>>>> - What should be changed in Rainer patch is the SSL reset,
>>>> indeed for HC4.5 we need to close idle and expired connections to really
>>>> reset SSL State
>>>> - What I don't get about Rainer patch is wether we should do the
>>>> close on all HTTPCLIENT Instances of VUs or not. I think we should but It
>>>> would be great if another brain or more would confirm
>>>>
>>>> *Let's zoom to PROBLEM A:*
>>>>
>>>> In summary, whenever we make a new Thread Group iteration, we must
>>>> start with new connections and not reuse existing ones.
>>>>
>>>> So Rainer patch looks ok to me.
>>>>
>>>> But:
>>>>
>>>> 1/ I would personally set default to
>>>>
>>>> -
>>>>
>>>> reuse.http.connections=false
>>>>
>>>>
>>>> 2/ I would not introduce reuse.http.client
>>>>
>>>> 3/ I would not as a consequence reset HttpClient as it is not needed to reset state if I am not wrong and it would
>>>>
>>>> introduce "initialization cost" in requests timing
>>>>
>>>> What's your thoughts ?
>>>>
>>>>
>>>> --
>>>> Regards
>>>> Philippe M.
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Cordialement.
>>> Philippe Mouawad.
>>>
>>>
>>>
>>
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>>
>>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: Bug 58807
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello,
Anybody had a chance to look at committed code regarding this bug ?
Thanks
On Sun, Feb 25, 2018 at 7:21 PM, Philippe Mouawad <
philippe.mouawad@gmail.com> wrote:
> Hello,
> After further thinking,
>
> https.use.cached.ssl.context should be false by default and maybe instead
> of having 2 properties we should only have 1 property:
>
> - httpclient.reset_state_on_thread_group_iteration
> - It true we reset SSL Context and close connection on ThreadGroup
> iteration
> - If false we allow reuse of both
>
> And we would deprecate https.use.cached.ssl.context
>
> Thoughts ?
>
> Regards
>
> On Sat, Feb 24, 2018 at 10:48 PM, Philippe Mouawad <
> philippe.mouawad@gmail.com> wrote:
>
>> Hello,
>> I attached a patch to:
>> - https://bz.apache.org/bugzilla/show_bug.cgi?id=58807
>>
>> Hope somebody can have a look.
>>
>> Regards
>>
>> On Sat, Feb 24, 2018 at 7:06 PM, Philippe Mouawad <
>> philippe.mouawad@gmail.com> wrote:
>>
>>> Hello,
>>> While working on Migration to last HC4.5 APIs and testing Client
>>> Certificate (see another question on how to include tests for it), I
>>> confirmed that the bug reported by Rainer was still here.
>>>
>>> This bug has the following impact:
>>>
>>> - Whenever you set "https.use.cached.ssl.context=false " which is
>>> required for Client Certificate Testing, you end up :
>>> - Replaying SSL Handshake on every sample while we shouldn't
>>> - Losing KeepAlive
>>> - Another thing he tackles is that fact that within Thread Group
>>> iteration we reuse objects we shouldn't
>>>
>>> IMO, we need to tackle it, looking at Rainer patch I agree with almost
>>> all his proposals except for what I describe below.
>>>
>>>
>>> Let's see use most common cases :
>>>
>>> - 0/ We are testing HTTP, Rainer is correct saying that when we
>>> switch to next iteration and are using KeepAlive we use the same connection
>>> while we shouldn't , this would be PROBLEM A below
>>> - 1/ We are testing HTTPS only without client certificate, then
>>> https.use.cached.ssl.context=true is ok within 1 thread group
>>> iteration. What would be unrealistic is the fact that SSL Handshake will
>>> not occur for next iteration. Let's call this case PROBLEM A.
>>> - 2/ We are testing HTTPS only WITH client certificate, then since
>>> https.use.cached.ssl.context=false is needed we have a problem
>>> PROBLEM B.
>>>
>>> *ASSUMPTION:*
>>>
>>> First, I think that we should consider that Each Thread Group Iteration
>>> is a new user, indeed, If one wants to make the same user loop, he can
>>> easily add a Loop Controller as child of Thread Group and it's ok.
>>>
>>> *So let's zoom to PROBLEM B first:*
>>>
>>> - If we make the assumption above, part of the patch of Rainer can
>>> be reused:
>>> - Have a thread local hold the fact that reset occured, but Maybe
>>> using JMeterVariables would be better particularly if one day we decide to
>>> switch to different model 1 VU == 1 Thread.
>>> - What should be changed in Rainer patch is the SSL reset, indeed
>>> for HC4.5 we need to close idle and expired connections to really reset SSL
>>> State
>>> - What I don't get about Rainer patch is wether we should do the
>>> close on all HTTPCLIENT Instances of VUs or not. I think we should but It
>>> would be great if another brain or more would confirm
>>>
>>> *Let's zoom to PROBLEM A:*
>>>
>>> In summary, whenever we make a new Thread Group iteration, we must start
>>> with new connections and not reuse existing ones.
>>>
>>> So Rainer patch looks ok to me.
>>>
>>> But:
>>>
>>> 1/ I would personally set default to
>>>
>>> -
>>>
>>> reuse.http.connections=false
>>>
>>>
>>> 2/ I would not introduce reuse.http.client
>>>
>>> 3/ I would not as a consequence reset HttpClient as it is not needed to reset state if I am not wrong and it would
>>>
>>> introduce "initialization cost" in requests timing
>>>
>>> What's your thoughts ?
>>>
>>>
>>> --
>>> Regards
>>> Philippe M.
>>>
>>>
>>>
>>
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>>
>>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: Bug 58807
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello,
After further thinking,
https.use.cached.ssl.context should be false by default and maybe instead
of having 2 properties we should only have 1 property:
- httpclient.reset_state_on_thread_group_iteration
- It true we reset SSL Context and close connection on ThreadGroup
iteration
- If false we allow reuse of both
And we would deprecate https.use.cached.ssl.context
Thoughts ?
Regards
On Sat, Feb 24, 2018 at 10:48 PM, Philippe Mouawad <
philippe.mouawad@gmail.com> wrote:
> Hello,
> I attached a patch to:
> - https://bz.apache.org/bugzilla/show_bug.cgi?id=58807
>
> Hope somebody can have a look.
>
> Regards
>
> On Sat, Feb 24, 2018 at 7:06 PM, Philippe Mouawad <
> philippe.mouawad@gmail.com> wrote:
>
>> Hello,
>> While working on Migration to last HC4.5 APIs and testing Client
>> Certificate (see another question on how to include tests for it), I
>> confirmed that the bug reported by Rainer was still here.
>>
>> This bug has the following impact:
>>
>> - Whenever you set "https.use.cached.ssl.context=false " which is
>> required for Client Certificate Testing, you end up :
>> - Replaying SSL Handshake on every sample while we shouldn't
>> - Losing KeepAlive
>> - Another thing he tackles is that fact that within Thread Group
>> iteration we reuse objects we shouldn't
>>
>> IMO, we need to tackle it, looking at Rainer patch I agree with almost
>> all his proposals except for what I describe below.
>>
>>
>> Let's see use most common cases :
>>
>> - 0/ We are testing HTTP, Rainer is correct saying that when we
>> switch to next iteration and are using KeepAlive we use the same connection
>> while we shouldn't , this would be PROBLEM A below
>> - 1/ We are testing HTTPS only without client certificate, then
>> https.use.cached.ssl.context=true is ok within 1 thread group
>> iteration. What would be unrealistic is the fact that SSL Handshake will
>> not occur for next iteration. Let's call this case PROBLEM A.
>> - 2/ We are testing HTTPS only WITH client certificate, then since
>> https.use.cached.ssl.context=false is needed we have a problem
>> PROBLEM B.
>>
>> *ASSUMPTION:*
>>
>> First, I think that we should consider that Each Thread Group Iteration
>> is a new user, indeed, If one wants to make the same user loop, he can
>> easily add a Loop Controller as child of Thread Group and it's ok.
>>
>> *So let's zoom to PROBLEM B first:*
>>
>> - If we make the assumption above, part of the patch of Rainer can be
>> reused:
>> - Have a thread local hold the fact that reset occured, but Maybe
>> using JMeterVariables would be better particularly if one day we decide to
>> switch to different model 1 VU == 1 Thread.
>> - What should be changed in Rainer patch is the SSL reset, indeed
>> for HC4.5 we need to close idle and expired connections to really reset SSL
>> State
>> - What I don't get about Rainer patch is wether we should do the
>> close on all HTTPCLIENT Instances of VUs or not. I think we should but It
>> would be great if another brain or more would confirm
>>
>> *Let's zoom to PROBLEM A:*
>>
>> In summary, whenever we make a new Thread Group iteration, we must start
>> with new connections and not reuse existing ones.
>>
>> So Rainer patch looks ok to me.
>>
>> But:
>>
>> 1/ I would personally set default to
>>
>> -
>>
>> reuse.http.connections=false
>>
>>
>> 2/ I would not introduce reuse.http.client
>>
>> 3/ I would not as a consequence reset HttpClient as it is not needed to reset state if I am not wrong and it would
>>
>> introduce "initialization cost" in requests timing
>>
>> What's your thoughts ?
>>
>>
>> --
>> Regards
>> Philippe M.
>>
>>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: Bug 58807
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello,
I attached a patch to:
- https://bz.apache.org/bugzilla/show_bug.cgi?id=58807
Hope somebody can have a look.
Regards
On Sat, Feb 24, 2018 at 7:06 PM, Philippe Mouawad <
philippe.mouawad@gmail.com> wrote:
> Hello,
> While working on Migration to last HC4.5 APIs and testing Client
> Certificate (see another question on how to include tests for it), I
> confirmed that the bug reported by Rainer was still here.
>
> This bug has the following impact:
>
> - Whenever you set "https.use.cached.ssl.context=false " which is
> required for Client Certificate Testing, you end up :
> - Replaying SSL Handshake on every sample while we shouldn't
> - Losing KeepAlive
> - Another thing he tackles is that fact that within Thread Group
> iteration we reuse objects we shouldn't
>
> IMO, we need to tackle it, looking at Rainer patch I agree with almost
> all his proposals except for what I describe below.
>
>
> Let's see use most common cases :
>
> - 0/ We are testing HTTP, Rainer is correct saying that when we switch
> to next iteration and are using KeepAlive we use the same connection while
> we shouldn't , this would be PROBLEM A below
> - 1/ We are testing HTTPS only without client certificate, then
> https.use.cached.ssl.context=true is ok within 1 thread group
> iteration. What would be unrealistic is the fact that SSL Handshake will
> not occur for next iteration. Let's call this case PROBLEM A.
> - 2/ We are testing HTTPS only WITH client certificate, then since
> https.use.cached.ssl.context=false is needed we have a problem PROBLEM
> B.
>
> *ASSUMPTION:*
>
> First, I think that we should consider that Each Thread Group Iteration is
> a new user, indeed, If one wants to make the same user loop, he can easily
> add a Loop Controller as child of Thread Group and it's ok.
>
> *So let's zoom to PROBLEM B first:*
>
> - If we make the assumption above, part of the patch of Rainer can be
> reused:
> - Have a thread local hold the fact that reset occured, but Maybe
> using JMeterVariables would be better particularly if one day we decide to
> switch to different model 1 VU == 1 Thread.
> - What should be changed in Rainer patch is the SSL reset, indeed
> for HC4.5 we need to close idle and expired connections to really reset SSL
> State
> - What I don't get about Rainer patch is wether we should do the
> close on all HTTPCLIENT Instances of VUs or not. I think we should but It
> would be great if another brain or more would confirm
>
> *Let's zoom to PROBLEM A:*
>
> In summary, whenever we make a new Thread Group iteration, we must start
> with new connections and not reuse existing ones.
>
> So Rainer patch looks ok to me.
>
> But:
>
> 1/ I would personally set default to
>
> -
>
> reuse.http.connections=false
>
>
> 2/ I would not introduce reuse.http.client
>
> 3/ I would not as a consequence reset HttpClient as it is not needed to reset state if I am not wrong and it would
>
> introduce "initialization cost" in requests timing
>
> What's your thoughts ?
>
>
> --
> Regards
> Philippe M.
>
>
>
--
Cordialement.
Philippe Mouawad.