You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@jmeter.apache.org by Gabriele D'Antona <da...@gmail.com> on 2021/05/26 19:00:16 UTC
Comparison between navigation times registered by JMeter and other tools
Hello,
I'm a consultant working for a customer that wants to test JMeter as an
alternative to HP BSM (now owned by Microfocus).
I registered one navigation that is (or at least shoudl be) identical to
the BSM one (same URLs, same methods, same parameters). When comparing the
results, even on single transactions, the times registered by JMeter are 2x
the BSM ones. I'm talking of simple HTTPs calls, nothing fancy. I'm also
launching the jmx project from the command line (no GUI).
I noticed that, if I run the jmx project with 3 or 4 iterations, the first
iteration is 2x longer than the BSM one, the subsequent iterations are
faster, but still 1.5x if compared to BSM.
I hope you can give me some suggestions on this problem.
Also, is there a way to fill the resulting jtl file only with the last
transaction? This would help having lower response times.
Many thanks.
RE: Comparison between navigation times registered by JMeter and
other tools
Posted by Robert Bourgault du Coudray <ro...@bankwest.com.au>.
Sometimes, there may be much slower response times than usual because of initial setup tasks, IIS pool refresh etc. which may bias your comparisons.
I would suggest you parse your test results for outliers with the Outlier Detector plugin before making comparisons.
Kind regards,
Robert
-----Original Message-----
From: Gabriele D'Antona <da...@gmail.com>
Sent: Thursday, 27 May 2021 2:27 PM
To: JMeter Users List <us...@jmeter.apache.org>
Subject: Re: Comparison between navigation times registered by JMeter and other tools
Thanks for your answer Tong Sun, but I already know that the times exported by JMeter aren't "correct".
In fact, as I said, the second, third, fourth, etc. iterations of JMeter's script give much lower times than the first one. So there is some kind of overhead/time lost in the first iteration.
It's strange that, in a tool that has more than 20 years of life, nobody did comparisons with other tools or noticed this.
Gabriele
Il giorno gio 27 mag 2021 alle ore 01:47 Tong Sun <su...@gmail.com> ha
scritto:
> On Wed, May 26, 2021 at 3:37 PM Felix Schumacher wrote:
> >
> > Can you describe the test elements, that you are using? What is
> > inside
> the transaction that takes eight seconds?
>
> I think in such a situation, the best way forward is to test a (simple
> & reliable) public service using JMeter and HP BSM.
>
> If the two still can't agree with each other this way, record the
> public service visiting with fiddler. The closer to fiddler's
> reporting wins.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com ______________________________________________________________________
_______________________________________________________________________________
This email has been scanned by the Bankwest Email Security System.
_______________________________________________________________________________
_______________________________________________________________________________
Unencrypted electronic mail is not secure and may not be authentic.
If you have any doubts as to the contents please telephone to confirm.
This electronic transmission including any attachments is intended only
for those to whom it is addressed. It may contain copyright material or
information that is confidential, privileged or exempt from disclosure by law.
Any claim to privilege is not waived or lost by reason of mistaken transmission
of this information. If you are not the intended recipient you must not
distribute or copy this transmission and should please notify the sender.
Your costs for doing this will be reimbursed by the sender.
We do not accept liability in connection with computer virus, data corruption,
delay, interruption, unauthorised access or unauthorised amendment.
_______________________________________________________________________________
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
Re: Comparison between navigation times registered by JMeter and
other tools
Posted by Gabriele D'Antona <da...@gmail.com>.
Thanks for your answer Tong Sun, but I already know that the times exported
by JMeter aren't "correct".
In fact, as I said, the second, third, fourth, etc. iterations of JMeter's
script give much lower times than the first one. So there is some kind of
overhead/time lost in the first iteration.
It's strange that, in a tool that has more than 20 years of life, nobody
did comparisons with other tools or noticed this.
Gabriele
Il giorno gio 27 mag 2021 alle ore 01:47 Tong Sun <su...@gmail.com> ha
scritto:
> On Wed, May 26, 2021 at 3:37 PM Felix Schumacher wrote:
> >
> > Can you describe the test elements, that you are using? What is inside
> the transaction that takes eight seconds?
>
> I think in such a situation, the best way forward is to test a (simple
> & reliable) public service using JMeter and HP BSM.
>
> If the two still can't agree with each other this way, record the
> public service visiting with fiddler. The closer to fiddler's
> reporting wins.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>
Re: Comparison between navigation times registered by JMeter and
other tools
Posted by Tong Sun <su...@gmail.com>.
On Wed, May 26, 2021 at 3:37 PM Felix Schumacher wrote:
>
> Can you describe the test elements, that you are using? What is inside the transaction that takes eight seconds?
I think in such a situation, the best way forward is to test a (simple
& reliable) public service using JMeter and HP BSM.
If the two still can't agree with each other this way, record the
public service visiting with fiddler. The closer to fiddler's
reporting wins.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
Re: Comparison between navigation times registered by JMeter and
other tools
Posted by Kiran Badi <ki...@yahoo.com.INVALID>.
I have done quite a work with both tools. I used to often get requests to test the use case with JMeter whenever i gave bad LR timings to the project. Then i use to repeat the test with Jmeter and i used to get difference of around 2~5% variation.
So i believed JMeter is developer's tool and LR is QA's tool that's real reason for 5% variation though they do same things just that they are implemented differently.
Please see generally blaming the toolset is not a good approach to take.
n fact, as I said, the second, third, fourth, etc. iterations of JMeter's
script give much lower times than the first one. So there is some kind of
overhead/time lost in the first iteration
This is always and almost due to caching and its good thing to do.
I suggest you revisit your test and see if you might be overlooking. Ask yourself these questions ?
How many requests do you make in each iterations ? how many of them are static ? how many of them involves caching ? What is size of each request ? Are they http or https calls ? if yes how many of them are http or https calls ? how many external systems does your request touch ? .. i can think of many such general questions..
On Thursday, May 27, 2021, 11:42:42 AM EDT, Felix Schumacher <fe...@internetallee.de> wrote:
Am 26.05.21 um 21:55 schrieb Gabriele D'Antona:
> Hello Felix,
> I will share the script if we really need it later.
>
> I'm using just one thread and one or more iterations.
>
> This is the result with 3 iterations:
WIth one thread?
Still, we can only guess, what is happening here, as we don't know what
kind of elements you are using.
Assuming you use http samplers, only, than all requests above 100 ms
will probably be the problem of the server and not the client (assuming
further, that you have a good connection between them).
In my eyes the 1.2 seconds for the initial connection are way too long,
When I test a http sampler against googles main page, I get a "Connect"
values of 40-80.
If you want to help us help you, you have to give more information about
the test plan. Sorry.
Apart from that, always remember, that
* Java will perform best after a warm up period (that means, you should
let your tests run long enough to get over the initial warm up phase)
* Java has a Garbage Collector, which can introduce spikes in your
measured response time (remove those outliers or be aware of them)
* It might not be the problem of the client (at least think about it :))
* Different elements of JMeter have different performance implications
(XPath might be more expensive than a plain text comparison)
* If you are familiar with Java, you can try to look deeper into your
suspected problems using tools like jstack, visualvm, jmc or any other
profiler.
Felix
>
> timeStamp,elapsed,label,responseCode,responseMessage,success,failureMessage,Latency,SampleCount,ErrorCount,Hostname,Connect
> 2021/05/26 21:49:58.014,1789,1_MFP_Homepage,200,"Number of samples in
> transaction : 9, number of failing samples : 0",true,,0,1,0,SPXMIW3825,1247
> 2021/05/26 21:50:00.012,8838,2_MFP_Login,200,"Number of samples in
> transaction : 13, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:08.888,201,3_MFP_InfoOfferta,200,"Number of samples in
> transaction : 1, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:09.094,297,4_MFP_InfoConto,200,"Number of samples in
> transaction : 2, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:09.393,784,5_MFP_InfoConsumi,200,"Number of samples in
> transaction : 3, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:10.181,4132,6_MFP_Logout,200,"Number of samples in
> transaction : 16, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:14.331,429,1_MFP_Homepage,200,"Number of samples in
> transaction : 9, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:14.767,6270,2_MFP_Login,200,"Number of samples in
> transaction : 13, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:21.051,182,3_MFP_InfoOfferta,200,"Number of samples in
> transaction : 1, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:21.235,1353,4_MFP_InfoConto,200,"Number of samples in
> transaction : 2, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:22.593,619,5_MFP_InfoConsumi,200,"Number of samples in
> transaction : 3, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:23.215,3483,6_MFP_Logout,200,"Number of samples in
> transaction : 16, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:26.710,438,1_MFP_Homepage,200,"Number of samples in
> transaction : 9, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:27.152,6397,2_MFP_Login,200,"Number of samples in
> transaction : 13, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:33.558,194,3_MFP_InfoOfferta,200,"Number of samples in
> transaction : 1, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:33.754,281,4_MFP_InfoConto,200,"Number of samples in
> transaction : 2, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:34.038,549,5_MFP_InfoConsumi,200,"Number of samples in
> transaction : 3, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:34.590,3719,6_MFP_Logout,200,"Number of samples in
> transaction : 16, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
>
> As you can see, the 2_MFP_Login transaction starts with a time of 8.8
> seconds, then goes down to 6.2-6.3 in the second and third iteration.
> I tried to switch the implementation from "Java" to "HttpClient4", maybe
> this helped a bit.
>
> Thanks
>
>
>
> Il giorno mer 26 mag 2021 alle ore 21:37 Felix Schumacher <
> felix.schumacher@internetallee.de> ha scritto:
>
>> Am 26.05.21 um 21:17 schrieb Gabriele D'Antona:
>>
>> Hello Felix,
>> thanks for your answer.
>>
>> Unfortunately, I can't share the script because it contains private data
>> like passwords and URLs.
>>
>> Maybe you can obscure/anonymize/remove those?
>>
>>
>> My java version is 11.0.3 2019-04-16 LTS, the OS is Windows 10.
>>
>> Which version of JMeter? How may threads are you using, how many requests
>> per second are you trying to reach, how much network traffic do you expect?
>>
>>
>> I can share the jtl file with one iteration of the test. As you can see,
>> the second transaction (2_MFP_Login) takes almost 8 seconds, while on the
>> BSM it takes something like 3,5 seconds.
>> As I said, if I increment the number of iterations, I get better times on
>> the subsequent runs (6.5 seconds for that transaction), but still far from
>> BSM.
>>
>> The list tends to strip of attachements. It seems to have removed yours,
>> too.
>>
>> Can you describe the test elements, that you are using? What is inside the
>> transaction that takes eight seconds?
>>
>> Felix
>>
>> Thanks
>>
>>
>> Il giorno mer 26 mag 2021 alle ore 21:10 Felix Schumacher <
>> felix.schumacher@internetallee.de> ha scritto:
>>
>>> Am 26.05.21 um 21:00 schrieb Gabriele D'Antona:
>>>> Hello,
>>>> I'm a consultant working for a customer that wants to test JMeter as an
>>>> alternative to HP BSM (now owned by Microfocus).
>>>>
>>>> I registered one navigation that is (or at least shoudl be) identical to
>>>> the BSM one (same URLs, same methods, same parameters). When comparing
>>> the
>>>> results, even on single transactions, the times registered by JMeter
>>> are 2x
>>>> the BSM ones. I'm talking of simple HTTPs calls, nothing fancy. I'm also
>>>> launching the jmx project from the command line (no GUI).
>>>>
>>>> I noticed that, if I run the jmx project with 3 or 4 iterations, the
>>> first
>>>> iteration is 2x longer than the BSM one, the subsequent iterations are
>>>> faster, but still 1.5x if compared to BSM.
>>>>
>>>> I hope you can give me some suggestions on this problem.
>>>>
>>>> Also, is there a way to fill the resulting jtl file only with the last
>>>> transaction? This would help having lower response times.
>>> You are quite vague on how you are setting up the plan and what versions
>>> of Java, JMeter or OS you are using.
>>>
>>> Can you be more specific? Maybe give a real example?
>>>
>>> Felix
>>>
>>>> Many thanks.
>>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>>
Re: Comparison between navigation times registered by JMeter and
other tools
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 26.05.21 um 21:55 schrieb Gabriele D'Antona:
> Hello Felix,
> I will share the script if we really need it later.
>
> I'm using just one thread and one or more iterations.
>
> This is the result with 3 iterations:
WIth one thread?
Still, we can only guess, what is happening here, as we don't know what
kind of elements you are using.
Assuming you use http samplers, only, than all requests above 100 ms
will probably be the problem of the server and not the client (assuming
further, that you have a good connection between them).
In my eyes the 1.2 seconds for the initial connection are way too long,
When I test a http sampler against googles main page, I get a "Connect"
values of 40-80.
If you want to help us help you, you have to give more information about
the test plan. Sorry.
Apart from that, always remember, that
* Java will perform best after a warm up period (that means, you should
let your tests run long enough to get over the initial warm up phase)
* Java has a Garbage Collector, which can introduce spikes in your
measured response time (remove those outliers or be aware of them)
* It might not be the problem of the client (at least think about it :))
* Different elements of JMeter have different performance implications
(XPath might be more expensive than a plain text comparison)
* If you are familiar with Java, you can try to look deeper into your
suspected problems using tools like jstack, visualvm, jmc or any other
profiler.
Felix
>
> timeStamp,elapsed,label,responseCode,responseMessage,success,failureMessage,Latency,SampleCount,ErrorCount,Hostname,Connect
> 2021/05/26 21:49:58.014,1789,1_MFP_Homepage,200,"Number of samples in
> transaction : 9, number of failing samples : 0",true,,0,1,0,SPXMIW3825,1247
> 2021/05/26 21:50:00.012,8838,2_MFP_Login,200,"Number of samples in
> transaction : 13, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:08.888,201,3_MFP_InfoOfferta,200,"Number of samples in
> transaction : 1, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:09.094,297,4_MFP_InfoConto,200,"Number of samples in
> transaction : 2, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:09.393,784,5_MFP_InfoConsumi,200,"Number of samples in
> transaction : 3, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:10.181,4132,6_MFP_Logout,200,"Number of samples in
> transaction : 16, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:14.331,429,1_MFP_Homepage,200,"Number of samples in
> transaction : 9, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:14.767,6270,2_MFP_Login,200,"Number of samples in
> transaction : 13, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:21.051,182,3_MFP_InfoOfferta,200,"Number of samples in
> transaction : 1, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:21.235,1353,4_MFP_InfoConto,200,"Number of samples in
> transaction : 2, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:22.593,619,5_MFP_InfoConsumi,200,"Number of samples in
> transaction : 3, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:23.215,3483,6_MFP_Logout,200,"Number of samples in
> transaction : 16, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:26.710,438,1_MFP_Homepage,200,"Number of samples in
> transaction : 9, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:27.152,6397,2_MFP_Login,200,"Number of samples in
> transaction : 13, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:33.558,194,3_MFP_InfoOfferta,200,"Number of samples in
> transaction : 1, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:33.754,281,4_MFP_InfoConto,200,"Number of samples in
> transaction : 2, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:34.038,549,5_MFP_InfoConsumi,200,"Number of samples in
> transaction : 3, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:34.590,3719,6_MFP_Logout,200,"Number of samples in
> transaction : 16, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
>
> As you can see, the 2_MFP_Login transaction starts with a time of 8.8
> seconds, then goes down to 6.2-6.3 in the second and third iteration.
> I tried to switch the implementation from "Java" to "HttpClient4", maybe
> this helped a bit.
>
> Thanks
>
>
>
> Il giorno mer 26 mag 2021 alle ore 21:37 Felix Schumacher <
> felix.schumacher@internetallee.de> ha scritto:
>
>> Am 26.05.21 um 21:17 schrieb Gabriele D'Antona:
>>
>> Hello Felix,
>> thanks for your answer.
>>
>> Unfortunately, I can't share the script because it contains private data
>> like passwords and URLs.
>>
>> Maybe you can obscure/anonymize/remove those?
>>
>>
>> My java version is 11.0.3 2019-04-16 LTS, the OS is Windows 10.
>>
>> Which version of JMeter? How may threads are you using, how many requests
>> per second are you trying to reach, how much network traffic do you expect?
>>
>>
>> I can share the jtl file with one iteration of the test. As you can see,
>> the second transaction (2_MFP_Login) takes almost 8 seconds, while on the
>> BSM it takes something like 3,5 seconds.
>> As I said, if I increment the number of iterations, I get better times on
>> the subsequent runs (6.5 seconds for that transaction), but still far from
>> BSM.
>>
>> The list tends to strip of attachements. It seems to have removed yours,
>> too.
>>
>> Can you describe the test elements, that you are using? What is inside the
>> transaction that takes eight seconds?
>>
>> Felix
>>
>> Thanks
>>
>>
>> Il giorno mer 26 mag 2021 alle ore 21:10 Felix Schumacher <
>> felix.schumacher@internetallee.de> ha scritto:
>>
>>> Am 26.05.21 um 21:00 schrieb Gabriele D'Antona:
>>>> Hello,
>>>> I'm a consultant working for a customer that wants to test JMeter as an
>>>> alternative to HP BSM (now owned by Microfocus).
>>>>
>>>> I registered one navigation that is (or at least shoudl be) identical to
>>>> the BSM one (same URLs, same methods, same parameters). When comparing
>>> the
>>>> results, even on single transactions, the times registered by JMeter
>>> are 2x
>>>> the BSM ones. I'm talking of simple HTTPs calls, nothing fancy. I'm also
>>>> launching the jmx project from the command line (no GUI).
>>>>
>>>> I noticed that, if I run the jmx project with 3 or 4 iterations, the
>>> first
>>>> iteration is 2x longer than the BSM one, the subsequent iterations are
>>>> faster, but still 1.5x if compared to BSM.
>>>>
>>>> I hope you can give me some suggestions on this problem.
>>>>
>>>> Also, is there a way to fill the resulting jtl file only with the last
>>>> transaction? This would help having lower response times.
>>> You are quite vague on how you are setting up the plan and what versions
>>> of Java, JMeter or OS you are using.
>>>
>>> Can you be more specific? Maybe give a real example?
>>>
>>> Felix
>>>
>>>> Many thanks.
>>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>>
Re: Comparison between navigation times registered by JMeter and
other tools
Posted by Gabriele D'Antona <da...@gmail.com>.
Hello Felix,
I will share the script if we really need it later.
I'm using just one thread and one or more iterations.
This is the result with 3 iterations:
timeStamp,elapsed,label,responseCode,responseMessage,success,failureMessage,Latency,SampleCount,ErrorCount,Hostname,Connect
2021/05/26 21:49:58.014,1789,1_MFP_Homepage,200,"Number of samples in
transaction : 9, number of failing samples : 0",true,,0,1,0,SPXMIW3825,1247
2021/05/26 21:50:00.012,8838,2_MFP_Login,200,"Number of samples in
transaction : 13, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
2021/05/26 21:50:08.888,201,3_MFP_InfoOfferta,200,"Number of samples in
transaction : 1, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
2021/05/26 21:50:09.094,297,4_MFP_InfoConto,200,"Number of samples in
transaction : 2, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
2021/05/26 21:50:09.393,784,5_MFP_InfoConsumi,200,"Number of samples in
transaction : 3, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
2021/05/26 21:50:10.181,4132,6_MFP_Logout,200,"Number of samples in
transaction : 16, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
2021/05/26 21:50:14.331,429,1_MFP_Homepage,200,"Number of samples in
transaction : 9, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
2021/05/26 21:50:14.767,6270,2_MFP_Login,200,"Number of samples in
transaction : 13, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
2021/05/26 21:50:21.051,182,3_MFP_InfoOfferta,200,"Number of samples in
transaction : 1, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
2021/05/26 21:50:21.235,1353,4_MFP_InfoConto,200,"Number of samples in
transaction : 2, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
2021/05/26 21:50:22.593,619,5_MFP_InfoConsumi,200,"Number of samples in
transaction : 3, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
2021/05/26 21:50:23.215,3483,6_MFP_Logout,200,"Number of samples in
transaction : 16, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
2021/05/26 21:50:26.710,438,1_MFP_Homepage,200,"Number of samples in
transaction : 9, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
2021/05/26 21:50:27.152,6397,2_MFP_Login,200,"Number of samples in
transaction : 13, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
2021/05/26 21:50:33.558,194,3_MFP_InfoOfferta,200,"Number of samples in
transaction : 1, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
2021/05/26 21:50:33.754,281,4_MFP_InfoConto,200,"Number of samples in
transaction : 2, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
2021/05/26 21:50:34.038,549,5_MFP_InfoConsumi,200,"Number of samples in
transaction : 3, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
2021/05/26 21:50:34.590,3719,6_MFP_Logout,200,"Number of samples in
transaction : 16, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
As you can see, the 2_MFP_Login transaction starts with a time of 8.8
seconds, then goes down to 6.2-6.3 in the second and third iteration.
I tried to switch the implementation from "Java" to "HttpClient4", maybe
this helped a bit.
Thanks
Il giorno mer 26 mag 2021 alle ore 21:37 Felix Schumacher <
felix.schumacher@internetallee.de> ha scritto:
>
> Am 26.05.21 um 21:17 schrieb Gabriele D'Antona:
>
> Hello Felix,
> thanks for your answer.
>
> Unfortunately, I can't share the script because it contains private data
> like passwords and URLs.
>
> Maybe you can obscure/anonymize/remove those?
>
>
> My java version is 11.0.3 2019-04-16 LTS, the OS is Windows 10.
>
> Which version of JMeter? How may threads are you using, how many requests
> per second are you trying to reach, how much network traffic do you expect?
>
>
> I can share the jtl file with one iteration of the test. As you can see,
> the second transaction (2_MFP_Login) takes almost 8 seconds, while on the
> BSM it takes something like 3,5 seconds.
> As I said, if I increment the number of iterations, I get better times on
> the subsequent runs (6.5 seconds for that transaction), but still far from
> BSM.
>
> The list tends to strip of attachements. It seems to have removed yours,
> too.
>
> Can you describe the test elements, that you are using? What is inside the
> transaction that takes eight seconds?
>
> Felix
>
> Thanks
>
>
> Il giorno mer 26 mag 2021 alle ore 21:10 Felix Schumacher <
> felix.schumacher@internetallee.de> ha scritto:
>
>>
>> Am 26.05.21 um 21:00 schrieb Gabriele D'Antona:
>> > Hello,
>> > I'm a consultant working for a customer that wants to test JMeter as an
>> > alternative to HP BSM (now owned by Microfocus).
>> >
>> > I registered one navigation that is (or at least shoudl be) identical to
>> > the BSM one (same URLs, same methods, same parameters). When comparing
>> the
>> > results, even on single transactions, the times registered by JMeter
>> are 2x
>> > the BSM ones. I'm talking of simple HTTPs calls, nothing fancy. I'm also
>> > launching the jmx project from the command line (no GUI).
>> >
>> > I noticed that, if I run the jmx project with 3 or 4 iterations, the
>> first
>> > iteration is 2x longer than the BSM one, the subsequent iterations are
>> > faster, but still 1.5x if compared to BSM.
>> >
>> > I hope you can give me some suggestions on this problem.
>> >
>> > Also, is there a way to fill the resulting jtl file only with the last
>> > transaction? This would help having lower response times.
>>
>> You are quite vague on how you are setting up the plan and what versions
>> of Java, JMeter or OS you are using.
>>
>> Can you be more specific? Maybe give a real example?
>>
>> Felix
>>
>> >
>> > Many thanks.
>> >
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>
Re: Comparison between navigation times registered by JMeter and
other tools
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 26.05.21 um 21:17 schrieb Gabriele D'Antona:
> Hello Felix,
> thanks for your answer.
>
> Unfortunately, I can't share the script because it contains private
> data like passwords and URLs.
Maybe you can obscure/anonymize/remove those?
>
> My java version is 11.0.3 2019-04-16 LTS, the OS is Windows 10.
Which version of JMeter? How may threads are you using, how many
requests per second are you trying to reach, how much network traffic do
you expect?
>
> I can share the jtl file with one iteration of the test. As you can
> see, the second transaction (2_MFP_Login) takes almost 8 seconds,
> while on the BSM it takes something like 3,5 seconds.
> As I said, if I increment the number of iterations, I get better times
> on the subsequent runs (6.5 seconds for that transaction), but still
> far from BSM.
The list tends to strip of attachements. It seems to have removed yours,
too.
Can you describe the test elements, that you are using? What is inside
the transaction that takes eight seconds?
Felix
> Thanks
>
>
> Il giorno mer 26 mag 2021 alle ore 21:10 Felix Schumacher
> <felix.schumacher@internetallee.de
> <ma...@internetallee.de>> ha scritto:
>
>
> Am 26.05.21 um 21:00 schrieb Gabriele D'Antona:
> > Hello,
> > I'm a consultant working for a customer that wants to test
> JMeter as an
> > alternative to HP BSM (now owned by Microfocus).
> >
> > I registered one navigation that is (or at least shoudl be)
> identical to
> > the BSM one (same URLs, same methods, same parameters). When
> comparing the
> > results, even on single transactions, the times registered by
> JMeter are 2x
> > the BSM ones. I'm talking of simple HTTPs calls, nothing fancy.
> I'm also
> > launching the jmx project from the command line (no GUI).
> >
> > I noticed that, if I run the jmx project with 3 or 4 iterations,
> the first
> > iteration is 2x longer than the BSM one, the subsequent
> iterations are
> > faster, but still 1.5x if compared to BSM.
> >
> > I hope you can give me some suggestions on this problem.
> >
> > Also, is there a way to fill the resulting jtl file only with
> the last
> > transaction? This would help having lower response times.
>
> You are quite vague on how you are setting up the plan and what
> versions
> of Java, JMeter or OS you are using.
>
> Can you be more specific? Maybe give a real example?
>
> Felix
>
> >
> > Many thanks.
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
Re: Comparison between navigation times registered by JMeter and
other tools
Posted by Gabriele D'Antona <da...@gmail.com>.
Hello Felix,
thanks for your answer.
Unfortunately, I can't share the script because it contains private data
like passwords and URLs.
My java version is 11.0.3 2019-04-16 LTS, the OS is Windows 10.
I can share the jtl file with one iteration of the test. As you can see,
the second transaction (2_MFP_Login) takes almost 8 seconds, while on the
BSM it takes something like 3,5 seconds.
As I said, if I increment the number of iterations, I get better times on
the subsequent runs (6.5 seconds for that transaction), but still far from
BSM.
Thanks
Il giorno mer 26 mag 2021 alle ore 21:10 Felix Schumacher <
felix.schumacher@internetallee.de> ha scritto:
>
> Am 26.05.21 um 21:00 schrieb Gabriele D'Antona:
> > Hello,
> > I'm a consultant working for a customer that wants to test JMeter as an
> > alternative to HP BSM (now owned by Microfocus).
> >
> > I registered one navigation that is (or at least shoudl be) identical to
> > the BSM one (same URLs, same methods, same parameters). When comparing
> the
> > results, even on single transactions, the times registered by JMeter are
> 2x
> > the BSM ones. I'm talking of simple HTTPs calls, nothing fancy. I'm also
> > launching the jmx project from the command line (no GUI).
> >
> > I noticed that, if I run the jmx project with 3 or 4 iterations, the
> first
> > iteration is 2x longer than the BSM one, the subsequent iterations are
> > faster, but still 1.5x if compared to BSM.
> >
> > I hope you can give me some suggestions on this problem.
> >
> > Also, is there a way to fill the resulting jtl file only with the last
> > transaction? This would help having lower response times.
>
> You are quite vague on how you are setting up the plan and what versions
> of Java, JMeter or OS you are using.
>
> Can you be more specific? Maybe give a real example?
>
> Felix
>
> >
> > Many thanks.
> >
>
>
Re: Comparison between navigation times registered by JMeter and
other tools
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 26.05.21 um 21:00 schrieb Gabriele D'Antona:
> Hello,
> I'm a consultant working for a customer that wants to test JMeter as an
> alternative to HP BSM (now owned by Microfocus).
>
> I registered one navigation that is (or at least shoudl be) identical to
> the BSM one (same URLs, same methods, same parameters). When comparing the
> results, even on single transactions, the times registered by JMeter are 2x
> the BSM ones. I'm talking of simple HTTPs calls, nothing fancy. I'm also
> launching the jmx project from the command line (no GUI).
>
> I noticed that, if I run the jmx project with 3 or 4 iterations, the first
> iteration is 2x longer than the BSM one, the subsequent iterations are
> faster, but still 1.5x if compared to BSM.
>
> I hope you can give me some suggestions on this problem.
>
> Also, is there a way to fill the resulting jtl file only with the last
> transaction? This would help having lower response times.
You are quite vague on how you are setting up the plan and what versions
of Java, JMeter or OS you are using.
Can you be more specific? Maybe give a real example?
Felix
>
> Many thanks.
>