You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@jmeter.apache.org by "Olofsson, Lars" <La...@sonyericsson.com> on 2009/10/08 16:18:42 UTC
Big difference between non-gui and gui result ?
Hi
I am new to jMeter and to this list ?
Is it normal that the average response difference a lot ( twice as h high) between
a non-gui and an gui run.
We have 5 sec average on gui and 10 sec on non-gui run.
1 Testcase, 2*40 users, 2 jmeter-servers.
I just wonder if this big difference is normal and can be explained by the the fact
that in non-gui the jmeter can send the request much faster to the jmeter-servers.
Or we have another problem in our testcase, environment or another problem.
best regards
Lars Olofsson
Re: Big difference between non-gui and gui result ?
Posted by Deepak Shetty <sh...@gmail.com>.
There may be some difference but shouldnt normally be this large. Do you
have some listeners on in the GUI mode? if so can you disable and retry?
On Thu, Oct 8, 2009 at 7:18 AM, Olofsson, Lars <
Lars.Olofsson@sonyericsson.com> wrote:
> Hi
>
> I am new to jMeter and to this list ?
> Is it normal that the average response difference a lot ( twice as h high)
> between
> a non-gui and an gui run.
> We have 5 sec average on gui and 10 sec on non-gui run.
> 1 Testcase, 2*40 users, 2 jmeter-servers.
>
> I just wonder if this big difference is normal and can be explained by the
> the fact
> that in non-gui the jmeter can send the request much faster to the
> jmeter-servers.
>
> Or we have another problem in our testcase, environment or another problem.
>
> best regards
> Lars Olofsson
>
Re: Big difference between non-gui and gui result ?
Posted by kirk <ki...@kodewerk.com>.
Hi all,
The problem here is not in a difference between GUI and headless
performance, it is in one of two other problems. Either your test is ill
defined or you are starving your test for threads. Let me explain.
First the JMeter Thread Group equate a user to be a thread. IMHO, this
is the wrong way to think about a load test. A better way to think about
this is arrival rates for requests to the server. The throughput measure
is a rough estimate of this but do remember that even that measure is
flawed because it reflect rates of exit from the system which in JMeter
happens to equal rates of entry into the a system. This behavior is
typical of closed systems. For example, think call center where
operators are the end users. In that system an operator will service a
customer from the queue and when they are done with that customer, they
go and get another customer from the queue. From the computers systems
point of view, there are a fixed number of users on the system at all
times (ok, not a lower loads but I think you get the point) and that
once a user does leave the system, it is almost immediately replaced. If
this is your situation than your timings in your jmeter script are not
stable or otherwise broken.
The other situation is an open system in which rate of entry is not tied
to rate of exit. This type of system is difficult to model in JMeter.
Since there is no direct support for open system simulation in JMeter
(not picking on JMeter here, this is a common omission in many test
harnesses) you have to figure out a different way to maintain some
constant inter-request arrival rate on the server. My idea in the past
has been to.
1) Calculate the frequency at which users arrive to the system. Lets
call this F
2) Determine the duration that you want to run your test. Let call this D
3) in the properties sheet for the thread group, set the number of
threads to F*D, the ramp up to the test duration, and the repeat count to 1.
This will cause JMeter to pulse the first request of the thread group to
the server on a schedule determined by the rampup behavior. However this
behavior being constant, isn't desirable either. To counter this you
need to add a timer to effectively randomization into the pulse. The
spread of the randomization should match the expected spread on the
average rate at which users will enter the system.
If you do this I think you should see the responses time difference
between headless and GUI disappear. In other words, this effect is due
entirely to the harness and that is a sure fire sign that the harness is
broken or mis configured.
Good luck,
Kirk
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
Re: Big difference between non-gui and gui result ?
Posted by Jakub Białek <ra...@gmail.com>.
Yes it can be the reason if you have some fixed times between the
block. Try with 2*1 users and check the differences between gui and
non-gui model, should be lower than for 2*40 user.
Best regards,
Jakub
2009/10/19 Olofsson, Lars <La...@sonyericsson.com>:
> Hi
> Thruput is about the same, but that is probably becaus of of the long "Think Time", betwen each block, about 3-6 seconds
> We have 2*40 users simulated, so I guess it can generate 2*40 request faster in non-gui, but then it has to wait for ThinkTime, until next burst.
>
> Br
> Lars Olofsson
>
> -----Original Message-----
> From: Jakub Białek [mailto:ragnor84@gmail.com]
> Sent: Monday, October 19, 2009 8:52 AM
> To: JMeter Users List
> Subject: Re: Big difference between non-gui and gui result ?
>
> Hi,
>
> how about throughput? Does it grow up in non gui mode? If yes then explanation is that: non gui jmeter can generate more requests in the same time than gui jmeter so the target server load is higher and response time become longer.
>
> Best regards,
> Jakub
>
> 2009/10/19 Olofsson, Lars <La...@sonyericsson.com>:
>> Clarification...
>>
>> All time seems to be the same for each Request/Response, botg in guin and non-gui.
>>
>> But all summary times for each block are 10-15 % higher in non-gui compared to gui-run !
>>
>> I only checked/compare these summary times.
>>
>> Why does this happens ?
>> I know there are processing time included in summary times, but why
>> Should this be higher for non-gui ?
>>
>> Best regards
>> Lars "newbie" Olofsson
>>
>> -----Original Message-----
>> From: sebb [mailto:sebbaz@gmail.com]
>> Sent: Thursday, October 08, 2009 11:47 PM
>> To: JMeter Users List
>> Subject: Re: Big difference between non-gui and gui result ?
>>
>> On 08/10/2009, Olofsson, Lars <La...@sonyericsson.com> wrote:
>>> Hi
>>>
>>> I am new to jMeter and to this list ?
>>> Is it normal that the average response difference a lot ( twice as h
>>> high) between a non-gui and an gui run.
>>> We have 5 sec average on gui and 10 sec on non-gui run.
>>> 1 Testcase, 2*40 users, 2 jmeter-servers.
>>>
>>> I just wonder if this big difference is normal and can be explained
>>> by the the fact that in non-gui the jmeter can send the request much faster to the jmeter-servers.
>>
>> In client-server mode, JMeter sends the test plan to the servers which then execute the test independently, and send back the results.
>>
>> If the JMeter server system is seriously overloaded, then the sample times can be affected, but this is unlikely to be the case here.
>> Non-GUI mode uses far fewer resources.
>>
>> The client JMeter is only responsible for reading and processing the sample responses, and cannot directly affect the sample timings of the servers.
>>
>>> Or we have another problem in our testcase, environment or another problem.
>>
>> Yes, that seems the most likely.
>>
>>> best regards
>>>
>>> Lars Olofsson
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
RE: Big difference between non-gui and gui result ?
Posted by "Olofsson, Lars" <La...@sonyericsson.com>.
Hi
Thruput is about the same, but that is probably becaus of of the long "Think Time", betwen each block, about 3-6 seconds
We have 2*40 users simulated, so I guess it can generate 2*40 request faster in non-gui, but then it has to wait for ThinkTime, until next burst.
Br
Lars Olofsson
-----Original Message-----
From: Jakub Białek [mailto:ragnor84@gmail.com]
Sent: Monday, October 19, 2009 8:52 AM
To: JMeter Users List
Subject: Re: Big difference between non-gui and gui result ?
Hi,
how about throughput? Does it grow up in non gui mode? If yes then explanation is that: non gui jmeter can generate more requests in the same time than gui jmeter so the target server load is higher and response time become longer.
Best regards,
Jakub
2009/10/19 Olofsson, Lars <La...@sonyericsson.com>:
> Clarification...
>
> All time seems to be the same for each Request/Response, botg in guin and non-gui.
>
> But all summary times for each block are 10-15 % higher in non-gui compared to gui-run !
>
> I only checked/compare these summary times.
>
> Why does this happens ?
> I know there are processing time included in summary times, but why
> Should this be higher for non-gui ?
>
> Best regards
> Lars "newbie" Olofsson
>
> -----Original Message-----
> From: sebb [mailto:sebbaz@gmail.com]
> Sent: Thursday, October 08, 2009 11:47 PM
> To: JMeter Users List
> Subject: Re: Big difference between non-gui and gui result ?
>
> On 08/10/2009, Olofsson, Lars <La...@sonyericsson.com> wrote:
>> Hi
>>
>> I am new to jMeter and to this list ?
>> Is it normal that the average response difference a lot ( twice as h
>> high) between a non-gui and an gui run.
>> We have 5 sec average on gui and 10 sec on non-gui run.
>> 1 Testcase, 2*40 users, 2 jmeter-servers.
>>
>> I just wonder if this big difference is normal and can be explained
>> by the the fact that in non-gui the jmeter can send the request much faster to the jmeter-servers.
>
> In client-server mode, JMeter sends the test plan to the servers which then execute the test independently, and send back the results.
>
> If the JMeter server system is seriously overloaded, then the sample times can be affected, but this is unlikely to be the case here.
> Non-GUI mode uses far fewer resources.
>
> The client JMeter is only responsible for reading and processing the sample responses, and cannot directly affect the sample timings of the servers.
>
>> Or we have another problem in our testcase, environment or another problem.
>
> Yes, that seems the most likely.
>
>> best regards
>>
>> Lars Olofsson
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
Re: Big difference between non-gui and gui result ?
Posted by Jakub Białek <ra...@gmail.com>.
Hi,
how about throughput? Does it grow up in non gui mode? If yes then
explanation is that: non gui jmeter can generate more requests in the
same time than gui jmeter so the target server load is higher and
response time become longer.
Best regards,
Jakub
2009/10/19 Olofsson, Lars <La...@sonyericsson.com>:
> Clarification...
>
> All time seems to be the same for each Request/Response, botg in guin and non-gui.
>
> But all summary times for each block are 10-15 % higher in non-gui compared to gui-run !
>
> I only checked/compare these summary times.
>
> Why does this happens ?
> I know there are processing time included in summary times, but why
> Should this be higher for non-gui ?
>
> Best regards
> Lars "newbie" Olofsson
>
> -----Original Message-----
> From: sebb [mailto:sebbaz@gmail.com]
> Sent: Thursday, October 08, 2009 11:47 PM
> To: JMeter Users List
> Subject: Re: Big difference between non-gui and gui result ?
>
> On 08/10/2009, Olofsson, Lars <La...@sonyericsson.com> wrote:
>> Hi
>>
>> I am new to jMeter and to this list ?
>> Is it normal that the average response difference a lot ( twice as h
>> high) between a non-gui and an gui run.
>> We have 5 sec average on gui and 10 sec on non-gui run.
>> 1 Testcase, 2*40 users, 2 jmeter-servers.
>>
>> I just wonder if this big difference is normal and can be explained
>> by the the fact that in non-gui the jmeter can send the request much faster to the jmeter-servers.
>
> In client-server mode, JMeter sends the test plan to the servers which then execute the test independently, and send back the results.
>
> If the JMeter server system is seriously overloaded, then the sample times can be affected, but this is unlikely to be the case here.
> Non-GUI mode uses far fewer resources.
>
> The client JMeter is only responsible for reading and processing the sample responses, and cannot directly affect the sample timings of the servers.
>
>> Or we have another problem in our testcase, environment or another problem.
>
> Yes, that seems the most likely.
>
>> best regards
>>
>> Lars Olofsson
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
RE: Big difference between non-gui and gui result ?
Posted by "Olofsson, Lars" <La...@sonyericsson.com>.
Clarification...
All time seems to be the same for each Request/Response, botg in guin and non-gui.
But all summary times for each block are 10-15 % higher in non-gui compared to gui-run !
I only checked/compare these summary times.
Why does this happens ?
I know there are processing time included in summary times, but why
Should this be higher for non-gui ?
Best regards
Lars "newbie" Olofsson
-----Original Message-----
From: sebb [mailto:sebbaz@gmail.com]
Sent: Thursday, October 08, 2009 11:47 PM
To: JMeter Users List
Subject: Re: Big difference between non-gui and gui result ?
On 08/10/2009, Olofsson, Lars <La...@sonyericsson.com> wrote:
> Hi
>
> I am new to jMeter and to this list ?
> Is it normal that the average response difference a lot ( twice as h
> high) between a non-gui and an gui run.
> We have 5 sec average on gui and 10 sec on non-gui run.
> 1 Testcase, 2*40 users, 2 jmeter-servers.
>
> I just wonder if this big difference is normal and can be explained
> by the the fact that in non-gui the jmeter can send the request much faster to the jmeter-servers.
In client-server mode, JMeter sends the test plan to the servers which then execute the test independently, and send back the results.
If the JMeter server system is seriously overloaded, then the sample times can be affected, but this is unlikely to be the case here.
Non-GUI mode uses far fewer resources.
The client JMeter is only responsible for reading and processing the sample responses, and cannot directly affect the sample timings of the servers.
> Or we have another problem in our testcase, environment or another problem.
Yes, that seems the most likely.
> best regards
>
> Lars Olofsson
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
Re: Big difference between non-gui and gui result ?
Posted by sebb <se...@gmail.com>.
On 08/10/2009, Olofsson, Lars <La...@sonyericsson.com> wrote:
> Hi
>
> I am new to jMeter and to this list ?
> Is it normal that the average response difference a lot ( twice as h high) between
> a non-gui and an gui run.
> We have 5 sec average on gui and 10 sec on non-gui run.
> 1 Testcase, 2*40 users, 2 jmeter-servers.
>
> I just wonder if this big difference is normal and can be explained by the the fact
> that in non-gui the jmeter can send the request much faster to the jmeter-servers.
In client-server mode, JMeter sends the test plan to the servers which
then execute the test independently, and send back the results.
If the JMeter server system is seriously overloaded, then the sample
times can be affected, but this is unlikely to be the case here.
Non-GUI mode uses far fewer resources.
The client JMeter is only responsible for reading and processing the
sample responses, and cannot directly affect the sample timings of the
servers.
> Or we have another problem in our testcase, environment or another problem.
Yes, that seems the most likely.
> best regards
>
> Lars Olofsson
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org