You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@jmeter.apache.org by ziohausam <zi...@gmail.com> on 2009/05/18 20:17:33 UTC
Analyze difference between 1 user and 25 users
Hi there,
I have run a test to one of our search functions using 1 user then had the
following results
# Samples 147
Average 271
Min 210
Max 5236
Std. Dev. 418.3129775
Error % 0
Throughput 3.643212967
KB/sec 52.09771792
Avg. Bytes 14643.13605
then using 25 users and i have this result:
# Samples 3115
Average 840
Min 207
Max 69535
Std. Dev. 3528.651571
Error % 9.63E-04
Throughput 18.89711235
KB/sec 249.9800706
Avg. Bytes 13545.9634
Actually i can't analyze these results. first this i was expecting when
running the 25 users to have 25*147 = 3675. but you can see that i have
3115. is there a logical reason that makes this occurs rather than an
internal error or anything ?
also throughput. the variation between 3.643212967 and 18.89711235. how can
i calculate the percentage of of the effect of 25 users on the throughput
and how can i deduce the performance indicator whether it is good or bad.
finally what you can see from this results? is it good? bad?
a note: i was working good on the application with no noticeable drawbacks
in the performance on the site.
Regards,
Hosam
--
View this message in context: http://www.nabble.com/Analyze-difference-between-1-user-and-25-users-tp23602736p23602736.html
Sent from the JMeter - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
Re: Analyze difference between 1 user and 25 users
Posted by sebb <se...@gmail.com>.
On 19/05/2009, ziohausam <zi...@gmail.com> wrote:
>
>
> Thanks for your response
>
>
>
> sebb-2-2 wrote:
> >
> >
> > Perhaps the test ended abruptly?
> > Did you set the test to continue on error?
> >
> > Are there any conditional controllers in the test plan?
> >
> > Are there any errors in the jmeter log file?
> >
> >
>
>
> no continue on error, no conditional controller.
> yes there are some exceptions in the JMeter log filte
>
And what are these?
Don't paste the whole file here, but a few snippets would be useful.
>
>
> sebb-2-2 wrote:
> >
> >
> > Not sure why you need that information, or whether it would mean
> > anything if calculated.
> >
>
>
> when i have tried this on some famous web applications, i have found that
> the variation between optimal throughput with 25 users and actual throughput
> is not huge.
> for example:
> with 1 user : throughput = 3/sec
> with 25 users : throughput(optimal) = 3*25 = 75/sec
> with 25 users : throughput(Actual) = 60
> performance drawback from speed point of view = 100-60*100/75 = 20%
This depends on the test plan, and the hardware.
There will be some reduction in the actual throughput that JMeter can
generate, as the threads have to share resourcs in the JVM, OS and
network.
However, it's largely irrelevant for server testing whether doubling
the number of JMeter threads doubles the throughput or not.
What's generally important is how the variations on load affect the application.
For example, with the "famous web applications" did the average
response times vary much? What about Max times?
If the response times don't vary significantly, then the application
is not being affected by the load.
If the response times do go up significantly (e.g. over 20%) then
there may be a problem with the application, but again that depends on
what the application is supposed to handle.
Unless the test plan has a throughput timer or similar in it, then the
throughput JMeter can generate will decrease as the average response
time increases, because JMeter needs to wait for the response before
it can send the next request.
> but with my application, the drawback reaches 70%. i need to know if i am:
> 1- thinking correctly:
> 2- thinking correctly but missing some other factors to be added to reach an
> accurate drawback percentage
> 3- thinking faulty ( not correctly )
That depends on what conclusions you are drawing.
Have you looked at:
http://wiki.apache.org/jakarta-jmeter/
In particular the links under "General Articles on Performance testing etc. "
>
>
>
> sebb-2-2 wrote:
> >
> >
> > Impossible to say without knowing what the performance requirements of
> > the server are, and what the test plan is trying to emulate.
> >
> > However, the fact that there are some errors when increasing the
> > number of threads to 25 could indicate a problem with the server. Or
> > it could just be an error in the test plan - e.g. it is not correctly
> > set up for multiple threads.
> >
>
>
>
> Test plan is trying to perform a performance test on each function in our
> application. this test specifically is for search function. we have maximum
> target to keep 2000 concurrent requests works on this function. ( actually,
> some customers hits the DB using web services that calls the search function
> with thousands of requests )
>
> how can i setup the plan for multiple threads? i just use direct settings by
> choosing number of users, ramb-up time, and loop count and even sets all to
> 0 except number of threats to 25.
>
>
>
> sebb-2-2 wrote:
> >
> > Try running 2, 3 and 4 threads. If these all work, then the test plan
> > at least seems to be suitable.
> >
>
>
>
> yes i will
>
>
>
> sebb-2-2 wrote:
> >
> > Also, the large increase in the maximum sample time from 5secs to
> > 69secs needs to be investigated.
> >
>
>
> yes good point
>
>
> Thanks for your response
>
>
> --
> View this message in context: http://www.nabble.com/Analyze-difference-between-1-user-and-25-users-tp23602736p23619555.html
>
> Sent from the JMeter - User mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
Re: Analyze difference between 1 user and 25 users
Posted by ziohausam <zi...@gmail.com>.
Thanks for your response
sebb-2-2 wrote:
>
>
> Perhaps the test ended abruptly?
> Did you set the test to continue on error?
>
> Are there any conditional controllers in the test plan?
>
> Are there any errors in the jmeter log file?
>
>
no continue on error, no conditional controller.
yes there are some exceptions in the JMeter log filte
sebb-2-2 wrote:
>
>
> Not sure why you need that information, or whether it would mean
> anything if calculated.
>
when i have tried this on some famous web applications, i have found that
the variation between optimal throughput with 25 users and actual throughput
is not huge.
for example:
with 1 user : throughput = 3/sec
with 25 users : throughput(optimal) = 3*25 = 75/sec
with 25 users : throughput(Actual) = 60
performance drawback from speed point of view = 100-60*100/75 = 20%
but with my application, the drawback reaches 70%. i need to know if i am:
1- thinking correctly:
2- thinking correctly but missing some other factors to be added to reach an
accurate drawback percentage
3- thinking faulty ( not correctly )
sebb-2-2 wrote:
>
>
> Impossible to say without knowing what the performance requirements of
> the server are, and what the test plan is trying to emulate.
>
> However, the fact that there are some errors when increasing the
> number of threads to 25 could indicate a problem with the server. Or
> it could just be an error in the test plan - e.g. it is not correctly
> set up for multiple threads.
>
Test plan is trying to perform a performance test on each function in our
application. this test specifically is for search function. we have maximum
target to keep 2000 concurrent requests works on this function. ( actually,
some customers hits the DB using web services that calls the search function
with thousands of requests )
how can i setup the plan for multiple threads? i just use direct settings by
choosing number of users, ramb-up time, and loop count and even sets all to
0 except number of threats to 25.
sebb-2-2 wrote:
>
> Try running 2, 3 and 4 threads. If these all work, then the test plan
> at least seems to be suitable.
>
yes i will
sebb-2-2 wrote:
>
> Also, the large increase in the maximum sample time from 5secs to
> 69secs needs to be investigated.
>
yes good point
Thanks for your response
--
View this message in context: http://www.nabble.com/Analyze-difference-between-1-user-and-25-users-tp23602736p23619555.html
Sent from the JMeter - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
Re: Analyze difference between 1 user and 25 users
Posted by sebb <se...@gmail.com>.
On 18/05/2009, ziohausam <zi...@gmail.com> wrote:
>
> Hi there,
>
> I have run a test to one of our search functions using 1 user then had the
> following results
>
> # Samples 147
> Average 271
> Min 210
> Max 5236
> Std. Dev. 418.3129775
> Error % 0
> Throughput 3.643212967
> KB/sec 52.09771792
> Avg. Bytes 14643.13605
>
> then using 25 users and i have this result:
>
> # Samples 3115
> Average 840
> Min 207
> Max 69535
> Std. Dev. 3528.651571
> Error % 9.63E-04
You need to analyse what these errors are.
> Throughput 18.89711235
> KB/sec 249.9800706
> Avg. Bytes 13545.9634
>
>
>
> Actually i can't analyze these results. first this i was expecting when
> running the 25 users to have 25*147 = 3675. but you can see that i have
> 3115. is there a logical reason that makes this occurs rather than an
> internal error or anything ?
Perhaps the test ended abruptly?
Did you set the test to continue on error?
Are there any conditional controllers in the test plan?
Are there any errors in the jmeter log file?
>
> also throughput. the variation between 3.643212967 and 18.89711235. how can
> i calculate the percentage of of the effect of 25 users on the throughput
Not sure why you need that information, or whether it would mean
anything if calculated.
> and how can i deduce the performance indicator whether it is good or bad.
Impossible to say without knowing what the performance requirements of
the server are, and what the test plan is trying to emulate.
However, the fact that there are some errors when increasing the
number of threads to 25 could indicate a problem with the server. Or
it could just be an error in the test plan - e.g. it is not correctly
set up for multiple threads.
Try running 2, 3 and 4 threads. If these all work, then the test plan
at least seems to be suitable.
Also, the large increase in the maximum sample time from 5secs to
69secs needs to be investigated.
> finally what you can see from this results? is it good? bad?
The figures say nothing much in isolation.
What load is the application under test supposed to support?
Are there max. response times?
What is the test trying to show?
> a note: i was working good on the application with no noticeable drawbacks
> in the performance on the site.
That at least is mildly encouraging.
> Regards,
> Hosam
>
>
> --
> View this message in context: http://www.nabble.com/Analyze-difference-between-1-user-and-25-users-tp23602736p23602736.html
> Sent from the JMeter - User mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org