You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@jmeter.apache.org by bobMeliev <b....@gmail.com> on 2013/10/01 15:54:02 UTC
Re: Understanding Hits per second
Strange. 10 users can't generate load with 21 req/s. Provide more information
about thread properties.
--
View this message in context: http://jmeter.512774.n5.nabble.com/Understanding-Hits-per-second-tp5718263p5718265.html
Sent from the JMeter - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
Re: Understanding Hits per second
Posted by Kirk Pepperdine <ki...@gmail.com>.
Hi Phillppe,
Like I mentioned, I came in late on the thread.. So, I think the question here isn't can 10 thread generate 21 req/s but more to the point, if I need 21 req/s how can I setup JMeter to deliver this without having the server interfere with it doing so. Given that server response time in this case could be a huge source of CO, you'd not want to have the harness depend upon a thread being freed by the server before a new request could be issued.
Regards,
Kirk
On 2013-10-03, at 4:02 PM, Philippe Mouawad <ph...@gmail.com> wrote:
> Hello Kirk,
> In this case , the question was can 10 Threads generate 21 req/s ? Do you
> agree that it is quite possible ?
> User was not saying he had a variation.
> I understand Coordinated Omission (and thanks for your contribution to
> this) but I am not sure it's the case here , or do I misunderstand ?
>
> Thanks
>
>
> On Thu, Oct 3, 2013 at 3:58 PM, Kirk Pepperdine
> <ki...@gmail.com>wrote:
>
>> Hi all,
>>
>> Jumping in on this thread late but (very bluntly) if your test harness
>> throughput is dependent upon the response time of your application, you
>> test system is broken. This is one of the conditions that we've recently
>> named coordinated omission (CO). It is a state where the test harness and
>> the server (accidentally or unintentionally) collaborate to miss injecting
>> samples when they should have been injected. The net result is that the
>> harness backs off allowing the server to drain and that in turn allows your
>> server to report on numbers that are much better than they would be in
>> reality. From a performance testing point of view, the reduced load is akin
>> to a scale down study of your system. Scale down studies often produce
>> different results than testing at scale. It is but one of the many ways
>> UATs fail to recognize problems that leak out into prod but it's an
>> important one.
>>
>> How to fix this? For starters, always set the loop count for a thread to
>> 1. Set the number of threads to the number of transactions you want to run
>> and then set the warmup time to the duration of the test. This won't fix
>> all of the problems that come with using JMeter but it gets rid of one of
>> the bigger problems. BTW, all of the other tools in the space suffer from
>> the exact same issues.
>>
>> Regards,
>> Kirk
>>
>>
>> On 2013-10-02, at 9:05 PM, Philippe Mouawad <ph...@gmail.com>
>> wrote:
>>
>>> Hello,
>>> Of course you can generate even much more than 21req / s.
>>> If fully depends on your response times.
>>>
>>> Want to test:
>>> - Create a mirror server listening on 8081
>>> - Create an HTTP Sampler hitting localhost:8081
>>> - Add only an aggregate report
>>>
>>>
>>> Running on a last generation Mac Book Pro with default configuration =>
>> 517
>>> req/s .
>>>
>>> This test is stupid but just to confirm you can hit this rate.
>>> The results you mention are strange and I think wrong.
>>>
>>> Regards
>>> Philippe
>>> @philmdot
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Oct 2, 2013 at 12:47 PM, bobMeliev
>>> <b.meliev@gmail.com<javascript:_e({}, 'cvml', 'b.meliev@gmail.com');>
>>>> wrote:
>>>
>>>> Nope. Even by default Dummy Sampler couldn't generate such load.
>> Screenshot
>>>> attached.
>>>>
>>>> <http://jmeter.512774.n5.nabble.com/file/n5718286/screenshot_233.png>
>>>>
>>>> 21 req/s is huge load just for 10 users. Check this report
>>>>
>>>>
>> http://blazemeter.com/blog/increasing-productivity-wordpress-site-when-using-blazemeter-its-easy-task
>>>> with 300 users generated only 5 req/s load and with such load CPU was
>> 75%
>>>> busy.
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>>
>> http://jmeter.512774.n5.nabble.com/Understanding-Hits-per-second-tp5718263p5718286.html
>>>> Sent from the JMeter - User mailing list archive at Nabble.com.
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org<javascript:_e({},
>> 'cvml', 'user-unsubscribe@jmeter.apache.org');>
>>>> For additional commands, e-mail: user-help@jmeter.apache.org<javascript:_e({},
>> 'cvml', 'user-help@jmeter.apache.org');>
>>>>
>>>>
>>>
>>>
>>> --
>>> Cordialement.
>>> Philippe Mouawad.
>>>
>>>
>>>
>>>
>>> --
>>> Cordialement.
>>> Philippe Mouawad.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
Re: Understanding Hits per second
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello Kirk,
In this case , the question was can 10 Threads generate 21 req/s ? Do you
agree that it is quite possible ?
User was not saying he had a variation.
I understand Coordinated Omission (and thanks for your contribution to
this) but I am not sure it's the case here , or do I misunderstand ?
Thanks
On Thu, Oct 3, 2013 at 3:58 PM, Kirk Pepperdine
<ki...@gmail.com>wrote:
> Hi all,
>
> Jumping in on this thread late but (very bluntly) if your test harness
> throughput is dependent upon the response time of your application, you
> test system is broken. This is one of the conditions that we've recently
> named coordinated omission (CO). It is a state where the test harness and
> the server (accidentally or unintentionally) collaborate to miss injecting
> samples when they should have been injected. The net result is that the
> harness backs off allowing the server to drain and that in turn allows your
> server to report on numbers that are much better than they would be in
> reality. From a performance testing point of view, the reduced load is akin
> to a scale down study of your system. Scale down studies often produce
> different results than testing at scale. It is but one of the many ways
> UATs fail to recognize problems that leak out into prod but it's an
> important one.
>
> How to fix this? For starters, always set the loop count for a thread to
> 1. Set the number of threads to the number of transactions you want to run
> and then set the warmup time to the duration of the test. This won't fix
> all of the problems that come with using JMeter but it gets rid of one of
> the bigger problems. BTW, all of the other tools in the space suffer from
> the exact same issues.
>
> Regards,
> Kirk
>
>
> On 2013-10-02, at 9:05 PM, Philippe Mouawad <ph...@gmail.com>
> wrote:
>
> > Hello,
> > Of course you can generate even much more than 21req / s.
> > If fully depends on your response times.
> >
> > Want to test:
> > - Create a mirror server listening on 8081
> > - Create an HTTP Sampler hitting localhost:8081
> > - Add only an aggregate report
> >
> >
> > Running on a last generation Mac Book Pro with default configuration =>
> 517
> > req/s .
> >
> > This test is stupid but just to confirm you can hit this rate.
> > The results you mention are strange and I think wrong.
> >
> > Regards
> > Philippe
> > @philmdot
> >
> >
> >
> >
> >
> > On Wed, Oct 2, 2013 at 12:47 PM, bobMeliev
> > <b.meliev@gmail.com<javascript:_e({}, 'cvml', 'b.meliev@gmail.com');>
> >> wrote:
> >
> >> Nope. Even by default Dummy Sampler couldn't generate such load.
> Screenshot
> >> attached.
> >>
> >> <http://jmeter.512774.n5.nabble.com/file/n5718286/screenshot_233.png>
> >>
> >> 21 req/s is huge load just for 10 users. Check this report
> >>
> >>
> http://blazemeter.com/blog/increasing-productivity-wordpress-site-when-using-blazemeter-its-easy-task
> >> with 300 users generated only 5 req/s load and with such load CPU was
> 75%
> >> busy.
> >>
> >>
> >>
> >>
> >> --
> >> View this message in context:
> >>
> http://jmeter.512774.n5.nabble.com/Understanding-Hits-per-second-tp5718263p5718286.html
> >> Sent from the JMeter - User mailing list archive at Nabble.com.
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org<javascript:_e({},
> 'cvml', 'user-unsubscribe@jmeter.apache.org');>
> >> For additional commands, e-mail: user-help@jmeter.apache.org<javascript:_e({},
> 'cvml', 'user-help@jmeter.apache.org');>
> >>
> >>
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
> >
> >
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>
--
Cordialement.
Philippe Mouawad.
Re: Understanding Hits per second
Posted by Kirk Pepperdine <ki...@gmail.com>.
Hi all,
Jumping in on this thread late but (very bluntly) if your test harness throughput is dependent upon the response time of your application, you test system is broken. This is one of the conditions that we've recently named coordinated omission (CO). It is a state where the test harness and the server (accidentally or unintentionally) collaborate to miss injecting samples when they should have been injected. The net result is that the harness backs off allowing the server to drain and that in turn allows your server to report on numbers that are much better than they would be in reality. From a performance testing point of view, the reduced load is akin to a scale down study of your system. Scale down studies often produce different results than testing at scale. It is but one of the many ways UATs fail to recognize problems that leak out into prod but it's an important one.
How to fix this? For starters, always set the loop count for a thread to 1. Set the number of threads to the number of transactions you want to run and then set the warmup time to the duration of the test. This won't fix all of the problems that come with using JMeter but it gets rid of one of the bigger problems. BTW, all of the other tools in the space suffer from the exact same issues.
Regards,
Kirk
On 2013-10-02, at 9:05 PM, Philippe Mouawad <ph...@gmail.com> wrote:
> Hello,
> Of course you can generate even much more than 21req / s.
> If fully depends on your response times.
>
> Want to test:
> - Create a mirror server listening on 8081
> - Create an HTTP Sampler hitting localhost:8081
> - Add only an aggregate report
>
>
> Running on a last generation Mac Book Pro with default configuration => 517
> req/s .
>
> This test is stupid but just to confirm you can hit this rate.
> The results you mention are strange and I think wrong.
>
> Regards
> Philippe
> @philmdot
>
>
>
>
>
> On Wed, Oct 2, 2013 at 12:47 PM, bobMeliev
> <b.meliev@gmail.com<javascript:_e({}, 'cvml', 'b.meliev@gmail.com');>
>> wrote:
>
>> Nope. Even by default Dummy Sampler couldn't generate such load. Screenshot
>> attached.
>>
>> <http://jmeter.512774.n5.nabble.com/file/n5718286/screenshot_233.png>
>>
>> 21 req/s is huge load just for 10 users. Check this report
>>
>> http://blazemeter.com/blog/increasing-productivity-wordpress-site-when-using-blazemeter-its-easy-task
>> with 300 users generated only 5 req/s load and with such load CPU was 75%
>> busy.
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://jmeter.512774.n5.nabble.com/Understanding-Hits-per-second-tp5718263p5718286.html
>> Sent from the JMeter - User mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org<javascript:_e({}, 'cvml', 'user-unsubscribe@jmeter.apache.org');>
>> For additional commands, e-mail: user-help@jmeter.apache.org<javascript:_e({}, 'cvml', 'user-help@jmeter.apache.org');>
>>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
Re: Understanding Hits per second
Posted by Kirk Pepperdine <ki...@gmail.com>.
3ms response time which means that 333.33 makes sense.. until you see that you've only one sample and then.... this is a calculation that shouldn't be performed with only one sample.
Regards,
Kirk
On 2013-10-04, at 6:00 AM, bobMeliev <b....@gmail.com> wrote:
> I tried your example and got this:
>
> <http://jmeter.512774.n5.nabble.com/file/n5718320/screenshot_234.png>
>
> One thread sampler and 333/sec throuput is not compatible I think. My
> expected result was Throuput=1 req/s, am I wrong?
>
> Also found old sebb's message in archive
> http://www.mail-archive.com/jmeter-user@jakarta.apache.org/msg18348.html
>
>
>
> --
> View this message in context: http://jmeter.512774.n5.nabble.com/Understanding-Hits-per-second-tp5718263p5718320.html
> Sent from the JMeter - User mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
Re: Understanding Hits per second
Posted by Philippe Mouawad <ph...@gmail.com>.
Read the full answer by sebb, he says it's fixed in next release in 2007,
so IMHO it is fixed.
On Friday, October 4, 2013, bobMeliev wrote:
> I tried your example and got this:
>
> <http://jmeter.512774.n5.nabble.com/file/n5718320/screenshot_234.png>
>
> One thread sampler and 333/sec throuput is not compatible I think. My
> expected result was Throuput=1 req/s, am I wrong?
>
> Also found old sebb's message in archive
> http://www.mail-archive.com/jmeter-user@jakarta.apache.org/msg18348.html
>
>
>
> --
> View this message in context:
> http://jmeter.512774.n5.nabble.com/Understanding-Hits-per-second-tp5718263p5718320.html
> Sent from the JMeter - User mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org <javascript:;>
> For additional commands, e-mail: user-help@jmeter.apache.org<javascript:;>
>
>
--
Cordialement.
Philippe Mouawad.
Re: Understanding Hits per second
Posted by bobMeliev <b....@gmail.com>.
I tried your example and got this:
<http://jmeter.512774.n5.nabble.com/file/n5718320/screenshot_234.png>
One thread sampler and 333/sec throuput is not compatible I think. My
expected result was Throuput=1 req/s, am I wrong?
Also found old sebb's message in archive
http://www.mail-archive.com/jmeter-user@jakarta.apache.org/msg18348.html
--
View this message in context: http://jmeter.512774.n5.nabble.com/Understanding-Hits-per-second-tp5718263p5718320.html
Sent from the JMeter - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
Re: Understanding Hits per second
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello,
Of course you can generate even much more than 21req / s.
If fully depends on your response times.
Want to test:
- Create a mirror server listening on 8081
- Create an HTTP Sampler hitting localhost:8081
- Add only an aggregate report
Running on a last generation Mac Book Pro with default configuration => 517
req/s .
This test is stupid but just to confirm you can hit this rate.
The results you mention are strange and I think wrong.
Regards
Philippe
@philmdot
On Wed, Oct 2, 2013 at 12:47 PM, bobMeliev
<b.meliev@gmail.com<javascript:_e({}, 'cvml', 'b.meliev@gmail.com');>
> wrote:
> Nope. Even by default Dummy Sampler couldn't generate such load. Screenshot
> attached.
>
> <http://jmeter.512774.n5.nabble.com/file/n5718286/screenshot_233.png>
>
> 21 req/s is huge load just for 10 users. Check this report
>
> http://blazemeter.com/blog/increasing-productivity-wordpress-site-when-using-blazemeter-its-easy-task
> with 300 users generated only 5 req/s load and with such load CPU was 75%
> busy.
>
>
>
>
> --
> View this message in context:
> http://jmeter.512774.n5.nabble.com/Understanding-Hits-per-second-tp5718263p5718286.html
> Sent from the JMeter - User mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org<javascript:_e({}, 'cvml', 'user-unsubscribe@jmeter.apache.org');>
> For additional commands, e-mail: user-help@jmeter.apache.org<javascript:_e({}, 'cvml', 'user-help@jmeter.apache.org');>
>
>
--
Cordialement.
Philippe Mouawad.
--
Cordialement.
Philippe Mouawad.
Re: Understanding Hits per second
Posted by bobMeliev <b....@gmail.com>.
Nope. Even by default Dummy Sampler couldn't generate such load. Screenshot
attached.
<http://jmeter.512774.n5.nabble.com/file/n5718286/screenshot_233.png>
21 req/s is huge load just for 10 users. Check this report
http://blazemeter.com/blog/increasing-productivity-wordpress-site-when-using-blazemeter-its-easy-task
with 300 users generated only 5 req/s load and with such load CPU was 75%
busy.
--
View this message in context: http://jmeter.512774.n5.nabble.com/Understanding-Hits-per-second-tp5718263p5718286.html
Sent from the JMeter - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org
Re: Understanding Hits per second
Posted by Flavio Cysne <fl...@gmail.com>.
10 users CAN generate load with 21 req/s, because the measure is per
second. If 1 request gets its response in less than a half second then 21
req/s is a possible measure.
2013/10/1 bobMeliev <b....@gmail.com>
> Strange. 10 users can't generate load with 21 req/s. Provide more
> information
> about thread properties.
>
>
>
> --
> View this message in context:
> http://jmeter.512774.n5.nabble.com/Understanding-Hits-per-second-tp5718263p5718265.html
> Sent from the JMeter - User mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>