You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@jmeter.apache.org by Shay Ginsbourg <sg...@gmail.com> on 2012/06/12 10:56:43 UTC

TCP Sampler Read Exception

Hi all,

Can anyone suggest what does the following error mean ?

After each TCP sampler, we receive "Response code: 500" and

"Response message: org.apache.jmeter.protocol.tcp.sampler.ReadException:"

Any help would be greatly appreciated.

thanks,
Shay

Re: TCP Sampler Read Exception

Posted by Shmuel Krakower <sh...@gmail.com>.

Feel free to copy and paste here the complete error.

Shmuel.


On Tue, Jun 12, 2012 at 12:02 PM, sebb <se...@gmail.com> wrote:

> On 12 June 2012 09:56, Shay Ginsbourg <sg...@gmail.com> wrote:
> > Hi all,
> >
> > Can anyone suggest what does the following error mean ?
> >
> > After each TCP sampler, we receive "Response code: 500" and
> >
> > "Response message: org.apache.jmeter.protocol.tcp.sampler.ReadException:"
> >
> > Any help would be greatly appreciated.
>
> Most likely the read terminated unexpectedly:
>
>
> http://jmeter.apache.org/api/org/apache/jmeter/protocol/tcp/sampler/TCPClient.html#read%28java.io.InputStream%29
>
> > thanks,
> > Shay
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>

Re: JMeter threading model

Posted by Flavio Cysne <fl...@gmail.com>.

Hi Kirk,

    just a thought on this case, and if you start as many JMeter instances
(using servers or not) as needed, once at a time, to increase the load,
let's say, pseudo-dynamically?

    I think this way you could, in some way, achieve the desired load. The
"hard work" will be on merging jtl files in only one big jtl file to
generate reports.

    I have to agree with you that increasing/decreasing dynamically the
load is better than starting many tests to pin-point a spot on the graph.

Hope it helps you.
Flávio Cysne

2012/6/13 Kirk Pepperdine <ki...@gmail.com>

> Hi Sebb,
>
> We've had this conversation before and I did some preliminary work to
> setup a different type of thread group but the couplings between the
> existing thread group and the model meant that an extensive refactoring
> would be involved. Since that involves a *lot* more than just a simple
> plugin...
>
> So, the current implementation supports a closed system model meaning,
> rate of entry into the system equals rate of exit from the system. This is
> exactly what you want if you're load testing a call centre where the number
> of servers (operators) is fixed and gate entry into the system. However,
> I'm often simulating open systems which means I do not want rate of entry
> into the system to be controlled by the performance of the system (rate of
> exit). More over, those that attend my performance tuning seminars come to
> understand why this is an important aspect of getting your test environment
> right and test harness correctly setup as it can adversely affect the
> quality of your test which can and often does, change the results of the
> test.
>
> As an example, today I will show a group how to tune an application by a
> partner company. That application has a number of "performance problems"
> backed into it. If I use the traditional means of using JMeter I will find
> a different set of performance issues than if I load with a pattern that is
> similar that found in production. In other words, with this particular
> application, JMeter exposes "problems" that are artifacts of how it wants
> to inject load on a system. I can fix all of these problems and eventually
> I'll get to a point where I'll fix everything that needs to be fixed. That
> said, if I can coerce JMeter to load as an open system I'll get to the
> problems without having to fix the artifacts (the things that really don't
> need fixing). To coerce JMeter into being an open system requires one to
> use a large number of very short lived threads. So I may only have 400-500
> active threads at any point in time but in order to achieve that load over
> a 1 or two hour test I may have to specify 10s of thousands of threads.
> Since all of the threads are created up front, this simply doesn't work.
>
> You might ask why not just specify 400-500 threads and loop over them? in
> theory you'd think it would work but as you tune the system and the
> performance characteristics change. Going back to the baked application,
> before I start tuning, the active user count is several thousand. In other
> words, the tuned system is better able to clear users out and that changes
> the performance profile in a way that hard to emulate with the current
> looping behaviour. Using a setting of looping 400 or so threads isn't
> adequate for the initial load tests as the test harness will become thread
> starved and that releases pressure on the server which in turn changes the
> performance profile.
>
> With all due respect to the wonderful work that everyone on the project
> has done, it is my opinion that the one user == one thread is a design
> mistake that has a huge impact on both the usability of the tool and the
> quality of the results one can achieve when using it. IMHO, moving to an
> thread pool/event heap based model would be an enormous improvement over
> the current implementation.
>
> Regards,
> Kirk
>
> On 2012-06-13, at 1:02 AM, sebb wrote:
>
> > On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com>
> wrote:
> >>
> >> On 2012-06-13, at 12:54 AM, sebb wrote:
> >>
> >>> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com>
> wrote:
> >>>> Hi,
> >>>>
> >>>> I figured thread pooling would be revolutionary so I wasn't
> suggesting that. I would be very useful just delay the creation of a thread
> until it was asked for.
> >>>
> >>> Not sure I understand how it would help to delay the thread creation,
> >>> except perhaps for the case where the first threads have finished
> >>> processing by the time the last threads start running samples.
> >>
> >> Bingo!!! ;-)
> >
> > So what percentage of use cases need to follow this model?
> >
> > Most of the JMeter testing I have done was long running tests where
> > all threads were active for most of the run.
> >
> >> Kirk
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >> For additional commands, e-mail: user-help@jmeter.apache.org
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> > For additional commands, e-mail: user-help@jmeter.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>

Re: JMeter threading model

Posted by Shay Ginsbourg <sg...@gmail.com>.

Looking forward!

Thanks!


www.ginsbourg.com | 054-6690915 |  Sent from my BlackBerry® smartphone from orange. 

-----Original Message-----
From: sebb <se...@gmail.com>
Date: Fri, 31 Aug 2012 16:52:48 
To: JMeter Users List<us...@jmeter.apache.org>
Reply-To: "JMeter Users List" <us...@jmeter.apache.org>
Subject: Re: JMeter threading model

On 31 August 2012 16:12, Shay Ginsbourg <sg...@gmail.com> wrote:
> Well done!
>
> Which JMeter version has all that implemented in?
>

The next version - which will likely be 2.8.

>
>
>
>
> On Fri, Aug 31, 2012 at 3:04 PM, sebb <se...@gmail.com> wrote:
>>
>> Just to follow up - we have now implemented changes to the
>> TestCompiler and JavaSamplers to reduce the long-term memory storage
>> needed.
>>
>> The memory requirement imposed by JMeter itself is now proportional to
>> the number of active threads; when a thread completes, its memory
>> should be released.
>>
>> In conjunction with the Thread Group delayed thread creation option
>> and a long ramp-up, it's now possible to run a test with hundreds of
>> thousands of threads (possibly over a million), so long as the number
>> of concurrent (active) threads is kept to a reasonable number.
>>
>> On 7 August 2012 08:54, Philippe Mouawad <ph...@gmail.com> wrote:
>> > Hello Kirk,
>> > Thank you very much for your feedback.
>> > Waiting for the big test result.
>> >
>> > Regards
>> > Philippe
>> >
>> > On Monday, August 6, 2012, Philippe Mouawad wrote:
>> >
>> >> From kirk (bad subject due to vacation message :) ) :
>> >>
>> >>
>> >>
>> >> Hi Phillippe,
>> >>
>> >> The on demand thread group worked brilliantly for my short test. I'll repeat with an 8 hour
>> >> run to give it an even better test and let you know how that worked.
>> >>
>> >> -- Kirk
>> >>
>> >>
>> >> ---------- Forwarded message ----------
>> >> From: *Kirk Pepperdine*
>> >> Date: Monday, August 6, 2012
>> >> Subject: JMeter threading model
>> >> To: Philippe Mouawad <ph...@gmail.com>
>> >> Cc: JMeter Users List <us...@jmeter.apache.org>
>> >>
>> >>
>> >> Hi Philippe,
>> >>
>> >> Just got back from vacation going through email and I noticed that I
>> >> missed this one. So I've now downloaded the nightly build and loaded a test
>> >> plan to check it out. Unfortunately the benchmark I decided to setup and
>> >> run against is borked. Well, I don't know if JMeter is the source of the
>> >> problem my recent upgrade to Mountain Lion (and what it did to Java). I'll
>> >> retry on an older machine where the bench should still behave as expected
>> >> and then see how this new ThreadGroup functions. My guess is that it should
>> >> be ok as it didn't complain as it would have using a regular ThreadGroup.
>> >> However, I would like to run a much much longer test to make sure. I'll
>> >> report back once this is done.
>> >>
>> >> Regards,
>> >> Kirk
>> >>
>> >> On 2012-07-14, at 3:36 PM, Philippe Mouawad <ph...@gmail.com>
>> >> wrote:
>> >>
>> >> Hello,
>> >> Just a little mail to inform you that this feature has been implemented as
>> >> part of:
>> >>
>> >>    - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418
>> >>
>> >> It is currently available in nightly build:
>> >>
>> >>    - http://ci.apache.org/projects/jmeter/nightlies/
>> >>
>> >>
>> >> @Kirk, it would be nice from you to look at this feature to see if it
>> >> answers your initial request and give us some feedback.
>> >>
>> >> *Note that this feature may be subject to changes until next release, but
>> >> getting some feedback would be very useful.
>> >> *
>> >>
>> >> Regards
>> >> Philippe M.
>> >>
>> >> On Sun, Jun 17, 2012 at 9:06 PM, sebb <se...@gmail.com> wrote:
>> >>
>> >> On 17 June 2012 13:57, Flavio Cysne <fl...@gmail.com> wrote:
>> >> > Post entry in my blog:
>> >> >
>> >> http://flaviocysne.blogspot.com.br/2012/06/how-to-dynamically-increasedecrease.html
>> >> >
>> >> > It's an unfinished work, but it's suffice to make what is proposed.
>> >> > Comments and constructive criticism are welcome.
>> >>
>> >> Thanks!
>> >>
>> >> The property used to control the thread count can also be set using
>> >> the BeanShell server, see:
>> >>
>> >> http://jmeter.apache.org/usermanual/best-practices.html#beanshell_server
>> >>
>> >> Or since properties are global, you could run the property update
>> >> sampler ("load controller") in a separate parallel thread group with
>> >> one thread.
>> >>
>> >> However, JMeter will still have to start all the threads - even if
>> >> they are not actively sampling - which will take up memory.
>> >>
>> >> It may not be too hard to change JMeter so it delays the thread
>> >> creation until required.
>> >> That may help in some use cases.
>> >>
>> >> But it would probably be somewhat harder to allow threads to be
>> >> created later on demand.
>> >> And converting to an event-based model is definitely a lot of work.
>> >>
>> >> > Hope it helps you.
>> >> > Flávio Cysne
>> >> >
>> >> > 2012/6/15 Flavio Cysne <fl...@gmail.com>
>> >> >
>> >> >> Errata:
>> >> >>
>> >> >> The first "If Controller" is executed and update a property named
>> >> >> "currentThreads" ...
>> >> >>
>> >> >>
>> >> >> 2012/6/15 Flavio Cysne <fl...@gmail.com>
>> >> >>
>> >> >>> I have tested something in JMeter that could achieve a dynamic
>> >> threading
>> >> >>> model and there was no need of new implementations.
>> >> >>>
>> >> >>> Basically, I have this structure:
>> >> >>>
>> >> >>> 1. Test Plan: { variables : [ {name: "maxThreads", value: 5}, {name:
>> >> >>> "currentThreads", value: 1} ] }
>> >> >>> 2. | - Thread Group: { threads: "${maxThreads}", rampup: 0, infinite:
>> >> >>> true }
>> >> >>> 3.     | - If Controller: { condition: "(${__threadNum()} ===
>> >> >>> ${maxThreads})" }
>> >> >>> 4.         | - Sampler /* I've used HTTP Request, but anyone would be
>> >> >>> used */
>> >> >>> 5.             | - RegEx Extractor: {refName: "th
>> >>
>> >>
>> >>
>> >> --
>> >> Cordialement.
>> >> Philippe Mouawad.
>> >>
>> >>
>> >>
>> >>
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>
>
>
> --
>
> Regards,
>
>
> Shay Ginsbourg
>
> Regulatory & Testing Affairs Consultant
>
>
> WWW.GINSBOURG.COM
>
>
> Providing Regulatory, Medical & Performance Testing services since 2008:
>
>
> * IEC 62304 Medical Device Software Life Cycle
>
> * IEEE 829 Software Test Documentation
>
> * ISO 14971 Medical Device Risk Management
>
> * FDA 21 CFR Part 11 Software Validation
>
> * IEC 60601-1:2005 3rd ED PEMS - Medical Electrical Equipment
>
> * End-to-end verification, validation, and testing (VV&T)
>
> * FDA and CE submissions
>
> * Open source free testing tools implementation
>
> * Functionality and regression testing
>
> * Software Performance & Load testing
>
> * Software Testing Advanced Automation
>
> * Medical Software Verification & Validation
>
> * Medical Device Verification & Validation
>
> * Medical Device Regulatory Submission
>
> * Organizational Regulatory Qualification
>
>
> Formerly QA Manager of LoadRunner at Mercury Interactive
>
>
> M.Sc. cum laude in Bio-Medical Engineering
>
> M.Sc. in Mechanical Engineering
>
>
> Work:   +972(0)3-5185873
>
> Mobile:  +972(0)54-6690915
>
>
> Email: sginsbourg@gmail.com
>
>
> Visit my personal page on LinkedIn at: http://www.linkedin.com/in/shayginsbourg
>
>
> Please consider your environmental responsibility before printing this e-mail.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by sebb <se...@gmail.com>.

On 31 August 2012 16:12, Shay Ginsbourg <sg...@gmail.com> wrote:
> Well done!
>
> Which JMeter version has all that implemented in?
>

The next version - which will likely be 2.8.

>
>
>
>
> On Fri, Aug 31, 2012 at 3:04 PM, sebb <se...@gmail.com> wrote:
>>
>> Just to follow up - we have now implemented changes to the
>> TestCompiler and JavaSamplers to reduce the long-term memory storage
>> needed.
>>
>> The memory requirement imposed by JMeter itself is now proportional to
>> the number of active threads; when a thread completes, its memory
>> should be released.
>>
>> In conjunction with the Thread Group delayed thread creation option
>> and a long ramp-up, it's now possible to run a test with hundreds of
>> thousands of threads (possibly over a million), so long as the number
>> of concurrent (active) threads is kept to a reasonable number.
>>
>> On 7 August 2012 08:54, Philippe Mouawad <ph...@gmail.com> wrote:
>> > Hello Kirk,
>> > Thank you very much for your feedback.
>> > Waiting for the big test result.
>> >
>> > Regards
>> > Philippe
>> >
>> > On Monday, August 6, 2012, Philippe Mouawad wrote:
>> >
>> >> From kirk (bad subject due to vacation message :) ) :
>> >>
>> >>
>> >>
>> >> Hi Phillippe,
>> >>
>> >> The on demand thread group worked brilliantly for my short test. I'll repeat with an 8 hour
>> >> run to give it an even better test and let you know how that worked.
>> >>
>> >> -- Kirk
>> >>
>> >>
>> >> ---------- Forwarded message ----------
>> >> From: *Kirk Pepperdine*
>> >> Date: Monday, August 6, 2012
>> >> Subject: JMeter threading model
>> >> To: Philippe Mouawad <ph...@gmail.com>
>> >> Cc: JMeter Users List <us...@jmeter.apache.org>
>> >>
>> >>
>> >> Hi Philippe,
>> >>
>> >> Just got back from vacation going through email and I noticed that I
>> >> missed this one. So I've now downloaded the nightly build and loaded a test
>> >> plan to check it out. Unfortunately the benchmark I decided to setup and
>> >> run against is borked. Well, I don't know if JMeter is the source of the
>> >> problem my recent upgrade to Mountain Lion (and what it did to Java). I'll
>> >> retry on an older machine where the bench should still behave as expected
>> >> and then see how this new ThreadGroup functions. My guess is that it should
>> >> be ok as it didn't complain as it would have using a regular ThreadGroup.
>> >> However, I would like to run a much much longer test to make sure. I'll
>> >> report back once this is done.
>> >>
>> >> Regards,
>> >> Kirk
>> >>
>> >> On 2012-07-14, at 3:36 PM, Philippe Mouawad <ph...@gmail.com>
>> >> wrote:
>> >>
>> >> Hello,
>> >> Just a little mail to inform you that this feature has been implemented as
>> >> part of:
>> >>
>> >>    - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418
>> >>
>> >> It is currently available in nightly build:
>> >>
>> >>    - http://ci.apache.org/projects/jmeter/nightlies/
>> >>
>> >>
>> >> @Kirk, it would be nice from you to look at this feature to see if it
>> >> answers your initial request and give us some feedback.
>> >>
>> >> *Note that this feature may be subject to changes until next release, but
>> >> getting some feedback would be very useful.
>> >> *
>> >>
>> >> Regards
>> >> Philippe M.
>> >>
>> >> On Sun, Jun 17, 2012 at 9:06 PM, sebb <se...@gmail.com> wrote:
>> >>
>> >> On 17 June 2012 13:57, Flavio Cysne <fl...@gmail.com> wrote:
>> >> > Post entry in my blog:
>> >> >
>> >> http://flaviocysne.blogspot.com.br/2012/06/how-to-dynamically-increasedecrease.html
>> >> >
>> >> > It's an unfinished work, but it's suffice to make what is proposed.
>> >> > Comments and constructive criticism are welcome.
>> >>
>> >> Thanks!
>> >>
>> >> The property used to control the thread count can also be set using
>> >> the BeanShell server, see:
>> >>
>> >> http://jmeter.apache.org/usermanual/best-practices.html#beanshell_server
>> >>
>> >> Or since properties are global, you could run the property update
>> >> sampler ("load controller") in a separate parallel thread group with
>> >> one thread.
>> >>
>> >> However, JMeter will still have to start all the threads - even if
>> >> they are not actively sampling - which will take up memory.
>> >>
>> >> It may not be too hard to change JMeter so it delays the thread
>> >> creation until required.
>> >> That may help in some use cases.
>> >>
>> >> But it would probably be somewhat harder to allow threads to be
>> >> created later on demand.
>> >> And converting to an event-based model is definitely a lot of work.
>> >>
>> >> > Hope it helps you.
>> >> > Flávio Cysne
>> >> >
>> >> > 2012/6/15 Flavio Cysne <fl...@gmail.com>
>> >> >
>> >> >> Errata:
>> >> >>
>> >> >> The first "If Controller" is executed and update a property named
>> >> >> "currentThreads" ...
>> >> >>
>> >> >>
>> >> >> 2012/6/15 Flavio Cysne <fl...@gmail.com>
>> >> >>
>> >> >>> I have tested something in JMeter that could achieve a dynamic
>> >> threading
>> >> >>> model and there was no need of new implementations.
>> >> >>>
>> >> >>> Basically, I have this structure:
>> >> >>>
>> >> >>> 1. Test Plan: { variables : [ {name: "maxThreads", value: 5}, {name:
>> >> >>> "currentThreads", value: 1} ] }
>> >> >>> 2. | - Thread Group: { threads: "${maxThreads}", rampup: 0, infinite:
>> >> >>> true }
>> >> >>> 3.     | - If Controller: { condition: "(${__threadNum()} ===
>> >> >>> ${maxThreads})" }
>> >> >>> 4.         | - Sampler /* I've used HTTP Request, but anyone would be
>> >> >>> used */
>> >> >>> 5.             | - RegEx Extractor: {refName: "th
>> >>
>> >>
>> >>
>> >> --
>> >> Cordialement.
>> >> Philippe Mouawad.
>> >>
>> >>
>> >>
>> >>
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>
>
>
> --
>
> Regards,
>
>
> Shay Ginsbourg
>
> Regulatory & Testing Affairs Consultant
>
>
> WWW.GINSBOURG.COM
>
>
> Providing Regulatory, Medical & Performance Testing services since 2008:
>
>
> * IEC 62304 Medical Device Software Life Cycle
>
> * IEEE 829 Software Test Documentation
>
> * ISO 14971 Medical Device Risk Management
>
> * FDA 21 CFR Part 11 Software Validation
>
> * IEC 60601-1:2005 3rd ED PEMS - Medical Electrical Equipment
>
> * End-to-end verification, validation, and testing (VV&T)
>
> * FDA and CE submissions
>
> * Open source free testing tools implementation
>
> * Functionality and regression testing
>
> * Software Performance & Load testing
>
> * Software Testing Advanced Automation
>
> * Medical Software Verification & Validation
>
> * Medical Device Verification & Validation
>
> * Medical Device Regulatory Submission
>
> * Organizational Regulatory Qualification
>
>
> Formerly QA Manager of LoadRunner at Mercury Interactive
>
>
> M.Sc. cum laude in Bio-Medical Engineering
>
> M.Sc. in Mechanical Engineering
>
>
> Work:   +972(0)3-5185873
>
> Mobile:  +972(0)54-6690915
>
>
> Email: sginsbourg@gmail.com
>
>
> Visit my personal page on LinkedIn at: http://www.linkedin.com/in/shayginsbourg
>
>
> Please consider your environmental responsibility before printing this e-mail.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by Shay Ginsbourg <sg...@gmail.com>.

Well done!

Which JMeter version has all that implemented in?






On Fri, Aug 31, 2012 at 3:04 PM, sebb <se...@gmail.com> wrote:
>
> Just to follow up - we have now implemented changes to the
> TestCompiler and JavaSamplers to reduce the long-term memory storage
> needed.
>
> The memory requirement imposed by JMeter itself is now proportional to
> the number of active threads; when a thread completes, its memory
> should be released.
>
> In conjunction with the Thread Group delayed thread creation option
> and a long ramp-up, it's now possible to run a test with hundreds of
> thousands of threads (possibly over a million), so long as the number
> of concurrent (active) threads is kept to a reasonable number.
>
> On 7 August 2012 08:54, Philippe Mouawad <ph...@gmail.com> wrote:
> > Hello Kirk,
> > Thank you very much for your feedback.
> > Waiting for the big test result.
> >
> > Regards
> > Philippe
> >
> > On Monday, August 6, 2012, Philippe Mouawad wrote:
> >
> >> From kirk (bad subject due to vacation message :) ) :
> >>
> >>
> >>
> >> Hi Phillippe,
> >>
> >> The on demand thread group worked brilliantly for my short test. I'll repeat with an 8 hour
> >> run to give it an even better test and let you know how that worked.
> >>
> >> -- Kirk
> >>
> >>
> >> ---------- Forwarded message ----------
> >> From: *Kirk Pepperdine*
> >> Date: Monday, August 6, 2012
> >> Subject: JMeter threading model
> >> To: Philippe Mouawad <ph...@gmail.com>
> >> Cc: JMeter Users List <us...@jmeter.apache.org>
> >>
> >>
> >> Hi Philippe,
> >>
> >> Just got back from vacation going through email and I noticed that I
> >> missed this one. So I've now downloaded the nightly build and loaded a test
> >> plan to check it out. Unfortunately the benchmark I decided to setup and
> >> run against is borked. Well, I don't know if JMeter is the source of the
> >> problem my recent upgrade to Mountain Lion (and what it did to Java). I'll
> >> retry on an older machine where the bench should still behave as expected
> >> and then see how this new ThreadGroup functions. My guess is that it should
> >> be ok as it didn't complain as it would have using a regular ThreadGroup.
> >> However, I would like to run a much much longer test to make sure. I'll
> >> report back once this is done.
> >>
> >> Regards,
> >> Kirk
> >>
> >> On 2012-07-14, at 3:36 PM, Philippe Mouawad <ph...@gmail.com>
> >> wrote:
> >>
> >> Hello,
> >> Just a little mail to inform you that this feature has been implemented as
> >> part of:
> >>
> >>    - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418
> >>
> >> It is currently available in nightly build:
> >>
> >>    - http://ci.apache.org/projects/jmeter/nightlies/
> >>
> >>
> >> @Kirk, it would be nice from you to look at this feature to see if it
> >> answers your initial request and give us some feedback.
> >>
> >> *Note that this feature may be subject to changes until next release, but
> >> getting some feedback would be very useful.
> >> *
> >>
> >> Regards
> >> Philippe M.
> >>
> >> On Sun, Jun 17, 2012 at 9:06 PM, sebb <se...@gmail.com> wrote:
> >>
> >> On 17 June 2012 13:57, Flavio Cysne <fl...@gmail.com> wrote:
> >> > Post entry in my blog:
> >> >
> >> http://flaviocysne.blogspot.com.br/2012/06/how-to-dynamically-increasedecrease.html
> >> >
> >> > It's an unfinished work, but it's suffice to make what is proposed.
> >> > Comments and constructive criticism are welcome.
> >>
> >> Thanks!
> >>
> >> The property used to control the thread count can also be set using
> >> the BeanShell server, see:
> >>
> >> http://jmeter.apache.org/usermanual/best-practices.html#beanshell_server
> >>
> >> Or since properties are global, you could run the property update
> >> sampler ("load controller") in a separate parallel thread group with
> >> one thread.
> >>
> >> However, JMeter will still have to start all the threads - even if
> >> they are not actively sampling - which will take up memory.
> >>
> >> It may not be too hard to change JMeter so it delays the thread
> >> creation until required.
> >> That may help in some use cases.
> >>
> >> But it would probably be somewhat harder to allow threads to be
> >> created later on demand.
> >> And converting to an event-based model is definitely a lot of work.
> >>
> >> > Hope it helps you.
> >> > Flávio Cysne
> >> >
> >> > 2012/6/15 Flavio Cysne <fl...@gmail.com>
> >> >
> >> >> Errata:
> >> >>
> >> >> The first "If Controller" is executed and update a property named
> >> >> "currentThreads" ...
> >> >>
> >> >>
> >> >> 2012/6/15 Flavio Cysne <fl...@gmail.com>
> >> >>
> >> >>> I have tested something in JMeter that could achieve a dynamic
> >> threading
> >> >>> model and there was no need of new implementations.
> >> >>>
> >> >>> Basically, I have this structure:
> >> >>>
> >> >>> 1. Test Plan: { variables : [ {name: "maxThreads", value: 5}, {name:
> >> >>> "currentThreads", value: 1} ] }
> >> >>> 2. | - Thread Group: { threads: "${maxThreads}", rampup: 0, infinite:
> >> >>> true }
> >> >>> 3.     | - If Controller: { condition: "(${__threadNum()} ===
> >> >>> ${maxThreads})" }
> >> >>> 4.         | - Sampler /* I've used HTTP Request, but anyone would be
> >> >>> used */
> >> >>> 5.             | - RegEx Extractor: {refName: "th
> >>
> >>
> >>
> >> --
> >> Cordialement.
> >> Philippe Mouawad.
> >>
> >>
> >>
> >>
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>



--

Regards,


Shay Ginsbourg

Regulatory & Testing Affairs Consultant


WWW.GINSBOURG.COM


Providing Regulatory, Medical & Performance Testing services since 2008:


* IEC 62304 Medical Device Software Life Cycle

* IEEE 829 Software Test Documentation

* ISO 14971 Medical Device Risk Management

* FDA 21 CFR Part 11 Software Validation

* IEC 60601-1:2005 3rd ED PEMS - Medical Electrical Equipment

* End-to-end verification, validation, and testing (VV&T)

* FDA and CE submissions

* Open source free testing tools implementation

* Functionality and regression testing

* Software Performance & Load testing

* Software Testing Advanced Automation

* Medical Software Verification & Validation

* Medical Device Verification & Validation

* Medical Device Regulatory Submission

* Organizational Regulatory Qualification


Formerly QA Manager of LoadRunner at Mercury Interactive


M.Sc. cum laude in Bio-Medical Engineering

M.Sc. in Mechanical Engineering


Work:   +972(0)3-5185873

Mobile:  +972(0)54-6690915


Email: sginsbourg@gmail.com


Visit my personal page on LinkedIn at: http://www.linkedin.com/in/shayginsbourg


Please consider your environmental responsibility before printing this e-mail.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by sebb <se...@gmail.com>.

Just to follow up - we have now implemented changes to the
TestCompiler and JavaSamplers to reduce the long-term memory storage
needed.

The memory requirement imposed by JMeter itself is now proportional to
the number of active threads; when a thread completes, its memory
should be released.

In conjunction with the Thread Group delayed thread creation option
and a long ramp-up, it's now possible to run a test with hundreds of
thousands of threads (possibly over a million), so long as the number
of concurrent (active) threads is kept to a reasonable number.

On 7 August 2012 08:54, Philippe Mouawad <ph...@gmail.com> wrote:
> Hello Kirk,
> Thank you very much for your feedback.
> Waiting for the big test result.
>
> Regards
> Philippe
>
> On Monday, August 6, 2012, Philippe Mouawad wrote:
>
>> From kirk (bad subject due to vacation message :) ) :
>>
>>
>>
>> Hi Phillippe,
>>
>> The on demand thread group worked brilliantly for my short test. I'll repeat with an 8 hour
>> run to give it an even better test and let you know how that worked.
>>
>> -- Kirk
>>
>>
>> ---------- Forwarded message ----------
>> From: *Kirk Pepperdine*
>> Date: Monday, August 6, 2012
>> Subject: JMeter threading model
>> To: Philippe Mouawad <ph...@gmail.com>
>> Cc: JMeter Users List <us...@jmeter.apache.org>
>>
>>
>> Hi Philippe,
>>
>> Just got back from vacation going through email and I noticed that I
>> missed this one. So I've now downloaded the nightly build and loaded a test
>> plan to check it out. Unfortunately the benchmark I decided to setup and
>> run against is borked. Well, I don't know if JMeter is the source of the
>> problem my recent upgrade to Mountain Lion (and what it did to Java). I'll
>> retry on an older machine where the bench should still behave as expected
>> and then see how this new ThreadGroup functions. My guess is that it should
>> be ok as it didn't complain as it would have using a regular ThreadGroup.
>> However, I would like to run a much much longer test to make sure. I'll
>> report back once this is done.
>>
>> Regards,
>> Kirk
>>
>> On 2012-07-14, at 3:36 PM, Philippe Mouawad <ph...@gmail.com>
>> wrote:
>>
>> Hello,
>> Just a little mail to inform you that this feature has been implemented as
>> part of:
>>
>>    - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418
>>
>> It is currently available in nightly build:
>>
>>    - http://ci.apache.org/projects/jmeter/nightlies/
>>
>>
>> @Kirk, it would be nice from you to look at this feature to see if it
>> answers your initial request and give us some feedback.
>>
>> *Note that this feature may be subject to changes until next release, but
>> getting some feedback would be very useful.
>> *
>>
>> Regards
>> Philippe M.
>>
>> On Sun, Jun 17, 2012 at 9:06 PM, sebb <se...@gmail.com> wrote:
>>
>> On 17 June 2012 13:57, Flavio Cysne <fl...@gmail.com> wrote:
>> > Post entry in my blog:
>> >
>> http://flaviocysne.blogspot.com.br/2012/06/how-to-dynamically-increasedecrease.html
>> >
>> > It's an unfinished work, but it's suffice to make what is proposed.
>> > Comments and constructive criticism are welcome.
>>
>> Thanks!
>>
>> The property used to control the thread count can also be set using
>> the BeanShell server, see:
>>
>> http://jmeter.apache.org/usermanual/best-practices.html#beanshell_server
>>
>> Or since properties are global, you could run the property update
>> sampler ("load controller") in a separate parallel thread group with
>> one thread.
>>
>> However, JMeter will still have to start all the threads - even if
>> they are not actively sampling - which will take up memory.
>>
>> It may not be too hard to change JMeter so it delays the thread
>> creation until required.
>> That may help in some use cases.
>>
>> But it would probably be somewhat harder to allow threads to be
>> created later on demand.
>> And converting to an event-based model is definitely a lot of work.
>>
>> > Hope it helps you.
>> > Flávio Cysne
>> >
>> > 2012/6/15 Flavio Cysne <fl...@gmail.com>
>> >
>> >> Errata:
>> >>
>> >> The first "If Controller" is executed and update a property named
>> >> "currentThreads" ...
>> >>
>> >>
>> >> 2012/6/15 Flavio Cysne <fl...@gmail.com>
>> >>
>> >>> I have tested something in JMeter that could achieve a dynamic
>> threading
>> >>> model and there was no need of new implementations.
>> >>>
>> >>> Basically, I have this structure:
>> >>>
>> >>> 1. Test Plan: { variables : [ {name: "maxThreads", value: 5}, {name:
>> >>> "currentThreads", value: 1} ] }
>> >>> 2. | - Thread Group: { threads: "${maxThreads}", rampup: 0, infinite:
>> >>> true }
>> >>> 3.     | - If Controller: { condition: "(${__threadNum()} ===
>> >>> ${maxThreads})" }
>> >>> 4.         | - Sampler /* I've used HTTP Request, but anyone would be
>> >>> used */
>> >>> 5.             | - RegEx Extractor: {refName: "th
>>
>>
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>>
>>
>>
>>
>
> --
> Cordialement.
> Philippe Mouawad.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by Philippe Mouawad <ph...@gmail.com>.

Hello Kirk,
Thank you very much for your feedback.
Waiting for the big test result.

Regards
Philippe

On Monday, August 6, 2012, Philippe Mouawad wrote:

> From kirk (bad subject due to vacation message :) ) :
>
>
>
> Hi Phillippe,
>
> The on demand thread group worked brilliantly for my short test. I'll repeat with an 8 hour
> run to give it an even better test and let you know how that worked.
>
> -- Kirk
>
>
> ---------- Forwarded message ----------
> From: *Kirk Pepperdine*
> Date: Monday, August 6, 2012
> Subject: JMeter threading model
> To: Philippe Mouawad <ph...@gmail.com>
> Cc: JMeter Users List <us...@jmeter.apache.org>
>
>
> Hi Philippe,
>
> Just got back from vacation going through email and I noticed that I
> missed this one. So I've now downloaded the nightly build and loaded a test
> plan to check it out. Unfortunately the benchmark I decided to setup and
> run against is borked. Well, I don't know if JMeter is the source of the
> problem my recent upgrade to Mountain Lion (and what it did to Java). I'll
> retry on an older machine where the bench should still behave as expected
> and then see how this new ThreadGroup functions. My guess is that it should
> be ok as it didn't complain as it would have using a regular ThreadGroup.
> However, I would like to run a much much longer test to make sure. I'll
> report back once this is done.
>
> Regards,
> Kirk
>
> On 2012-07-14, at 3:36 PM, Philippe Mouawad <ph...@gmail.com>
> wrote:
>
> Hello,
> Just a little mail to inform you that this feature has been implemented as
> part of:
>
>    - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418
>
> It is currently available in nightly build:
>
>    - http://ci.apache.org/projects/jmeter/nightlies/
>
>
> @Kirk, it would be nice from you to look at this feature to see if it
> answers your initial request and give us some feedback.
>
> *Note that this feature may be subject to changes until next release, but
> getting some feedback would be very useful.
> *
>
> Regards
> Philippe M.
>
> On Sun, Jun 17, 2012 at 9:06 PM, sebb <se...@gmail.com> wrote:
>
> On 17 June 2012 13:57, Flavio Cysne <fl...@gmail.com> wrote:
> > Post entry in my blog:
> >
> http://flaviocysne.blogspot.com.br/2012/06/how-to-dynamically-increasedecrease.html
> >
> > It's an unfinished work, but it's suffice to make what is proposed.
> > Comments and constructive criticism are welcome.
>
> Thanks!
>
> The property used to control the thread count can also be set using
> the BeanShell server, see:
>
> http://jmeter.apache.org/usermanual/best-practices.html#beanshell_server
>
> Or since properties are global, you could run the property update
> sampler ("load controller") in a separate parallel thread group with
> one thread.
>
> However, JMeter will still have to start all the threads - even if
> they are not actively sampling - which will take up memory.
>
> It may not be too hard to change JMeter so it delays the thread
> creation until required.
> That may help in some use cases.
>
> But it would probably be somewhat harder to allow threads to be
> created later on demand.
> And converting to an event-based model is definitely a lot of work.
>
> > Hope it helps you.
> > Flávio Cysne
> >
> > 2012/6/15 Flavio Cysne <fl...@gmail.com>
> >
> >> Errata:
> >>
> >> The first "If Controller" is executed and update a property named
> >> "currentThreads" ...
> >>
> >>
> >> 2012/6/15 Flavio Cysne <fl...@gmail.com>
> >>
> >>> I have tested something in JMeter that could achieve a dynamic
> threading
> >>> model and there was no need of new implementations.
> >>>
> >>> Basically, I have this structure:
> >>>
> >>> 1. Test Plan: { variables : [ {name: "maxThreads", value: 5}, {name:
> >>> "currentThreads", value: 1} ] }
> >>> 2. | - Thread Group: { threads: "${maxThreads}", rampup: 0, infinite:
> >>> true }
> >>> 3.     | - If Controller: { condition: "(${__threadNum()} ===
> >>> ${maxThreads})" }
> >>> 4.         | - Sampler /* I've used HTTP Request, but anyone would be
> >>> used */
> >>> 5.             | - RegEx Extractor: {refName: "th
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
>

-- 
Cordialement.
Philippe Mouawad.

Fwd: JMeter threading model

Posted by Philippe Mouawad <ph...@gmail.com>.

>From kirk (bad subject due to vacation message :) ) :

Hi Phillippe,

The on demand thread group worked brilliantly for my short test. I'll
repeat with an 8 hour
run to give it an even better test and let you know how that worked.

-- Kirk

---------- Forwarded message ----------
From: *Kirk Pepperdine*
Date: Monday, August 6, 2012
Subject: JMeter threading model
To: Philippe Mouawad <ph...@gmail.com>
Cc: JMeter Users List <us...@jmeter.apache.org>

Hi Philippe,

Just got back from vacation going through email and I noticed that I missed
this one. So I've now downloaded the nightly build and loaded a test plan
to check it out. Unfortunately the benchmark I decided to setup and run
against is borked. Well, I don't know if JMeter is the source of the
problem my recent upgrade to Mountain Lion (and what it did to Java). I'll
retry on an older machine where the bench should still behave as expected
and then see how this new ThreadGroup functions. My guess is that it should
be ok as it didn't complain as it would have using a regular ThreadGroup.
However, I would like to run a much much longer test to make sure. I'll
report back once this is done.

Regards,
Kirk

On 2012-07-14, at 3:36 PM, Philippe Mouawad <ph...@gmail.com>
wrote:

Hello,
Just a little mail to inform you that this feature has been implemented as
part of:

   - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418

It is currently available in nightly build:

   - http://ci.apache.org/projects/jmeter/nightlies/

@Kirk, it would be nice from you to look at this feature to see if it
answers your initial request and give us some feedback.

*Note that this feature may be subject to changes until next release, but
getting some feedback would be very useful.
*

Regards
Philippe M.

On Sun, Jun 17, 2012 at 9:06 PM, sebb <se...@gmail.com> wrote:

On 17 June 2012 13:57, Flavio Cysne <fl...@gmail.com> wrote:
> Post entry in my blog:
>
http://flaviocysne.blogspot.com.br/2012/06/how-to-dynamically-increasedecrease.html
>
> It's an unfinished work, but it's suffice to make what is proposed.
> Comments and constructive criticism are welcome.

Thanks!

The property used to control the thread count can also be set using
the BeanShell server, see:

http://jmeter.apache.org/usermanual/best-practices.html#beanshell_server

Or since properties are global, you could run the property update
sampler ("load controller") in a separate parallel thread group with
one thread.

However, JMeter will still have to start all the threads - even if
they are not actively sampling - which will take up memory.

It may not be too hard to change JMeter so it delays the thread
creation until required.
That may help in some use cases.

But it would probably be somewhat harder to allow threads to be
created later on demand.
And converting to an event-based model is definitely a lot of work.

> Hope it helps you.
> Flávio Cysne
>
> 2012/6/15 Flavio Cysne <fl...@gmail.com>
>
>> Errata:
>>
>> The first "If Controller" is executed and update a property named
>> "currentThreads" ...
>>
>>
>> 2012/6/15 Flavio Cysne <fl...@gmail.com>
>>
>>> I have tested something in JMeter that could achieve a dynamic threading
>>> model and there was no need of new implementations.
>>>
>>> Basically, I have this structure:
>>>
>>> 1. Test Plan: { variables : [ {name: "maxThreads", value: 5}, {name:
>>> "currentThreads", value: 1} ] }
>>> 2. | - Thread Group: { threads: "${maxThreads}", rampup: 0, infinite:
>>> true }
>>> 3.     | - If Controller: { condition: "(${__threadNum()} ===
>>> ${maxThreads})" }
>>> 4.         | - Sampler /* I've used HTTP Request, but anyone would be
>>> used */
>>> 5.             | - RegEx Extractor: {refName: "threads", expression:
>>> "(.*)", model: "$1$", matchNo: "1", defaultValue: "0"}
>>> 6.             | - BeanShell Post-Processor: {language: "javascript",
>>> script: "var threads =
vars.get(\"threads\");\nprops.put(\"currentThreads
>>> \",threads);"}
>>> 7.     | - If Controller: { condition: "(${__threadNum()} <
>>> ${maxThreads})" }
>>> 8.         | - If Controller: { condition: "(${__threadNum()} <= ${__P(
>>> currentThreads,0)})" }
>>> 9.             | - All other Samplers go here
>>>
>>>
>>> The idea is starting the maximum numbe

-- 
Cordialement.
Philippe Mouawad.

Re: JMeter threading model

Posted by Kirk Pepperdine <ki...@gmail.com>.

Hi Philippe,

Just got back from vacation going through email and I noticed that I missed this one. So I've now downloaded the nightly build and loaded a test plan to check it out. Unfortunately the benchmark I decided to setup and run against is borked. Well, I don't know if JMeter is the source of the problem my recent upgrade to Mountain Lion (and what it did to Java). I'll retry on an older machine where the bench should still behave as expected and then see how this new ThreadGroup functions. My guess is that it should be ok as it didn't complain as it would have using a regular ThreadGroup. However, I would like to run a much much longer test to make sure. I'll report back once this is done.

Regards,
Kirk

On 2012-07-14, at 3:36 PM, Philippe Mouawad <ph...@gmail.com> wrote:

> Hello,
> Just a little mail to inform you that this feature has been implemented as part of:
> https://issues.apache.org/bugzilla/show_bug.cgi?id=53418
> It is currently available in nightly build:
> http://ci.apache.org/projects/jmeter/nightlies/
> 
> @Kirk, it would be nice from you to look at this feature to see if it answers your initial request and give us some feedback.
> 
> Note that this feature may be subject to changes until next release, but getting some feedback would be very useful.
> 
> 
> Regards
> Philippe M.
> 
> On Sun, Jun 17, 2012 at 9:06 PM, sebb <se...@gmail.com> wrote:
> On 17 June 2012 13:57, Flavio Cysne <fl...@gmail.com> wrote:
> > Post entry in my blog:
> > http://flaviocysne.blogspot.com.br/2012/06/how-to-dynamically-increasedecrease.html
> >
> > It's an unfinished work, but it's suffice to make what is proposed.
> > Comments and constructive criticism are welcome.
> 
> Thanks!
> 
> The property used to control the thread count can also be set using
> the BeanShell server, see:
> 
> http://jmeter.apache.org/usermanual/best-practices.html#beanshell_server
> 
> Or since properties are global, you could run the property update
> sampler ("load controller") in a separate parallel thread group with
> one thread.
> 
> However, JMeter will still have to start all the threads - even if
> they are not actively sampling - which will take up memory.
> 
> It may not be too hard to change JMeter so it delays the thread
> creation until required.
> That may help in some use cases.
> 
> But it would probably be somewhat harder to allow threads to be
> created later on demand.
> And converting to an event-based model is definitely a lot of work.
> 
> > Hope it helps you.
> > Flávio Cysne
> >
> > 2012/6/15 Flavio Cysne <fl...@gmail.com>
> >
> >> Errata:
> >>
> >> The first "If Controller" is executed and update a property named
> >> "currentThreads" ...
> >>
> >>
> >> 2012/6/15 Flavio Cysne <fl...@gmail.com>
> >>
> >>> I have tested something in JMeter that could achieve a dynamic threading
> >>> model and there was no need of new implementations.
> >>>
> >>> Basically, I have this structure:
> >>>
> >>> 1. Test Plan: { variables : [ {name: "maxThreads", value: 5}, {name:
> >>> "currentThreads", value: 1} ] }
> >>> 2. | - Thread Group: { threads: "${maxThreads}", rampup: 0, infinite:
> >>> true }
> >>> 3.     | - If Controller: { condition: "(${__threadNum()} ===
> >>> ${maxThreads})" }
> >>> 4.         | - Sampler /* I've used HTTP Request, but anyone would be
> >>> used */
> >>> 5.             | - RegEx Extractor: {refName: "threads", expression:
> >>> "(.*)", model: "$1$", matchNo: "1", defaultValue: "0"}
> >>> 6.             | - BeanShell Post-Processor: {language: "javascript",
> >>> script: "var threads = vars.get(\"threads\");\nprops.put(\"currentThreads
> >>> \",threads);"}
> >>> 7.     | - If Controller: { condition: "(${__threadNum()} <
> >>> ${maxThreads})" }
> >>> 8.         | - If Controller: { condition: "(${__threadNum()} <= ${__P(
> >>> currentThreads,0)})" }
> >>> 9.             | - All other Samplers go here
> >>>
> >>>
> >>> The idea is starting the maximum number of desired threads at the
> >>> beginning but only use what you want.
> >>> The first "If Controller" is executed and update a property named
> >>> "threads" with the value retrieved from a response (HTTP, SOAP, file,
> >>> whatever JMeter can access)
> >>> When the first loop iteration is done line 9 will be skipped because
> >>> ${__P(currentThreads,0)} will return 0. At the same iteration, lines 4
> >>> through 6 will be executed and will update the "currentThreads" property
> >>> using the value in "threads" variable.
> >>> After the first iteration, if the "currentThreads" property has been
> >>> updated, line 9 will be executed using only the number of threads
> >>> of "currentThreads" property.
> >>> To increase/decrease the "currentThreads" property all you have to do is
> >>> edit the file accessed by the Sampler at line 4 and wait for the next
> >>> iteration to see the new "threading model" running.
> >>>
> >>> The approach used by distributed testing and this pseudo-dynamic
> >>> threading model will include ${__machineName()} or ${__machineIP()} in "If
> >>> Controller" conditions.
> >>>
> >>> I'll try to make a entry in my blog to explain in detail this scenario.
> >>>
> >>> I used JMeter 2.7 and you can download jmx file here<https://docs.google.com/open?id=0Bwr5j-BqUUDobnFTSHc1eTltdmM>
> >>> .
> >>>
> >>> Hope it helps you.
> >>> Flávio Cysne
> >>>
> >>> 2012/6/15 D G <dm...@gmail.com>
> >>>
> >>>> Details are very welcome as I'm still not sure whether I fully
> >>>> understand the problem :)
> >>>>
> >>>> Thread pooling does sound nice and it might enable a feature I'd like:
> >>>>
> >>>> On-demand creation of threads. Say you're looping 10 threads, system
> >>>> is under no load.
> >>>> Currently: if you want 10 more users you need to stop the test, add 10
> >>>> users and start it again.
> >>>>
> >>>> It would be nice if it where possible to simply say '+10', followed by
> >>>> some monitoring and then another '+10' (30 threads running total) etc.
> >>>> (10 is an example number, +/- n users would be the goal)
> >>>> Mostly for the 'getting a feel for the system' part of the test, the
> >>>> initial setup of it all.
> >>>>
> >>>> Regards,
> >>>> Daniel
> >>>>
> >>>> On Fri, Jun 15, 2012 at 7:24 AM, Kirk Pepperdine
> >>>> <ki...@gmail.com> wrote:
> >>>> > Hi,
> >>>> >
> >>>> > I'm very happy to add details if need be.
> >>>> >
> >>>> > Regards,
> >>>> > Kirk
> >>>> >
> >>>> > On 2012-06-15, at 12:13 AM, Philippe Mouawad wrote:
> >>>> >
> >>>> >> Bugzilla created:
> >>>> >>
> >>>> >>   - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418
> >>>> >>
> >>>> >>
> >>>> >> Regards
> >>>> >>
> >>>> >> Philippe M.
> >>>> >>
> >>>> >> On Thu, Jun 14, 2012 at 10:30 PM, Philippe Mouawad <
> >>>> >> philippe.mouawad@gmail.com> wrote:
> >>>> >>
> >>>> >>> Hi M. Pepperdine,
> >>>> >>>
> >>>> >>> On Wed, Jun 13, 2012 at 7:05 AM, Kirk Pepperdine <
> >>>> >>> kirk.pepperdine@gmail.com> wrote:
> >>>> >>>
> >>>> >>>> Hi Sebb,
> >>>> >>>>
> >>>> >>>> We've had this conversation before and I did some preliminary work
> >>>> to
> >>>> >>>> setup a different type of thread group but the couplings between the
> >>>> >>>> existing thread group and the model meant that an extensive
> >>>> refactoring
> >>>> >>>> would be involved. Since that involves a *lot* more than just a
> >>>> simple
> >>>> >>>> plugin...
> >>>> >>>>
> >>>> >>>> So, the current implementation supports a closed system model
> >>>> meaning,
> >>>> >>>> rate of entry into the system equals rate of exit from the
> >>>> system.This is
> >>>> >>>> exactly what you want if you're load testing a call centre where
> >>>> the number
> >>>> >>>> of servers (operators) is fixed and gate entry into the system.
> >>>> However,
> >>>> >>>> I'm often simulating open systems which means I do not want rate of
> >>>> entry
> >>>> >>>> into the system to be controlled by the performance of the system
> >>>> (rate of
> >>>> >>>> exit).
> >>>> >>>
> >>>> >>> What makes you think JMeter does this ?
> >>>> >>>
> >>>> >>>
> >>>> >>>> More over, those that attend my performance tuning seminars come to
> >>>> >>>> understand why this is an important aspect of getting your test
> >>>> environment
> >>>> >>>> right and test harness correctly setup as it can adversely affect
> >>>> the
> >>>> >>>> quality of your test which can and often does, change the results
> >>>> of the
> >>>> >>>> test.
> >>>> >>>>
> >>>> >>>
> >>>> >>>> As an example, today I will show a group how to tune an application
> >>>> by a
> >>>> >>>> partner company. That application has a number of "performance
> >>>> problems"
> >>>> >>>> backed into it. If I use the traditional means of using JMeter I
> >>>> will find
> >>>> >>>> a different set of performance issues than if I load with a pattern
> >>>> that is
> >>>> >>>> similar that found in production.
> >>>> >>>
> >>>> >>> Can you clarify this point ? a figure might be better than a long
> >>>> text
> >>>> >>>
> >>>> >>>
> >>>> >>>> In other words, with this particular application, JMeter exposes
> >>>> >>>> "problems" that are artifacts of how it wants to inject load on a
> >>>> system.
> >>>> >>>
> >>>> >>> Not clear for me.
> >>>> >>>
> >>>> >>> I can fix all of these problems
> >>>> >>>
> >>>> >>> What are these problems ? and how do you fix these ?
> >>>> >>>
> >>>> >>>
> >>>> >>>> and eventually I'll get to a point where I'll fix everything that
> >>>> needs
> >>>> >>>> to be fixed. That said, if I can coerce JMeter to load as an open
> >>>> system
> >>>> >>>> I'll get to the problems without having to fix the artifacts (the
> >>>> things
> >>>> >>>> that really don't need fixing).
> >>>> >>>
> >>>> >>> Still not clear
> >>>> >>>
> >>>> >>>> To coerce JMeter into being an open system requires one to use a
> >>>> large
> >>>> >>>> number of very short lived threads. So I may only have 400-500
> >>>> active
> >>>> >>>> threads at any point in time but in order to achieve that load over
> >>>> a 1 or
> >>>> >>>> two hour test I may have to specify 10s of thousands of threads.
> >>>> Since all
> >>>> >>>> of the threads are created up front, this simply doesn't work.
> >>>> >>>>
> >>>> >>>> You might ask why not just specify 400-500 threads and loop over
> >>>> them? in
> >>>> >>>> theory you'd think it would work but as you tune the system and the
> >>>> >>>> performance characteristics change. Going back to the baked
> >>>> application,
> >>>> >>>> before I start tuning, the active user count is several thousand.
> >>>> In other
> >>>> >>>> words, the tuned system is better able to clear users out and that
> >>>> changes
> >>>> >>>> the performance profile in a way that hard to emulate with the
> >>>> current
> >>>> >>>> looping behaviour. Using a setting of looping 400 or so threads
> >>>> isn't
> >>>> >>>> adequate for the initial load tests as the test harness will become
> >>>> thread
> >>>> >>>> starved and that releases pressure on the server which in turn
> >>>> changes the
> >>>> >>>> performance profile.
> >>>> >>>>
> >>>> >>>
> >>>> >>>
> >>>> >>>> With all due respect to the wonderful work that everyone on the
> >>>> project
> >>>> >>>> has done, it is my opinion that the one user == one thread is a
> >>>> design
> >>>> >>>> mistake that has a huge impact on both the usability of the tool
> >>>> >>>>
> >>>> >>> Examples ?
> >>>> >>>
> >>>> >>>> and the quality of the results
> >>>> >>>
> >>>> >>> I disagree with this assertion . We have been using JMeter for load
> >>>> >>> testing all kind of applications Intranets / Large ECommerce Systems
> >>>> /
> >>>> >>> Backoffice systems / , and quality of results is good provided you
> >>>> >>> configure it properly.
> >>>> >>> Particularly when using Remote  Testing. Lot of users in this
> >>>> mailing list
> >>>> >>> use it and are satisfied (I think).
> >>>> >>>
> >>>> >>>
> >>>> >>>> one can achieve when using it. IMHO, moving to an thread pool/event
> >>>> heap
> >>>> >>>> based model would be an enormous improvement over the current
> >>>> >>>> implementation.
> >>>> >>>>
> >>>> >>>> Agree it would be better. We will work on it.
> >>>> >>>
> >>>> >>>> Regards,
> >>>> >>>> Kirk
> >>>> >>>>
> >>>> >>>> On 2012-06-13, at 1:02 AM, sebb wrote:
> >>>> >>>>
> >>>> >>>>> On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com>
> >>>> >>>> wrote:
> >>>> >>>>>>
> >>>> >>>>>> On 2012-06-13, at 12:54 AM, sebb wrote:
> >>>> >>>>>>
> >>>> >>>>>>> On 12 June 2012 22:06, Kirk Pepperdine <
> >>>> kirk.pepperdine@gmail.com>
> >>>> >>>> wrote:
> >>>> >>>>>>>> Hi,
> >>>> >>>>>>>>
> >>>> >>>>>>>> I figured thread pooling would be revolutionary so I wasn't
> >>>> >>>> suggesting that. I would be very useful just delay the creation of
> >>>> a thread
> >>>> >>>> until it was asked for.
> >>>> >>>>>>>
> >>>> >>>>>>> Not sure I understand how it would help to delay the thread
> >>>> creation,
> >>>> >>>>>>> except perhaps for the case where the first threads have finished
> >>>> >>>>>>> processing by the time the last threads start running samples.
> >>>> >>>>>>
> >>>> >>>>>> Bingo!!! ;-)
> >>>> >>>>>
> >>>> >>>>> So what percentage of use cases need to follow this model?
> >>>> >>>>>
> >>>> >>>>> Most of the JMeter testing I have done was long running tests where
> >>>> >>>>> all threads were active for most of the run.
> >>>> >>>>>
> >>>> >>>>>> Kirk
> >>>> >>>>>>
> >>>> >>>>>>
> >>>> >>>>>>
> >>>> ---------------------------------------------------------------------
> >>>> >>>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >>>> >>>>>> For additional commands, e-mail: user-help@jmeter.apache.org
> >>>> >>>>>>
> >>>> >>>>>
> >>>> >>>>>
> >>>> ---------------------------------------------------------------------
> >>>> >>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >>>> >>>>> For additional commands, e-mail: user-help@jmeter.apache.org
> >>>> >>>>>
> >>>> >>>>
> >>>> >>>>
> >>>> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> >>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >>>> >>>> For additional commands, e-mail: user-help@jmeter.apache.org
> >>>> >>>>
> >>>> >>>>
> >>>> >>>
> >>>> >>>
> >>>> >>> --
> >>>> >>> Cordialement.
> >>>> >>> Philippe Mouawad.
> >>>> >>>
> >>>> >>>
> >>>> >>>
> >>>> >>>
> >>>> >>
> >>>> >>
> >>>> >> --
> >>>> >> Cordialement.
> >>>> >> Philippe Mouawad.
> >>>> >
> >>>> >
> >>>> > ---------------------------------------------------------------------
> >>>> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >>>> > For additional commands, e-mail: user-help@jmeter.apache.org
> >>>> >
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >>>> For additional commands, e-mail: user-help@jmeter.apache.org
> >>>>
> >>>>
> >>>
> >>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
> 
> 
> 
> 
> -- 
> Cordialement.
> Philippe Mouawad.
> 
> 
>

Re: JMeter threading model

Posted by Philippe Mouawad <ph...@gmail.com>.

Hello,
Just a little mail to inform you that this feature has been implemented as
part of:

   - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418

It is currently available in nightly build:

   - http://ci.apache.org/projects/jmeter/nightlies/


@Kirk, it would be nice from you to look at this feature to see if it
answers your initial request and give us some feedback.

*Note that this feature may be subject to changes until next release, but
getting some feedback would be very useful.
*

Regards
Philippe M.

On Sun, Jun 17, 2012 at 9:06 PM, sebb <se...@gmail.com> wrote:

> On 17 June 2012 13:57, Flavio Cysne <fl...@gmail.com> wrote:
> > Post entry in my blog:
> >
> http://flaviocysne.blogspot.com.br/2012/06/how-to-dynamically-increasedecrease.html
> >
> > It's an unfinished work, but it's suffice to make what is proposed.
> > Comments and constructive criticism are welcome.
>
> Thanks!
>
> The property used to control the thread count can also be set using
> the BeanShell server, see:
>
> http://jmeter.apache.org/usermanual/best-practices.html#beanshell_server
>
> Or since properties are global, you could run the property update
> sampler ("load controller") in a separate parallel thread group with
> one thread.
>
> However, JMeter will still have to start all the threads - even if
> they are not actively sampling - which will take up memory.
>
> It may not be too hard to change JMeter so it delays the thread
> creation until required.
> That may help in some use cases.
>
> But it would probably be somewhat harder to allow threads to be
> created later on demand.
> And converting to an event-based model is definitely a lot of work.
>
> > Hope it helps you.
> > Flávio Cysne
> >
> > 2012/6/15 Flavio Cysne <fl...@gmail.com>
> >
> >> Errata:
> >>
> >> The first "If Controller" is executed and update a property named
> >> "currentThreads" ...
> >>
> >>
> >> 2012/6/15 Flavio Cysne <fl...@gmail.com>
> >>
> >>> I have tested something in JMeter that could achieve a dynamic
> threading
> >>> model and there was no need of new implementations.
> >>>
> >>> Basically, I have this structure:
> >>>
> >>> 1. Test Plan: { variables : [ {name: "maxThreads", value: 5}, {name:
> >>> "currentThreads", value: 1} ] }
> >>> 2. | - Thread Group: { threads: "${maxThreads}", rampup: 0, infinite:
> >>> true }
> >>> 3.     | - If Controller: { condition: "(${__threadNum()} ===
> >>> ${maxThreads})" }
> >>> 4.         | - Sampler /* I've used HTTP Request, but anyone would be
> >>> used */
> >>> 5.             | - RegEx Extractor: {refName: "threads", expression:
> >>> "(.*)", model: "$1$", matchNo: "1", defaultValue: "0"}
> >>> 6.             | - BeanShell Post-Processor: {language: "javascript",
> >>> script: "var threads =
> vars.get(\"threads\");\nprops.put(\"currentThreads
> >>> \",threads);"}
> >>> 7.     | - If Controller: { condition: "(${__threadNum()} <
> >>> ${maxThreads})" }
> >>> 8.         | - If Controller: { condition: "(${__threadNum()} <= ${__P(
> >>> currentThreads,0)})" }
> >>> 9.             | - All other Samplers go here
> >>>
> >>>
> >>> The idea is starting the maximum number of desired threads at the
> >>> beginning but only use what you want.
> >>> The first "If Controller" is executed and update a property named
> >>> "threads" with the value retrieved from a response (HTTP, SOAP, file,
> >>> whatever JMeter can access)
> >>> When the first loop iteration is done line 9 will be skipped because
> >>> ${__P(currentThreads,0)} will return 0. At the same iteration, lines 4
> >>> through 6 will be executed and will update the "currentThreads"
> property
> >>> using the value in "threads" variable.
> >>> After the first iteration, if the "currentThreads" property has been
> >>> updated, line 9 will be executed using only the number of threads
> >>> of "currentThreads" property.
> >>> To increase/decrease the "currentThreads" property all you have to do
> is
> >>> edit the file accessed by the Sampler at line 4 and wait for the next
> >>> iteration to see the new "threading model" running.
> >>>
> >>> The approach used by distributed testing and this pseudo-dynamic
> >>> threading model will include ${__machineName()} or ${__machineIP()} in
> "If
> >>> Controller" conditions.
> >>>
> >>> I'll try to make a entry in my blog to explain in detail this scenario.
> >>>
> >>> I used JMeter 2.7 and you can download jmx file here<
> https://docs.google.com/open?id=0Bwr5j-BqUUDobnFTSHc1eTltdmM>
> >>> .
> >>>
> >>> Hope it helps you.
> >>> Flávio Cysne
> >>>
> >>> 2012/6/15 D G <dm...@gmail.com>
> >>>
> >>>> Details are very welcome as I'm still not sure whether I fully
> >>>> understand the problem :)
> >>>>
> >>>> Thread pooling does sound nice and it might enable a feature I'd like:
> >>>>
> >>>> On-demand creation of threads. Say you're looping 10 threads, system
> >>>> is under no load.
> >>>> Currently: if you want 10 more users you need to stop the test, add 10
> >>>> users and start it again.
> >>>>
> >>>> It would be nice if it where possible to simply say '+10', followed by
> >>>> some monitoring and then another '+10' (30 threads running total) etc.
> >>>> (10 is an example number, +/- n users would be the goal)
> >>>> Mostly for the 'getting a feel for the system' part of the test, the
> >>>> initial setup of it all.
> >>>>
> >>>> Regards,
> >>>> Daniel
> >>>>
> >>>> On Fri, Jun 15, 2012 at 7:24 AM, Kirk Pepperdine
> >>>> <ki...@gmail.com> wrote:
> >>>> > Hi,
> >>>> >
> >>>> > I'm very happy to add details if need be.
> >>>> >
> >>>> > Regards,
> >>>> > Kirk
> >>>> >
> >>>> > On 2012-06-15, at 12:13 AM, Philippe Mouawad wrote:
> >>>> >
> >>>> >> Bugzilla created:
> >>>> >>
> >>>> >>   - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418
> >>>> >>
> >>>> >>
> >>>> >> Regards
> >>>> >>
> >>>> >> Philippe M.
> >>>> >>
> >>>> >> On Thu, Jun 14, 2012 at 10:30 PM, Philippe Mouawad <
> >>>> >> philippe.mouawad@gmail.com> wrote:
> >>>> >>
> >>>> >>> Hi M. Pepperdine,
> >>>> >>>
> >>>> >>> On Wed, Jun 13, 2012 at 7:05 AM, Kirk Pepperdine <
> >>>> >>> kirk.pepperdine@gmail.com> wrote:
> >>>> >>>
> >>>> >>>> Hi Sebb,
> >>>> >>>>
> >>>> >>>> We've had this conversation before and I did some preliminary
> work
> >>>> to
> >>>> >>>> setup a different type of thread group but the couplings between
> the
> >>>> >>>> existing thread group and the model meant that an extensive
> >>>> refactoring
> >>>> >>>> would be involved. Since that involves a *lot* more than just a
> >>>> simple
> >>>> >>>> plugin...
> >>>> >>>>
> >>>> >>>> So, the current implementation supports a closed system model
> >>>> meaning,
> >>>> >>>> rate of entry into the system equals rate of exit from the
> >>>> system.This is
> >>>> >>>> exactly what you want if you're load testing a call centre where
> >>>> the number
> >>>> >>>> of servers (operators) is fixed and gate entry into the system.
> >>>> However,
> >>>> >>>> I'm often simulating open systems which means I do not want rate
> of
> >>>> entry
> >>>> >>>> into the system to be controlled by the performance of the system
> >>>> (rate of
> >>>> >>>> exit).
> >>>> >>>
> >>>> >>> What makes you think JMeter does this ?
> >>>> >>>
> >>>> >>>
> >>>> >>>> More over, those that attend my performance tuning seminars come
> to
> >>>> >>>> understand why this is an important aspect of getting your test
> >>>> environment
> >>>> >>>> right and test harness correctly setup as it can adversely affect
> >>>> the
> >>>> >>>> quality of your test which can and often does, change the results
> >>>> of the
> >>>> >>>> test.
> >>>> >>>>
> >>>> >>>
> >>>> >>>> As an example, today I will show a group how to tune an
> application
> >>>> by a
> >>>> >>>> partner company. That application has a number of "performance
> >>>> problems"
> >>>> >>>> backed into it. If I use the traditional means of using JMeter I
> >>>> will find
> >>>> >>>> a different set of performance issues than if I load with a
> pattern
> >>>> that is
> >>>> >>>> similar that found in production.
> >>>> >>>
> >>>> >>> Can you clarify this point ? a figure might be better than a long
> >>>> text
> >>>> >>>
> >>>> >>>
> >>>> >>>> In other words, with this particular application, JMeter exposes
> >>>> >>>> "problems" that are artifacts of how it wants to inject load on a
> >>>> system.
> >>>> >>>
> >>>> >>> Not clear for me.
> >>>> >>>
> >>>> >>> I can fix all of these problems
> >>>> >>>
> >>>> >>> What are these problems ? and how do you fix these ?
> >>>> >>>
> >>>> >>>
> >>>> >>>> and eventually I'll get to a point where I'll fix everything that
> >>>> needs
> >>>> >>>> to be fixed. That said, if I can coerce JMeter to load as an open
> >>>> system
> >>>> >>>> I'll get to the problems without having to fix the artifacts (the
> >>>> things
> >>>> >>>> that really don't need fixing).
> >>>> >>>
> >>>> >>> Still not clear
> >>>> >>>
> >>>> >>>> To coerce JMeter into being an open system requires one to use a
> >>>> large
> >>>> >>>> number of very short lived threads. So I may only have 400-500
> >>>> active
> >>>> >>>> threads at any point in time but in order to achieve that load
> over
> >>>> a 1 or
> >>>> >>>> two hour test I may have to specify 10s of thousands of threads.
> >>>> Since all
> >>>> >>>> of the threads are created up front, this simply doesn't work.
> >>>> >>>>
> >>>> >>>> You might ask why not just specify 400-500 threads and loop over
> >>>> them? in
> >>>> >>>> theory you'd think it would work but as you tune the system and
> the
> >>>> >>>> performance characteristics change. Going back to the baked
> >>>> application,
> >>>> >>>> before I start tuning, the active user count is several thousand.
> >>>> In other
> >>>> >>>> words, the tuned system is better able to clear users out and
> that
> >>>> changes
> >>>> >>>> the performance profile in a way that hard to emulate with the
> >>>> current
> >>>> >>>> looping behaviour. Using a setting of looping 400 or so threads
> >>>> isn't
> >>>> >>>> adequate for the initial load tests as the test harness will
> become
> >>>> thread
> >>>> >>>> starved and that releases pressure on the server which in turn
> >>>> changes the
> >>>> >>>> performance profile.
> >>>> >>>>
> >>>> >>>
> >>>> >>>
> >>>> >>>> With all due respect to the wonderful work that everyone on the
> >>>> project
> >>>> >>>> has done, it is my opinion that the one user == one thread is a
> >>>> design
> >>>> >>>> mistake that has a huge impact on both the usability of the tool
> >>>> >>>>
> >>>> >>> Examples ?
> >>>> >>>
> >>>> >>>> and the quality of the results
> >>>> >>>
> >>>> >>> I disagree with this assertion . We have been using JMeter for
> load
> >>>> >>> testing all kind of applications Intranets / Large ECommerce
> Systems
> >>>> /
> >>>> >>> Backoffice systems / , and quality of results is good provided you
> >>>> >>> configure it properly.
> >>>> >>> Particularly when using Remote  Testing. Lot of users in this
> >>>> mailing list
> >>>> >>> use it and are satisfied (I think).
> >>>> >>>
> >>>> >>>
> >>>> >>>> one can achieve when using it. IMHO, moving to an thread
> pool/event
> >>>> heap
> >>>> >>>> based model would be an enormous improvement over the current
> >>>> >>>> implementation.
> >>>> >>>>
> >>>> >>>> Agree it would be better. We will work on it.
> >>>> >>>
> >>>> >>>> Regards,
> >>>> >>>> Kirk
> >>>> >>>>
> >>>> >>>> On 2012-06-13, at 1:02 AM, sebb wrote:
> >>>> >>>>
> >>>> >>>>> On 12 June 2012 22:57, Kirk Pepperdine <
> kirk.pepperdine@gmail.com>
> >>>> >>>> wrote:
> >>>> >>>>>>
> >>>> >>>>>> On 2012-06-13, at 12:54 AM, sebb wrote:
> >>>> >>>>>>
> >>>> >>>>>>> On 12 June 2012 22:06, Kirk Pepperdine <
> >>>> kirk.pepperdine@gmail.com>
> >>>> >>>> wrote:
> >>>> >>>>>>>> Hi,
> >>>> >>>>>>>>
> >>>> >>>>>>>> I figured thread pooling would be revolutionary so I wasn't
> >>>> >>>> suggesting that. I would be very useful just delay the creation
> of
> >>>> a thread
> >>>> >>>> until it was asked for.
> >>>> >>>>>>>
> >>>> >>>>>>> Not sure I understand how it would help to delay the thread
> >>>> creation,
> >>>> >>>>>>> except perhaps for the case where the first threads have
> finished
> >>>> >>>>>>> processing by the time the last threads start running samples.
> >>>> >>>>>>
> >>>> >>>>>> Bingo!!! ;-)
> >>>> >>>>>
> >>>> >>>>> So what percentage of use cases need to follow this model?
> >>>> >>>>>
> >>>> >>>>> Most of the JMeter testing I have done was long running tests
> where
> >>>> >>>>> all threads were active for most of the run.
> >>>> >>>>>
> >>>> >>>>>> Kirk
> >>>> >>>>>>
> >>>> >>>>>>
> >>>> >>>>>>
> >>>> ---------------------------------------------------------------------
> >>>> >>>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >>>> >>>>>> For additional commands, e-mail: user-help@jmeter.apache.org
> >>>> >>>>>>
> >>>> >>>>>
> >>>> >>>>>
> >>>> ---------------------------------------------------------------------
> >>>> >>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >>>> >>>>> For additional commands, e-mail: user-help@jmeter.apache.org
> >>>> >>>>>
> >>>> >>>>
> >>>> >>>>
> >>>> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> >>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >>>> >>>> For additional commands, e-mail: user-help@jmeter.apache.org
> >>>> >>>>
> >>>> >>>>
> >>>> >>>
> >>>> >>>
> >>>> >>> --
> >>>> >>> Cordialement.
> >>>> >>> Philippe Mouawad.
> >>>> >>>
> >>>> >>>
> >>>> >>>
> >>>> >>>
> >>>> >>
> >>>> >>
> >>>> >> --
> >>>> >> Cordialement.
> >>>> >> Philippe Mouawad.
> >>>> >
> >>>> >
> >>>> >
> ---------------------------------------------------------------------
> >>>> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >>>> > For additional commands, e-mail: user-help@jmeter.apache.org
> >>>> >
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >>>> For additional commands, e-mail: user-help@jmeter.apache.org
> >>>>
> >>>>
> >>>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>


-- 
Cordialement.
Philippe Mouawad.

Re: JMeter threading model

Posted by sebb <se...@gmail.com>.

On 17 June 2012 13:57, Flavio Cysne <fl...@gmail.com> wrote:
> Post entry in my blog:
> http://flaviocysne.blogspot.com.br/2012/06/how-to-dynamically-increasedecrease.html
>
> It's an unfinished work, but it's suffice to make what is proposed.
> Comments and constructive criticism are welcome.

Thanks!

The property used to control the thread count can also be set using
the BeanShell server, see:

http://jmeter.apache.org/usermanual/best-practices.html#beanshell_server

Or since properties are global, you could run the property update
sampler ("load controller") in a separate parallel thread group with
one thread.

However, JMeter will still have to start all the threads - even if
they are not actively sampling - which will take up memory.

It may not be too hard to change JMeter so it delays the thread
creation until required.
That may help in some use cases.

But it would probably be somewhat harder to allow threads to be
created later on demand.
And converting to an event-based model is definitely a lot of work.

> Hope it helps you.
> Flávio Cysne
>
> 2012/6/15 Flavio Cysne <fl...@gmail.com>
>
>> Errata:
>>
>> The first "If Controller" is executed and update a property named
>> "currentThreads" ...
>>
>>
>> 2012/6/15 Flavio Cysne <fl...@gmail.com>
>>
>>> I have tested something in JMeter that could achieve a dynamic threading
>>> model and there was no need of new implementations.
>>>
>>> Basically, I have this structure:
>>>
>>> 1. Test Plan: { variables : [ {name: "maxThreads", value: 5}, {name:
>>> "currentThreads", value: 1} ] }
>>> 2. | - Thread Group: { threads: "${maxThreads}", rampup: 0, infinite:
>>> true }
>>> 3.     | - If Controller: { condition: "(${__threadNum()} ===
>>> ${maxThreads})" }
>>> 4.         | - Sampler /* I've used HTTP Request, but anyone would be
>>> used */
>>> 5.             | - RegEx Extractor: {refName: "threads", expression:
>>> "(.*)", model: "$1$", matchNo: "1", defaultValue: "0"}
>>> 6.             | - BeanShell Post-Processor: {language: "javascript",
>>> script: "var threads = vars.get(\"threads\");\nprops.put(\"currentThreads
>>> \",threads);"}
>>> 7.     | - If Controller: { condition: "(${__threadNum()} <
>>> ${maxThreads})" }
>>> 8.         | - If Controller: { condition: "(${__threadNum()} <= ${__P(
>>> currentThreads,0)})" }
>>> 9.             | - All other Samplers go here
>>>
>>>
>>> The idea is starting the maximum number of desired threads at the
>>> beginning but only use what you want.
>>> The first "If Controller" is executed and update a property named
>>> "threads" with the value retrieved from a response (HTTP, SOAP, file,
>>> whatever JMeter can access)
>>> When the first loop iteration is done line 9 will be skipped because
>>> ${__P(currentThreads,0)} will return 0. At the same iteration, lines 4
>>> through 6 will be executed and will update the "currentThreads" property
>>> using the value in "threads" variable.
>>> After the first iteration, if the "currentThreads" property has been
>>> updated, line 9 will be executed using only the number of threads
>>> of "currentThreads" property.
>>> To increase/decrease the "currentThreads" property all you have to do is
>>> edit the file accessed by the Sampler at line 4 and wait for the next
>>> iteration to see the new "threading model" running.
>>>
>>> The approach used by distributed testing and this pseudo-dynamic
>>> threading model will include ${__machineName()} or ${__machineIP()} in "If
>>> Controller" conditions.
>>>
>>> I'll try to make a entry in my blog to explain in detail this scenario.
>>>
>>> I used JMeter 2.7 and you can download jmx file here<https://docs.google.com/open?id=0Bwr5j-BqUUDobnFTSHc1eTltdmM>
>>> .
>>>
>>> Hope it helps you.
>>> Flávio Cysne
>>>
>>> 2012/6/15 D G <dm...@gmail.com>
>>>
>>>> Details are very welcome as I'm still not sure whether I fully
>>>> understand the problem :)
>>>>
>>>> Thread pooling does sound nice and it might enable a feature I'd like:
>>>>
>>>> On-demand creation of threads. Say you're looping 10 threads, system
>>>> is under no load.
>>>> Currently: if you want 10 more users you need to stop the test, add 10
>>>> users and start it again.
>>>>
>>>> It would be nice if it where possible to simply say '+10', followed by
>>>> some monitoring and then another '+10' (30 threads running total) etc.
>>>> (10 is an example number, +/- n users would be the goal)
>>>> Mostly for the 'getting a feel for the system' part of the test, the
>>>> initial setup of it all.
>>>>
>>>> Regards,
>>>> Daniel
>>>>
>>>> On Fri, Jun 15, 2012 at 7:24 AM, Kirk Pepperdine
>>>> <ki...@gmail.com> wrote:
>>>> > Hi,
>>>> >
>>>> > I'm very happy to add details if need be.
>>>> >
>>>> > Regards,
>>>> > Kirk
>>>> >
>>>> > On 2012-06-15, at 12:13 AM, Philippe Mouawad wrote:
>>>> >
>>>> >> Bugzilla created:
>>>> >>
>>>> >>   - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418
>>>> >>
>>>> >>
>>>> >> Regards
>>>> >>
>>>> >> Philippe M.
>>>> >>
>>>> >> On Thu, Jun 14, 2012 at 10:30 PM, Philippe Mouawad <
>>>> >> philippe.mouawad@gmail.com> wrote:
>>>> >>
>>>> >>> Hi M. Pepperdine,
>>>> >>>
>>>> >>> On Wed, Jun 13, 2012 at 7:05 AM, Kirk Pepperdine <
>>>> >>> kirk.pepperdine@gmail.com> wrote:
>>>> >>>
>>>> >>>> Hi Sebb,
>>>> >>>>
>>>> >>>> We've had this conversation before and I did some preliminary work
>>>> to
>>>> >>>> setup a different type of thread group but the couplings between the
>>>> >>>> existing thread group and the model meant that an extensive
>>>> refactoring
>>>> >>>> would be involved. Since that involves a *lot* more than just a
>>>> simple
>>>> >>>> plugin...
>>>> >>>>
>>>> >>>> So, the current implementation supports a closed system model
>>>> meaning,
>>>> >>>> rate of entry into the system equals rate of exit from the
>>>> system.This is
>>>> >>>> exactly what you want if you're load testing a call centre where
>>>> the number
>>>> >>>> of servers (operators) is fixed and gate entry into the system.
>>>> However,
>>>> >>>> I'm often simulating open systems which means I do not want rate of
>>>> entry
>>>> >>>> into the system to be controlled by the performance of the system
>>>> (rate of
>>>> >>>> exit).
>>>> >>>
>>>> >>> What makes you think JMeter does this ?
>>>> >>>
>>>> >>>
>>>> >>>> More over, those that attend my performance tuning seminars come to
>>>> >>>> understand why this is an important aspect of getting your test
>>>> environment
>>>> >>>> right and test harness correctly setup as it can adversely affect
>>>> the
>>>> >>>> quality of your test which can and often does, change the results
>>>> of the
>>>> >>>> test.
>>>> >>>>
>>>> >>>
>>>> >>>> As an example, today I will show a group how to tune an application
>>>> by a
>>>> >>>> partner company. That application has a number of "performance
>>>> problems"
>>>> >>>> backed into it. If I use the traditional means of using JMeter I
>>>> will find
>>>> >>>> a different set of performance issues than if I load with a pattern
>>>> that is
>>>> >>>> similar that found in production.
>>>> >>>
>>>> >>> Can you clarify this point ? a figure might be better than a long
>>>> text
>>>> >>>
>>>> >>>
>>>> >>>> In other words, with this particular application, JMeter exposes
>>>> >>>> "problems" that are artifacts of how it wants to inject load on a
>>>> system.
>>>> >>>
>>>> >>> Not clear for me.
>>>> >>>
>>>> >>> I can fix all of these problems
>>>> >>>
>>>> >>> What are these problems ? and how do you fix these ?
>>>> >>>
>>>> >>>
>>>> >>>> and eventually I'll get to a point where I'll fix everything that
>>>> needs
>>>> >>>> to be fixed. That said, if I can coerce JMeter to load as an open
>>>> system
>>>> >>>> I'll get to the problems without having to fix the artifacts (the
>>>> things
>>>> >>>> that really don't need fixing).
>>>> >>>
>>>> >>> Still not clear
>>>> >>>
>>>> >>>> To coerce JMeter into being an open system requires one to use a
>>>> large
>>>> >>>> number of very short lived threads. So I may only have 400-500
>>>> active
>>>> >>>> threads at any point in time but in order to achieve that load over
>>>> a 1 or
>>>> >>>> two hour test I may have to specify 10s of thousands of threads.
>>>> Since all
>>>> >>>> of the threads are created up front, this simply doesn't work.
>>>> >>>>
>>>> >>>> You might ask why not just specify 400-500 threads and loop over
>>>> them? in
>>>> >>>> theory you'd think it would work but as you tune the system and the
>>>> >>>> performance characteristics change. Going back to the baked
>>>> application,
>>>> >>>> before I start tuning, the active user count is several thousand.
>>>> In other
>>>> >>>> words, the tuned system is better able to clear users out and that
>>>> changes
>>>> >>>> the performance profile in a way that hard to emulate with the
>>>> current
>>>> >>>> looping behaviour. Using a setting of looping 400 or so threads
>>>> isn't
>>>> >>>> adequate for the initial load tests as the test harness will become
>>>> thread
>>>> >>>> starved and that releases pressure on the server which in turn
>>>> changes the
>>>> >>>> performance profile.
>>>> >>>>
>>>> >>>
>>>> >>>
>>>> >>>> With all due respect to the wonderful work that everyone on the
>>>> project
>>>> >>>> has done, it is my opinion that the one user == one thread is a
>>>> design
>>>> >>>> mistake that has a huge impact on both the usability of the tool
>>>> >>>>
>>>> >>> Examples ?
>>>> >>>
>>>> >>>> and the quality of the results
>>>> >>>
>>>> >>> I disagree with this assertion . We have been using JMeter for load
>>>> >>> testing all kind of applications Intranets / Large ECommerce Systems
>>>> /
>>>> >>> Backoffice systems / , and quality of results is good provided you
>>>> >>> configure it properly.
>>>> >>> Particularly when using Remote  Testing. Lot of users in this
>>>> mailing list
>>>> >>> use it and are satisfied (I think).
>>>> >>>
>>>> >>>
>>>> >>>> one can achieve when using it. IMHO, moving to an thread pool/event
>>>> heap
>>>> >>>> based model would be an enormous improvement over the current
>>>> >>>> implementation.
>>>> >>>>
>>>> >>>> Agree it would be better. We will work on it.
>>>> >>>
>>>> >>>> Regards,
>>>> >>>> Kirk
>>>> >>>>
>>>> >>>> On 2012-06-13, at 1:02 AM, sebb wrote:
>>>> >>>>
>>>> >>>>> On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com>
>>>> >>>> wrote:
>>>> >>>>>>
>>>> >>>>>> On 2012-06-13, at 12:54 AM, sebb wrote:
>>>> >>>>>>
>>>> >>>>>>> On 12 June 2012 22:06, Kirk Pepperdine <
>>>> kirk.pepperdine@gmail.com>
>>>> >>>> wrote:
>>>> >>>>>>>> Hi,
>>>> >>>>>>>>
>>>> >>>>>>>> I figured thread pooling would be revolutionary so I wasn't
>>>> >>>> suggesting that. I would be very useful just delay the creation of
>>>> a thread
>>>> >>>> until it was asked for.
>>>> >>>>>>>
>>>> >>>>>>> Not sure I understand how it would help to delay the thread
>>>> creation,
>>>> >>>>>>> except perhaps for the case where the first threads have finished
>>>> >>>>>>> processing by the time the last threads start running samples.
>>>> >>>>>>
>>>> >>>>>> Bingo!!! ;-)
>>>> >>>>>
>>>> >>>>> So what percentage of use cases need to follow this model?
>>>> >>>>>
>>>> >>>>> Most of the JMeter testing I have done was long running tests where
>>>> >>>>> all threads were active for most of the run.
>>>> >>>>>
>>>> >>>>>> Kirk
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> ---------------------------------------------------------------------
>>>> >>>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>>> >>>>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>> >>>>>>
>>>> >>>>>
>>>> >>>>>
>>>> ---------------------------------------------------------------------
>>>> >>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>>> >>>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>> >>>>>
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> ---------------------------------------------------------------------
>>>> >>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>>> >>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>> >>>>
>>>> >>>>
>>>> >>>
>>>> >>>
>>>> >>> --
>>>> >>> Cordialement.
>>>> >>> Philippe Mouawad.
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Cordialement.
>>>> >> Philippe Mouawad.
>>>> >
>>>> >
>>>> > ---------------------------------------------------------------------
>>>> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>>> > For additional commands, e-mail: user-help@jmeter.apache.org
>>>> >
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>>
>>>>
>>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by Flavio Cysne <fl...@gmail.com>.

Post entry in my blog:
http://flaviocysne.blogspot.com.br/2012/06/how-to-dynamically-increasedecrease.html

It's an unfinished work, but it's suffice to make what is proposed.
Comments and constructive criticism are welcome.

Hope it helps you.
Flávio Cysne

2012/6/15 Flavio Cysne <fl...@gmail.com>

> Errata:
>
> The first "If Controller" is executed and update a property named
> "currentThreads" ...
>
>
> 2012/6/15 Flavio Cysne <fl...@gmail.com>
>
>> I have tested something in JMeter that could achieve a dynamic threading
>> model and there was no need of new implementations.
>>
>> Basically, I have this structure:
>>
>> 1. Test Plan: { variables : [ {name: "maxThreads", value: 5}, {name:
>> "currentThreads", value: 1} ] }
>> 2. | - Thread Group: { threads: "${maxThreads}", rampup: 0, infinite:
>> true }
>> 3.     | - If Controller: { condition: "(${__threadNum()} ===
>> ${maxThreads})" }
>> 4.         | - Sampler /* I've used HTTP Request, but anyone would be
>> used */
>> 5.             | - RegEx Extractor: {refName: "threads", expression:
>> "(.*)", model: "$1$", matchNo: "1", defaultValue: "0"}
>> 6.             | - BeanShell Post-Processor: {language: "javascript",
>> script: "var threads = vars.get(\"threads\");\nprops.put(\"currentThreads
>> \",threads);"}
>> 7.     | - If Controller: { condition: "(${__threadNum()} <
>> ${maxThreads})" }
>> 8.         | - If Controller: { condition: "(${__threadNum()} <= ${__P(
>> currentThreads,0)})" }
>> 9.             | - All other Samplers go here
>>
>>
>> The idea is starting the maximum number of desired threads at the
>> beginning but only use what you want.
>> The first "If Controller" is executed and update a property named
>> "threads" with the value retrieved from a response (HTTP, SOAP, file,
>> whatever JMeter can access)
>> When the first loop iteration is done line 9 will be skipped because
>> ${__P(currentThreads,0)} will return 0. At the same iteration, lines 4
>> through 6 will be executed and will update the "currentThreads" property
>> using the value in "threads" variable.
>> After the first iteration, if the "currentThreads" property has been
>> updated, line 9 will be executed using only the number of threads
>> of "currentThreads" property.
>> To increase/decrease the "currentThreads" property all you have to do is
>> edit the file accessed by the Sampler at line 4 and wait for the next
>> iteration to see the new "threading model" running.
>>
>> The approach used by distributed testing and this pseudo-dynamic
>> threading model will include ${__machineName()} or ${__machineIP()} in "If
>> Controller" conditions.
>>
>> I'll try to make a entry in my blog to explain in detail this scenario.
>>
>> I used JMeter 2.7 and you can download jmx file here<https://docs.google.com/open?id=0Bwr5j-BqUUDobnFTSHc1eTltdmM>
>> .
>>
>> Hope it helps you.
>> Flávio Cysne
>>
>> 2012/6/15 D G <dm...@gmail.com>
>>
>>> Details are very welcome as I'm still not sure whether I fully
>>> understand the problem :)
>>>
>>> Thread pooling does sound nice and it might enable a feature I'd like:
>>>
>>> On-demand creation of threads. Say you're looping 10 threads, system
>>> is under no load.
>>> Currently: if you want 10 more users you need to stop the test, add 10
>>> users and start it again.
>>>
>>> It would be nice if it where possible to simply say '+10', followed by
>>> some monitoring and then another '+10' (30 threads running total) etc.
>>> (10 is an example number, +/- n users would be the goal)
>>> Mostly for the 'getting a feel for the system' part of the test, the
>>> initial setup of it all.
>>>
>>> Regards,
>>> Daniel
>>>
>>> On Fri, Jun 15, 2012 at 7:24 AM, Kirk Pepperdine
>>> <ki...@gmail.com> wrote:
>>> > Hi,
>>> >
>>> > I'm very happy to add details if need be.
>>> >
>>> > Regards,
>>> > Kirk
>>> >
>>> > On 2012-06-15, at 12:13 AM, Philippe Mouawad wrote:
>>> >
>>> >> Bugzilla created:
>>> >>
>>> >>   - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418
>>> >>
>>> >>
>>> >> Regards
>>> >>
>>> >> Philippe M.
>>> >>
>>> >> On Thu, Jun 14, 2012 at 10:30 PM, Philippe Mouawad <
>>> >> philippe.mouawad@gmail.com> wrote:
>>> >>
>>> >>> Hi M. Pepperdine,
>>> >>>
>>> >>> On Wed, Jun 13, 2012 at 7:05 AM, Kirk Pepperdine <
>>> >>> kirk.pepperdine@gmail.com> wrote:
>>> >>>
>>> >>>> Hi Sebb,
>>> >>>>
>>> >>>> We've had this conversation before and I did some preliminary work
>>> to
>>> >>>> setup a different type of thread group but the couplings between the
>>> >>>> existing thread group and the model meant that an extensive
>>> refactoring
>>> >>>> would be involved. Since that involves a *lot* more than just a
>>> simple
>>> >>>> plugin...
>>> >>>>
>>> >>>> So, the current implementation supports a closed system model
>>> meaning,
>>> >>>> rate of entry into the system equals rate of exit from the
>>> system.This is
>>> >>>> exactly what you want if you're load testing a call centre where
>>> the number
>>> >>>> of servers (operators) is fixed and gate entry into the system.
>>> However,
>>> >>>> I'm often simulating open systems which means I do not want rate of
>>> entry
>>> >>>> into the system to be controlled by the performance of the system
>>> (rate of
>>> >>>> exit).
>>> >>>
>>> >>> What makes you think JMeter does this ?
>>> >>>
>>> >>>
>>> >>>> More over, those that attend my performance tuning seminars come to
>>> >>>> understand why this is an important aspect of getting your test
>>> environment
>>> >>>> right and test harness correctly setup as it can adversely affect
>>> the
>>> >>>> quality of your test which can and often does, change the results
>>> of the
>>> >>>> test.
>>> >>>>
>>> >>>
>>> >>>> As an example, today I will show a group how to tune an application
>>> by a
>>> >>>> partner company. That application has a number of "performance
>>> problems"
>>> >>>> backed into it. If I use the traditional means of using JMeter I
>>> will find
>>> >>>> a different set of performance issues than if I load with a pattern
>>> that is
>>> >>>> similar that found in production.
>>> >>>
>>> >>> Can you clarify this point ? a figure might be better than a long
>>> text
>>> >>>
>>> >>>
>>> >>>> In other words, with this particular application, JMeter exposes
>>> >>>> "problems" that are artifacts of how it wants to inject load on a
>>> system.
>>> >>>
>>> >>> Not clear for me.
>>> >>>
>>> >>> I can fix all of these problems
>>> >>>
>>> >>> What are these problems ? and how do you fix these ?
>>> >>>
>>> >>>
>>> >>>> and eventually I'll get to a point where I'll fix everything that
>>> needs
>>> >>>> to be fixed. That said, if I can coerce JMeter to load as an open
>>> system
>>> >>>> I'll get to the problems without having to fix the artifacts (the
>>> things
>>> >>>> that really don't need fixing).
>>> >>>
>>> >>> Still not clear
>>> >>>
>>> >>>> To coerce JMeter into being an open system requires one to use a
>>> large
>>> >>>> number of very short lived threads. So I may only have 400-500
>>> active
>>> >>>> threads at any point in time but in order to achieve that load over
>>> a 1 or
>>> >>>> two hour test I may have to specify 10s of thousands of threads.
>>> Since all
>>> >>>> of the threads are created up front, this simply doesn't work.
>>> >>>>
>>> >>>> You might ask why not just specify 400-500 threads and loop over
>>> them? in
>>> >>>> theory you'd think it would work but as you tune the system and the
>>> >>>> performance characteristics change. Going back to the baked
>>> application,
>>> >>>> before I start tuning, the active user count is several thousand.
>>> In other
>>> >>>> words, the tuned system is better able to clear users out and that
>>> changes
>>> >>>> the performance profile in a way that hard to emulate with the
>>> current
>>> >>>> looping behaviour. Using a setting of looping 400 or so threads
>>> isn't
>>> >>>> adequate for the initial load tests as the test harness will become
>>> thread
>>> >>>> starved and that releases pressure on the server which in turn
>>> changes the
>>> >>>> performance profile.
>>> >>>>
>>> >>>
>>> >>>
>>> >>>> With all due respect to the wonderful work that everyone on the
>>> project
>>> >>>> has done, it is my opinion that the one user == one thread is a
>>> design
>>> >>>> mistake that has a huge impact on both the usability of the tool
>>> >>>>
>>> >>> Examples ?
>>> >>>
>>> >>>> and the quality of the results
>>> >>>
>>> >>> I disagree with this assertion . We have been using JMeter for load
>>> >>> testing all kind of applications Intranets / Large ECommerce Systems
>>> /
>>> >>> Backoffice systems / , and quality of results is good provided you
>>> >>> configure it properly.
>>> >>> Particularly when using Remote  Testing. Lot of users in this
>>> mailing list
>>> >>> use it and are satisfied (I think).
>>> >>>
>>> >>>
>>> >>>> one can achieve when using it. IMHO, moving to an thread pool/event
>>> heap
>>> >>>> based model would be an enormous improvement over the current
>>> >>>> implementation.
>>> >>>>
>>> >>>> Agree it would be better. We will work on it.
>>> >>>
>>> >>>> Regards,
>>> >>>> Kirk
>>> >>>>
>>> >>>> On 2012-06-13, at 1:02 AM, sebb wrote:
>>> >>>>
>>> >>>>> On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com>
>>> >>>> wrote:
>>> >>>>>>
>>> >>>>>> On 2012-06-13, at 12:54 AM, sebb wrote:
>>> >>>>>>
>>> >>>>>>> On 12 June 2012 22:06, Kirk Pepperdine <
>>> kirk.pepperdine@gmail.com>
>>> >>>> wrote:
>>> >>>>>>>> Hi,
>>> >>>>>>>>
>>> >>>>>>>> I figured thread pooling would be revolutionary so I wasn't
>>> >>>> suggesting that. I would be very useful just delay the creation of
>>> a thread
>>> >>>> until it was asked for.
>>> >>>>>>>
>>> >>>>>>> Not sure I understand how it would help to delay the thread
>>> creation,
>>> >>>>>>> except perhaps for the case where the first threads have finished
>>> >>>>>>> processing by the time the last threads start running samples.
>>> >>>>>>
>>> >>>>>> Bingo!!! ;-)
>>> >>>>>
>>> >>>>> So what percentage of use cases need to follow this model?
>>> >>>>>
>>> >>>>> Most of the JMeter testing I have done was long running tests where
>>> >>>>> all threads were active for most of the run.
>>> >>>>>
>>> >>>>>> Kirk
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> ---------------------------------------------------------------------
>>> >>>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>> >>>>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>> >>>>>>
>>> >>>>>
>>> >>>>>
>>> ---------------------------------------------------------------------
>>> >>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>> >>>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>> >>>>>
>>> >>>>
>>> >>>>
>>> >>>>
>>> ---------------------------------------------------------------------
>>> >>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>> >>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>> >>>>
>>> >>>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Cordialement.
>>> >>> Philippe Mouawad.
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>
>>> >>
>>> >> --
>>> >> Cordialement.
>>> >> Philippe Mouawad.
>>> >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>> > For additional commands, e-mail: user-help@jmeter.apache.org
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>
>>>
>>
>

Re: JMeter threading model

Posted by Flavio Cysne <fl...@gmail.com>.

Errata:

The first "If Controller" is executed and update a property named
"currentThreads" ...

2012/6/15 Flavio Cysne <fl...@gmail.com>

> I have tested something in JMeter that could achieve a dynamic threading
> model and there was no need of new implementations.
>
> Basically, I have this structure:
>
> 1. Test Plan: { variables : [ {name: "maxThreads", value: 5}, {name:
> "currentThreads", value: 1} ] }
> 2. | - Thread Group: { threads: "${maxThreads}", rampup: 0, infinite: true
> }
> 3.     | - If Controller: { condition: "(${__threadNum()} ===
> ${maxThreads})" }
> 4.         | - Sampler /* I've used HTTP Request, but anyone would be used
> */
> 5.             | - RegEx Extractor: {refName: "threads", expression:
> "(.*)", model: "$1$", matchNo: "1", defaultValue: "0"}
> 6.             | - BeanShell Post-Processor: {language: "javascript",
> script: "var threads = vars.get(\"threads\");\nprops.put(\"currentThreads
> \",threads);"}
> 7.     | - If Controller: { condition: "(${__threadNum()} <
> ${maxThreads})" }
> 8.         | - If Controller: { condition: "(${__threadNum()} <= ${__P(
> currentThreads,0)})" }
> 9.             | - All other Samplers go here
>
>
> The idea is starting the maximum number of desired threads at the
> beginning but only use what you want.
> The first "If Controller" is executed and update a property named
> "threads" with the value retrieved from a response (HTTP, SOAP, file,
> whatever JMeter can access)
> When the first loop iteration is done line 9 will be skipped because
> ${__P(currentThreads,0)} will return 0. At the same iteration, lines 4
> through 6 will be executed and will update the "currentThreads" property
> using the value in "threads" variable.
> After the first iteration, if the "currentThreads" property has been
> updated, line 9 will be executed using only the number of threads
> of "currentThreads" property.
> To increase/decrease the "currentThreads" property all you have to do is
> edit the file accessed by the Sampler at line 4 and wait for the next
> iteration to see the new "threading model" running.
>
> The approach used by distributed testing and this pseudo-dynamic threading
> model will include ${__machineName()} or ${__machineIP()} in "If
> Controller" conditions.
>
> I'll try to make a entry in my blog to explain in detail this scenario.
>
> I used JMeter 2.7 and you can download jmx file here<https://docs.google.com/open?id=0Bwr5j-BqUUDobnFTSHc1eTltdmM>
> .
>
> Hope it helps you.
> Flávio Cysne
>
> 2012/6/15 D G <dm...@gmail.com>
>
>> Details are very welcome as I'm still not sure whether I fully
>> understand the problem :)
>>
>> Thread pooling does sound nice and it might enable a feature I'd like:
>>
>> On-demand creation of threads. Say you're looping 10 threads, system
>> is under no load.
>> Currently: if you want 10 more users you need to stop the test, add 10
>> users and start it again.
>>
>> It would be nice if it where possible to simply say '+10', followed by
>> some monitoring and then another '+10' (30 threads running total) etc.
>> (10 is an example number, +/- n users would be the goal)
>> Mostly for the 'getting a feel for the system' part of the test, the
>> initial setup of it all.
>>
>> Regards,
>> Daniel
>>
>> On Fri, Jun 15, 2012 at 7:24 AM, Kirk Pepperdine
>> <ki...@gmail.com> wrote:
>> > Hi,
>> >
>> > I'm very happy to add details if need be.
>> >
>> > Regards,
>> > Kirk
>> >
>> > On 2012-06-15, at 12:13 AM, Philippe Mouawad wrote:
>> >
>> >> Bugzilla created:
>> >>
>> >>   - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418
>> >>
>> >>
>> >> Regards
>> >>
>> >> Philippe M.
>> >>
>> >> On Thu, Jun 14, 2012 at 10:30 PM, Philippe Mouawad <
>> >> philippe.mouawad@gmail.com> wrote:
>> >>
>> >>> Hi M. Pepperdine,
>> >>>
>> >>> On Wed, Jun 13, 2012 at 7:05 AM, Kirk Pepperdine <
>> >>> kirk.pepperdine@gmail.com> wrote:
>> >>>
>> >>>> Hi Sebb,
>> >>>>
>> >>>> We've had this conversation before and I did some preliminary work to
>> >>>> setup a different type of thread group but the couplings between the
>> >>>> existing thread group and the model meant that an extensive
>> refactoring
>> >>>> would be involved. Since that involves a *lot* more than just a
>> simple
>> >>>> plugin...
>> >>>>
>> >>>> So, the current implementation supports a closed system model
>> meaning,
>> >>>> rate of entry into the system equals rate of exit from the
>> system.This is
>> >>>> exactly what you want if you're load testing a call centre where the
>> number
>> >>>> of servers (operators) is fixed and gate entry into the system.
>> However,
>> >>>> I'm often simulating open systems which means I do not want rate of
>> entry
>> >>>> into the system to be controlled by the performance of the system
>> (rate of
>> >>>> exit).
>> >>>
>> >>> What makes you think JMeter does this ?
>> >>>
>> >>>
>> >>>> More over, those that attend my performance tuning seminars come to
>> >>>> understand why this is an important aspect of getting your test
>> environment
>> >>>> right and test harness correctly setup as it can adversely affect the
>> >>>> quality of your test which can and often does, change the results of
>> the
>> >>>> test.
>> >>>>
>> >>>
>> >>>> As an example, today I will show a group how to tune an application
>> by a
>> >>>> partner company. That application has a number of "performance
>> problems"
>> >>>> backed into it. If I use the traditional means of using JMeter I
>> will find
>> >>>> a different set of performance issues than if I load with a pattern
>> that is
>> >>>> similar that found in production.
>> >>>
>> >>> Can you clarify this point ? a figure might be better than a long text
>> >>>
>> >>>
>> >>>> In other words, with this particular application, JMeter exposes
>> >>>> "problems" that are artifacts of how it wants to inject load on a
>> system.
>> >>>
>> >>> Not clear for me.
>> >>>
>> >>> I can fix all of these problems
>> >>>
>> >>> What are these problems ? and how do you fix these ?
>> >>>
>> >>>
>> >>>> and eventually I'll get to a point where I'll fix everything that
>> needs
>> >>>> to be fixed. That said, if I can coerce JMeter to load as an open
>> system
>> >>>> I'll get to the problems without having to fix the artifacts (the
>> things
>> >>>> that really don't need fixing).
>> >>>
>> >>> Still not clear
>> >>>
>> >>>> To coerce JMeter into being an open system requires one to use a
>> large
>> >>>> number of very short lived threads. So I may only have 400-500 active
>> >>>> threads at any point in time but in order to achieve that load over
>> a 1 or
>> >>>> two hour test I may have to specify 10s of thousands of threads.
>> Since all
>> >>>> of the threads are created up front, this simply doesn't work.
>> >>>>
>> >>>> You might ask why not just specify 400-500 threads and loop over
>> them? in
>> >>>> theory you'd think it would work but as you tune the system and the
>> >>>> performance characteristics change. Going back to the baked
>> application,
>> >>>> before I start tuning, the active user count is several thousand. In
>> other
>> >>>> words, the tuned system is better able to clear users out and that
>> changes
>> >>>> the performance profile in a way that hard to emulate with the
>> current
>> >>>> looping behaviour. Using a setting of looping 400 or so threads isn't
>> >>>> adequate for the initial load tests as the test harness will become
>> thread
>> >>>> starved and that releases pressure on the server which in turn
>> changes the
>> >>>> performance profile.
>> >>>>
>> >>>
>> >>>
>> >>>> With all due respect to the wonderful work that everyone on the
>> project
>> >>>> has done, it is my opinion that the one user == one thread is a
>> design
>> >>>> mistake that has a huge impact on both the usability of the tool
>> >>>>
>> >>> Examples ?
>> >>>
>> >>>> and the quality of the results
>> >>>
>> >>> I disagree with this assertion . We have been using JMeter for load
>> >>> testing all kind of applications Intranets / Large ECommerce Systems /
>> >>> Backoffice systems / , and quality of results is good provided you
>> >>> configure it properly.
>> >>> Particularly when using Remote  Testing. Lot of users in this mailing
>> list
>> >>> use it and are satisfied (I think).
>> >>>
>> >>>
>> >>>> one can achieve when using it. IMHO, moving to an thread pool/event
>> heap
>> >>>> based model would be an enormous improvement over the current
>> >>>> implementation.
>> >>>>
>> >>>> Agree it would be better. We will work on it.
>> >>>
>> >>>> Regards,
>> >>>> Kirk
>> >>>>
>> >>>> On 2012-06-13, at 1:02 AM, sebb wrote:
>> >>>>
>> >>>>> On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com>
>> >>>> wrote:
>> >>>>>>
>> >>>>>> On 2012-06-13, at 12:54 AM, sebb wrote:
>> >>>>>>
>> >>>>>>> On 12 June 2012 22:06, Kirk Pepperdine <kirk.pepperdine@gmail.com
>> >
>> >>>> wrote:
>> >>>>>>>> Hi,
>> >>>>>>>>
>> >>>>>>>> I figured thread pooling would be revolutionary so I wasn't
>> >>>> suggesting that. I would be very useful just delay the creation of a
>> thread
>> >>>> until it was asked for.
>> >>>>>>>
>> >>>>>>> Not sure I understand how it would help to delay the thread
>> creation,
>> >>>>>>> except perhaps for the case where the first threads have finished
>> >>>>>>> processing by the time the last threads start running samples.
>> >>>>>>
>> >>>>>> Bingo!!! ;-)
>> >>>>>
>> >>>>> So what percentage of use cases need to follow this model?
>> >>>>>
>> >>>>> Most of the JMeter testing I have done was long running tests where
>> >>>>> all threads were active for most of the run.
>> >>>>>
>> >>>>>> Kirk
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> ---------------------------------------------------------------------
>> >>>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> >>>>>> For additional commands, e-mail: user-help@jmeter.apache.org
>> >>>>>>
>> >>>>>
>> >>>>>
>> ---------------------------------------------------------------------
>> >>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> >>>>> For additional commands, e-mail: user-help@jmeter.apache.org
>> >>>>>
>> >>>>
>> >>>>
>> >>>> ---------------------------------------------------------------------
>> >>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> >>>> For additional commands, e-mail: user-help@jmeter.apache.org
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>> --
>> >>> Cordialement.
>> >>> Philippe Mouawad.
>> >>>
>> >>>
>> >>>
>> >>>
>> >>
>> >>
>> >> --
>> >> Cordialement.
>> >> Philippe Mouawad.
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> > For additional commands, e-mail: user-help@jmeter.apache.org
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>>
>

Re: JMeter threading model

Posted by Flavio Cysne <fl...@gmail.com>.

I have tested something in JMeter that could achieve a dynamic threading
model and there was no need of new implementations.

Basically, I have this structure:

1. Test Plan: { variables : [ {name: "maxThreads", value: 5}, {name:
"currentThreads", value: 1} ] }
2. | - Thread Group: { threads: "${maxThreads}", rampup: 0, infinite: true }
3.     | - If Controller: { condition: "(${__threadNum()} ===
${maxThreads})" }
4.         | - Sampler /* I've used HTTP Request, but anyone would be used
*/
5.             | - RegEx Extractor: {refName: "threads", expression:
"(.*)", model: "$1$", matchNo: "1", defaultValue: "0"}
6.             | - BeanShell Post-Processor: {language: "javascript",
script: "var threads = vars.get(\"threads\");\nprops.put(\"currentThreads
\",threads);"}
7.     | - If Controller: { condition: "(${__threadNum()} < ${maxThreads})"
}
8.         | - If Controller: { condition: "(${__threadNum()} <= ${__P(
currentThreads,0)})" }
9.             | - All other Samplers go here


The idea is starting the maximum number of desired threads at the beginning
but only use what you want.
The first "If Controller" is executed and update a property named "threads"
with the value retrieved from a response (HTTP, SOAP, file, whatever JMeter
can access)
When the first loop iteration is done line 9 will be skipped because ${__P(
currentThreads,0)} will return 0. At the same iteration, lines 4 through 6
will be executed and will update the "currentThreads" property using the
value in "threads" variable.
After the first iteration, if the "currentThreads" property has been
updated, line 9 will be executed using only the number of threads
of "currentThreads" property.
To increase/decrease the "currentThreads" property all you have to do is
edit the file accessed by the Sampler at line 4 and wait for the next
iteration to see the new "threading model" running.

The approach used by distributed testing and this pseudo-dynamic threading
model will include ${__machineName()} or ${__machineIP()} in "If
Controller" conditions.

I'll try to make a entry in my blog to explain in detail this scenario.

I used JMeter 2.7 and you can download jmx file
here<https://docs.google.com/open?id=0Bwr5j-BqUUDobnFTSHc1eTltdmM>
.

Hope it helps you.
Flávio Cysne

2012/6/15 D G <dm...@gmail.com>

> Details are very welcome as I'm still not sure whether I fully
> understand the problem :)
>
> Thread pooling does sound nice and it might enable a feature I'd like:
>
> On-demand creation of threads. Say you're looping 10 threads, system
> is under no load.
> Currently: if you want 10 more users you need to stop the test, add 10
> users and start it again.
>
> It would be nice if it where possible to simply say '+10', followed by
> some monitoring and then another '+10' (30 threads running total) etc.
> (10 is an example number, +/- n users would be the goal)
> Mostly for the 'getting a feel for the system' part of the test, the
> initial setup of it all.
>
> Regards,
> Daniel
>
> On Fri, Jun 15, 2012 at 7:24 AM, Kirk Pepperdine
> <ki...@gmail.com> wrote:
> > Hi,
> >
> > I'm very happy to add details if need be.
> >
> > Regards,
> > Kirk
> >
> > On 2012-06-15, at 12:13 AM, Philippe Mouawad wrote:
> >
> >> Bugzilla created:
> >>
> >>   - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418
> >>
> >>
> >> Regards
> >>
> >> Philippe M.
> >>
> >> On Thu, Jun 14, 2012 at 10:30 PM, Philippe Mouawad <
> >> philippe.mouawad@gmail.com> wrote:
> >>
> >>> Hi M. Pepperdine,
> >>>
> >>> On Wed, Jun 13, 2012 at 7:05 AM, Kirk Pepperdine <
> >>> kirk.pepperdine@gmail.com> wrote:
> >>>
> >>>> Hi Sebb,
> >>>>
> >>>> We've had this conversation before and I did some preliminary work to
> >>>> setup a different type of thread group but the couplings between the
> >>>> existing thread group and the model meant that an extensive
> refactoring
> >>>> would be involved. Since that involves a *lot* more than just a simple
> >>>> plugin...
> >>>>
> >>>> So, the current implementation supports a closed system model meaning,
> >>>> rate of entry into the system equals rate of exit from the
> system.This is
> >>>> exactly what you want if you're load testing a call centre where the
> number
> >>>> of servers (operators) is fixed and gate entry into the system.
> However,
> >>>> I'm often simulating open systems which means I do not want rate of
> entry
> >>>> into the system to be controlled by the performance of the system
> (rate of
> >>>> exit).
> >>>
> >>> What makes you think JMeter does this ?
> >>>
> >>>
> >>>> More over, those that attend my performance tuning seminars come to
> >>>> understand why this is an important aspect of getting your test
> environment
> >>>> right and test harness correctly setup as it can adversely affect the
> >>>> quality of your test which can and often does, change the results of
> the
> >>>> test.
> >>>>
> >>>
> >>>> As an example, today I will show a group how to tune an application
> by a
> >>>> partner company. That application has a number of "performance
> problems"
> >>>> backed into it. If I use the traditional means of using JMeter I will
> find
> >>>> a different set of performance issues than if I load with a pattern
> that is
> >>>> similar that found in production.
> >>>
> >>> Can you clarify this point ? a figure might be better than a long text
> >>>
> >>>
> >>>> In other words, with this particular application, JMeter exposes
> >>>> "problems" that are artifacts of how it wants to inject load on a
> system.
> >>>
> >>> Not clear for me.
> >>>
> >>> I can fix all of these problems
> >>>
> >>> What are these problems ? and how do you fix these ?
> >>>
> >>>
> >>>> and eventually I'll get to a point where I'll fix everything that
> needs
> >>>> to be fixed. That said, if I can coerce JMeter to load as an open
> system
> >>>> I'll get to the problems without having to fix the artifacts (the
> things
> >>>> that really don't need fixing).
> >>>
> >>> Still not clear
> >>>
> >>>> To coerce JMeter into being an open system requires one to use a large
> >>>> number of very short lived threads. So I may only have 400-500 active
> >>>> threads at any point in time but in order to achieve that load over a
> 1 or
> >>>> two hour test I may have to specify 10s of thousands of threads.
> Since all
> >>>> of the threads are created up front, this simply doesn't work.
> >>>>
> >>>> You might ask why not just specify 400-500 threads and loop over
> them? in
> >>>> theory you'd think it would work but as you tune the system and the
> >>>> performance characteristics change. Going back to the baked
> application,
> >>>> before I start tuning, the active user count is several thousand. In
> other
> >>>> words, the tuned system is better able to clear users out and that
> changes
> >>>> the performance profile in a way that hard to emulate with the current
> >>>> looping behaviour. Using a setting of looping 400 or so threads isn't
> >>>> adequate for the initial load tests as the test harness will become
> thread
> >>>> starved and that releases pressure on the server which in turn
> changes the
> >>>> performance profile.
> >>>>
> >>>
> >>>
> >>>> With all due respect to the wonderful work that everyone on the
> project
> >>>> has done, it is my opinion that the one user == one thread is a design
> >>>> mistake that has a huge impact on both the usability of the tool
> >>>>
> >>> Examples ?
> >>>
> >>>> and the quality of the results
> >>>
> >>> I disagree with this assertion . We have been using JMeter for load
> >>> testing all kind of applications Intranets / Large ECommerce Systems /
> >>> Backoffice systems / , and quality of results is good provided you
> >>> configure it properly.
> >>> Particularly when using Remote  Testing. Lot of users in this mailing
> list
> >>> use it and are satisfied (I think).
> >>>
> >>>
> >>>> one can achieve when using it. IMHO, moving to an thread pool/event
> heap
> >>>> based model would be an enormous improvement over the current
> >>>> implementation.
> >>>>
> >>>> Agree it would be better. We will work on it.
> >>>
> >>>> Regards,
> >>>> Kirk
> >>>>
> >>>> On 2012-06-13, at 1:02 AM, sebb wrote:
> >>>>
> >>>>> On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com>
> >>>> wrote:
> >>>>>>
> >>>>>> On 2012-06-13, at 12:54 AM, sebb wrote:
> >>>>>>
> >>>>>>> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com>
> >>>> wrote:
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> I figured thread pooling would be revolutionary so I wasn't
> >>>> suggesting that. I would be very useful just delay the creation of a
> thread
> >>>> until it was asked for.
> >>>>>>>
> >>>>>>> Not sure I understand how it would help to delay the thread
> creation,
> >>>>>>> except perhaps for the case where the first threads have finished
> >>>>>>> processing by the time the last threads start running samples.
> >>>>>>
> >>>>>> Bingo!!! ;-)
> >>>>>
> >>>>> So what percentage of use cases need to follow this model?
> >>>>>
> >>>>> Most of the JMeter testing I have done was long running tests where
> >>>>> all threads were active for most of the run.
> >>>>>
> >>>>>> Kirk
> >>>>>>
> >>>>>>
> >>>>>>
> ---------------------------------------------------------------------
> >>>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >>>>>> For additional commands, e-mail: user-help@jmeter.apache.org
> >>>>>>
> >>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >>>>> For additional commands, e-mail: user-help@jmeter.apache.org
> >>>>>
> >>>>
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >>>> For additional commands, e-mail: user-help@jmeter.apache.org
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> Cordialement.
> >>> Philippe Mouawad.
> >>>
> >>>
> >>>
> >>>
> >>
> >>
> >> --
> >> Cordialement.
> >> Philippe Mouawad.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> > For additional commands, e-mail: user-help@jmeter.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>

Re: JMeter threading model

Posted by D G <dm...@gmail.com>.

Details are very welcome as I'm still not sure whether I fully
understand the problem :)

Thread pooling does sound nice and it might enable a feature I'd like:

On-demand creation of threads. Say you're looping 10 threads, system
is under no load.
Currently: if you want 10 more users you need to stop the test, add 10
users and start it again.

It would be nice if it where possible to simply say '+10', followed by
some monitoring and then another '+10' (30 threads running total) etc.
(10 is an example number, +/- n users would be the goal)
Mostly for the 'getting a feel for the system' part of the test, the
initial setup of it all.

Regards,
Daniel

On Fri, Jun 15, 2012 at 7:24 AM, Kirk Pepperdine
<ki...@gmail.com> wrote:
> Hi,
>
> I'm very happy to add details if need be.
>
> Regards,
> Kirk
>
> On 2012-06-15, at 12:13 AM, Philippe Mouawad wrote:
>
>> Bugzilla created:
>>
>>   - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418
>>
>>
>> Regards
>>
>> Philippe M.
>>
>> On Thu, Jun 14, 2012 at 10:30 PM, Philippe Mouawad <
>> philippe.mouawad@gmail.com> wrote:
>>
>>> Hi M. Pepperdine,
>>>
>>> On Wed, Jun 13, 2012 at 7:05 AM, Kirk Pepperdine <
>>> kirk.pepperdine@gmail.com> wrote:
>>>
>>>> Hi Sebb,
>>>>
>>>> We've had this conversation before and I did some preliminary work to
>>>> setup a different type of thread group but the couplings between the
>>>> existing thread group and the model meant that an extensive refactoring
>>>> would be involved. Since that involves a *lot* more than just a simple
>>>> plugin...
>>>>
>>>> So, the current implementation supports a closed system model meaning,
>>>> rate of entry into the system equals rate of exit from the system.This is
>>>> exactly what you want if you're load testing a call centre where the number
>>>> of servers (operators) is fixed and gate entry into the system. However,
>>>> I'm often simulating open systems which means I do not want rate of entry
>>>> into the system to be controlled by the performance of the system (rate of
>>>> exit).
>>>
>>> What makes you think JMeter does this ?
>>>
>>>
>>>> More over, those that attend my performance tuning seminars come to
>>>> understand why this is an important aspect of getting your test environment
>>>> right and test harness correctly setup as it can adversely affect the
>>>> quality of your test which can and often does, change the results of the
>>>> test.
>>>>
>>>
>>>> As an example, today I will show a group how to tune an application by a
>>>> partner company. That application has a number of "performance problems"
>>>> backed into it. If I use the traditional means of using JMeter I will find
>>>> a different set of performance issues than if I load with a pattern that is
>>>> similar that found in production.
>>>
>>> Can you clarify this point ? a figure might be better than a long text
>>>
>>>
>>>> In other words, with this particular application, JMeter exposes
>>>> "problems" that are artifacts of how it wants to inject load on a system.
>>>
>>> Not clear for me.
>>>
>>> I can fix all of these problems
>>>
>>> What are these problems ? and how do you fix these ?
>>>
>>>
>>>> and eventually I'll get to a point where I'll fix everything that needs
>>>> to be fixed. That said, if I can coerce JMeter to load as an open system
>>>> I'll get to the problems without having to fix the artifacts (the things
>>>> that really don't need fixing).
>>>
>>> Still not clear
>>>
>>>> To coerce JMeter into being an open system requires one to use a large
>>>> number of very short lived threads. So I may only have 400-500 active
>>>> threads at any point in time but in order to achieve that load over a 1 or
>>>> two hour test I may have to specify 10s of thousands of threads. Since all
>>>> of the threads are created up front, this simply doesn't work.
>>>>
>>>> You might ask why not just specify 400-500 threads and loop over them? in
>>>> theory you'd think it would work but as you tune the system and the
>>>> performance characteristics change. Going back to the baked application,
>>>> before I start tuning, the active user count is several thousand. In other
>>>> words, the tuned system is better able to clear users out and that changes
>>>> the performance profile in a way that hard to emulate with the current
>>>> looping behaviour. Using a setting of looping 400 or so threads isn't
>>>> adequate for the initial load tests as the test harness will become thread
>>>> starved and that releases pressure on the server which in turn changes the
>>>> performance profile.
>>>>
>>>
>>>
>>>> With all due respect to the wonderful work that everyone on the project
>>>> has done, it is my opinion that the one user == one thread is a design
>>>> mistake that has a huge impact on both the usability of the tool
>>>>
>>> Examples ?
>>>
>>>> and the quality of the results
>>>
>>> I disagree with this assertion . We have been using JMeter for load
>>> testing all kind of applications Intranets / Large ECommerce Systems /
>>> Backoffice systems / , and quality of results is good provided you
>>> configure it properly.
>>> Particularly when using Remote  Testing. Lot of users in this mailing list
>>> use it and are satisfied (I think).
>>>
>>>
>>>> one can achieve when using it. IMHO, moving to an thread pool/event heap
>>>> based model would be an enormous improvement over the current
>>>> implementation.
>>>>
>>>> Agree it would be better. We will work on it.
>>>
>>>> Regards,
>>>> Kirk
>>>>
>>>> On 2012-06-13, at 1:02 AM, sebb wrote:
>>>>
>>>>> On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com>
>>>> wrote:
>>>>>>
>>>>>> On 2012-06-13, at 12:54 AM, sebb wrote:
>>>>>>
>>>>>>> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com>
>>>> wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I figured thread pooling would be revolutionary so I wasn't
>>>> suggesting that. I would be very useful just delay the creation of a thread
>>>> until it was asked for.
>>>>>>>
>>>>>>> Not sure I understand how it would help to delay the thread creation,
>>>>>>> except perhaps for the case where the first threads have finished
>>>>>>> processing by the time the last threads start running samples.
>>>>>>
>>>>>> Bingo!!! ;-)
>>>>>
>>>>> So what percentage of use cases need to follow this model?
>>>>>
>>>>> Most of the JMeter testing I have done was long running tests where
>>>>> all threads were active for most of the run.
>>>>>
>>>>>> Kirk
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>>>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>>
>>>>
>>>
>>>
>>> --
>>> Cordialement.
>>> Philippe Mouawad.
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by Kirk Pepperdine <ki...@gmail.com>.

Hi,

I'm very happy to add details if need be.

Regards,
Kirk

On 2012-06-15, at 12:13 AM, Philippe Mouawad wrote:

> Bugzilla created:
> 
>   - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418
> 
> 
> Regards
> 
> Philippe M.
> 
> On Thu, Jun 14, 2012 at 10:30 PM, Philippe Mouawad <
> philippe.mouawad@gmail.com> wrote:
> 
>> Hi M. Pepperdine,
>> 
>> On Wed, Jun 13, 2012 at 7:05 AM, Kirk Pepperdine <
>> kirk.pepperdine@gmail.com> wrote:
>> 
>>> Hi Sebb,
>>> 
>>> We've had this conversation before and I did some preliminary work to
>>> setup a different type of thread group but the couplings between the
>>> existing thread group and the model meant that an extensive refactoring
>>> would be involved. Since that involves a *lot* more than just a simple
>>> plugin...
>>> 
>>> So, the current implementation supports a closed system model meaning,
>>> rate of entry into the system equals rate of exit from the system.This is
>>> exactly what you want if you're load testing a call centre where the number
>>> of servers (operators) is fixed and gate entry into the system. However,
>>> I'm often simulating open systems which means I do not want rate of entry
>>> into the system to be controlled by the performance of the system (rate of
>>> exit).
>> 
>> What makes you think JMeter does this ?
>> 
>> 
>>> More over, those that attend my performance tuning seminars come to
>>> understand why this is an important aspect of getting your test environment
>>> right and test harness correctly setup as it can adversely affect the
>>> quality of your test which can and often does, change the results of the
>>> test.
>>> 
>> 
>>> As an example, today I will show a group how to tune an application by a
>>> partner company. That application has a number of "performance problems"
>>> backed into it. If I use the traditional means of using JMeter I will find
>>> a different set of performance issues than if I load with a pattern that is
>>> similar that found in production.
>> 
>> Can you clarify this point ? a figure might be better than a long text
>> 
>> 
>>> In other words, with this particular application, JMeter exposes
>>> "problems" that are artifacts of how it wants to inject load on a system.
>> 
>> Not clear for me.
>> 
>> I can fix all of these problems
>> 
>> What are these problems ? and how do you fix these ?
>> 
>> 
>>> and eventually I'll get to a point where I'll fix everything that needs
>>> to be fixed. That said, if I can coerce JMeter to load as an open system
>>> I'll get to the problems without having to fix the artifacts (the things
>>> that really don't need fixing).
>> 
>> Still not clear
>> 
>>> To coerce JMeter into being an open system requires one to use a large
>>> number of very short lived threads. So I may only have 400-500 active
>>> threads at any point in time but in order to achieve that load over a 1 or
>>> two hour test I may have to specify 10s of thousands of threads. Since all
>>> of the threads are created up front, this simply doesn't work.
>>> 
>>> You might ask why not just specify 400-500 threads and loop over them? in
>>> theory you'd think it would work but as you tune the system and the
>>> performance characteristics change. Going back to the baked application,
>>> before I start tuning, the active user count is several thousand. In other
>>> words, the tuned system is better able to clear users out and that changes
>>> the performance profile in a way that hard to emulate with the current
>>> looping behaviour. Using a setting of looping 400 or so threads isn't
>>> adequate for the initial load tests as the test harness will become thread
>>> starved and that releases pressure on the server which in turn changes the
>>> performance profile.
>>> 
>> 
>> 
>>> With all due respect to the wonderful work that everyone on the project
>>> has done, it is my opinion that the one user == one thread is a design
>>> mistake that has a huge impact on both the usability of the tool
>>> 
>> Examples ?
>> 
>>> and the quality of the results
>> 
>> I disagree with this assertion . We have been using JMeter for load
>> testing all kind of applications Intranets / Large ECommerce Systems /
>> Backoffice systems / , and quality of results is good provided you
>> configure it properly.
>> Particularly when using Remote  Testing. Lot of users in this mailing list
>> use it and are satisfied (I think).
>> 
>> 
>>> one can achieve when using it. IMHO, moving to an thread pool/event heap
>>> based model would be an enormous improvement over the current
>>> implementation.
>>> 
>>> Agree it would be better. We will work on it.
>> 
>>> Regards,
>>> Kirk
>>> 
>>> On 2012-06-13, at 1:02 AM, sebb wrote:
>>> 
>>>> On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com>
>>> wrote:
>>>>> 
>>>>> On 2012-06-13, at 12:54 AM, sebb wrote:
>>>>> 
>>>>>> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com>
>>> wrote:
>>>>>>> Hi,
>>>>>>> 
>>>>>>> I figured thread pooling would be revolutionary so I wasn't
>>> suggesting that. I would be very useful just delay the creation of a thread
>>> until it was asked for.
>>>>>> 
>>>>>> Not sure I understand how it would help to delay the thread creation,
>>>>>> except perhaps for the case where the first threads have finished
>>>>>> processing by the time the last threads start running samples.
>>>>> 
>>>>> Bingo!!! ;-)
>>>> 
>>>> So what percentage of use cases need to follow this model?
>>>> 
>>>> Most of the JMeter testing I have done was long running tests where
>>>> all threads were active for most of the run.
>>>> 
>>>>> Kirk
>>>>> 
>>>>> 
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>>> 
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>> 
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>> 
>>> 
>> 
>> 
>> --
>> Cordialement.
>> Philippe Mouawad.
>> 
>> 
>> 
>> 
> 
> 
> -- 
> Cordialement.
> Philippe Mouawad.


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by Philippe Mouawad <ph...@gmail.com>.

Bugzilla created:

   - https://issues.apache.org/bugzilla/show_bug.cgi?id=53418


Regards

Philippe M.

On Thu, Jun 14, 2012 at 10:30 PM, Philippe Mouawad <
philippe.mouawad@gmail.com> wrote:

> Hi M. Pepperdine,
>
> On Wed, Jun 13, 2012 at 7:05 AM, Kirk Pepperdine <
> kirk.pepperdine@gmail.com> wrote:
>
>> Hi Sebb,
>>
>> We've had this conversation before and I did some preliminary work to
>> setup a different type of thread group but the couplings between the
>> existing thread group and the model meant that an extensive refactoring
>> would be involved. Since that involves a *lot* more than just a simple
>> plugin...
>>
>> So, the current implementation supports a closed system model meaning,
>> rate of entry into the system equals rate of exit from the system.This is
>> exactly what you want if you're load testing a call centre where the number
>> of servers (operators) is fixed and gate entry into the system. However,
>> I'm often simulating open systems which means I do not want rate of entry
>> into the system to be controlled by the performance of the system (rate of
>> exit).
>
> What makes you think JMeter does this ?
>
>
>> More over, those that attend my performance tuning seminars come to
>> understand why this is an important aspect of getting your test environment
>> right and test harness correctly setup as it can adversely affect the
>> quality of your test which can and often does, change the results of the
>> test.
>>
>
>> As an example, today I will show a group how to tune an application by a
>> partner company. That application has a number of "performance problems"
>> backed into it. If I use the traditional means of using JMeter I will find
>> a different set of performance issues than if I load with a pattern that is
>> similar that found in production.
>
> Can you clarify this point ? a figure might be better than a long text
>
>
>> In other words, with this particular application, JMeter exposes
>> "problems" that are artifacts of how it wants to inject load on a system.
>
> Not clear for me.
>
> I can fix all of these problems
>
> What are these problems ? and how do you fix these ?
>
>
>> and eventually I'll get to a point where I'll fix everything that needs
>> to be fixed. That said, if I can coerce JMeter to load as an open system
>> I'll get to the problems without having to fix the artifacts (the things
>> that really don't need fixing).
>
> Still not clear
>
>> To coerce JMeter into being an open system requires one to use a large
>> number of very short lived threads. So I may only have 400-500 active
>> threads at any point in time but in order to achieve that load over a 1 or
>> two hour test I may have to specify 10s of thousands of threads. Since all
>> of the threads are created up front, this simply doesn't work.
>>
>> You might ask why not just specify 400-500 threads and loop over them? in
>> theory you'd think it would work but as you tune the system and the
>> performance characteristics change. Going back to the baked application,
>> before I start tuning, the active user count is several thousand. In other
>> words, the tuned system is better able to clear users out and that changes
>> the performance profile in a way that hard to emulate with the current
>> looping behaviour. Using a setting of looping 400 or so threads isn't
>> adequate for the initial load tests as the test harness will become thread
>> starved and that releases pressure on the server which in turn changes the
>> performance profile.
>>
>
>
>> With all due respect to the wonderful work that everyone on the project
>> has done, it is my opinion that the one user == one thread is a design
>> mistake that has a huge impact on both the usability of the tool
>>
> Examples ?
>
>> and the quality of the results
>
> I disagree with this assertion . We have been using JMeter for load
> testing all kind of applications Intranets / Large ECommerce Systems /
> Backoffice systems / , and quality of results is good provided you
> configure it properly.
> Particularly when using Remote  Testing. Lot of users in this mailing list
> use it and are satisfied (I think).
>
>
>> one can achieve when using it. IMHO, moving to an thread pool/event heap
>> based model would be an enormous improvement over the current
>> implementation.
>>
>> Agree it would be better. We will work on it.
>
>> Regards,
>> Kirk
>>
>> On 2012-06-13, at 1:02 AM, sebb wrote:
>>
>> > On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com>
>> wrote:
>> >>
>> >> On 2012-06-13, at 12:54 AM, sebb wrote:
>> >>
>> >>> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com>
>> wrote:
>> >>>> Hi,
>> >>>>
>> >>>> I figured thread pooling would be revolutionary so I wasn't
>> suggesting that. I would be very useful just delay the creation of a thread
>> until it was asked for.
>> >>>
>> >>> Not sure I understand how it would help to delay the thread creation,
>> >>> except perhaps for the case where the first threads have finished
>> >>> processing by the time the last threads start running samples.
>> >>
>> >> Bingo!!! ;-)
>> >
>> > So what percentage of use cases need to follow this model?
>> >
>> > Most of the JMeter testing I have done was long running tests where
>> > all threads were active for most of the run.
>> >
>> >> Kirk
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> >> For additional commands, e-mail: user-help@jmeter.apache.org
>> >>
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> > For additional commands, e-mail: user-help@jmeter.apache.org
>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
>


-- 
Cordialement.
Philippe Mouawad.

Re: JMeter threading model

Posted by Kirk Pepperdine <ki...@gmail.com>.

>> 
>> of servers (operators) is fixed and gate entry into the system. However,
>> I'm often simulating open systems which means I do not want rate of entry
>> into the system to be controlled by the performance of the system (rate of
>> exit).
> 
> What makes you think JMeter does this ?

math (queuing theory) and observation from profiling JMeter.

> 
> 
> 
>> and the quality of the results
> 
> I disagree with this assertion . We have been using JMeter for load testing
> all kind of applications Intranets / Large ECommerce Systems / Backoffice
> systems / , and quality of results is good provided you configure it
> properly.
> Particularly when using Remote  Testing. Lot of users in this mailing list
> use it and are satisfied (I think).

I'm happy to engage JMeter project developers in a constructive conversation if they are interested and I shan't be offended if they choose to ignore me. What I don't feel is a the need to get into this type of a long protracted discussion. As I've said, JMeter has a lot of admiral qualities which is why I use it and have been introducing people how to use it for years. All I'm suggesting here are ways in which the tool can be improved so that scales better and has less of an artificial influence on profiling results. For example, I've tried to use it to drive a websocket app that was driving traffic to 1,000,000 simulated end users. We tried using JMeter but the threading model simply prevented it so I took on the arduous task of custom rolling an injector (not a route that I wanted to take). As any of the JMeter developers will tell you, load injectors are not trivial applications that can simply be thrown together and they require a boat load of testing. It took a number of iterations to understand the load pattern it was producing and then be able to tweak it to remove artificial artifacts that were simply a result of internal mechanics of the injector. To see this you have to study the output of the injector. Without this study of the output of the injector the bench would have failed. To their credit, the client in this case learned quite a bit from the exercise and went on to use that information to their advantage in some pretty tough competitive situations.  So, with all due respect, I doubt the vast majority of JMeter users ever bother to do any analysis of the load generated by the tool. They just simply use it and if that works, good for them. That said, the vast majority of benchmarks I deal with are busted in some fundamental way and one common way in which they are busted is in how the load injector pushed load into the unit of test. In this case JMeter's thread model can artificially bottleneck the test due to it's threading model and when that happens you can't rely on the results of that bench.. IOWs, the bench is busted and it's busted because of how the load is generated.

Regards,
Kirk


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by Philippe Mouawad <ph...@gmail.com>.

Hi M. Pepperdine,

On Wed, Jun 13, 2012 at 7:05 AM, Kirk Pepperdine
<ki...@gmail.com>wrote:

> Hi Sebb,
>
> We've had this conversation before and I did some preliminary work to
> setup a different type of thread group but the couplings between the
> existing thread group and the model meant that an extensive refactoring
> would be involved. Since that involves a *lot* more than just a simple
> plugin...
>
> So, the current implementation supports a closed system model meaning,
> rate of entry into the system equals rate of exit from the system.This is
> exactly what you want if you're load testing a call centre where the number
> of servers (operators) is fixed and gate entry into the system. However,
> I'm often simulating open systems which means I do not want rate of entry
> into the system to be controlled by the performance of the system (rate of
> exit).

What makes you think JMeter does this ?


> More over, those that attend my performance tuning seminars come to
> understand why this is an important aspect of getting your test environment
> right and test harness correctly setup as it can adversely affect the
> quality of your test which can and often does, change the results of the
> test.
>

> As an example, today I will show a group how to tune an application by a
> partner company. That application has a number of "performance problems"
> backed into it. If I use the traditional means of using JMeter I will find
> a different set of performance issues than if I load with a pattern that is
> similar that found in production.

Can you clarify this point ? a figure might be better than a long text


> In other words, with this particular application, JMeter exposes
> "problems" that are artifacts of how it wants to inject load on a system.

Not clear for me.

I can fix all of these problems

What are these problems ? and how do you fix these ?


> and eventually I'll get to a point where I'll fix everything that needs to
> be fixed. That said, if I can coerce JMeter to load as an open system I'll
> get to the problems without having to fix the artifacts (the things that
> really don't need fixing).

Still not clear

> To coerce JMeter into being an open system requires one to use a large
> number of very short lived threads. So I may only have 400-500 active
> threads at any point in time but in order to achieve that load over a 1 or
> two hour test I may have to specify 10s of thousands of threads. Since all
> of the threads are created up front, this simply doesn't work.
>
> You might ask why not just specify 400-500 threads and loop over them? in
> theory you'd think it would work but as you tune the system and the
> performance characteristics change. Going back to the baked application,
> before I start tuning, the active user count is several thousand. In other
> words, the tuned system is better able to clear users out and that changes
> the performance profile in a way that hard to emulate with the current
> looping behaviour. Using a setting of looping 400 or so threads isn't
> adequate for the initial load tests as the test harness will become thread
> starved and that releases pressure on the server which in turn changes the
> performance profile.
>


> With all due respect to the wonderful work that everyone on the project
> has done, it is my opinion that the one user == one thread is a design
> mistake that has a huge impact on both the usability of the tool
>
Examples ?

> and the quality of the results

I disagree with this assertion . We have been using JMeter for load testing
all kind of applications Intranets / Large ECommerce Systems / Backoffice
systems / , and quality of results is good provided you configure it
properly.
Particularly when using Remote  Testing. Lot of users in this mailing list
use it and are satisfied (I think).


> one can achieve when using it. IMHO, moving to an thread pool/event heap
> based model would be an enormous improvement over the current
> implementation.
>
> Agree it would be better. We will work on it.

> Regards,
> Kirk
>
> On 2012-06-13, at 1:02 AM, sebb wrote:
>
> > On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com>
> wrote:
> >>
> >> On 2012-06-13, at 12:54 AM, sebb wrote:
> >>
> >>> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com>
> wrote:
> >>>> Hi,
> >>>>
> >>>> I figured thread pooling would be revolutionary so I wasn't
> suggesting that. I would be very useful just delay the creation of a thread
> until it was asked for.
> >>>
> >>> Not sure I understand how it would help to delay the thread creation,
> >>> except perhaps for the case where the first threads have finished
> >>> processing by the time the last threads start running samples.
> >>
> >> Bingo!!! ;-)
> >
> > So what percentage of use cases need to follow this model?
> >
> > Most of the JMeter testing I have done was long running tests where
> > all threads were active for most of the run.
> >
> >> Kirk
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >> For additional commands, e-mail: user-help@jmeter.apache.org
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> > For additional commands, e-mail: user-help@jmeter.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>


-- 
Cordialement.
Philippe Mouawad.

Re: JMeter threading model

Posted by sebb <se...@gmail.com>.

On 13 June 2012 06:05, Kirk Pepperdine <ki...@gmail.com> wrote:
> Hi Sebb,
>
> We've had this conversation before and I did some preliminary work to setup a different type of thread group but the couplings between the existing thread group and the model meant that an extensive refactoring would be involved. Since that involves a *lot* more than just a simple plugin...
>
> So, the current implementation supports a closed system model meaning, rate of entry into the system equals rate of exit from the system. This is exactly what you want if you're load testing a call centre where the number of servers (operators) is fixed and gate entry into the system. However, I'm often simulating open systems which means I do not want rate of entry into the system to be controlled by the performance of the system (rate of exit). More over, those that attend my performance tuning seminars come to understand why this is an important aspect of getting your test environment right and test harness correctly setup as it can adversely affect the quality of your test which can and often does, change the results of the test.
>
> As an example, today I will show a group how to tune an application by a partner company. That application has a number of "performance problems" backed into it. If I use the traditional means of using JMeter I will find a different set of performance issues than if I load with a pattern that is similar that found in production. In other words, with this particular application, JMeter exposes "problems" that are artifacts of how it wants to inject load on a system. I can fix all of these problems and eventually I'll get to a point where I'll fix everything that needs to be fixed. That said, if I can coerce JMeter to load as an open system I'll get to the problems without having to fix the artifacts (the things that really don't need fixing). To coerce JMeter into being an open system requires one to use a large number of very short lived threads. So I may only have 400-500 active threads at any point in time but in order to achieve that load over a 1 or two hour test I may have to specify 10s of thousands of threads. Since all of the threads are created up front, this simply doesn't work.
>
> You might ask why not just specify 400-500 threads and loop over them? in theory you'd think it would work but as you tune the system and the performance characteristics change. Going back to the baked application, before I start tuning, the active user count is several thousand. In other words, the tuned system is better able to clear users out and that changes the performance profile in a way that hard to emulate with the current looping behaviour. Using a setting of looping 400 or so threads isn't adequate for the initial load tests as the test harness will become thread starved and that releases pressure on the server which in turn changes the performance profile.

Not sure I follow how using a thread pool will help, if at some point
you still need to have thousands of active users. Surely that will
need 1000s of pool entries?

> With all due respect to the wonderful work that everyone on the project has done, it is my opinion that the one user == one thread is a design mistake that has a huge impact on both the usability of the tool and the quality of the results one can achieve when using it. IMHO, moving to an thread pool/event heap based model would be an enormous improvement over the current implementation.

That may be so, but probably means a complete redesign.
This would likely mean all 3rd party plugins would have to be reworked as well.

> Regards,
> Kirk
>
> On 2012-06-13, at 1:02 AM, sebb wrote:
>
>> On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com> wrote:
>>>
>>> On 2012-06-13, at 12:54 AM, sebb wrote:
>>>
>>>> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> I figured thread pooling would be revolutionary so I wasn't suggesting that. I would be very useful just delay the creation of a thread until it was asked for.
>>>>
>>>> Not sure I understand how it would help to delay the thread creation,
>>>> except perhaps for the case where the first threads have finished
>>>> processing by the time the last threads start running samples.
>>>
>>> Bingo!!! ;-)
>>
>> So what percentage of use cases need to follow this model?
>>
>> Most of the JMeter testing I have done was long running tests where
>> all threads were active for most of the run.
>>
>>> Kirk
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by Kirk Pepperdine <ki...@gmail.com>.

Hi Sebb,

We've had this conversation before and I did some preliminary work to setup a different type of thread group but the couplings between the existing thread group and the model meant that an extensive refactoring would be involved. Since that involves a *lot* more than just a simple plugin...

So, the current implementation supports a closed system model meaning, rate of entry into the system equals rate of exit from the system. This is exactly what you want if you're load testing a call centre where the number of servers (operators) is fixed and gate entry into the system. However, I'm often simulating open systems which means I do not want rate of entry into the system to be controlled by the performance of the system (rate of exit). More over, those that attend my performance tuning seminars come to understand why this is an important aspect of getting your test environment right and test harness correctly setup as it can adversely affect the quality of your test which can and often does, change the results of the test.

As an example, today I will show a group how to tune an application by a partner company. That application has a number of "performance problems" backed into it. If I use the traditional means of using JMeter I will find a different set of performance issues than if I load with a pattern that is similar that found in production. In other words, with this particular application, JMeter exposes "problems" that are artifacts of how it wants to inject load on a system. I can fix all of these problems and eventually I'll get to a point where I'll fix everything that needs to be fixed. That said, if I can coerce JMeter to load as an open system I'll get to the problems without having to fix the artifacts (the things that really don't need fixing). To coerce JMeter into being an open system requires one to use a large number of very short lived threads. So I may only have 400-500 active threads at any point in time but in order to achieve that load over a 1 or two hour test I may have to specify 10s of thousands of threads. Since all of the threads are created up front, this simply doesn't work.

You might ask why not just specify 400-500 threads and loop over them? in theory you'd think it would work but as you tune the system and the performance characteristics change. Going back to the baked application, before I start tuning, the active user count is several thousand. In other words, the tuned system is better able to clear users out and that changes the performance profile in a way that hard to emulate with the current looping behaviour. Using a setting of looping 400 or so threads isn't adequate for the initial load tests as the test harness will become thread starved and that releases pressure on the server which in turn changes the performance profile.

With all due respect to the wonderful work that everyone on the project has done, it is my opinion that the one user == one thread is a design mistake that has a huge impact on both the usability of the tool and the quality of the results one can achieve when using it. IMHO, moving to an thread pool/event heap based model would be an enormous improvement over the current implementation. 

Regards,
Kirk

On 2012-06-13, at 1:02 AM, sebb wrote:

> On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com> wrote:
>> 
>> On 2012-06-13, at 12:54 AM, sebb wrote:
>> 
>>> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com> wrote:
>>>> Hi,
>>>> 
>>>> I figured thread pooling would be revolutionary so I wasn't suggesting that. I would be very useful just delay the creation of a thread until it was asked for.
>>> 
>>> Not sure I understand how it would help to delay the thread creation,
>>> except perhaps for the case where the first threads have finished
>>> processing by the time the last threads start running samples.
>> 
>> Bingo!!! ;-)
> 
> So what percentage of use cases need to follow this model?
> 
> Most of the JMeter testing I have done was long running tests where
> all threads were active for most of the run.
> 
>> Kirk
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by sebb <se...@gmail.com>.

On 13 June 2012 00:39, Robin D. Wilson <rw...@gmail.com> wrote:
> sebb wrote:
>>For tests where a CSV data file is not appropriate or convenient, it
>>might make sense to implement a feature which counts total loops and
>>causes the test to stop when a pre-specified limit is reached. Perhaps
>>at thread group level, which already has a duration limit. Or a test
>>element of some kind.
>
> At the thread group level is exactly what I'm suggesting...
>
> In the meantime, I will setup a csv file with 0-9999 in it (one number per row), and see if I can get that to work like you suggest.

I would start with a shorter file and simple test plan ...

> Seems a little clunky, but if it works - no issue.

Just realised, another possibility is:

Thread Group
+ If Controller, condition: ${__counter(FALSE)} > 10000
+ + Test Action/Stop Thread.
+ rest of thread samples

As the Perl mantra goes: TMTOWTDI

> --
> Robin D. Wilson
> Sr. Director of Web Development
> KingsIsle Entertainment, Inc.
> VOICE: 512-777-1861
> www.KingsIsle.com
>
>
> -----Original Message-----
> From: sebb [mailto:sebbaz@gmail.com]
> Sent: Tuesday, June 12, 2012 6:07 PM
> To: JMeter Users List
> Subject: Re: JMeter threading model
>
> On 12 June 2012 23:42, Robin D. Wilson <rw...@gmail.com> wrote:
>> Currently, JMeter will run 100 threads, for 100 iterations each in order to reach 10,000 total passes through my test case. If
>> thread 1 completes its 100 iterations before any of the rest of the threads, it terminates and then I only have 99 threads running
>> until the 10,000 iterations complete. My test will show a performance change at the end - as the number of total simultaneous
>> threads begins to trail off, the performance will appear to improve. For example, I will show the sampler as having completed 9200
>> requests when the first thread finishes its 100 iterations. This means for the remaining 800 requests, I only have a maximum 99
>> threads operating. Then at 9300 samples, 19 more threads might have died off - so I only have 80 threads max operating for the
>> remaining 700 requests. As you can imagine, this means that by the end of the test - the performance numbers will start to
>> progressively improve (since fewer threads means less workload on the process being measured, and therefore faster response
> times).
>>
>> It would be really nice if JMeter just kept a pool of 100 threads operating on requests for the duration of the 10,000 iterations,
>> so that threads would only die off during the final iteration, leaving the server at more-or-less peak load throughout the test.
>
> JMeter can do this already.
> Rather than specify a fixed number of loops, use a CSV Config file
> with 10,000 entries, and set it to stop the thread (not the test, as
> that would not let some threads complete) when EOF is reached.
>
> For tests where a CSV data file is not appropriate or convenient, it
> might make sense to implement a feature which counts total loops and
> causes the test to stop when a pre-specified limit is reached. Perhaps
> at thread group level, which already has a duration limit. Or a test
> element of some kind.
>
>> From a code standpoint, this doesn't seem like it would be too hard to setup - just identify how many total iterations need to be
>> run through the thread group, startup the total number of threads you need, and let each thread keep going until all the
> iterations
>> have been started. (Of course, I say that knowing that I'm just a 'manager' type, and won't be coding it myself...)
>>
>> --
>> Robin D. Wilson
>> Sr. Director of Web Development
>> KingsIsle Entertainment, Inc.
>> VOICE: 512-777-1861
>> www.KingsIsle.com
>>
>>
>> -----Original Message-----
>> From: sebb [mailto:sebbaz@gmail.com]
>> Sent: Tuesday, June 12, 2012 5:02 PM
>> To: JMeter Users List
>> Subject: Re: JMeter threading model
>>
>> On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com> wrote:
>>>
>>> On 2012-06-13, at 12:54 AM, sebb wrote:
>>>
>>>> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> I figured thread pooling would be revolutionary so I wasn't suggesting that. I would be very useful just delay the creation of
> a
>> thread until it was asked for.
>>>>
>>>> Not sure I understand how it would help to delay the thread creation,
>>>> except perhaps for the case where the first threads have finished
>>>> processing by the time the last threads start running samples.
>>>
>>> Bingo!!! ;-)
>>
>> So what percentage of use cases need to follow this model?
>>
>> Most of the JMeter testing I have done was long running tests where
>> all threads were active for most of the run.
>>
>>> Kirk
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

RE: JMeter threading model

Posted by "Robin D. Wilson" <rw...@gmail.com>.

sebb wrote:
>For tests where a CSV data file is not appropriate or convenient, it
>might make sense to implement a feature which counts total loops and
>causes the test to stop when a pre-specified limit is reached. Perhaps
>at thread group level, which already has a duration limit. Or a test
>element of some kind.

At the thread group level is exactly what I'm suggesting...

In the meantime, I will setup a csv file with 0-9999 in it (one number per row), and see if I can get that to work like you suggest.
Seems a little clunky, but if it works - no issue.

--
Robin D. Wilson
Sr. Director of Web Development
KingsIsle Entertainment, Inc.
VOICE: 512-777-1861
www.KingsIsle.com


-----Original Message-----
From: sebb [mailto:sebbaz@gmail.com] 
Sent: Tuesday, June 12, 2012 6:07 PM
To: JMeter Users List
Subject: Re: JMeter threading model

On 12 June 2012 23:42, Robin D. Wilson <rw...@gmail.com> wrote:
> Currently, JMeter will run 100 threads, for 100 iterations each in order to reach 10,000 total passes through my test case. If
> thread 1 completes its 100 iterations before any of the rest of the threads, it terminates and then I only have 99 threads running
> until the 10,000 iterations complete. My test will show a performance change at the end - as the number of total simultaneous
> threads begins to trail off, the performance will appear to improve. For example, I will show the sampler as having completed 9200
> requests when the first thread finishes its 100 iterations. This means for the remaining 800 requests, I only have a maximum 99
> threads operating. Then at 9300 samples, 19 more threads might have died off - so I only have 80 threads max operating for the
> remaining 700 requests. As you can imagine, this means that by the end of the test - the performance numbers will start to
> progressively improve (since fewer threads means less workload on the process being measured, and therefore faster response
times).
>
> It would be really nice if JMeter just kept a pool of 100 threads operating on requests for the duration of the 10,000 iterations,
> so that threads would only die off during the final iteration, leaving the server at more-or-less peak load throughout the test.

JMeter can do this already.
Rather than specify a fixed number of loops, use a CSV Config file
with 10,000 entries, and set it to stop the thread (not the test, as
that would not let some threads complete) when EOF is reached.

For tests where a CSV data file is not appropriate or convenient, it
might make sense to implement a feature which counts total loops and
causes the test to stop when a pre-specified limit is reached. Perhaps
at thread group level, which already has a duration limit. Or a test
element of some kind.

> From a code standpoint, this doesn't seem like it would be too hard to setup - just identify how many total iterations need to be
> run through the thread group, startup the total number of threads you need, and let each thread keep going until all the
iterations
> have been started. (Of course, I say that knowing that I'm just a 'manager' type, and won't be coding it myself...)
>
> --
> Robin D. Wilson
> Sr. Director of Web Development
> KingsIsle Entertainment, Inc.
> VOICE: 512-777-1861
> www.KingsIsle.com
>
>
> -----Original Message-----
> From: sebb [mailto:sebbaz@gmail.com]
> Sent: Tuesday, June 12, 2012 5:02 PM
> To: JMeter Users List
> Subject: Re: JMeter threading model
>
> On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com> wrote:
>>
>> On 2012-06-13, at 12:54 AM, sebb wrote:
>>
>>> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> I figured thread pooling would be revolutionary so I wasn't suggesting that. I would be very useful just delay the creation of
a
> thread until it was asked for.
>>>
>>> Not sure I understand how it would help to delay the thread creation,
>>> except perhaps for the case where the first threads have finished
>>> processing by the time the last threads start running samples.
>>
>> Bingo!!! ;-)
>
> So what percentage of use cases need to follow this model?
>
> Most of the JMeter testing I have done was long running tests where
> all threads were active for most of the run.
>
>> Kirk
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by sebb <se...@gmail.com>.

On 13 June 2012 01:14, Anthony Johnson <an...@gmail.com> wrote:
> On Tue, Jun 12, 2012 at 7:06 PM, sebb <se...@gmail.com> wrote:
>>
>> On 12 June 2012 23:42, Robin D. Wilson <rw...@gmail.com> wrote:
>> > Currently, JMeter will run 100 threads, for 100 iterations each in order
>> > to reach 10,000 total passes through my test case. If
>> > thread 1 completes its 100 iterations before any of the rest of the
>> > threads, it terminates and then I only have 99 threads running
>> > until the 10,000 iterations complete. My test will show a performance
>> > change at the end - as the number of total simultaneous
>> > threads begins to trail off, the performance will appear to improve. For
>> > example, I will show the sampler as having completed 9200
>> > requests when the first thread finishes its 100 iterations. This means
>> > for the remaining 800 requests, I only have a maximum 99
>> > threads operating. Then at 9300 samples, 19 more threads might have died
>> > off - so I only have 80 threads max operating for the
>> > remaining 700 requests. As you can imagine, this means that by the end
>> > of the test - the performance numbers will start to
>> > progressively improve (since fewer threads means less workload on the
>> > process being measured, and therefore faster response times).
>> >
>> > It would be really nice if JMeter just kept a pool of 100 threads
>> > operating on requests for the duration of the 10,000 iterations,
>> > so that threads would only die off during the final iteration, leaving
>> > the server at more-or-less peak load throughout the test.
>>
>> JMeter can do this already.
>> Rather than specify a fixed number of loops, use a CSV Config file
>> with 10,000 entries, and set it to stop the thread (not the test, as
>> that would not let some threads complete) when EOF is reached.
>>
>> For tests where a CSV data file is not appropriate or convenient, it
>> might make sense to implement a feature which counts total loops and
>> causes the test to stop when a pre-specified limit is reached. Perhaps
>> at thread group level, which already has a duration limit. Or a test
>> element of some kind.
>
> It would be nice if their was a drop-in element for this.  My team
> created a test flow that did exactly this, but it was more work than
> it should of been.

There's a drop-in element pair:

+ If Controller, condition: ${__counter(FALSE)} > 10000
+ + Test Action/Stop Thread.

Theres a trade-off here: the work to add specialised elements (and
documenting/maintaining them) versus the user needing to combine
existing elements to perform the same function. And of course the more
elements there are, the more the user needs to read to find the exact
element they need.

>>
>> > From a code standpoint, this doesn't seem like it would be too hard to
>> > setup - just identify how many total iterations need to be
>> > run through the thread group, startup the total number of threads you
>> > need, and let each thread keep going until all the iterations
>> > have been started. (Of course, I say that knowing that I'm just a
>> > 'manager' type, and won't be coding it myself...)
>> >
>> > --
>> > Robin D. Wilson
>> > Sr. Director of Web Development
>> > KingsIsle Entertainment, Inc.
>> > VOICE: 512-777-1861
>> > www.KingsIsle.com
>> >
>> >
>> > -----Original Message-----
>> > From: sebb [mailto:sebbaz@gmail.com]
>> > Sent: Tuesday, June 12, 2012 5:02 PM
>> > To: JMeter Users List
>> > Subject: Re: JMeter threading model
>> >
>> > On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com>
>> > wrote:
>> >>
>> >> On 2012-06-13, at 12:54 AM, sebb wrote:
>> >>
>> >>> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com>
>> >>> wrote:
>> >>>> Hi,
>> >>>>
>> >>>> I figured thread pooling would be revolutionary so I wasn't
>> >>>> suggesting that. I would be very useful just delay the creation of a
>> > thread until it was asked for.
>> >>>
>> >>> Not sure I understand how it would help to delay the thread creation,
>> >>> except perhaps for the case where the first threads have finished
>> >>> processing by the time the last threads start running samples.
>> >>
>> >> Bingo!!! ;-)
>> >
>> > So what percentage of use cases need to follow this model?
>> >
>> > Most of the JMeter testing I have done was long running tests where
>> > all threads were active for most of the run.
>> >
>> >> Kirk
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> >> For additional commands, e-mail: user-help@jmeter.apache.org
>> >>
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> > For additional commands, e-mail: user-help@jmeter.apache.org
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> > For additional commands, e-mail: user-help@jmeter.apache.org
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by Anthony Johnson <an...@gmail.com>.

On Tue, Jun 12, 2012 at 7:06 PM, sebb <se...@gmail.com> wrote:
>
> On 12 June 2012 23:42, Robin D. Wilson <rw...@gmail.com> wrote:
> > Currently, JMeter will run 100 threads, for 100 iterations each in order
> > to reach 10,000 total passes through my test case. If
> > thread 1 completes its 100 iterations before any of the rest of the
> > threads, it terminates and then I only have 99 threads running
> > until the 10,000 iterations complete. My test will show a performance
> > change at the end - as the number of total simultaneous
> > threads begins to trail off, the performance will appear to improve. For
> > example, I will show the sampler as having completed 9200
> > requests when the first thread finishes its 100 iterations. This means
> > for the remaining 800 requests, I only have a maximum 99
> > threads operating. Then at 9300 samples, 19 more threads might have died
> > off - so I only have 80 threads max operating for the
> > remaining 700 requests. As you can imagine, this means that by the end
> > of the test - the performance numbers will start to
> > progressively improve (since fewer threads means less workload on the
> > process being measured, and therefore faster response times).
> >
> > It would be really nice if JMeter just kept a pool of 100 threads
> > operating on requests for the duration of the 10,000 iterations,
> > so that threads would only die off during the final iteration, leaving
> > the server at more-or-less peak load throughout the test.
>
> JMeter can do this already.
> Rather than specify a fixed number of loops, use a CSV Config file
> with 10,000 entries, and set it to stop the thread (not the test, as
> that would not let some threads complete) when EOF is reached.
>
> For tests where a CSV data file is not appropriate or convenient, it
> might make sense to implement a feature which counts total loops and
> causes the test to stop when a pre-specified limit is reached. Perhaps
> at thread group level, which already has a duration limit. Or a test
> element of some kind.

It would be nice if their was a drop-in element for this.  My team
created a test flow that did exactly this, but it was more work than
it should of been.

>
> > From a code standpoint, this doesn't seem like it would be too hard to
> > setup - just identify how many total iterations need to be
> > run through the thread group, startup the total number of threads you
> > need, and let each thread keep going until all the iterations
> > have been started. (Of course, I say that knowing that I'm just a
> > 'manager' type, and won't be coding it myself...)
> >
> > --
> > Robin D. Wilson
> > Sr. Director of Web Development
> > KingsIsle Entertainment, Inc.
> > VOICE: 512-777-1861
> > www.KingsIsle.com
> >
> >
> > -----Original Message-----
> > From: sebb [mailto:sebbaz@gmail.com]
> > Sent: Tuesday, June 12, 2012 5:02 PM
> > To: JMeter Users List
> > Subject: Re: JMeter threading model
> >
> > On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com>
> > wrote:
> >>
> >> On 2012-06-13, at 12:54 AM, sebb wrote:
> >>
> >>> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com>
> >>> wrote:
> >>>> Hi,
> >>>>
> >>>> I figured thread pooling would be revolutionary so I wasn't
> >>>> suggesting that. I would be very useful just delay the creation of a
> > thread until it was asked for.
> >>>
> >>> Not sure I understand how it would help to delay the thread creation,
> >>> except perhaps for the case where the first threads have finished
> >>> processing by the time the last threads start running samples.
> >>
> >> Bingo!!! ;-)
> >
> > So what percentage of use cases need to follow this model?
> >
> > Most of the JMeter testing I have done was long running tests where
> > all threads were active for most of the run.
> >
> >> Kirk
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> >> For additional commands, e-mail: user-help@jmeter.apache.org
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> > For additional commands, e-mail: user-help@jmeter.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> > For additional commands, e-mail: user-help@jmeter.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by sebb <se...@gmail.com>.

On 12 June 2012 23:42, Robin D. Wilson <rw...@gmail.com> wrote:
> Currently, JMeter will run 100 threads, for 100 iterations each in order to reach 10,000 total passes through my test case. If
> thread 1 completes its 100 iterations before any of the rest of the threads, it terminates and then I only have 99 threads running
> until the 10,000 iterations complete. My test will show a performance change at the end - as the number of total simultaneous
> threads begins to trail off, the performance will appear to improve. For example, I will show the sampler as having completed 9200
> requests when the first thread finishes its 100 iterations. This means for the remaining 800 requests, I only have a maximum 99
> threads operating. Then at 9300 samples, 19 more threads might have died off - so I only have 80 threads max operating for the
> remaining 700 requests. As you can imagine, this means that by the end of the test - the performance numbers will start to
> progressively improve (since fewer threads means less workload on the process being measured, and therefore faster response times).
>
> It would be really nice if JMeter just kept a pool of 100 threads operating on requests for the duration of the 10,000 iterations,
> so that threads would only die off during the final iteration, leaving the server at more-or-less peak load throughout the test.

JMeter can do this already.
Rather than specify a fixed number of loops, use a CSV Config file
with 10,000 entries, and set it to stop the thread (not the test, as
that would not let some threads complete) when EOF is reached.

For tests where a CSV data file is not appropriate or convenient, it
might make sense to implement a feature which counts total loops and
causes the test to stop when a pre-specified limit is reached. Perhaps
at thread group level, which already has a duration limit. Or a test
element of some kind.

> From a code standpoint, this doesn't seem like it would be too hard to setup - just identify how many total iterations need to be
> run through the thread group, startup the total number of threads you need, and let each thread keep going until all the iterations
> have been started. (Of course, I say that knowing that I'm just a 'manager' type, and won't be coding it myself...)
>
> --
> Robin D. Wilson
> Sr. Director of Web Development
> KingsIsle Entertainment, Inc.
> VOICE: 512-777-1861
> www.KingsIsle.com
>
>
> -----Original Message-----
> From: sebb [mailto:sebbaz@gmail.com]
> Sent: Tuesday, June 12, 2012 5:02 PM
> To: JMeter Users List
> Subject: Re: JMeter threading model
>
> On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com> wrote:
>>
>> On 2012-06-13, at 12:54 AM, sebb wrote:
>>
>>> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> I figured thread pooling would be revolutionary so I wasn't suggesting that. I would be very useful just delay the creation of a
> thread until it was asked for.
>>>
>>> Not sure I understand how it would help to delay the thread creation,
>>> except perhaps for the case where the first threads have finished
>>> processing by the time the last threads start running samples.
>>
>> Bingo!!! ;-)
>
> So what percentage of use cases need to follow this model?
>
> Most of the JMeter testing I have done was long running tests where
> all threads were active for most of the run.
>
>> Kirk
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by Deepak Shetty <sh...@gmail.com>.

sorry i think i misread your email.

>But it would make my test a better measure of operating
>characteristics of my system 'under load' to have it consistently maintain
the full number of requesting threads throughout the
>test.
Cant you already do this? fix the duration of the test instead of the
number of times you want to run .
still seems unrelated to kirk's request :)



On Tue, Jun 12, 2012 at 4:01 PM, Robin D. Wilson <rw...@gmail.com> wrote:

> I'm not sure I follow...
>
> My JMeter test runs on a single Win7 box. My test environment consists of
> a web server, multiple Java App servers and a database
> server. The performance appears to improve because I'm hitting the test
> environment with fewer and fewer threads as the test begins
> to wrap up. I'm just saying that it would be nice if I could delay that
> drop-off until I get to the last 100 samples, rather than
> several hundred samples before the test ends. It probably doesn't make a
> big difference to me - since I'm really just comparing the
> results of the last time I ran the test against the results of this time.
> But it would make my test a better measure of operating
> characteristics of my system 'under load' to have it consistently maintain
> the full number of requesting threads throughout the
> test.
>
> --
> Robin D. Wilson
> Sr. Director of Web Development
> KingsIsle Entertainment, Inc.
> VOICE: 512-777-1861
> www.KingsIsle.com
>
>
> -----Original Message-----
> From: Deepak Shetty [mailto:shettyd@gmail.com]
> Sent: Tuesday, June 12, 2012 5:47 PM
> To: JMeter Users List
> Subject: Re: JMeter threading model
>
> >My test will show a performance change at the end - as the number of total
> simultaneous threads begins to trail off, the performance will appear to
> improve.
> If so , your test needs more than 1 machine to run and the threads on
> demand wont really help your use case.
>
>
> On Tue, Jun 12, 2012 at 3:42 PM, Robin D. Wilson <rw...@gmail.com>
> wrote:
>
> > Currently, JMeter will run 100 threads, for 100 iterations each in order
> > to reach 10,000 total passes through my test case. If
> > thread 1 completes its 100 iterations before any of the rest of the
> > threads, it terminates and then I only have 99 threads running
> > until the 10,000 iterations complete. My test will show a performance
> > change at the end - as the number of total simultaneous
> > threads begins to trail off, the performance will appear to improve. For
> > example, I will show the sampler as having completed 9200
> > requests when the first thread finishes its 100 iterations. This means
> for
> > the remaining 800 requests, I only have a maximum 99
> > threads operating. Then at 9300 samples, 19 more threads might have died
> > off - so I only have 80 threads max operating for the
> > remaining 700 requests. As you can imagine, this means that by the end of
> > the test - the performance numbers will start to
> > progressively improve (since fewer threads means less workload on the
> > process being measured, and therefore faster response times).
> >
> > It would be really nice if JMeter just kept a pool of 100 threads
> > operating on requests for the duration of the 10,000 iterations,
> > so that threads would only die off during the final iteration, leaving
> the
> > server at more-or-less peak load throughout the test.
> >
> > From a code standpoint, this doesn't seem like it would be too hard to
> > setup - just identify how many total iterations need to be
> > run through the thread group, startup the total number of threads you
> > need, and let each thread keep going until all the iterations
> > have been started. (Of course, I say that knowing that I'm just a
> > 'manager' type, and won't be coding it myself...)
> >
> > --
> > Robin D. Wilson
> > Sr. Director of Web Development
> > KingsIsle Entertainment, Inc.
> > VOICE: 512-777-1861
> > www.KingsIsle.com
> >
> >
> > -----Original Message-----
> > From: sebb [mailto:sebbaz@gmail.com]
> > Sent: Tuesday, June 12, 2012 5:02 PM
> > To: JMeter Users List
> > Subject: Re: JMeter threading model
> >
> > On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com>
> wrote:
> > >
> > > On 2012-06-13, at 12:54 AM, sebb wrote:
> > >
> > >> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com>
> > wrote:
> > >>> Hi,
> > >>>
> > >>> I figured thread pooling would be revolutionary so I wasn't
> suggesting
> > that. I would be very useful just delay the creation of a
> > thread until it was asked for.
> > >>
> > >> Not sure I understand how it would help to delay the thread creation,
> > >> except perhaps for the case where the first threads have finished
> > >> processing by the time the last threads start running samples.
> > >
> > > Bingo!!! ;-)
> >
> > So what percentage of use cases need to follow this model?
> >
> > Most of the JMeter testing I have done was long running tests where
> > all threads were active for most of the run.
> >
> > > Kirk
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> > > For additional commands, e-mail: user-help@jmeter.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> > For additional commands, e-mail: user-help@jmeter.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> > For additional commands, e-mail: user-help@jmeter.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>

RE: JMeter threading model

Posted by "Robin D. Wilson" <rw...@gmail.com>.

I'm not sure I follow... 

My JMeter test runs on a single Win7 box. My test environment consists of a web server, multiple Java App servers and a database
server. The performance appears to improve because I'm hitting the test environment with fewer and fewer threads as the test begins
to wrap up. I'm just saying that it would be nice if I could delay that drop-off until I get to the last 100 samples, rather than
several hundred samples before the test ends. It probably doesn't make a big difference to me - since I'm really just comparing the
results of the last time I ran the test against the results of this time. But it would make my test a better measure of operating
characteristics of my system 'under load' to have it consistently maintain the full number of requesting threads throughout the
test.

--
Robin D. Wilson
Sr. Director of Web Development
KingsIsle Entertainment, Inc.
VOICE: 512-777-1861
www.KingsIsle.com


-----Original Message-----
From: Deepak Shetty [mailto:shettyd@gmail.com] 
Sent: Tuesday, June 12, 2012 5:47 PM
To: JMeter Users List
Subject: Re: JMeter threading model

>My test will show a performance change at the end - as the number of total
simultaneous threads begins to trail off, the performance will appear to
improve.
If so , your test needs more than 1 machine to run and the threads on
demand wont really help your use case.


On Tue, Jun 12, 2012 at 3:42 PM, Robin D. Wilson <rw...@gmail.com> wrote:

> Currently, JMeter will run 100 threads, for 100 iterations each in order
> to reach 10,000 total passes through my test case. If
> thread 1 completes its 100 iterations before any of the rest of the
> threads, it terminates and then I only have 99 threads running
> until the 10,000 iterations complete. My test will show a performance
> change at the end - as the number of total simultaneous
> threads begins to trail off, the performance will appear to improve. For
> example, I will show the sampler as having completed 9200
> requests when the first thread finishes its 100 iterations. This means for
> the remaining 800 requests, I only have a maximum 99
> threads operating. Then at 9300 samples, 19 more threads might have died
> off - so I only have 80 threads max operating for the
> remaining 700 requests. As you can imagine, this means that by the end of
> the test - the performance numbers will start to
> progressively improve (since fewer threads means less workload on the
> process being measured, and therefore faster response times).
>
> It would be really nice if JMeter just kept a pool of 100 threads
> operating on requests for the duration of the 10,000 iterations,
> so that threads would only die off during the final iteration, leaving the
> server at more-or-less peak load throughout the test.
>
> From a code standpoint, this doesn't seem like it would be too hard to
> setup - just identify how many total iterations need to be
> run through the thread group, startup the total number of threads you
> need, and let each thread keep going until all the iterations
> have been started. (Of course, I say that knowing that I'm just a
> 'manager' type, and won't be coding it myself...)
>
> --
> Robin D. Wilson
> Sr. Director of Web Development
> KingsIsle Entertainment, Inc.
> VOICE: 512-777-1861
> www.KingsIsle.com
>
>
> -----Original Message-----
> From: sebb [mailto:sebbaz@gmail.com]
> Sent: Tuesday, June 12, 2012 5:02 PM
> To: JMeter Users List
> Subject: Re: JMeter threading model
>
> On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com> wrote:
> >
> > On 2012-06-13, at 12:54 AM, sebb wrote:
> >
> >> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com>
> wrote:
> >>> Hi,
> >>>
> >>> I figured thread pooling would be revolutionary so I wasn't suggesting
> that. I would be very useful just delay the creation of a
> thread until it was asked for.
> >>
> >> Not sure I understand how it would help to delay the thread creation,
> >> except perhaps for the case where the first threads have finished
> >> processing by the time the last threads start running samples.
> >
> > Bingo!!! ;-)
>
> So what percentage of use cases need to follow this model?
>
> Most of the JMeter testing I have done was long running tests where
> all threads were active for most of the run.
>
> > Kirk
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> > For additional commands, e-mail: user-help@jmeter.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by Deepak Shetty <sh...@gmail.com>.

>My test will show a performance change at the end - as the number of total
simultaneous threads begins to trail off, the performance will appear to
improve.
If so , your test needs more than 1 machine to run and the threads on
demand wont really help your use case.


On Tue, Jun 12, 2012 at 3:42 PM, Robin D. Wilson <rw...@gmail.com> wrote:

> Currently, JMeter will run 100 threads, for 100 iterations each in order
> to reach 10,000 total passes through my test case. If
> thread 1 completes its 100 iterations before any of the rest of the
> threads, it terminates and then I only have 99 threads running
> until the 10,000 iterations complete. My test will show a performance
> change at the end - as the number of total simultaneous
> threads begins to trail off, the performance will appear to improve. For
> example, I will show the sampler as having completed 9200
> requests when the first thread finishes its 100 iterations. This means for
> the remaining 800 requests, I only have a maximum 99
> threads operating. Then at 9300 samples, 19 more threads might have died
> off - so I only have 80 threads max operating for the
> remaining 700 requests. As you can imagine, this means that by the end of
> the test - the performance numbers will start to
> progressively improve (since fewer threads means less workload on the
> process being measured, and therefore faster response times).
>
> It would be really nice if JMeter just kept a pool of 100 threads
> operating on requests for the duration of the 10,000 iterations,
> so that threads would only die off during the final iteration, leaving the
> server at more-or-less peak load throughout the test.
>
> From a code standpoint, this doesn't seem like it would be too hard to
> setup - just identify how many total iterations need to be
> run through the thread group, startup the total number of threads you
> need, and let each thread keep going until all the iterations
> have been started. (Of course, I say that knowing that I'm just a
> 'manager' type, and won't be coding it myself...)
>
> --
> Robin D. Wilson
> Sr. Director of Web Development
> KingsIsle Entertainment, Inc.
> VOICE: 512-777-1861
> www.KingsIsle.com
>
>
> -----Original Message-----
> From: sebb [mailto:sebbaz@gmail.com]
> Sent: Tuesday, June 12, 2012 5:02 PM
> To: JMeter Users List
> Subject: Re: JMeter threading model
>
> On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com> wrote:
> >
> > On 2012-06-13, at 12:54 AM, sebb wrote:
> >
> >> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com>
> wrote:
> >>> Hi,
> >>>
> >>> I figured thread pooling would be revolutionary so I wasn't suggesting
> that. I would be very useful just delay the creation of a
> thread until it was asked for.
> >>
> >> Not sure I understand how it would help to delay the thread creation,
> >> except perhaps for the case where the first threads have finished
> >> processing by the time the last threads start running samples.
> >
> > Bingo!!! ;-)
>
> So what percentage of use cases need to follow this model?
>
> Most of the JMeter testing I have done was long running tests where
> all threads were active for most of the run.
>
> > Kirk
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> > For additional commands, e-mail: user-help@jmeter.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>

RE: JMeter threading model

Posted by "Robin D. Wilson" <rw...@gmail.com>.

Currently, JMeter will run 100 threads, for 100 iterations each in order to reach 10,000 total passes through my test case. If
thread 1 completes its 100 iterations before any of the rest of the threads, it terminates and then I only have 99 threads running
until the 10,000 iterations complete. My test will show a performance change at the end - as the number of total simultaneous
threads begins to trail off, the performance will appear to improve. For example, I will show the sampler as having completed 9200
requests when the first thread finishes its 100 iterations. This means for the remaining 800 requests, I only have a maximum 99
threads operating. Then at 9300 samples, 19 more threads might have died off - so I only have 80 threads max operating for the
remaining 700 requests. As you can imagine, this means that by the end of the test - the performance numbers will start to
progressively improve (since fewer threads means less workload on the process being measured, and therefore faster response times).

It would be really nice if JMeter just kept a pool of 100 threads operating on requests for the duration of the 10,000 iterations,
so that threads would only die off during the final iteration, leaving the server at more-or-less peak load throughout the test.

>From a code standpoint, this doesn't seem like it would be too hard to setup - just identify how many total iterations need to be
run through the thread group, startup the total number of threads you need, and let each thread keep going until all the iterations
have been started. (Of course, I say that knowing that I'm just a 'manager' type, and won't be coding it myself...)

--
Robin D. Wilson
Sr. Director of Web Development
KingsIsle Entertainment, Inc.
VOICE: 512-777-1861
www.KingsIsle.com

-----Original Message-----
From: sebb [mailto:sebbaz@gmail.com] 
Sent: Tuesday, June 12, 2012 5:02 PM
To: JMeter Users List
Subject: Re: JMeter threading model

On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com> wrote:
>
> On 2012-06-13, at 12:54 AM, sebb wrote:
>
>> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com> wrote:
>>> Hi,
>>>
>>> I figured thread pooling would be revolutionary so I wasn't suggesting that. I would be very useful just delay the creation of a
thread until it was asked for.
>>
>> Not sure I understand how it would help to delay the thread creation,
>> except perhaps for the case where the first threads have finished
>> processing by the time the last threads start running samples.
>
> Bingo!!! ;-)

So what percentage of use cases need to follow this model?

Most of the JMeter testing I have done was long running tests where
all threads were active for most of the run.

> Kirk
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by sebb <se...@gmail.com>.

On 12 June 2012 22:57, Kirk Pepperdine <ki...@gmail.com> wrote:
>
> On 2012-06-13, at 12:54 AM, sebb wrote:
>
>> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com> wrote:
>>> Hi,
>>>
>>> I figured thread pooling would be revolutionary so I wasn't suggesting that. I would be very useful just delay the creation of a thread until it was asked for.
>>
>> Not sure I understand how it would help to delay the thread creation,
>> except perhaps for the case where the first threads have finished
>> processing by the time the last threads start running samples.
>
> Bingo!!! ;-)

So what percentage of use cases need to follow this model?

Most of the JMeter testing I have done was long running tests where
all threads were active for most of the run.

> Kirk
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by Kirk Pepperdine <ki...@gmail.com>.

On 2012-06-13, at 12:54 AM, sebb wrote:

> On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com> wrote:
>> Hi,
>> 
>> I figured thread pooling would be revolutionary so I wasn't suggesting that. I would be very useful just delay the creation of a thread until it was asked for.
> 
> Not sure I understand how it would help to delay the thread creation,
> except perhaps for the case where the first threads have finished
> processing by the time the last threads start running samples.

Bingo!!! ;-)

Kirk


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by sebb <se...@gmail.com>.

On 12 June 2012 22:06, Kirk Pepperdine <ki...@gmail.com> wrote:
> Hi,
>
> I figured thread pooling would be revolutionary so I wasn't suggesting that. I would be very useful just delay the creation of a thread until it was asked for.

Not sure I understand how it would help to delay the thread creation,
except perhaps for the case where the first threads have finished
processing by the time the last threads start running samples.

> Regards,
> Kirk
>
> On 2012-06-12, at 8:46 PM, Philippe Mouawad wrote:
>
>> Hello M. Pepperdine,
>> There is no way today to do that.
>> There is an enhancement request that has not been implemented yet although
>> started some time ago:
>>
>>   - https://issues.apache.org/bugzilla/show_bug.cgi?id=47886
>>
>> JMeter core is build around the paradigm:
>>
>>   - 1 user == 1 Thread
>>   - And lot of its internal behaviour makes this supposition
>>
>> This could be changed in 2 ways:
>>
>>   - 1) use for example a ThreadPoolExecutor without limiting its growth to
>>   avoid thread starvation during a Load Test but to create as little threads
>>   as possible
>>   - 2) Use a completely different approach based on something like netty
>>   but this one would be a revolution and a too big impact in my opinion
>>
>>
>> But even with 1) we have to handle the ThreadLocal objects currently used
>> so this need through investigation.
>>
>> Regards
>> Philippe M.
>> http://www.ubik-ingenierie.com
>>
>> On Tue, Jun 12, 2012 at 6:28 PM, Shmuel Krakower <sh...@gmail.com> wrote:
>>
>>> Hi,
>>> I was thinking to suggest working with JP@GC Ultimate thread group, but it
>>> behaves the same (pre-creating the threads).
>>> Kirk, why does it matter to you?
>>>
>>> Regards,
>>> Shmuel.
>>>
>>> On Tue, Jun 12, 2012 at 6:31 PM, Kirk Pepperdine
>>> <ki...@gmail.com>wrote:
>>>
>>>> Hi,
>>>>
>>>> I guess this is mostly directed towards Sebb.. I spent a bit looking at
>>>> JMeter performance today. What I found was that the thread group starts
>>> all
>>>> of it's threads at the beginning and then uses them over time. Is there a
>>>> way to easily change this so that the thread group only creates threads
>>> on
>>>> demand?
>>>>
>>>> Regards,
>>>> Kirk
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>>>
>>>>
>>>
>>
>>
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by Kirk Pepperdine <ki...@gmail.com>.

Hi,

I figured thread pooling would be revolutionary so I wasn't suggesting that. I would be very useful just delay the creation of a thread until it was asked for.

Regards,
Kirk

On 2012-06-12, at 8:46 PM, Philippe Mouawad wrote:

> Hello M. Pepperdine,
> There is no way today to do that.
> There is an enhancement request that has not been implemented yet although
> started some time ago:
> 
>   - https://issues.apache.org/bugzilla/show_bug.cgi?id=47886
> 
> JMeter core is build around the paradigm:
> 
>   - 1 user == 1 Thread
>   - And lot of its internal behaviour makes this supposition
> 
> This could be changed in 2 ways:
> 
>   - 1) use for example a ThreadPoolExecutor without limiting its growth to
>   avoid thread starvation during a Load Test but to create as little threads
>   as possible
>   - 2) Use a completely different approach based on something like netty
>   but this one would be a revolution and a too big impact in my opinion
> 
> 
> But even with 1) we have to handle the ThreadLocal objects currently used
> so this need through investigation.
> 
> Regards
> Philippe M.
> http://www.ubik-ingenierie.com
> 
> On Tue, Jun 12, 2012 at 6:28 PM, Shmuel Krakower <sh...@gmail.com> wrote:
> 
>> Hi,
>> I was thinking to suggest working with JP@GC Ultimate thread group, but it
>> behaves the same (pre-creating the threads).
>> Kirk, why does it matter to you?
>> 
>> Regards,
>> Shmuel.
>> 
>> On Tue, Jun 12, 2012 at 6:31 PM, Kirk Pepperdine
>> <ki...@gmail.com>wrote:
>> 
>>> Hi,
>>> 
>>> I guess this is mostly directed towards Sebb.. I spent a bit looking at
>>> JMeter performance today. What I found was that the thread group starts
>> all
>>> of it's threads at the beginning and then uses them over time. Is there a
>>> way to easily change this so that the thread group only creates threads
>> on
>>> demand?
>>> 
>>> Regards,
>>> Kirk
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>>> For additional commands, e-mail: user-help@jmeter.apache.org
>>> 
>>> 
>> 
> 
> 
> 
> -- 
> Cordialement.
> Philippe Mouawad.


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: JMeter threading model

Posted by Philippe Mouawad <ph...@gmail.com>.

Hello M. Pepperdine,
There is no way today to do that.
There is an enhancement request that has not been implemented yet although
started some time ago:

   - https://issues.apache.org/bugzilla/show_bug.cgi?id=47886

JMeter core is build around the paradigm:

   - 1 user == 1 Thread
   - And lot of its internal behaviour makes this supposition

This could be changed in 2 ways:

   - 1) use for example a ThreadPoolExecutor without limiting its growth to
   avoid thread starvation during a Load Test but to create as little threads
   as possible
   - 2) Use a completely different approach based on something like netty
   but this one would be a revolution and a too big impact in my opinion


But even with 1) we have to handle the ThreadLocal objects currently used
so this need through investigation.

Regards
Philippe M.
http://www.ubik-ingenierie.com

On Tue, Jun 12, 2012 at 6:28 PM, Shmuel Krakower <sh...@gmail.com> wrote:

> Hi,
> I was thinking to suggest working with JP@GC Ultimate thread group, but it
> behaves the same (pre-creating the threads).
> Kirk, why does it matter to you?
>
> Regards,
> Shmuel.
>
> On Tue, Jun 12, 2012 at 6:31 PM, Kirk Pepperdine
> <ki...@gmail.com>wrote:
>
> > Hi,
> >
> > I guess this is mostly directed towards Sebb.. I spent a bit looking at
> > JMeter performance today. What I found was that the thread group starts
> all
> > of it's threads at the beginning and then uses them over time. Is there a
> > way to easily change this so that the thread group only creates threads
> on
> > demand?
> >
> > Regards,
> > Kirk
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> > For additional commands, e-mail: user-help@jmeter.apache.org
> >
> >
>



-- 
Cordialement.
Philippe Mouawad.

Re: JMeter threading model

Posted by Shmuel Krakower <sh...@gmail.com>.

Hi,
I was thinking to suggest working with JP@GC Ultimate thread group, but it
behaves the same (pre-creating the threads).
Kirk, why does it matter to you?

Regards,
Shmuel.

On Tue, Jun 12, 2012 at 6:31 PM, Kirk Pepperdine
<ki...@gmail.com>wrote:

> Hi,
>
> I guess this is mostly directed towards Sebb.. I spent a bit looking at
> JMeter performance today. What I found was that the thread group starts all
> of it's threads at the beginning and then uses them over time. Is there a
> way to easily change this so that the thread group only creates threads on
> demand?
>
> Regards,
> Kirk
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
> For additional commands, e-mail: user-help@jmeter.apache.org
>
>

JMeter threading model

Posted by Kirk Pepperdine <ki...@gmail.com>.

Hi,

I guess this is mostly directed towards Sebb.. I spent a bit looking at JMeter performance today. What I found was that the thread group starts all of it's threads at the beginning and then uses them over time. Is there a way to easily change this so that the thread group only creates threads on demand?

Regards,
Kirk


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org

Re: TCP Sampler Read Exception

Posted by sebb <se...@gmail.com>.

On 12 June 2012 09:56, Shay Ginsbourg <sg...@gmail.com> wrote:
> Hi all,
>
> Can anyone suggest what does the following error mean ?
>
> After each TCP sampler, we receive "Response code: 500" and
>
> "Response message: org.apache.jmeter.protocol.tcp.sampler.ReadException:"
>
> Any help would be greatly appreciated.

Most likely the read terminated unexpectedly:

http://jmeter.apache.org/api/org/apache/jmeter/protocol/tcp/sampler/TCPClient.html#read%28java.io.InputStream%29

> thanks,
> Shay

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org