jMeter performance, garbage collector - related problem

[Michal Kostrzewa <M....@pentacomp.com.pl>]

Hi list,

Perhaps you can say something about this:

My machine (1000MHz) can handle 200 threads without any problem. I proved it 
to myself doing simple test with http://host:80/ (apache welcome page) and 
average response time was almost the same when using 1 and 200 threads.

I created a scenario, not very long (about 30 requests) but quite 
sophisticated (functions, modifiers, assertions etc). One user runs this 
scenario in, say, *1 minute*. I ran 60 users and everything was fine until I 
realized that every user runs the scenario in over *10 minutes* (!) (keep in 
mind described 200 threads without time of thread swiching issues). And 
during this test I can run one user from other machine and receive short 
response times (so tested application is doing well). 

After some tweaking I discovered that the main reason for this is... garbage 
collector. The allocated memory during this test quickly achieves java 
maximum memory level, garbage collector starts doing it's job and 
considerably slows down jMeter. This situation isn't happening when running 
simple test, because jMeter slower eats memory. 
To provide some kind of proof of my "garbage collector" theory, I first ran 
jMeter with 60 users and wait for eating all java's memory, and then ran one 
user doing the same scenario on THE SAME machine. The result was, that "60 
users jMeter" was slow, and "one user jMeter" performed good. I monitored 
both memory consumption and "the one user jMeter" ate memory slower (garbage 
collector had less work)
Also analysis of response time showed me, that response time is considerable 
bigger, when gc (which actions I monitored) ran.

Conclusion to me: jMeter on one machine *can* do many threads/users but 
*cannot* fully load web application... It can send *many* requests from *many 
different* users, but it's doing it slow and application has no trouble to 
handle it. It would not happen if allocated objects were created right after 
use, or jMeter created less objects....

Am I right, or I'm doing something wrong? Can somebody confirm this? Is there 
a way to workaround this? Does jMeter have to create so much small objects? 

looking forward for any comments
best regards
Michal Kostrzewa



--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: jMeter performance, garbage collector - related problem

[Michal Kostrzewa <M....@pentacomp.com.pl>]

Hi Mike

Thank you very much for your answer.

>
> The way to fix this would be to use an object pool.  It would have to be
> relatively sophisticated in JMeter's case, but certainly doable.
>
I thought it's wonderful idea, but... How do you want to do that? there are 
objects, that can be only created and the cannot be reused (e.g. Strings). 
Please consider ResponseData. Web applications can produce pages with 100kb 
of data (altavista rearch result returns 25kb, and it's quite light). 100kb * 
60 threads=6MB (so without pool it's required at least 6MB at every scenario 
step. At least, because jMeter consumes many times more memory for it 
allocates many Strings with this data) 

I suspect response data to be the most memory eater in jMeter.

ResponseData is stored in jMeter as array of bytes. That's good, because we 
can reuse this. (BTW reading of this is quite memory ineficient: 

while ((x = in.read(buffer)) > -1)
		{
			w.write(buffer, 0, x);
		}
in.close();
w.flush();
w.close();
return w.toByteArray();

beacause stream 'w' internally creates second array.)

But memory allocation nightmare starts when using modifiers. Consider 
URLRewriting, modifyEntry:
String text = new String(responseText.getResponseData());
or AnchorModifier, modifyEntry
responseText = new String(result.getResponseData(),"8859_1");
...
HtmlParser.getDOM(responseText.substring(index));

and so on. jMeter needs this long string only for a second and then we cannot 
do anything with this object (cannot return to pool). When you run test with 
modifiers jMeter create several times more memory that it already needs just 
for that reason that parsers, DOM and whoewer requires String as an input.

What do you think about it? Can object pool be successful approach in this 
case? Or I'm missing something?

best regards
Michal Kostrzewa


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: jMeter performance, garbage collector - related problem

[Mike Stover <ms...@apache.org>]

The more functions and modifiers you use, the more JMeter has to work.  JMeter 
does create a lot of objects during test run because of all the cloning that goes on 
(JMeter has to replace the values of functions, but it would be undesirable to alter 
the "blueprint" of the test during the run).  

The way to fix this would be to use an object pool.  It would have to be relatively 
sophisticated in JMeter's case, but certainly doable.

-Mike

On 16 Oct 2002 at 1:48, Michal Kostrzewa wrote:

> Hi list,
> 
> Perhaps you can say something about this:
> 
> My machine (1000MHz) can handle 200 threads without any problem. I proved it 
> to myself doing simple test with http://host:80/ (apache welcome page) and 
> average response time was almost the same when using 1 and 200 threads.
> 
> I created a scenario, not very long (about 30 requests) but quite 
> sophisticated (functions, modifiers, assertions etc). One user runs this 
> scenario in, say, *1 minute*. I ran 60 users and everything was fine until I 
> realized that every user runs the scenario in over *10 minutes* (!) (keep in 
> mind described 200 threads without time of thread swiching issues). And 
> during this test I can run one user from other machine and receive short 
> response times (so tested application is doing well). 
> 
> After some tweaking I discovered that the main reason for this is... garbage 
> collector. The allocated memory during this test quickly achieves java 
> maximum memory level, garbage collector starts doing it's job and 
> considerably slows down jMeter. This situation isn't happening when running 
> simple test, because jMeter slower eats memory. 
> To provide some kind of proof of my "garbage collector" theory, I first ran 
> jMeter with 60 users and wait for eating all java's memory, and then ran one 
> user doing the same scenario on THE SAME machine. The result was, that "60 
> users jMeter" was slow, and "one user jMeter" performed good. I monitored 
> both memory consumption and "the one user jMeter" ate memory slower (garbage 
> collector had less work)
> Also analysis of response time showed me, that response time is considerable 
> bigger, when gc (which actions I monitored) ran.
> 
> Conclusion to me: jMeter on one machine *can* do many threads/users but 
> *cannot* fully load web application... It can send *many* requests from *many 
> different* users, but it's doing it slow and application has no trouble to 
> handle it. It would not happen if allocated objects were created right after 
> use, or jMeter created less objects....
> 
> Am I right, or I'm doing something wrong? Can somebody confirm this? Is there 
> a way to workaround this? Does jMeter have to create so much small objects? 
> 
> looking forward for any comments
> best regards
> Michal Kostrzewa
> 
> 
> 
> --
> To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
> 



--
Michael Stover
mstover1@apache.org
Yahoo IM: mstover_ya
ICQ: 152975688

--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>