RE: Running JMeter virtualised instances

[Sonam Chauhan <so...@ce.com.au>]

You're welcome Oliver. Glad to get the information out there. 

> Have you ever run a comparison VM to real? Did you get the same
results?

Hmmmm... Its been a long time. I was queasy about running JMeter from a
virtual instance, but I've gotta say I see no notable differences once
enough CPU and memory are reserved on ESX server. 

I did notice issues when one 'system under test' (as opposed to the
system JMeter runs on) switched from real hardware to a VM - tests would
start failing randomly. 

> That NTP issue you describe is an interesting one because I have had
that 
> on my physical test harness too! Didn't ever think to pin this on NTP 
> though. 

:-) Yes, I caught onto it by /var/log/messages entries. This is from an
old email back in 2006 when I found the issue. 

[[  By examining the JMeter logfile, I found a negative time value in
the last entry
======================================
<sampleResult timeStamp="1159440609176" dataType="text"
threadName="Integration Tests Thread Group 1-1" label="Post request to
translate ERP 'TU' UOM to xCBL2.0 standard" time="-4"
responseMessage="OK" responseCode="200" success="true"><assertionResult
failureMessage="" error="false" failure="false"/></sampleResult>
======================================

So I took a look at /var/log/messages on b21: it showed a the Network
time protocol log entries for NTP issues around the time the test ran.
======================================
Sep 28 21:07:53 b21 ntpd[950]: synchronisation lost
Sep 28 21:27:07 b21 ntpd[950]: time reset 3.355363 s
Sep 28 21:27:07 b21 ntpd[950]: synchronisation lost
Sep 28 22:22:00 b21 ntpd[950]: time reset -0.497615 s
Sep 28 22:22:00 b21 ntpd[950]: synchronisation lost
======================================
 ]]  


> You could also make your life easy by just switching off ntpd before 
> you run the test and activate it after the test again 
> (provided your tests don't run more than 12 hrs because then the 
> clock would be significantly off).

Good point - that's another option. My test suite runs against multiple
servers each night and lasts several hours. For us, the operations folks
disabled NTP in favour of VM host-based time synchronisation.

Regards,
Sonam Chauhan

-----Original Message-----
From: Oliver Erlewein [DATACOM] [mailto:Oliver.Erlewein@datacom.co.nz] 
Sent: Friday, 27 February 2009 10:43 AM
To: JMeter Users List
Subject: RE: Running JMeter virtualised instances

Hi Sonam,

Thanks for that! Lots of info in there. I'll mull over it in detail now.
Have you ever run a comparison VM to real? Did you get the same results?

That NTP issue you describe is an interesting one because I have had
that on my physical test harness too! Didn't ever think to pin this on
NTP though. It only happens once in a blue moon but I wondered where it
came from. So now I have an explanation. Thanks!!! You could also make
your life easy by just switching off ntpd before you run the test and
activate it after the test again (provided your tests don't run more
than 12 hrs because then the clock would be significantly off).

Cheers Oliver

-----Original Message-----
From: Sonam Chauhan [mailto:sonam.chauhan@ce.com.au] 
Sent: Thursday, 26 February 2009 4:05 p.m.
To: JMeter Users List
Subject: RE: Running JMeter virtualised instances

Hi Oliver- 

Like you, I've also built a load test harness that fires up independent
copies of JMeter :)... it run on virtualized Redhat Linux ES on ESX
server. 

Whether you hit a problem with ESX depends on your tests, the ESX server
hardware and how much memory and CPU you've allocated (or better
'reserved') for your test-runner system in ESX console. Based on your
numbers, you won't much have a problem if the server is not already
heavily laden doing other things for other people :) You can manage min.
CPU and memory 'reservations' in ESX server console.

I think we 'reserve' 4 GB RAM and 3GHz for each of the 2 CPU allocated
to the VM. We run around 30 independent JMeter instances in parallel.
Note, I had to reduce the JVM memory parameters each JMeter instance
sets (by tweaking Jmeter.sh I think) to a more 'reasonable' level (made
it around 1/4th - it depends on your tests though). The tests run with
no display (using the JMeter '-n' (non-GUI) parameter). CPU usage isn't
much and hits 20%. Network usage peaks at 5 Mbps during the testing.
While its hard to put a precise number (due to ramp times, etc) my
observations a while back showed upto 30,000 concurrent (running or
sleeping) JMeter threads on that VM.

One thing to watch out for - we hit problems due to NTP on VMWare not
working very well. If the time goes backward JMeter gets confused and
shows negative time for sampler requests etc. The solution was to use
some sort of VMWare kernel module to bypass NTP and get time from ESX
directly.

Regards,
Sonam Chauhan

-----Original Message-----
From: Oliver Erlewein [DATACOM] [mailto:Oliver.Erlewein@datacom.co.nz] 
Sent: Thursday, 26 February 2009 1:23 PM
To: JMeter Users List
Subject: Running JMeter virtualised instances

Hi all,

I'm being asked if I can run JMeter instances on a virtualised
environment. 

The detail is that we (will) have massive machines (8 core lots of RAM)
that will run VMware ESX server. On that I will need about 15-30
instances of JMeter running to generate the load needed. How does the
virtualisation affect my results? In the past I've always run native as
I don't trust virtualisation. JMeter will be the only service running.

Has anyone done tests between native and virtualised? Any experiences?
Pros / Cons? Any help is appreciated.

Best
Oliver



---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org


The information contained in this email and any attached files are
strictly
private and confidential. This email should be read by the intended
addressee
only.  If the recipient of this message is not the intended addressee,
please
call Corporate Express Australia Limited on +61 2 9335 0555 or Corporate
Express
New Zealand Limited on +64 9 279 2555 and promptly delete this email and
any
attachments.  The intended recipient of this email may only use,
reproduce,
disclose or distribute the information contained in this email and any
attached
files with Corporate Express' permission. If you are not the intended
addressee,
you are strictly prohibited from using, reproducing, disclosing or
distributing
the information contained in this email and any attached files.
Corporate
Express advises that this email and any attached files should be scanned
to
detect viruses. Corporate Express accepts no liability for loss or
damage
(whether caused by negligence or not) resulting from the use of any
attached
files.

---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org


The information contained in this email and any attached files are strictly
private and confidential. This email should be read by the intended addressee
only.  If the recipient of this message is not the intended addressee, please
call Corporate Express Australia Limited on +61 2 9335 0555 or Corporate Express
New Zealand Limited on +64 9 279 2555 and promptly delete this email and any
attachments.  The intended recipient of this email may only use, reproduce,
disclose or distribute the information contained in this email and any attached
files with Corporate Express' permission. If you are not the intended addressee,
you are strictly prohibited from using, reproducing, disclosing or distributing
the information contained in this email and any attached files.  Corporate
Express advises that this email and any attached files should be scanned to
detect viruses. Corporate Express accepts no liability for loss or damage
(whether caused by negligence or not) resulting from the use of any attached
files.

---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org