Different results when running JMeter on two different machines

[rats123 <ra...@tester.co.nz>]

I am getting different results (poorer response times) running exactly the
same test with the same settings and same version of JMeter on two machines.

First machine is an Intel E8400 with 3.46GB of RAM running XP. Second
machine is an Intel i5 with 3.16GB of RAM running XP.

The results on Machine 1 are good while the results on Machine 2 (the more
powerful machine) are much poorer.

I have tried:

1) Copying settings from Machine 1 to Machine 2
2) Swapping network cables

All efforts consistently give poor results on Machine 2.

Any ideas what's happening here? I need to distribute load tests to test
high thread counts.

--
View this message in context: http://jmeter.512774.n5.nabble.com/Different-results-when-running-JMeter-on-two-different-machines-tp4584518p4584518.html
Sent from the JMeter - User mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org

Re: Different results when running JMeter on two different machines

[Adrian Speteanu <as...@gmail.com>]

Hi,

Whenever I hear a situation like this, its usually a misconfiguration of
some sort (as already suggested) - i.e. you upload a file on one sampler,
and the location of it is not the same on the two machines => response times
differ a lot (add assertion, the requests fail and you know quickly -
problem solved).

But if its not the case it would be quite interesting. If I had to make a
tremendous and outrageously unverified guess, I would say that the OS
doesn't completely support the newer processor architecture (from personal
experience, XP doesn't handle multi-core situations very well, not as well
as windows 7 does, anyway, on the exact same machine - and I speculate that
they would sacrifice performance to ensure stability and reduce development
time on an OS that will soon not be supported) - this WOULD explain why it
does better on the older processor rather than on the newer one, but its
completely speculative and the difference should be around 10%, not more.

The reasons why response times would be affected lie in the amount of
processing required from the time the request is completely sent until the
moment the answer is completely received back. If you use the http sampler,
there is little that can affect the performance of this (or little that can
be improved) - personally I had reliable results with this, but if you use
something like BSH sampler, your response times certainly include some
processing time done client side (the approach to this is to move much of
the processing in a pre-processor, so that the impact on response times is
minimal - this is what we did anyway and then got pretty accurate results).

*The pragmatic approach.* If you can't figure this out quickly, you have to
be pragmatic about it. You can't rely on the response times JMeter reports
or you just don't know for sure what is going on, but its still the best
tool out there to generate huge loads: use it to generate the throughput you
need, but monitor response times as reported by the application. With the
java application we test, we have added some plugin to the application that
measures response times between methods (major flagpoints - like all methods
used for an action triggered with one jmeter sampler) and an external
service that gathers, aggregates and plots these response times (we keep
monitoring at a minimum level and also require this in production to see if
something goes wrong and where, the impact on performance is something like
1-35% at most). I recommend this approach, but I have to mention that the
results we get in JMeter differ by 3-10ms from the results in this plugin
(overhead added by network, way within acceptable values at response times
of 200-3000ms). See: http://jetm.void.fm/ as an example.

Regards,
Adrian S

On Thu, Jul 14, 2011 at 1:11 AM, rats123 <ra...@tester.co.nz> wrote:

> I am getting different results (poorer response times) running exactly the
> same test with the same settings and same version of JMeter on two
> machines.
>
> First machine is an Intel E8400 with 3.46GB of RAM running XP. Second
> machine is an Intel i5 with 3.16GB of RAM running XP.
>
> The results on Machine 1 are good while the results on Machine 2 (the more
> powerful machine) are much poorer.
>
> I have tried:
>
> 1) Copying settings from Machine 1 to Machine 2
> 2) Swapping network cables
>
> All efforts consistently give poor results on Machine 2.
>
> Any ideas what's happening here? I need to distribute load tests to test
> high thread counts.
>
> --
> View this message in context:
> http://jmeter.512774.n5.nabble.com/Different-results-when-running-JMeter-on-two-different-machines-tp4584518p4584518.html
> Sent from the JMeter - User mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>

Re: Different results when running JMeter on two different machines

[sebb <se...@gmail.com>]

On 18 July 2011 09:16, Felix Frank <ff...@mpexnet.de> wrote:
> On 07/14/2011 12:11 AM, rats123 wrote:
>> I am getting different results (poorer response times) running exactly the
>> same test with the same settings and same version of JMeter on two machines.
>>
>> First machine is an Intel E8400 with 3.46GB of RAM running XP. Second
>> machine is an Intel i5 with 3.16GB of RAM running XP.
>>
>> The results on Machine 1 are good while the results on Machine 2 (the more
>> powerful machine) are much poorer.
>
> Is the "more powerful" machine faster down to each last component (hard
> drive comes to mind, but there may be others involved).
>
> Otherwise, you will have to find the bottleneck. This may prove
> difficult with WinXP, but you probably just need the right tools. (I
> cannot really advise you here, as I never had the misfortune to have to
> do productive work using Windows.)
>
> Would it be possible to dual-boot a Linux installation on those boxes? I
> think I remember reading (on this list) that the Linux implementation of
> Java can perform better than the Windows one. Debugging may be easier too.
>
> Are both XPs 32 bit?

Are the two systems running the same Java version?

Check the jmeter.log file, which includes this information near the start.

Do the systems have equivalent network cards?

What is the load on the individual machines? Use Windows Task Manager
for a basic check.
Are both the hosts dedicated to running JMeter, or is other software
being run in the background?
Do they have the same network settings (apart from IP address, obviously)?

Try running a protocol analyser such as Wireshark to capture a short
test that shows the problem; check if there are any obvious
differences such as retries or errors.


> HTH,
> Felix
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org

Re: Different results when running JMeter on two different machines

[Felix Frank <ff...@mpexnet.de>]

On 07/14/2011 12:11 AM, rats123 wrote:
> I am getting different results (poorer response times) running exactly the
> same test with the same settings and same version of JMeter on two machines.
> 
> First machine is an Intel E8400 with 3.46GB of RAM running XP. Second
> machine is an Intel i5 with 3.16GB of RAM running XP.
> 
> The results on Machine 1 are good while the results on Machine 2 (the more
> powerful machine) are much poorer.

Is the "more powerful" machine faster down to each last component (hard
drive comes to mind, but there may be others involved).

Otherwise, you will have to find the bottleneck. This may prove
difficult with WinXP, but you probably just need the right tools. (I
cannot really advise you here, as I never had the misfortune to have to
do productive work using Windows.)

Would it be possible to dual-boot a Linux installation on those boxes? I
think I remember reading (on this list) that the Linux implementation of
Java can perform better than the Windows one. Debugging may be easier too.

Are both XPs 32 bit?

HTH,
Felix

---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org