Re: JMeter variables and the beginnings of a JMeter scripting lan guag e

[Jordi Salvat i Alabart <js...@atg.com>]

I absolutely agree.

Assertions are still a great tool in load testing: we may want to check 
that pages don't come back blank, or that they don't contain the words 
"Error" or "Exception".

Even some limited amount of cleverness (e.g. for forms whose fields 
dynamically change names) is sometimes necessary.

As you say, we just need to ensure these features don't get in the way 
of useful load testing.

Salut,

Jordi.

Stover, Michael wrote:
> Jordi,
>     Thanks for you analysis - I think you are very right in your comparison
> of the tools.  My vision of useful functional testing doesn't follow along
> the lines most such tools go for (thus my interest in writing my own).  In
> my opinion, a tool can do very little in terms of initial functional
> testing.  You can tweak your script endlessly, but if you ever believe
> you're getting decent coverage of the range of input, I think you're
> mistaken.  The development of new test cases requires a human.  A tool such
> as JMeter, however, can provide decent regression testing, however.  Once a
> test case has been discovered and recorded, it can be handed off to JMeter
> to run.  This means it isn't required to program JMeter to be too smart - it
> just does what it's told and checks the results against assertions provided
> to it.  Provide no assertions, and JMeter does no such extra work
> (stress-testing).  
> 
> You might say I have a vision of a minimalist functional tool.  I also
> happen to think that's the only practical kind.
> 
> -Mike
> 
> 
>>-----Original Message-----
>>From: Jordi Salvat i Alabart [mailto:jsalvata@atg.com]
>>Sent: Thursday, July 11, 2002 10:13 AM
>>To: JMeter Developers List
>>Subject: Re: JMeter variables and the beginnings of a JMeter scripting
>>languag e
>>
>>
>>
>>
>>>If ever what I do messes up stress testing - I hope people 
>>
>>let me know.  My intention 
>>
>>>is to add functional testing capabilities as needed without 
>>
>>degrading the stress testing 
>>
>>>capabilities.  Thus, I am willing to come up short in 
>>
>>functional testing, but not in stress 
>>
>>>testing.
>>
>>Some information on the reasons why I believe those products which do 
>>good functional testing fail to be good at load testing:
>>
>>1.- They generally try to test page functionality as a whole, 
>>including 
>>the effect of JavaScript code in the pages and, often, issues which 
>>depend on the rendering engine, such as the accessibility of form 
>>fields. As it turns out, there's only one reliable way to do 
>>this, which 
>>is to include the browser itself as part of the test runtime 
>>-- so that 
>>the JavaScript code is run by the actual browser and not by some 
>>"simulation" code in the test tool. This makes the test tool 
>>tremendously heavy in terms of resource needs -- a machine 
>>can't usually 
>>run more than half a dozen such browsers without significant 
>>performance 
>>penalties. Which means you would need 100 machines to simulate 1000 
>>users hitting your system :-(
>>
>>2.- Even if they stay away from testing JavaScript & 
>>rendering features, 
>>functional test scripts still tend to be (and probably need to be) 
>>expressed as:
>>
>>   1/ Get URL A.
>>   2/ Click on the link with text T.
>>   3/ Fill the three form fields in the resulting form with 
>>values x y 
>>z, then press button labelled "SUBMIT".
>>
>>which require parsing and analysis of the returned HTML. This is 
>>generally a costly operation, which plays against you when 
>>load testing.
>>
>>In contrast, the (current) JMeter way would be to do:
>>   1/ Get URL A
>>   2/ Get URL B.
>>   2/ POST URL C with parameters X=x, Y=y, Z=z.
>>which is not very good from a functional testing point of 
>>view, because 
>>it depends on knowledge of the URLs pointed to by "the link 
>>with text T" 
>>and "the action for the resulting form".
>>
>>Just hoping that this helps avoiding the same pitfalls.
>>
>>Salut,
>>
>>Jordi.
>>
>>
>>
>>
>>--
>>To unsubscribe, e-mail:   
>><ma...@jakarta.apache.org>
>>For additional commands, e-mail: 
>><ma...@jakarta.apache.org>
>>
> 
> --
> To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
> 
> 




--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>