What a nightmare!

[Jordi Salvat i Alabart <js...@atg.com>]

Hi.

As you may know, I've lately spent quite a bit of time trying out and 
learning how to use JMeter, with the intent of replacing the tool I've 
been using so far (which is very expensive but totally innapropriate for 
the job I usually do).

This week I've had my first opportunity to use JMeter in the real world, 
to help a customer load-test their application. Frankly, it's been a 
real nightmare, and it's not yet over.

To start with, the application is pretty complex. It uses all sorts of 
tricks to do its function: multiple forms chained up which carry data 
from one to the next, POST requests with additional parameters in the 
URI, fancy parameter encodings, lots of redirects, most of them with 
parameters, forms with dynamically-named fields,... Also, probably 
wasn't helping that I have never worked with the development team, and 
so I just didn't know about the internals -- I had to go guessing as I 
progressed.

I've not yet had time to isolate and rationalize the problems I've run 
into, so I can post them as specific bugs and enhancement proposals, but 
here's a quick description. Please don't take it for what it's not: it's 
intended to be constructive criticism: I appreciate the speed at which 
JMeter is improving and I hope knowledge of these limitations will help 
working in the right (or at least in the most urgent) direction:

- The HTTPSampler and the HTTP Proxy don't allow POST requests with 
additional parameters in the query string (in the URI).

- The HTTPSampler and the HTTP Proxy fail on redirects which carry 
parameters in the query string (they do the redirect but don't pass the 
parameters on).

- The HTTPSampler generates one single Sample even if it does a 
redirect. This makes debugging the scripts really difficult, because the 
information on the original request is lost (you only get the page at 
the end of the redirect).

- It's not possible to use any unusual characters in request parameters: 
the HTTPSampler will always URLEncode them -- even if they were not 
encoded when the HTTP Proxy received them.

- I would summarize all of the above in that they get too much in the 
way -- mainly for the HTTP Proxy.

- When trying to correct the above, I ran into lots of trouble because 
the protocol.http packages are too convoluted: looks like each class is 
tightly coupled to each other class there. I think those packages call 
for refactoring.

- The interface to edit the request parameters needs much improvement. I 
had lots of parameter names 30 or 40 characters long which was really 
hard to edit in that very small field which only accepts deleting and 
writing at the end. Add a __regexp function to cope with variable form 
field names and it's madness.

- Do we really need embedded functions? I have a feeling that just 
having placeholders (aka variables) and modifiers to define them would 
do the job just as well, and would be much simpler.

- The View Results Tree Listener is a very useful debugging tool. I was 
just missing the information sent in POST requests and a "browse" 
button. Would also be cool if it labelled events as per the originating 
sampler, instead of the URL. A "clean" button would also be extremely handy.

- I also saw the agregate report throwing lots of 
ConcurrentUpdateExceptions ... don't yet know if they are harmful.

- The counters in the User Parameter Modifier are not "thread safe": 
each counter goes its own way. To put a simpler equivalent example: I 
had registered users named "user1" to "user200" with passwords "pwd1" to 
"pwd200", and I found no way to ensure each thread would properly log 
in. My case was even worse because I needed to use the same number in 
two consecutive pages. We need variables (not only constants) real soon 
(in case they are not already there).

- The same goes for the need to use the same regexps again and again -- 
which very badly affected the performance of my scripts -- variables and 
modifiers would probably cure this.

Finally, less urgent but probably most important, I feel the code is 
generally too convoluted. I can't tell if it's only in the http packages 
as I mentioned before or it's a more general problem, but the result is 
that it makes the threshold for new contributors very high. I will make 
this the subject of a later e-mail when the dust settles down.

Faithfully,

Jordi Salvat i Alabart


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

How kool! [was: What a nightmare!]

[Jordi Salvat i Alabart <js...@atg.com>]

Hi.

I'm amazed. I've finally found time to try to isolate and reproduce, and 
eventually report, all the problems I mentioned in my e-mail.

To my grateful surprise, most of them (the most important, actually) are 
already solved in the latest release. Many many thanks to whoever has 
been working on these (mostly Mike, I guess).

> - The HTTPSampler and the HTTP Proxy don't allow POST requests with 
> additional parameters in the query string (in the URI).

This is solved. Thanks.

> - The HTTPSampler and the HTTP Proxy fail on redirects which carry 
> parameters in the query string (they do the redirect but don't pass the 
> parameters on).

This is solved. Thanks.

> - The HTTPSampler generates one single Sample even if it does a 
> redirect. This makes debugging the scripts really difficult, because the 
> information on the original request is lost (you only get the page at 
> the end of the redirect).

This is unchanged, but should be OK, since I always have the option of 
not following redirects (which is the default).

> - It's not possible to use any unusual characters in request parameters: 
> the HTTPSampler will always URLEncode them -- even if they were not 
> encoded when the HTTP Proxy received them.

This is still there and causing me trouble, but since automatic 
URL-encoding of parameter names and values is probably a desirable 
functionality, I'll probably file an enhancement request so that you can 
choose to URL-encode or not.

> - The interface to edit the request parameters needs much improvement. I 
> had lots of parameter names 30 or 40 characters long which was really 
> hard to edit in that very small field which only accepts deleting and 
> writing at the end. Add a __regexp function to cope with variable form 
> field names and it's madness.

The fields are now fully editable. Thanks.
The space devoted to parameters is WAY bigger now. Thanks.

> - The View Results Tree Listener is a very useful debugging tool. I was 
> just missing the information sent in POST requests and a "browse" 
> button. Would also be cool if it labelled events as per the originating 
> sampler, instead of the URL. A "clean" button would also be extremely 
> handy.

The information I was missing is now there. Cool!

I'll post enhancement requests for the browse button, event labelling 
and clean button.

> - I also saw the agregate report throwing lots of 
> ConcurrentUpdateExceptions ... don't yet know if they are harmful.

I still need to investigate this one.

> - The counters in the User Parameter Modifier are not "thread safe": 
> each counter goes its own way. To put a simpler equivalent example: I 
> had registered users named "user1" to "user200" with passwords "pwd1" to 
> "pwd200", and I found no way to ensure each thread would properly log 
> in. My case was even worse because I needed to use the same number in 
> two consecutive pages. We need variables (not only constants) real soon 
> (in case they are not already there).

The new __counter and __threadNum functions solve this -- at least in my 
case. I guess other functions can solve the issue in more general cases. 
Great!

Salut,

Jordi.


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>