Include + Module controller hybrid

[Lena Mulyar <le...@gmail.com>]

Hi!

I am in a strong need of having a Controller that allows to choose a
certain module from an external file, something like a hybrid of Module and
Include Controllers (I work with a complex microservices-based system that
requires organizing a nicely structured and convenient in usage test
framework). I have found a few requests for the similar feature here and
there, an old enhancement ticket at bugzilla: id=60264, but no action on
that. So I decided to try implementing it myself.

I am new to jmeter development, so might miss some obvious points. On the
first sight it looks like the desired controller should be a combination of
Module and Include controller. How I see it roughly -
* gui element should have includePanel similar to Include controller, and
treePanel+modulePanel from ModuleController, no expandButton as it's not
applicable for external tree.
* contoller's clone() loads a subtree/ sub test element/s from the selected
module of the external file
* gui reinitialize() method takes care of rebuilding moduleToRunTreeModel
gui element rendering the tree from external file - basically, it's a
representation of a subtree?
* on create element the treePanel is hidden
* on modify element, with adding change listener to includePanel, treePanel
becomes visible after external jmx file selected, and renders the external
jmx tree (if proper, otherwise raise error)
* the final/effective tree is being resolved only on run by engine?.. -
controller's resolveReplacementSubTree() should take care by that moment of
having a subtree set and selectedNode defined


First of all I would like to make sure there are no core pitfalls that
prevents this solution from implementation (I assume there might have been
a decent reason why it was not implemented over all these years?), so being
a novice I do not waste time trying to solve the problem that is hard to
solve by jmeter experts.

And then I would appreciate any hints / advice on what are the key
functional points I have to take care of to implement this kind of a
controller.

Thanks in advance!
//Lena

Re: Include + Module controller hybrid

[Lena Mulyar <le...@gmail.com>]

> Do you think relative path is a viable solution?
>
> An alternative option is to have "library paths" JMeter propery, so it
> would lookup libraries in the given set of paths.

Yup, I think either of options would work.

> When it comes to the UI, I would love to see the included elements, and it
> would be great to drill down into the implementation (e.g. to verify you
> call the right fragment).
>
> It would be great if UI suggested the available elements and highlighted
> errors like "calling a non-existing fragment" or "library file missing".

Absolutely, this is how I ideally see it as well. We'll see how far I
can go, but I can't wait to make it working :D
At the moment I am still on the stage of figuring out how the inner
jmeter machinery works though.

//Lena

Re: Include + Module controller hybrid

[Vladimir Sitnikov <si...@gmail.com>]

>test plan can
>be run in different envs with different jmeter installation paths

Do you think relative path is a viable solution?

An alternative option is to have "library paths" JMeter propery, so it
would lookup libraries in the given set of paths.



When it comes to the UI, I would love to see the included elements, and it
would be great to drill down into the implementation (e.g. to verify you
call the right fragment).

It would be great if UI suggested the available elements and highlighted
errors like "calling a non-existing fragment" or "library file missing".

I might be wrong, however, dynamic file names make it harder to implement
user-friendly UI.

>jmeter benefit in it's
visual way of building test scenarios in just a few clicks

That's fair.

Vladimir

Re: Include + Module controller hybrid

[Lena Mulyar <le...@gmail.com>]

> By the way, I wonder if we can explicitly forbid the use of ${...}
> expressions in file names.
Mm. I see it as a quite limiting change, considering the test plan can
be run in different envs with different jmeter installation paths, or
for ex. I run it locally on the pre-installed jmeter instance, and
ci/cd runs it via jmeter maven plugin - which requires the filenames
to be resolved dynamically.

> As a side note, programmatic test plan generation gets this "Include +
> Module controller" for free.
> I agree "JMeter libraries" is useful for UI, however, I wonder if you
> considered programmatic test plan generation.
Well, in my case the ambition is to use a jmeter benefit in it's
visual way of building test scenarios in just a few clicks having all
those reusables properly organized in the libraries.
(we are intended to use jmeter for both functional/feature and
load/performance backend testing, so the scenarios are supposed to be
quite complex and diverse)

//Lena

Re: Include + Module controller hybrid

[Vladimir Sitnikov <si...@gmail.com>]

>Gotcha, thanks for clarifying - then it might absolutely be an
>alternative solution.

In practice, there's not much difference overall :)

However, many programming languages have the concept of "exporting and
importing symbols" (e.g. import ... in Java),
so JMeter might be good to borrow the concept.

>Sounds promising, I'll consider both options then

Great.
By the way, I wonder if we can explicitly forbid the use of ${...}
expressions in file names.
It would simplify the implementation and maintenance.

---

As a side note, programmatic test plan generation gets this "Include +
Module controller" for free.
I agree "JMeter libraries" is useful for UI, however, I wonder if you
considered programmatic test plan generation.

Vladimir

Re: Include + Module controller hybrid

[Lena Mulyar <le...@gmail.com>]

> I suggest that "import element" would create a virtual node that references
> the external file symbolically.
> Then, the import could be "scoped to the subtree", so you might import
> "commons" library at the very top, and you could import more specific
> "libraries" somewhere inside the tree.

Gotcha, thanks for clarifying - then it might absolutely be an
alternative solution.

>> a challenge to
>> properly resolve a tree assuming the external modules could
>> potentially be cross-referenced
> Even though it sounds the most challenging task, it does not sound it is
> complicated.
> It should probably cache the referenced files to avoid reparsing on each
> click.

Sounds promising, I'll consider both options then.

//Lena

Re: Include + Module controller hybrid

[Vladimir Sitnikov <si...@gmail.com>]

> However, it doesn't seem to not be an optimal solution for
the case when I have 20+ "libraries" with a certain level of depth
each having multiple nested modules/elements that also can reference

I think there's misunderstanding.
I don't suggest copying the trees at import time.
I suggest that "import element" would create a virtual node that references
the external file symbolically.

Then, the import could be "scoped to the subtree", so you might import
"commons" library at the very top, and you could import more specific
"libraries" somewhere inside the tree.

> a challenge to
properly resolve a tree assuming the external modules could
potentially be cross-referenced

Even though it sounds the most challenging task, it does not sound it is
complicated.

It should probably cache the referenced files to avoid reparsing on each
click.

Vladimir

Re: Include + Module controller hybrid

[Lena Mulyar <le...@gmail.com>]

Thanks for your input!

> Have you explored the possibility of "Import Element"?
> In other words, an element that would "import a jmx file" into the
> namespace of the current test plan.
> Then Module Controllers should be able to reference the nodes of the
> imported JMX.
> For example, "Import auth_lib.jmx as UserAuthentication",  "Module
> Controller, element=UserAuthentication / login"

Yeah, I was thinking about something like that - using a merge
feature. However, it doesn't seem to not be an optimal solution for
the case when I have 20+ "libraries" with a certain level of depth
each having multiple nested modules/elements that also can reference
other external libraries. Eventually the test plan still would look
huge and hard to read/navigate. If I understand your idea correctly.

Getting back to the solution I'm thinking about - I see a challenge to
properly resolve a tree assuming the external modules could
potentially be cross-referenced. Do you think it makes sense?

//Lena

Re: Include + Module controller hybrid

[Vladimir Sitnikov <si...@gmail.com>]

Lena>a decent reason why it was not implemented over all these years?

I guess the reason is nobody invested time to make it happen.

Lena>I am in a strong need of having a Controller that allows to choose a
Lena>certain module from an external file, something like a hybrid of
Module and
Lena>Include Controllers

Have you explored the possibility of "Import Element"?
In other words, an element that would "import a jmx file" into the
namespace of the current test plan.
Then Module Controllers should be able to reference the nodes of the
imported JMX.
For example, "Import auth_lib.jmx as UserAuthentication",  "Module
Controller, element=UserAuthentication / login"

Vladimir