You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jmeter.apache.org by Philippe Mouawad <p....@ubik-ingenierie.com> on 2019/01/18 12:38:21 UTC
New feature : Readable overview of Test Plan
Hello,
I think there is a need in JMeter since we're not a DSL based tool, to
provide an overview of the test for documentation purpose.
It would also help a lot on issue reporting for example
Currently, I think the only available option is to use schematic.sh which
is:
- outdated
- very limited
- not friendly to use, you need to copy the test plan in extras folder
to make it work
As the expected overview might depend a lot on what somebody wants to see,
what do you think of this as a first step:
- Create in new Tools menu Item (where we would move Curl, Export
Transactions for Report, Create Heap/Thread dump) an entry "Export as
schematic overview"
- The feature would :
- open A dialog with Option path to enter Exporter
- I see at least 2 ways to implement exporter:
- XSL with schematic way:
- + : Not much work
- - : XSL is not very readable nor friendly
- - : JMeter has no contract on XML
- Add an interface, JMeter would discover implementation and propose it:
- We would use the Visitor pattern
- + : Very flexible
- + : Easier to maintain, we rely on Java Contract
- - : More work for user
- Note we can implement XSL Exporter as first step
- In the long term, maybe we could be proposing an exporter to YAML or
JSON
Exporter would look have:
export(File pathToJMeterFile, HashTree testPlan, Writer writer);
Interface for exporter to be implemented:
interface NodeExporter {
String exportElement(int level, TestElement element, HashTree testPlan);
}
Any thoughts ?
Regards
Philippe
<https://www.openstreetmap.org/#map=18/50.69454/3.16455>
Re: New feature : Readable overview of Test Plan
Posted by Philippe Mouawad <ph...@gmail.com>.
Hi Felix,
I added patch so you can have a look to it:
https://bz.apache.org/bugzilla/show_bug.cgi?id=63101
I think it is highly "improvable".
For now I put in Menu Item (alpha), this way it can easily be tested.
Regards
On Tue, Jan 22, 2019 at 9:03 PM Felix Schumacher <
felix.schumacher@internetallee.de> wrote:
>
> Am 22.01.19 um 20:36 schrieb Philippe Mouawad:
> > Hello,
> > Do you think this feature as of now (based on XSL) is worth being
> committed
> > to core as alpha?
>
> That depends on how alpha you think it is :)
>
> Maybe include it, but disable it by default with a clear warning, that
> it is there for testing the usability?
>
> >
> > In terms of implementation, it relies on Saxon BasicTransformer as it
> > supports xslt 2.0 functions.
>
> Are the used functions comfort functions or necessary ones? (Just out of
> curiosity).
>
> Felix
>
> >
> > Regards
> >
> > On Monday, January 21, 2019, Felix Schumacher <
> > felix.schumacher@internetallee.de> wrote:
> >
> >> Am 21.01.19 um 20:56 schrieb Philippe Mouawad:
> >>
> >>> On Monday, January 21, 2019, Felix Schumacher <
> >>> felix.schumacher@internetallee.de> wrote:
> >>>
> >>> Am 21.01.19 um 11:40 schrieb Philippe Mouawad:
> >>>> Hello,
> >>>>> This will better explain feature:
> >>>>> Menu:
> >>>>>
> >>>>> - http://ubikloadpack.com/demo/schematic/MenuSchematicView.png
> >>>>>
> >>>>> Plan:
> >>>>>
> >>>>> - http://ubikloadpack.com/demo/schematic/testRequest.jmx
> >>>>>
> >>>>> Schematic View:
> >>>>>
> >>>>> - http://ubikloadpack.com/demo/schematic/testRequest.html
> >>>>>
> >>>>> Where do the names "handle Cookies", "add Headers", "Virtual Users
> >>>> executing", ... come from? How would extensions (plugins) fit into
> this?
> >>>>
> >>> It currently comes from the xsl file.
> >>> There would be a default display using element name. For now plugins
> would
> >>> have a very basic display.
> >>>
> >>> There would be a need to provide registration for implementations, I
> see
> >>> how to do it for java implementation, but for XSLT, for now I don't.
> >>>
> >> Yes, I think the java implementation would have advantages over the
> >> xsl-only version, especially with respect to extensibility.
> >>
> >>
> >>> Working on this feature (within some consulting work, I was asked to
> >>> document a plan to provide a textual description and ease
> understanding of
> >>> errors (make other users find to what request an assertion was
> related)),
> >>> I
> >>> find that it makes the reading/understanding of a test plan more easy.
> >>> I ran it on previous plans, and I see at least this major benefit.
> >>> Another one (if we move forward) , would be to provide a more source
> >>> version friendly representation of a JMX test plan.
> >>>
> >> Would this lead to something like Vladimir described? A textual
> >> representation of the test plan, possibly with explanations for the
> >> elements, that is diff and reader friendly.
> >>
> >> On the other hand, this reminds me of something. I sometimes dream of
> >> having markers in the test plan tree, that could be used to show
> >>
> >> * errors that happened during execution of a test plan (linked to/by
> >> samples in result tree)
> >>
> >> * common misconfigurations like double timers (scoping issues mostly)
> or
> >> beanshell/javascript scripts :)
> >>
> >> * samplers with long response times
> >>
> >> * ...
> >>
> >> Felix
> >>
> >>
> >>>
> >>> Felix
> >>>>
> >>>> For now, it's html (I'm using XSLT), but a textual format like YAML
> would
> >>>>> be better for source comparison.
> >>>>>
> >>>>> Regards
> >>>>>
> >>>>> On Mon, Jan 21, 2019 at 11:04 AM Vladimir Sitnikov <
> >>>>> sitnikov.vladimir@gmail.com> wrote:
> >>>>>
> >>>>> Paulo Maia Borges> Why not export to standard HTML?
> >>>>>
> >>>>>> For instance, if jmx is stored in a Git repository (GitLab?
> GitHub?),
> >>>>>> then
> >>>>>> markdown would be much easier to review in diffs than HTML.
> >>>>>> E.g. https://twitter.com/jamiebuilds/status/1085377471057412096
> >>>>>>
> >>>>>> In other words, it could make sense to render markdown
> representation
> >>>>>> on
> >>>>>> save.
> >>>>>> However, it is not clear how tables should be rendered (e.g. HTTP
> >>>>>> parameters).
> >>>>>>
> >>>>>> Vladimir
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> <https://www.openstreetmap.org/#map=18/50.69454/3.16455>
> >>>>>
> >>>>>
> >>>>>
>
--
Cordialement.
Philippe Mouawad.
Re: New feature : Readable overview of Test Plan
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 22.01.19 um 20:36 schrieb Philippe Mouawad:
> Hello,
> Do you think this feature as of now (based on XSL) is worth being committed
> to core as alpha?
That depends on how alpha you think it is :)
Maybe include it, but disable it by default with a clear warning, that
it is there for testing the usability?
>
> In terms of implementation, it relies on Saxon BasicTransformer as it
> supports xslt 2.0 functions.
Are the used functions comfort functions or necessary ones? (Just out of
curiosity).
Felix
>
> Regards
>
> On Monday, January 21, 2019, Felix Schumacher <
> felix.schumacher@internetallee.de> wrote:
>
>> Am 21.01.19 um 20:56 schrieb Philippe Mouawad:
>>
>>> On Monday, January 21, 2019, Felix Schumacher <
>>> felix.schumacher@internetallee.de> wrote:
>>>
>>> Am 21.01.19 um 11:40 schrieb Philippe Mouawad:
>>>> Hello,
>>>>> This will better explain feature:
>>>>> Menu:
>>>>>
>>>>> - http://ubikloadpack.com/demo/schematic/MenuSchematicView.png
>>>>>
>>>>> Plan:
>>>>>
>>>>> - http://ubikloadpack.com/demo/schematic/testRequest.jmx
>>>>>
>>>>> Schematic View:
>>>>>
>>>>> - http://ubikloadpack.com/demo/schematic/testRequest.html
>>>>>
>>>>> Where do the names "handle Cookies", "add Headers", "Virtual Users
>>>> executing", ... come from? How would extensions (plugins) fit into this?
>>>>
>>> It currently comes from the xsl file.
>>> There would be a default display using element name. For now plugins would
>>> have a very basic display.
>>>
>>> There would be a need to provide registration for implementations, I see
>>> how to do it for java implementation, but for XSLT, for now I don't.
>>>
>> Yes, I think the java implementation would have advantages over the
>> xsl-only version, especially with respect to extensibility.
>>
>>
>>> Working on this feature (within some consulting work, I was asked to
>>> document a plan to provide a textual description and ease understanding of
>>> errors (make other users find to what request an assertion was related)),
>>> I
>>> find that it makes the reading/understanding of a test plan more easy.
>>> I ran it on previous plans, and I see at least this major benefit.
>>> Another one (if we move forward) , would be to provide a more source
>>> version friendly representation of a JMX test plan.
>>>
>> Would this lead to something like Vladimir described? A textual
>> representation of the test plan, possibly with explanations for the
>> elements, that is diff and reader friendly.
>>
>> On the other hand, this reminds me of something. I sometimes dream of
>> having markers in the test plan tree, that could be used to show
>>
>> * errors that happened during execution of a test plan (linked to/by
>> samples in result tree)
>>
>> * common misconfigurations like double timers (scoping issues mostly) or
>> beanshell/javascript scripts :)
>>
>> * samplers with long response times
>>
>> * ...
>>
>> Felix
>>
>>
>>>
>>> Felix
>>>>
>>>> For now, it's html (I'm using XSLT), but a textual format like YAML would
>>>>> be better for source comparison.
>>>>>
>>>>> Regards
>>>>>
>>>>> On Mon, Jan 21, 2019 at 11:04 AM Vladimir Sitnikov <
>>>>> sitnikov.vladimir@gmail.com> wrote:
>>>>>
>>>>> Paulo Maia Borges> Why not export to standard HTML?
>>>>>
>>>>>> For instance, if jmx is stored in a Git repository (GitLab? GitHub?),
>>>>>> then
>>>>>> markdown would be much easier to review in diffs than HTML.
>>>>>> E.g. https://twitter.com/jamiebuilds/status/1085377471057412096
>>>>>>
>>>>>> In other words, it could make sense to render markdown representation
>>>>>> on
>>>>>> save.
>>>>>> However, it is not clear how tables should be rendered (e.g. HTTP
>>>>>> parameters).
>>>>>>
>>>>>> Vladimir
>>>>>>
>>>>>>
>>>>>>
>>>>> <https://www.openstreetmap.org/#map=18/50.69454/3.16455>
>>>>>
>>>>>
>>>>>
Re: New feature : Readable overview of Test Plan
Posted by Philippe Mouawad <p....@ubik-ingenierie.com>.
Hello,
Do you think this feature as of now (based on XSL) is worth being committed
to core as alpha?
In terms of implementation, it relies on Saxon BasicTransformer as it
supports xslt 2.0 functions.
Regards
On Monday, January 21, 2019, Felix Schumacher <
felix.schumacher@internetallee.de> wrote:
>
> Am 21.01.19 um 20:56 schrieb Philippe Mouawad:
>
>> On Monday, January 21, 2019, Felix Schumacher <
>> felix.schumacher@internetallee.de> wrote:
>>
>> Am 21.01.19 um 11:40 schrieb Philippe Mouawad:
>>>
>>> Hello,
>>>> This will better explain feature:
>>>> Menu:
>>>>
>>>> - http://ubikloadpack.com/demo/schematic/MenuSchematicView.png
>>>>
>>>> Plan:
>>>>
>>>> - http://ubikloadpack.com/demo/schematic/testRequest.jmx
>>>>
>>>> Schematic View:
>>>>
>>>> - http://ubikloadpack.com/demo/schematic/testRequest.html
>>>>
>>>> Where do the names "handle Cookies", "add Headers", "Virtual Users
>>> executing", ... come from? How would extensions (plugins) fit into this?
>>>
>>
>> It currently comes from the xsl file.
>> There would be a default display using element name. For now plugins would
>> have a very basic display.
>>
>> There would be a need to provide registration for implementations, I see
>> how to do it for java implementation, but for XSLT, for now I don't.
>>
>
> Yes, I think the java implementation would have advantages over the
> xsl-only version, especially with respect to extensibility.
>
>
>> Working on this feature (within some consulting work, I was asked to
>> document a plan to provide a textual description and ease understanding of
>> errors (make other users find to what request an assertion was related)),
>> I
>> find that it makes the reading/understanding of a test plan more easy.
>> I ran it on previous plans, and I see at least this major benefit.
>> Another one (if we move forward) , would be to provide a more source
>> version friendly representation of a JMX test plan.
>>
>
> Would this lead to something like Vladimir described? A textual
> representation of the test plan, possibly with explanations for the
> elements, that is diff and reader friendly.
>
> On the other hand, this reminds me of something. I sometimes dream of
> having markers in the test plan tree, that could be used to show
>
> * errors that happened during execution of a test plan (linked to/by
> samples in result tree)
>
> * common misconfigurations like double timers (scoping issues mostly) or
> beanshell/javascript scripts :)
>
> * samplers with long response times
>
> * ...
>
> Felix
>
>
>>
>>
>> Felix
>>>
>>>
>>> For now, it's html (I'm using XSLT), but a textual format like YAML would
>>>> be better for source comparison.
>>>>
>>>> Regards
>>>>
>>>> On Mon, Jan 21, 2019 at 11:04 AM Vladimir Sitnikov <
>>>> sitnikov.vladimir@gmail.com> wrote:
>>>>
>>>> Paulo Maia Borges> Why not export to standard HTML?
>>>>
>>>>> For instance, if jmx is stored in a Git repository (GitLab? GitHub?),
>>>>> then
>>>>> markdown would be much easier to review in diffs than HTML.
>>>>> E.g. https://twitter.com/jamiebuilds/status/1085377471057412096
>>>>>
>>>>> In other words, it could make sense to render markdown representation
>>>>> on
>>>>> save.
>>>>> However, it is not clear how tables should be rendered (e.g. HTTP
>>>>> parameters).
>>>>>
>>>>> Vladimir
>>>>>
>>>>>
>>>>>
>>>>
>>>> <https://www.openstreetmap.org/#map=18/50.69454/3.16455>
>>>>
>>>>
>>>>
--
[image: logo Ubik Ingenierie] <https://www.ubik-ingenierie.com> Philippe
Mouawad
Senior Performance Expert
320914981 <+33320914981> | p.mouawad@ubik-ingenierie.com
[image: ubik-ingenierie.com] ubik-ingenierie.com
<https://www.ubik-ingenierie.com> | [image: 03.20.91.49.81] 03.20.91.49.81
<+33320914981> | [image: 23 rue du chemin de fer , 59100 , Roubaix] 23 rue
du chemin de fer, 59100, Roubaix
<https://www.openstreetmap.org/#map=18/50.69454/3.16455>
Re: New feature : Readable overview of Test Plan
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 21.01.19 um 20:56 schrieb Philippe Mouawad:
> On Monday, January 21, 2019, Felix Schumacher <
> felix.schumacher@internetallee.de> wrote:
>
>> Am 21.01.19 um 11:40 schrieb Philippe Mouawad:
>>
>>> Hello,
>>> This will better explain feature:
>>> Menu:
>>>
>>> - http://ubikloadpack.com/demo/schematic/MenuSchematicView.png
>>>
>>> Plan:
>>>
>>> - http://ubikloadpack.com/demo/schematic/testRequest.jmx
>>>
>>> Schematic View:
>>>
>>> - http://ubikloadpack.com/demo/schematic/testRequest.html
>>>
>> Where do the names "handle Cookies", "add Headers", "Virtual Users
>> executing", ... come from? How would extensions (plugins) fit into this?
>
> It currently comes from the xsl file.
> There would be a default display using element name. For now plugins would
> have a very basic display.
>
> There would be a need to provide registration for implementations, I see
> how to do it for java implementation, but for XSLT, for now I don't.
Yes, I think the java implementation would have advantages over the
xsl-only version, especially with respect to extensibility.
>
> Working on this feature (within some consulting work, I was asked to
> document a plan to provide a textual description and ease understanding of
> errors (make other users find to what request an assertion was related)), I
> find that it makes the reading/understanding of a test plan more easy.
> I ran it on previous plans, and I see at least this major benefit.
> Another one (if we move forward) , would be to provide a more source
> version friendly representation of a JMX test plan.
Would this lead to something like Vladimir described? A textual
representation of the test plan, possibly with explanations for the
elements, that is diff and reader friendly.
On the other hand, this reminds me of something. I sometimes dream of
having markers in the test plan tree, that could be used to show
* errors that happened during execution of a test plan (linked to/by
samples in result tree)
* common misconfigurations like double timers (scoping issues mostly)
or beanshell/javascript scripts :)
* samplers with long response times
* ...
Felix
>
>
>
>> Felix
>>
>>
>>> For now, it's html (I'm using XSLT), but a textual format like YAML would
>>> be better for source comparison.
>>>
>>> Regards
>>>
>>> On Mon, Jan 21, 2019 at 11:04 AM Vladimir Sitnikov <
>>> sitnikov.vladimir@gmail.com> wrote:
>>>
>>> Paulo Maia Borges> Why not export to standard HTML?
>>>> For instance, if jmx is stored in a Git repository (GitLab? GitHub?),
>>>> then
>>>> markdown would be much easier to review in diffs than HTML.
>>>> E.g. https://twitter.com/jamiebuilds/status/1085377471057412096
>>>>
>>>> In other words, it could make sense to render markdown representation on
>>>> save.
>>>> However, it is not clear how tables should be rendered (e.g. HTTP
>>>> parameters).
>>>>
>>>> Vladimir
>>>>
>>>>
>>>
>>>
>>> <https://www.openstreetmap.org/#map=18/50.69454/3.16455>
>>>
>>>
Re: New feature : Readable overview of Test Plan
Posted by Philippe Mouawad <ph...@gmail.com>.
On Monday, January 21, 2019, Felix Schumacher <
felix.schumacher@internetallee.de> wrote:
>
> Am 21.01.19 um 11:40 schrieb Philippe Mouawad:
>
>> Hello,
>> This will better explain feature:
>> Menu:
>>
>> - http://ubikloadpack.com/demo/schematic/MenuSchematicView.png
>>
>> Plan:
>>
>> - http://ubikloadpack.com/demo/schematic/testRequest.jmx
>>
>> Schematic View:
>>
>> - http://ubikloadpack.com/demo/schematic/testRequest.html
>>
>
> Where do the names "handle Cookies", "add Headers", "Virtual Users
> executing", ... come from? How would extensions (plugins) fit into this?
It currently comes from the xsl file.
There would be a default display using element name. For now plugins would
have a very basic display.
There would be a need to provide registration for implementations, I see
how to do it for java implementation, but for XSLT, for now I don't.
Working on this feature (within some consulting work, I was asked to
document a plan to provide a textual description and ease understanding of
errors (make other users find to what request an assertion was related)), I
find that it makes the reading/understanding of a test plan more easy.
I ran it on previous plans, and I see at least this major benefit.
Another one (if we move forward) , would be to provide a more source
version friendly representation of a JMX test plan.
> Felix
>
>
>> For now, it's html (I'm using XSLT), but a textual format like YAML would
>> be better for source comparison.
>>
>> Regards
>>
>> On Mon, Jan 21, 2019 at 11:04 AM Vladimir Sitnikov <
>> sitnikov.vladimir@gmail.com> wrote:
>>
>> Paulo Maia Borges> Why not export to standard HTML?
>>>
>>> For instance, if jmx is stored in a Git repository (GitLab? GitHub?),
>>> then
>>> markdown would be much easier to review in diffs than HTML.
>>> E.g. https://twitter.com/jamiebuilds/status/1085377471057412096
>>>
>>> In other words, it could make sense to render markdown representation on
>>> save.
>>> However, it is not clear how tables should be rendered (e.g. HTTP
>>> parameters).
>>>
>>> Vladimir
>>>
>>>
>>
>>
>>
>> <https://www.openstreetmap.org/#map=18/50.69454/3.16455>
>>
>>
Re: New feature : Readable overview of Test Plan
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 21.01.19 um 11:40 schrieb Philippe Mouawad:
> Hello,
> This will better explain feature:
> Menu:
>
> - http://ubikloadpack.com/demo/schematic/MenuSchematicView.png
>
> Plan:
>
> - http://ubikloadpack.com/demo/schematic/testRequest.jmx
>
> Schematic View:
>
> - http://ubikloadpack.com/demo/schematic/testRequest.html
Where do the names "handle Cookies", "add Headers", "Virtual Users
executing", ... come from? How would extensions (plugins) fit into this?
Felix
>
> For now, it's html (I'm using XSLT), but a textual format like YAML would
> be better for source comparison.
>
> Regards
>
> On Mon, Jan 21, 2019 at 11:04 AM Vladimir Sitnikov <
> sitnikov.vladimir@gmail.com> wrote:
>
>> Paulo Maia Borges> Why not export to standard HTML?
>>
>> For instance, if jmx is stored in a Git repository (GitLab? GitHub?), then
>> markdown would be much easier to review in diffs than HTML.
>> E.g. https://twitter.com/jamiebuilds/status/1085377471057412096
>>
>> In other words, it could make sense to render markdown representation on
>> save.
>> However, it is not clear how tables should be rendered (e.g. HTTP
>> parameters).
>>
>> Vladimir
>>
>
>
>
>
> <https://www.openstreetmap.org/#map=18/50.69454/3.16455>
>
Re: New feature : Readable overview of Test Plan
Posted by Philippe Mouawad <p....@ubik-ingenierie.com>.
Hello,
This will better explain feature:
Menu:
- http://ubikloadpack.com/demo/schematic/MenuSchematicView.png
Plan:
- http://ubikloadpack.com/demo/schematic/testRequest.jmx
Schematic View:
- http://ubikloadpack.com/demo/schematic/testRequest.html
For now, it's html (I'm using XSLT), but a textual format like YAML would
be better for source comparison.
Regards
On Mon, Jan 21, 2019 at 11:04 AM Vladimir Sitnikov <
sitnikov.vladimir@gmail.com> wrote:
> Paulo Maia Borges> Why not export to standard HTML?
>
> For instance, if jmx is stored in a Git repository (GitLab? GitHub?), then
> markdown would be much easier to review in diffs than HTML.
> E.g. https://twitter.com/jamiebuilds/status/1085377471057412096
>
> In other words, it could make sense to render markdown representation on
> save.
> However, it is not clear how tables should be rendered (e.g. HTTP
> parameters).
>
> Vladimir
>
<https://www.openstreetmap.org/#map=18/50.69454/3.16455>
Re: New feature : Readable overview of Test Plan
Posted by Vladimir Sitnikov <si...@gmail.com>.
Philippe> It's not my notes, it's Paulo's.
Oh, sorry for the confusion.
Philippe >Did you see what the schematic overview looks like ?
I've seen the above links.
Vladimir
Re: New feature : Readable overview of Test Plan
Posted by Philippe Mouawad <ph...@gmail.com>.
Vladimir,
It's not my notes, it's Paulo's.
Did you see what the schematic overview looks like ?
--------------------------------------------------------------------------------------------------------------
Menu:
- http://ubikloadpack.com/demo/schematic/MenuSchematicView.png
Plan:
- http://ubikloadpack.com/demo/schematic/testRequest.jmx
Schematic View:
- http://ubikloadpack.com/demo/schematic/testRequest.html
For now, it's html (I'm using XSLT), but a textual format like YAML would
be better for source comparison.
--------------------------------------------------------------------------------------------------------------
Thanks
On Mon, Jan 21, 2019 at 12:01 PM Vladimir Sitnikov <
sitnikov.vladimir@gmail.com> wrote:
> Philippe>itself in version control and, thus, fully tracked in terms of
> diffs.
>
> Frankly speaking, JMX is hard to review.
> For instance, if a developer commits a new test plan version, then I
> could wonder "what changed".
> JMX diff does not provide that option.
> I'm not sure if JMeter can load two JMX files and compare them somehow
> (I guess it is not implemented), however plain old diff for JMX is not
> very human-understandable.
>
> On the other hand, if we add (optionally) a side file that is
> "human-readable" representation, then it would be so much easier to
> review the diff.
>
> Philippe> Maybe you're right, if it was HTML it would be much harder
> to spot differences. Though I'm sceptical of the usefulness of that
> being of more value than the usefulness of having a graphical (HTML)
> way to see the test.
>
> I guess those two (markdown vs HTML) target different use cases:
> a) Markdown is best for incremental review. In other words, it helps
> to review the script by providing a small gist of the altered bits.
> b) HTML might shine in reviewing the full script at once. For
> instance, if I browse a folder with multiple JMX files, I might wander
> how those scripts "look like", and it might be cool to just open a
> HTML file rather than start JMeter.
>
> Vladimir
>
--
Cordialement.
Philippe Mouawad.
Re: New feature : Readable overview of Test Plan
Posted by Vladimir Sitnikov <si...@gmail.com>.
Philippe>itself in version control and, thus, fully tracked in terms of diffs.
Frankly speaking, JMX is hard to review.
For instance, if a developer commits a new test plan version, then I
could wonder "what changed".
JMX diff does not provide that option.
I'm not sure if JMeter can load two JMX files and compare them somehow
(I guess it is not implemented), however plain old diff for JMX is not
very human-understandable.
On the other hand, if we add (optionally) a side file that is
"human-readable" representation, then it would be so much easier to
review the diff.
Philippe> Maybe you're right, if it was HTML it would be much harder
to spot differences. Though I'm sceptical of the usefulness of that
being of more value than the usefulness of having a graphical (HTML)
way to see the test.
I guess those two (markdown vs HTML) target different use cases:
a) Markdown is best for incremental review. In other words, it helps
to review the script by providing a small gist of the altered bits.
b) HTML might shine in reviewing the full script at once. For
instance, if I browse a folder with multiple JMX files, I might wander
how those scripts "look like", and it might be cool to just open a
HTML file rather than start JMeter.
Vladimir
RE: New feature : Readable overview of Test Plan
Posted by Paulo Maia Borges <pm...@temenos.com>.
Hi Vladimir,
"if jmx is stored in a Git repository (GitLab? GitHub?), then markdown would be much easier to review in diffs than HTML"
^ did you mean the export as "xml" instead of "jmx"?
I agree with your point. However, I'm not sure what's the use of tracking diffs in a Test Plan export (version control, in essence), because we would likely have the JMX (JMeter "script") itself in version control and, thus, fully tracked in terms of diffs.
I reckon it might be useful to keep exporting both Test-Results *and* Test-Plan-Summary for test runs. Then, later, it would be useful to compare Test-Plan-Summaries between test runs. However, I'm not sure this warrants the extra "ease" of comparing XML vs comparing HTML. In fact, I'm not even sure it would be "easier" to compare XML - we'd need some kind of tool for that and, more importantly, the ability to interpret the meaning of the XML elements, which might or might not be trivial, but will certainly be daunting for the less tech-savvy person.
If a few whole XML sections dis/appear from one file to another, .., the comparison can be quite hard to interpret - I've been running into that myself with version diffs for our JMX files when reviewing the Git commits.
Maybe you're right, if it was HTML it would be much harder to spot differences. Though I'm sceptical of the usefulness of that being of more value than the usefulness of having a graphical (HTML) way to see the test.
Also, if we export a test-plan pattern to XML, I reckon that pretty much rules out the shareability of such exported documents with non-technical people...
Regards,
Paulo Augusto Maia Borges
WWVzLCBJJ20gZ2Vla3kuIEJ1dCBzbyBhcmUgeW91IQ==
-----Original Message-----
From: Vladimir Sitnikov <si...@gmail.com>
Sent: 21 January 2019 10:04
To: ApacheJMeter dev list <de...@jmeter.apache.org>
Subject: Re: New feature : Readable overview of Test Plan
...
The information in this e-mail and any attachments is confidential and may be legally privileged. It is intended solely for the addressee or addressees. Any use or disclosure of the contents of this e-mail/attachments by a not intended recipient is unauthorized and may be unlawful. If you have received this e-mail in error please notify the sender. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of TEMENOS. We recommend that you check this e-mail and any attachments against viruses. TEMENOS accepts no liability for any damage caused by any malicious code or virus transmitted by this e-mail.
Re: New feature : Readable overview of Test Plan
Posted by Vladimir Sitnikov <si...@gmail.com>.
Paulo Maia Borges> Why not export to standard HTML?
For instance, if jmx is stored in a Git repository (GitLab? GitHub?), then
markdown would be much easier to review in diffs than HTML.
E.g. https://twitter.com/jamiebuilds/status/1085377471057412096
In other words, it could make sense to render markdown representation on
save.
However, it is not clear how tables should be rendered (e.g. HTTP
parameters).
Vladimir
RE: New feature : Readable overview of Test Plan
Posted by Paulo Maia Borges <pm...@temenos.com>.
Hi Philippe,
(disclaimer - I don't know anything about that schematics script thingy and couldn't find documentation for it)
I like that idea a lot. It is quite common to require a documented form of what a test plan is like, either in preparation before a test or to «look back into the past».
You appear to be asking what people think about the export format. Why not export to standard HTML? JMeter already exports test results to HTML (? never tried it myself) - https://jmeter.apache.org/usermanual/generating-dashboard.html - so, following the pattern, a test-plan export should also export to HTML. Also, HTML is the most widely accepted format beyond text-file and is extremely versatile. Unless I didn't fully understand what you meant.
Another interesting question would be what kind of content to produce - high level or technical. Like you said and I corroborate, it is common to need to document how a test plan is. But for 2 different objectives. One is high level, to present to managers, stake-holders, etc, non-technical people who want to see how their product is going to be tested. The second is technical, to precisely document and later be studied by whoever is going to analyse the results (maybe even months later) or by another test scripter who needs to reproduce the test pattern into another website or so.
I wonder .. would it be possible to integrate that into the test *result* report? Results usually speak for themselves and are the ultimate "truth". However, somewhat often we analyse a test result and wish we could see exactly how that test was configured at the time...
Another consideration - the whole functionality might not achieve wonderful results, since many of the test settings might be outside of the test, like CSV files that change drastically the behaviour of the test, or imported command-line variables. <- despite me still liking the idea. I'm just recommending to consider how much that might, or not, decrease the value of such a functionality.
Regards,
Paulo Augusto Maia Borges
WWVzLCBJJ20gZ2Vla3kuIEJ1dCBzbyBhcmUgeW91IQ==
-----Original Message-----
From: Philippe Mouawad <p....@ubik-ingenierie.com>
Sent: 18 January 2019 12:38
To: dev@jmeter.apache.org
Subject: New feature : Readable overview of Test Plan
Hello,
I think there is a need in JMeter since we're not a DSL based tool, to provide an overview of the test for documentation purpose.
It would also help a lot on issue reporting for example
Currently, I think the only available option is to use schematic.sh which
is:
- outdated
- very limited
- not friendly to use, you need to copy the test plan in extras folder
to make it work
As the expected overview might depend a lot on what somebody wants to see, what do you think of this as a first step:
- Create in new Tools menu Item (where we would move Curl, Export
Transactions for Report, Create Heap/Thread dump) an entry "Export as
schematic overview"
- The feature would :
- open A dialog with Option path to enter Exporter
- I see at least 2 ways to implement exporter:
- XSL with schematic way:
- + : Not much work
- - : XSL is not very readable nor friendly
- - : JMeter has no contract on XML
- Add an interface, JMeter would discover implementation and propose it:
- We would use the Visitor pattern
- + : Very flexible
- + : Easier to maintain, we rely on Java Contract
- - : More work for user
- Note we can implement XSL Exporter as first step
- In the long term, maybe we could be proposing an exporter to YAML or
JSON
Exporter would look have:
export(File pathToJMeterFile, HashTree testPlan, Writer writer);
Interface for exporter to be implemented:
interface NodeExporter {
String exportElement(int level, TestElement element, HashTree testPlan); }
Any thoughts ?
Regards
Philippe
<https://www.openstreetmap.org/#map=18/50.69454/3.16455>
The information in this e-mail and any attachments is confidential and may be legally privileged. It is intended solely for the addressee or addressees. Any use or disclosure of the contents of this e-mail/attachments by a not intended recipient is unauthorized and may be unlawful. If you have received this e-mail in error please notify the sender. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of TEMENOS. We recommend that you check this e-mail and any attachments against viruses. TEMENOS accepts no liability for any damage caused by any malicious code or virus transmitted by this e-mail.