You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jmeter.apache.org by Philippe Mouawad <ph...@gmail.com> on 2015/02/03 13:38:05 UTC
New Listener to generate HTML report at end of load test Was APDEX
Computing and reporting
Hi,
Any additional opinions of dev team or users ?
I 'd like to know if I should continue efforts on this or stop it.
Thanks
Regards
On Monday, February 2, 2015, Philippe Mouawad <ph...@gmail.com>
wrote:
>
>
> On Monday, February 2, 2015, sebb <sebbaz@gmail.com
> <javascript:_e(%7B%7D,'cvml','sebbaz@gmail.com');>> wrote:
>
>> On 1 February 2015 at 22:41, Philippe Mouawad
>> <ph...@gmail.com> wrote:
>> > On Sunday, February 1, 2015, sebb <se...@gmail.com> wrote:
>> >
>> >> On 1 February 2015 at 19:31, Philippe Mouawad
>> >> <philippe.mouawad@gmail.com <javascript:;>> wrote:
>> >> > Hi ,
>> >> > I uploaded a screenshot and new patch code showing what I meant.
>> >> > I think including it in core should be considered seriously.
>> >>
>> >> If the code reads the CSV file at the end of a test to produce the
>> >> HTML file, there is no need for it to be included in the JMeter code
>> >> which is used to run tests.
>> >> It should be a separate application.
>> >>
>> >> I don't share your opinion sebb.
>> > Having a report ready at end of load test is nowadays a standard, look
>> at
>> > market tools wether open source or not.
>> > I don't see the problem here, can you give more details ?
>>
>> The problem is that it adds extra code to the JMeter application which
>> takes extra memory.
>
>
> My last implementation only uses memory at end of test not morz during it.
>
>
>> Also, if the user wants to process existing files, it will be easier
>
> to have a standalone application rather than having to start the
>> JMeter test application and then find the reporting tool.
>
>
> It is possible with it, just make a plan with 1 debug sampler and put
> listener. Report will be generated.
>
>>
>> It would be useful if the tool could be used in batch mode to process
>> a set of CSV files.
>
>
> Maybe, but I think it's an additional feature. Also having another tool
> may lead to what happened to the report package, it seems it was never
> used. But if you implement it it will be ok for me.
>
>
>>
>> It's also likely to be easier for others to contribute additional
>> reporting tools if they are part of a separate application.
>
> I don't understand this ? how is it easier if it's in jmeter core. If it's
> apart I don't support this option.
>
>
>
>> It should make the coding requirements simpler.
>
> how ?
>
>>
>> > Having another application( by the way do you mean provided as a tool in
>> > jmeter or third party) means in the first case you need to setup another
>> > tool and in the latter case need to develop your own, or use or pay one.
>>
>> Of course the code should be included with JMeter.
>> But it should be a separate tool.
>>
>> > This is the major drawback of JMeter I hear from customers and read
>> around
>> > the net.
>>
>> > I first though it would not be that easy but it appears we are able to
>> > reuse existing code.
>>
>> I don't follow.
>>
>> > If you disagree, I think we should wait for other team members their
>> > opinion and/or setup a vote for this.
>>
>> The main JMeter code should be reserved for setting up and running tests.
>>
> IMHO reporting must be part of JMeter, and anyway it is already through
> some Listeners , which although not perfect bring useful infos. The aim of
> this new listener is:
> 1/ Allow generation in gui and non gui mode
> 2/ Have a "sexy" (not yet ) report readable in browser with dynamic
> behaviour (zoom, select some samples...)
> 3/ Allow generation at end of load test
> 4/ Make it easily customizable as FTL will be a property
> 5/ Make it extensible in the future
>
>
>
>
>> >
>> >
>> >
>> >
>> >> > @Felix thanks for review of initial patch.
>> >> > Regards
>> >> >
>> >> > On Sunday, February 1, 2015, sebb <sebbaz@gmail.com <javascript:;>>
>> >> wrote:
>> >> >
>> >> >> On 31 January 2015 at 13:16, Philippe Mouawad
>> >> >> <philippe.mouawad@gmail.com <javascript:;> <javascript:;>> wrote:
>> >> >> > On Sat, Jan 31, 2015 at 12:30 PM, Felix Schumacher <
>> >> >> > felix.schumacher@internetallee.de <javascript:;> <javascript:;>>
>> >> wrote:
>> >> >> >
>> >> >> >> Am 30.01.2015 um 13:31 schrieb Philippe Mouawad:
>> >> >> >>
>> >> >> >>> Hi,
>> >> >> >>> I intend to commit a BackendListener client implementation that
>> >> >> computes
>> >> >> >>> APDEX at end of Load Test.
>> >> >> >>> It will take:
>> >> >> >>>
>> >> >> >>> - Acceptable Response Time (http://www.apdex.fr/) T
>> >> >> >>>
>> >> >> >> The link was in french (my french is not very good (really non
>> >> existant
>> >> >> >> :)), but luckily wikipedia had an article about apdex, which I
>> could
>> >> >> read.
>> >> >> >>
>> >> >> >
>> >> >> > Sorry for the french link some references:
>> >> >> > http://www.apdex.org/overview.html
>> >> >> >
>> >> >> > - Compute F as 4xT but allow customization
>> >> >> >>> - List of samples taken into account
>> >> >> >>>
>> >> >> >>>
>> >> >> >>> This listener will compute it during load test and generate an
>> HTML
>> >> >> file
>> >> >> >>> at
>> >> >> >>> end of test containing it.
>> >> >> >>>
>> >> >> >>> This listener could be later enhanced to :
>> >> >> >>>
>> >> >> >>> - Generate equivalent of Aggregate Report
>> >> >> >>> - Possibly graphs based on chartjs or some other graphing JS
>> >> >> Library
>> >> >> >>>
>> >> >> >> I understand, that apdex is a simple mean to have one metric. As
>> >> such I
>> >> >> >> wonder why we would want to graph it.
>> >> >> >
>> >> >> > In fact my proposal was larger than just APDEX.
>> >> >> > What I am proposing is to implement a report generator that at
>> end of
>> >> >> Test
>> >> >> > would compute HTML page with:
>> >> >> > - APDEX
>> >> >> > - Some graphs like:
>> >> >> > - Plot of OK/KO (http://www.chartjs.org/docs/#doughnut-pie-chart
>> )
>> >> >> > - Request Summary (what's currently in Aggregate Graph)
>> >> >> > http://www.chartjs.org/docs/#bar-chart
>> >> >> > - Response Time over time (
>> http://www.chartjs.org/docs/#line-chart)
>> >> >> >
>> >> >> > I think all this should in fact be computed at end of load test
>> >> instead
>> >> >> of
>> >> >> > during it.
>> >> >> > The input of this listener would be the generated CSV file, and at
>> >> end of
>> >> >> > Test it would read it an compute all this.
>> >> >>
>> >> >> In which case it should be a stand-alone app.
>> >> >>
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >> If we want to display a graph of apdex values for different
>> tolerated
>> >> >> >> times, than we would have to store all values, which would make
>> the
>> >> >> >> listener quite heavy on the memory side.
>> >> >> >>
>> >> >> >>>
>> >> >> >>> Thoughts ?
>> >> >> >>>
>> >> >> >> How will errors be counted? Sometimes I can cope with errors, as
>> >> long as
>> >> >> >> they are reported fast and sometimes an error would result in a
>> very
>> >> >> >> unhappy consumer.
>> >> >> >>
>> >> >> > Good catch, I think only successful samples must be taken into
>> >> account.
>> >> >> > Error will count only toward total requests (unsatisfied).
>> >> >> >
>> >> >> >>
>> >> >> >> Regards
>> >> >> >> Felix
>> >> >> >>
>> >> >> >>
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Cordialement.
>> >> >> > Philippe Mouawad.
>> >> >>
>> >> >
>> >> >
>> >> > --
>> >> > Cordialement.
>> >> > Philippe Mouawad.
>> >>
>> >
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: New Listener to generate HTML report at end of load test Was
APDEX Computing and reporting
Posted by Vladimir Sitnikov <si...@gmail.com>.
+1 for being able to run the tool from both within GUI and from the
command line.
Vladmir
Re: New Listener to generate HTML report at end of load test Was
APDEX Computing and reporting
Posted by Antonio Gomes Rodrigues <ra...@gmail.com>.
Hi,
Same opinion and all the other load testing tools (Gatling, LoadRunner,
Silkperformer, etc.) have it
Antonio
2015-02-10 9:56 GMT+01:00 <Pa...@t-systems.com>:
> As a user I think this feature would be very helpful.
>
> Reports are a JMeter weakest feature, so enhancements in this area are
> very welcome.
>
> It would also be nice if it as easy as possible to use. Imho this means
> that it should be added to JMeter and not to a separate commandline utility
> etc.
>
> Regards,
> Pascal
>
> -----Ursprüngliche Nachricht-----
> Von: Felix Schumacher [mailto:felix.schumacher@internetallee.de]
> Gesendet: Montag, 9. Februar 2015 18:55
> An: dev@jmeter.apache.org
> Betreff: Re: New Listener to generate HTML report at end of load test Was
> APDEX Computing and reporting
>
> Am 03.02.2015 um 13:38 schrieb Philippe Mouawad:
> > Hi,
> > Any additional opinions of dev team or users ?
> > I 'd like to know if I should continue efforts on this or stop it.
> I think it would be a nice feature to have a simple report at the end of a
> jmeter run. Maybe the user should ask for it by specifying a command line
> parameter like "--report".
>
> If this is integrated in the shell wrapper it could even be a program that
> is started after the jmeter run. But I don't think users will generally
> start another program themselves, so we shouldn't hide it too deep.
>
> Regards
> Felix
> >
> > Thanks
> > Regards
> >
> > On Monday, February 2, 2015, Philippe Mouawad
> > <ph...@gmail.com>
> > wrote:
> >
> >>
> >> On Monday, February 2, 2015, sebb <sebbaz@gmail.com
> >> <javascript:_e(%7B%7D,'cvml','sebbaz@gmail.com');>> wrote:
> >>
> >>> On 1 February 2015 at 22:41, Philippe Mouawad
> >>> <ph...@gmail.com> wrote:
> >>>> On Sunday, February 1, 2015, sebb <se...@gmail.com> wrote:
> >>>>
> >>>>> On 1 February 2015 at 19:31, Philippe Mouawad
> >>>>> <philippe.mouawad@gmail.com <javascript:;>> wrote:
> >>>>>> Hi ,
> >>>>>> I uploaded a screenshot and new patch code showing what I meant.
> >>>>>> I think including it in core should be considered seriously.
> >>>>> If the code reads the CSV file at the end of a test to produce the
> >>>>> HTML file, there is no need for it to be included in the JMeter
> >>>>> code which is used to run tests.
> >>>>> It should be a separate application.
> >>>>>
> >>>>> I don't share your opinion sebb.
> >>>> Having a report ready at end of load test is nowadays a standard,
> >>>> look
> >>> at
> >>>> market tools wether open source or not.
> >>>> I don't see the problem here, can you give more details ?
> >>> The problem is that it adds extra code to the JMeter application
> >>> which takes extra memory.
> >>
> >> My last implementation only uses memory at end of test not morz during
> it.
> >>
> >>
> >>> Also, if the user wants to process existing files, it will be easier
> >> to have a standalone application rather than having to start the
> >>> JMeter test application and then find the reporting tool.
> >>
> >> It is possible with it, just make a plan with 1 debug sampler and put
> >> listener. Report will be generated.
> >>
> >>> It would be useful if the tool could be used in batch mode to
> >>> process a set of CSV files.
> >>
> >> Maybe, but I think it's an additional feature. Also having another
> >> tool may lead to what happened to the report package, it seems it was
> >> never used. But if you implement it it will be ok for me.
> >>
> >>
> >>> It's also likely to be easier for others to contribute additional
> >>> reporting tools if they are part of a separate application.
> >> I don't understand this ? how is it easier if it's in jmeter core. If
> >> it's apart I don't support this option.
> >>
> >>
> >>
> >>> It should make the coding requirements simpler.
> >> how ?
> >>
> >>>> Having another application( by the way do you mean provided as a
> >>>> tool in jmeter or third party) means in the first case you need to
> >>>> setup another tool and in the latter case need to develop your own,
> or use or pay one.
> >>> Of course the code should be included with JMeter.
> >>> But it should be a separate tool.
> >>>
> >>>> This is the major drawback of JMeter I hear from customers and read
> >>> around
> >>>> the net.
> >>>> I first though it would not be that easy but it appears we are able
> >>>> to reuse existing code.
> >>> I don't follow.
> >>>
> >>>> If you disagree, I think we should wait for other team members
> >>>> their opinion and/or setup a vote for this.
> >>> The main JMeter code should be reserved for setting up and running
> tests.
> >>>
> >> IMHO reporting must be part of JMeter, and anyway it is already
> >> through some Listeners , which although not perfect bring useful
> >> infos. The aim of this new listener is:
> >> 1/ Allow generation in gui and non gui mode 2/ Have a "sexy" (not yet
> >> ) report readable in browser with dynamic behaviour (zoom, select
> >> some samples...) 3/ Allow generation at end of load test 4/ Make it
> >> easily customizable as FTL will be a property 5/ Make it extensible
> >> in the future
> >>
> >>
> >>
> >>
> >>>>
> >>>>
> >>>>
> >>>>>> @Felix thanks for review of initial patch.
> >>>>>> Regards
> >>>>>>
> >>>>>> On Sunday, February 1, 2015, sebb <sebbaz@gmail.com
> >>>>>> <javascript:;>>
> >>>>> wrote:
> >>>>>>> On 31 January 2015 at 13:16, Philippe Mouawad
> >>>>>>> <philippe.mouawad@gmail.com <javascript:;> <javascript:;>> wrote:
> >>>>>>>> On Sat, Jan 31, 2015 at 12:30 PM, Felix Schumacher <
> >>>>>>>> felix.schumacher@internetallee.de <javascript:;>
> >>>>>>>> <javascript:;>>
> >>>>> wrote:
> >>>>>>>>> Am 30.01.2015 um 13:31 schrieb Philippe Mouawad:
> >>>>>>>>>
> >>>>>>>>>> Hi,
> >>>>>>>>>> I intend to commit a BackendListener client implementation
> >>>>>>>>>> that
> >>>>>>> computes
> >>>>>>>>>> APDEX at end of Load Test.
> >>>>>>>>>> It will take:
> >>>>>>>>>>
> >>>>>>>>>> - Acceptable Response Time (http://www.apdex.fr/) T
> >>>>>>>>>>
> >>>>>>>>> The link was in french (my french is not very good (really non
> >>>>> existant
> >>>>>>>>> :)), but luckily wikipedia had an article about apdex, which I
> >>> could
> >>>>>>> read.
> >>>>>>>> Sorry for the french link some references:
> >>>>>>>> http://www.apdex.org/overview.html
> >>>>>>>>
> >>>>>>>> - Compute F as 4xT but allow customization
> >>>>>>>>>> - List of samples taken into account
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> This listener will compute it during load test and generate
> >>>>>>>>>> an
> >>> HTML
> >>>>>>> file
> >>>>>>>>>> at
> >>>>>>>>>> end of test containing it.
> >>>>>>>>>>
> >>>>>>>>>> This listener could be later enhanced to :
> >>>>>>>>>>
> >>>>>>>>>> - Generate equivalent of Aggregate Report
> >>>>>>>>>> - Possibly graphs based on chartjs or some other
> >>>>>>>>>> graphing JS
> >>>>>>> Library
> >>>>>>>>> I understand, that apdex is a simple mean to have one metric.
> >>>>>>>>> As
> >>>>> such I
> >>>>>>>>> wonder why we would want to graph it.
> >>>>>>>> In fact my proposal was larger than just APDEX.
> >>>>>>>> What I am proposing is to implement a report generator that at
> >>> end of
> >>>>>>> Test
> >>>>>>>> would compute HTML page with:
> >>>>>>>> - APDEX
> >>>>>>>> - Some graphs like:
> >>>>>>>> - Plot of OK/KO
> >>>>>>>> (http://www.chartjs.org/docs/#doughnut-pie-chart
> >>> )
> >>>>>>>> - Request Summary (what's currently in Aggregate Graph)
> >>>>>>>> http://www.chartjs.org/docs/#bar-chart
> >>>>>>>> - Response Time over time (
> >>> http://www.chartjs.org/docs/#line-chart)
> >>>>>>>> I think all this should in fact be computed at end of load test
> >>>>> instead
> >>>>>>> of
> >>>>>>>> during it.
> >>>>>>>> The input of this listener would be the generated CSV file, and
> >>>>>>>> at
> >>>>> end of
> >>>>>>>> Test it would read it an compute all this.
> >>>>>>> In which case it should be a stand-alone app.
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> If we want to display a graph of apdex values for different
> >>> tolerated
> >>>>>>>>> times, than we would have to store all values, which would
> >>>>>>>>> make
> >>> the
> >>>>>>>>> listener quite heavy on the memory side.
> >>>>>>>>>
> >>>>>>>>>> Thoughts ?
> >>>>>>>>>>
> >>>>>>>>> How will errors be counted? Sometimes I can cope with errors,
> >>>>>>>>> as
> >>>>> long as
> >>>>>>>>> they are reported fast and sometimes an error would result in
> >>>>>>>>> a
> >>> very
> >>>>>>>>> unhappy consumer.
> >>>>>>>>>
> >>>>>>>> Good catch, I think only successful samples must be taken into
> >>>>> account.
> >>>>>>>> Error will count only toward total requests (unsatisfied).
> >>>>>>>>
> >>>>>>>>> Regards
> >>>>>>>>> Felix
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Cordialement.
> >>>>>>>> Philippe Mouawad.
> >>>>>>
> >>>>>> --
> >>>>>> Cordialement.
> >>>>>> Philippe Mouawad.
> >>>>
> >>>> --
> >>>> Cordialement.
> >>>> Philippe Mouawad.
> >>
> >> --
> >> Cordialement.
> >> Philippe Mouawad.
> >>
> >>
> >>
> >>
>
>
AW: New Listener to generate HTML report at end of load test Was
APDEX Computing and reporting
Posted by Pa...@t-systems.com.
As a user I think this feature would be very helpful.
Reports are a JMeter weakest feature, so enhancements in this area are very welcome.
It would also be nice if it as easy as possible to use. Imho this means that it should be added to JMeter and not to a separate commandline utility etc.
Regards,
Pascal
-----Ursprüngliche Nachricht-----
Von: Felix Schumacher [mailto:felix.schumacher@internetallee.de]
Gesendet: Montag, 9. Februar 2015 18:55
An: dev@jmeter.apache.org
Betreff: Re: New Listener to generate HTML report at end of load test Was APDEX Computing and reporting
Am 03.02.2015 um 13:38 schrieb Philippe Mouawad:
> Hi,
> Any additional opinions of dev team or users ?
> I 'd like to know if I should continue efforts on this or stop it.
I think it would be a nice feature to have a simple report at the end of a jmeter run. Maybe the user should ask for it by specifying a command line parameter like "--report".
If this is integrated in the shell wrapper it could even be a program that is started after the jmeter run. But I don't think users will generally start another program themselves, so we shouldn't hide it too deep.
Regards
Felix
>
> Thanks
> Regards
>
> On Monday, February 2, 2015, Philippe Mouawad
> <ph...@gmail.com>
> wrote:
>
>>
>> On Monday, February 2, 2015, sebb <sebbaz@gmail.com
>> <javascript:_e(%7B%7D,'cvml','sebbaz@gmail.com');>> wrote:
>>
>>> On 1 February 2015 at 22:41, Philippe Mouawad
>>> <ph...@gmail.com> wrote:
>>>> On Sunday, February 1, 2015, sebb <se...@gmail.com> wrote:
>>>>
>>>>> On 1 February 2015 at 19:31, Philippe Mouawad
>>>>> <philippe.mouawad@gmail.com <javascript:;>> wrote:
>>>>>> Hi ,
>>>>>> I uploaded a screenshot and new patch code showing what I meant.
>>>>>> I think including it in core should be considered seriously.
>>>>> If the code reads the CSV file at the end of a test to produce the
>>>>> HTML file, there is no need for it to be included in the JMeter
>>>>> code which is used to run tests.
>>>>> It should be a separate application.
>>>>>
>>>>> I don't share your opinion sebb.
>>>> Having a report ready at end of load test is nowadays a standard,
>>>> look
>>> at
>>>> market tools wether open source or not.
>>>> I don't see the problem here, can you give more details ?
>>> The problem is that it adds extra code to the JMeter application
>>> which takes extra memory.
>>
>> My last implementation only uses memory at end of test not morz during it.
>>
>>
>>> Also, if the user wants to process existing files, it will be easier
>> to have a standalone application rather than having to start the
>>> JMeter test application and then find the reporting tool.
>>
>> It is possible with it, just make a plan with 1 debug sampler and put
>> listener. Report will be generated.
>>
>>> It would be useful if the tool could be used in batch mode to
>>> process a set of CSV files.
>>
>> Maybe, but I think it's an additional feature. Also having another
>> tool may lead to what happened to the report package, it seems it was
>> never used. But if you implement it it will be ok for me.
>>
>>
>>> It's also likely to be easier for others to contribute additional
>>> reporting tools if they are part of a separate application.
>> I don't understand this ? how is it easier if it's in jmeter core. If
>> it's apart I don't support this option.
>>
>>
>>
>>> It should make the coding requirements simpler.
>> how ?
>>
>>>> Having another application( by the way do you mean provided as a
>>>> tool in jmeter or third party) means in the first case you need to
>>>> setup another tool and in the latter case need to develop your own, or use or pay one.
>>> Of course the code should be included with JMeter.
>>> But it should be a separate tool.
>>>
>>>> This is the major drawback of JMeter I hear from customers and read
>>> around
>>>> the net.
>>>> I first though it would not be that easy but it appears we are able
>>>> to reuse existing code.
>>> I don't follow.
>>>
>>>> If you disagree, I think we should wait for other team members
>>>> their opinion and/or setup a vote for this.
>>> The main JMeter code should be reserved for setting up and running tests.
>>>
>> IMHO reporting must be part of JMeter, and anyway it is already
>> through some Listeners , which although not perfect bring useful
>> infos. The aim of this new listener is:
>> 1/ Allow generation in gui and non gui mode 2/ Have a "sexy" (not yet
>> ) report readable in browser with dynamic behaviour (zoom, select
>> some samples...) 3/ Allow generation at end of load test 4/ Make it
>> easily customizable as FTL will be a property 5/ Make it extensible
>> in the future
>>
>>
>>
>>
>>>>
>>>>
>>>>
>>>>>> @Felix thanks for review of initial patch.
>>>>>> Regards
>>>>>>
>>>>>> On Sunday, February 1, 2015, sebb <sebbaz@gmail.com
>>>>>> <javascript:;>>
>>>>> wrote:
>>>>>>> On 31 January 2015 at 13:16, Philippe Mouawad
>>>>>>> <philippe.mouawad@gmail.com <javascript:;> <javascript:;>> wrote:
>>>>>>>> On Sat, Jan 31, 2015 at 12:30 PM, Felix Schumacher <
>>>>>>>> felix.schumacher@internetallee.de <javascript:;>
>>>>>>>> <javascript:;>>
>>>>> wrote:
>>>>>>>>> Am 30.01.2015 um 13:31 schrieb Philippe Mouawad:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> I intend to commit a BackendListener client implementation
>>>>>>>>>> that
>>>>>>> computes
>>>>>>>>>> APDEX at end of Load Test.
>>>>>>>>>> It will take:
>>>>>>>>>>
>>>>>>>>>> - Acceptable Response Time (http://www.apdex.fr/) T
>>>>>>>>>>
>>>>>>>>> The link was in french (my french is not very good (really non
>>>>> existant
>>>>>>>>> :)), but luckily wikipedia had an article about apdex, which I
>>> could
>>>>>>> read.
>>>>>>>> Sorry for the french link some references:
>>>>>>>> http://www.apdex.org/overview.html
>>>>>>>>
>>>>>>>> - Compute F as 4xT but allow customization
>>>>>>>>>> - List of samples taken into account
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> This listener will compute it during load test and generate
>>>>>>>>>> an
>>> HTML
>>>>>>> file
>>>>>>>>>> at
>>>>>>>>>> end of test containing it.
>>>>>>>>>>
>>>>>>>>>> This listener could be later enhanced to :
>>>>>>>>>>
>>>>>>>>>> - Generate equivalent of Aggregate Report
>>>>>>>>>> - Possibly graphs based on chartjs or some other
>>>>>>>>>> graphing JS
>>>>>>> Library
>>>>>>>>> I understand, that apdex is a simple mean to have one metric.
>>>>>>>>> As
>>>>> such I
>>>>>>>>> wonder why we would want to graph it.
>>>>>>>> In fact my proposal was larger than just APDEX.
>>>>>>>> What I am proposing is to implement a report generator that at
>>> end of
>>>>>>> Test
>>>>>>>> would compute HTML page with:
>>>>>>>> - APDEX
>>>>>>>> - Some graphs like:
>>>>>>>> - Plot of OK/KO
>>>>>>>> (http://www.chartjs.org/docs/#doughnut-pie-chart
>>> )
>>>>>>>> - Request Summary (what's currently in Aggregate Graph)
>>>>>>>> http://www.chartjs.org/docs/#bar-chart
>>>>>>>> - Response Time over time (
>>> http://www.chartjs.org/docs/#line-chart)
>>>>>>>> I think all this should in fact be computed at end of load test
>>>>> instead
>>>>>>> of
>>>>>>>> during it.
>>>>>>>> The input of this listener would be the generated CSV file, and
>>>>>>>> at
>>>>> end of
>>>>>>>> Test it would read it an compute all this.
>>>>>>> In which case it should be a stand-alone app.
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> If we want to display a graph of apdex values for different
>>> tolerated
>>>>>>>>> times, than we would have to store all values, which would
>>>>>>>>> make
>>> the
>>>>>>>>> listener quite heavy on the memory side.
>>>>>>>>>
>>>>>>>>>> Thoughts ?
>>>>>>>>>>
>>>>>>>>> How will errors be counted? Sometimes I can cope with errors,
>>>>>>>>> as
>>>>> long as
>>>>>>>>> they are reported fast and sometimes an error would result in
>>>>>>>>> a
>>> very
>>>>>>>>> unhappy consumer.
>>>>>>>>>
>>>>>>>> Good catch, I think only successful samples must be taken into
>>>>> account.
>>>>>>>> Error will count only toward total requests (unsatisfied).
>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>> Felix
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Cordialement.
>>>>>>>> Philippe Mouawad.
>>>>>>
>>>>>> --
>>>>>> Cordialement.
>>>>>> Philippe Mouawad.
>>>>
>>>> --
>>>> Cordialement.
>>>> Philippe Mouawad.
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>>
>>
>>
>>
Re: New Listener to generate HTML report at end of load test Was
APDEX Computing and reporting
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 03.02.2015 um 13:38 schrieb Philippe Mouawad:
> Hi,
> Any additional opinions of dev team or users ?
> I 'd like to know if I should continue efforts on this or stop it.
I think it would be a nice feature to have a simple report at the end of
a jmeter run. Maybe the user should ask for it by specifying a command
line parameter like "--report".
If this is integrated in the shell wrapper it could even be a program
that is started after the jmeter run. But I don't think users will
generally start another program themselves, so we shouldn't hide it too
deep.
Regards
Felix
>
> Thanks
> Regards
>
> On Monday, February 2, 2015, Philippe Mouawad <ph...@gmail.com>
> wrote:
>
>>
>> On Monday, February 2, 2015, sebb <sebbaz@gmail.com
>> <javascript:_e(%7B%7D,'cvml','sebbaz@gmail.com');>> wrote:
>>
>>> On 1 February 2015 at 22:41, Philippe Mouawad
>>> <ph...@gmail.com> wrote:
>>>> On Sunday, February 1, 2015, sebb <se...@gmail.com> wrote:
>>>>
>>>>> On 1 February 2015 at 19:31, Philippe Mouawad
>>>>> <philippe.mouawad@gmail.com <javascript:;>> wrote:
>>>>>> Hi ,
>>>>>> I uploaded a screenshot and new patch code showing what I meant.
>>>>>> I think including it in core should be considered seriously.
>>>>> If the code reads the CSV file at the end of a test to produce the
>>>>> HTML file, there is no need for it to be included in the JMeter code
>>>>> which is used to run tests.
>>>>> It should be a separate application.
>>>>>
>>>>> I don't share your opinion sebb.
>>>> Having a report ready at end of load test is nowadays a standard, look
>>> at
>>>> market tools wether open source or not.
>>>> I don't see the problem here, can you give more details ?
>>> The problem is that it adds extra code to the JMeter application which
>>> takes extra memory.
>>
>> My last implementation only uses memory at end of test not morz during it.
>>
>>
>>> Also, if the user wants to process existing files, it will be easier
>> to have a standalone application rather than having to start the
>>> JMeter test application and then find the reporting tool.
>>
>> It is possible with it, just make a plan with 1 debug sampler and put
>> listener. Report will be generated.
>>
>>> It would be useful if the tool could be used in batch mode to process
>>> a set of CSV files.
>>
>> Maybe, but I think it's an additional feature. Also having another tool
>> may lead to what happened to the report package, it seems it was never
>> used. But if you implement it it will be ok for me.
>>
>>
>>> It's also likely to be easier for others to contribute additional
>>> reporting tools if they are part of a separate application.
>> I don't understand this ? how is it easier if it's in jmeter core. If it's
>> apart I don't support this option.
>>
>>
>>
>>> It should make the coding requirements simpler.
>> how ?
>>
>>>> Having another application( by the way do you mean provided as a tool in
>>>> jmeter or third party) means in the first case you need to setup another
>>>> tool and in the latter case need to develop your own, or use or pay one.
>>> Of course the code should be included with JMeter.
>>> But it should be a separate tool.
>>>
>>>> This is the major drawback of JMeter I hear from customers and read
>>> around
>>>> the net.
>>>> I first though it would not be that easy but it appears we are able to
>>>> reuse existing code.
>>> I don't follow.
>>>
>>>> If you disagree, I think we should wait for other team members their
>>>> opinion and/or setup a vote for this.
>>> The main JMeter code should be reserved for setting up and running tests.
>>>
>> IMHO reporting must be part of JMeter, and anyway it is already through
>> some Listeners , which although not perfect bring useful infos. The aim of
>> this new listener is:
>> 1/ Allow generation in gui and non gui mode
>> 2/ Have a "sexy" (not yet ) report readable in browser with dynamic
>> behaviour (zoom, select some samples...)
>> 3/ Allow generation at end of load test
>> 4/ Make it easily customizable as FTL will be a property
>> 5/ Make it extensible in the future
>>
>>
>>
>>
>>>>
>>>>
>>>>
>>>>>> @Felix thanks for review of initial patch.
>>>>>> Regards
>>>>>>
>>>>>> On Sunday, February 1, 2015, sebb <sebbaz@gmail.com <javascript:;>>
>>>>> wrote:
>>>>>>> On 31 January 2015 at 13:16, Philippe Mouawad
>>>>>>> <philippe.mouawad@gmail.com <javascript:;> <javascript:;>> wrote:
>>>>>>>> On Sat, Jan 31, 2015 at 12:30 PM, Felix Schumacher <
>>>>>>>> felix.schumacher@internetallee.de <javascript:;> <javascript:;>>
>>>>> wrote:
>>>>>>>>> Am 30.01.2015 um 13:31 schrieb Philippe Mouawad:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> I intend to commit a BackendListener client implementation that
>>>>>>> computes
>>>>>>>>>> APDEX at end of Load Test.
>>>>>>>>>> It will take:
>>>>>>>>>>
>>>>>>>>>> - Acceptable Response Time (http://www.apdex.fr/) T
>>>>>>>>>>
>>>>>>>>> The link was in french (my french is not very good (really non
>>>>> existant
>>>>>>>>> :)), but luckily wikipedia had an article about apdex, which I
>>> could
>>>>>>> read.
>>>>>>>> Sorry for the french link some references:
>>>>>>>> http://www.apdex.org/overview.html
>>>>>>>>
>>>>>>>> - Compute F as 4xT but allow customization
>>>>>>>>>> - List of samples taken into account
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> This listener will compute it during load test and generate an
>>> HTML
>>>>>>> file
>>>>>>>>>> at
>>>>>>>>>> end of test containing it.
>>>>>>>>>>
>>>>>>>>>> This listener could be later enhanced to :
>>>>>>>>>>
>>>>>>>>>> - Generate equivalent of Aggregate Report
>>>>>>>>>> - Possibly graphs based on chartjs or some other graphing JS
>>>>>>> Library
>>>>>>>>> I understand, that apdex is a simple mean to have one metric. As
>>>>> such I
>>>>>>>>> wonder why we would want to graph it.
>>>>>>>> In fact my proposal was larger than just APDEX.
>>>>>>>> What I am proposing is to implement a report generator that at
>>> end of
>>>>>>> Test
>>>>>>>> would compute HTML page with:
>>>>>>>> - APDEX
>>>>>>>> - Some graphs like:
>>>>>>>> - Plot of OK/KO (http://www.chartjs.org/docs/#doughnut-pie-chart
>>> )
>>>>>>>> - Request Summary (what's currently in Aggregate Graph)
>>>>>>>> http://www.chartjs.org/docs/#bar-chart
>>>>>>>> - Response Time over time (
>>> http://www.chartjs.org/docs/#line-chart)
>>>>>>>> I think all this should in fact be computed at end of load test
>>>>> instead
>>>>>>> of
>>>>>>>> during it.
>>>>>>>> The input of this listener would be the generated CSV file, and at
>>>>> end of
>>>>>>>> Test it would read it an compute all this.
>>>>>>> In which case it should be a stand-alone app.
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> If we want to display a graph of apdex values for different
>>> tolerated
>>>>>>>>> times, than we would have to store all values, which would make
>>> the
>>>>>>>>> listener quite heavy on the memory side.
>>>>>>>>>
>>>>>>>>>> Thoughts ?
>>>>>>>>>>
>>>>>>>>> How will errors be counted? Sometimes I can cope with errors, as
>>>>> long as
>>>>>>>>> they are reported fast and sometimes an error would result in a
>>> very
>>>>>>>>> unhappy consumer.
>>>>>>>>>
>>>>>>>> Good catch, I think only successful samples must be taken into
>>>>> account.
>>>>>>>> Error will count only toward total requests (unsatisfied).
>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>> Felix
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Cordialement.
>>>>>>>> Philippe Mouawad.
>>>>>>
>>>>>> --
>>>>>> Cordialement.
>>>>>> Philippe Mouawad.
>>>>
>>>> --
>>>> Cordialement.
>>>> Philippe Mouawad.
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>>
>>
>>
>>
Re: New Listener to generate HTML report at end of load test Was
APDEX Computing and reporting
Posted by Antonio Gomes Rodrigues <ra...@gmail.com>.
Hi,
For your information, APM tools use APDEX (and a little more) in UEM
features
And customers like it (easy to make some great dashboard)
And I think it will be great to have it in JMeter
Antonio
2015-02-03 13:38 GMT+01:00 Philippe Mouawad <ph...@gmail.com>:
> Hi,
> Any additional opinions of dev team or users ?
> I 'd like to know if I should continue efforts on this or stop it.
>
> Thanks
> Regards
>
> On Monday, February 2, 2015, Philippe Mouawad <ph...@gmail.com>
> wrote:
>
> >
> >
> > On Monday, February 2, 2015, sebb <sebbaz@gmail.com
> > <javascript:_e(%7B%7D,'cvml','sebbaz@gmail.com');>> wrote:
> >
> >> On 1 February 2015 at 22:41, Philippe Mouawad
> >> <ph...@gmail.com> wrote:
> >> > On Sunday, February 1, 2015, sebb <se...@gmail.com> wrote:
> >> >
> >> >> On 1 February 2015 at 19:31, Philippe Mouawad
> >> >> <philippe.mouawad@gmail.com <javascript:;>> wrote:
> >> >> > Hi ,
> >> >> > I uploaded a screenshot and new patch code showing what I meant.
> >> >> > I think including it in core should be considered seriously.
> >> >>
> >> >> If the code reads the CSV file at the end of a test to produce the
> >> >> HTML file, there is no need for it to be included in the JMeter code
> >> >> which is used to run tests.
> >> >> It should be a separate application.
> >> >>
> >> >> I don't share your opinion sebb.
> >> > Having a report ready at end of load test is nowadays a standard, look
> >> at
> >> > market tools wether open source or not.
> >> > I don't see the problem here, can you give more details ?
> >>
> >> The problem is that it adds extra code to the JMeter application which
> >> takes extra memory.
> >
> >
> > My last implementation only uses memory at end of test not morz during
> it.
> >
> >
> >> Also, if the user wants to process existing files, it will be easier
> >
> > to have a standalone application rather than having to start the
> >> JMeter test application and then find the reporting tool.
> >
> >
> > It is possible with it, just make a plan with 1 debug sampler and put
> > listener. Report will be generated.
> >
> >>
> >> It would be useful if the tool could be used in batch mode to process
> >> a set of CSV files.
> >
> >
> > Maybe, but I think it's an additional feature. Also having another tool
> > may lead to what happened to the report package, it seems it was never
> > used. But if you implement it it will be ok for me.
> >
> >
> >>
> >> It's also likely to be easier for others to contribute additional
> >> reporting tools if they are part of a separate application.
> >
> > I don't understand this ? how is it easier if it's in jmeter core. If
> it's
> > apart I don't support this option.
> >
> >
> >
> >> It should make the coding requirements simpler.
> >
> > how ?
> >
> >>
> >> > Having another application( by the way do you mean provided as a tool
> in
> >> > jmeter or third party) means in the first case you need to setup
> another
> >> > tool and in the latter case need to develop your own, or use or pay
> one.
> >>
> >> Of course the code should be included with JMeter.
> >> But it should be a separate tool.
> >>
> >> > This is the major drawback of JMeter I hear from customers and read
> >> around
> >> > the net.
> >>
> >> > I first though it would not be that easy but it appears we are able to
> >> > reuse existing code.
> >>
> >> I don't follow.
> >>
> >> > If you disagree, I think we should wait for other team members their
> >> > opinion and/or setup a vote for this.
> >>
> >> The main JMeter code should be reserved for setting up and running
> tests.
> >>
> > IMHO reporting must be part of JMeter, and anyway it is already through
> > some Listeners , which although not perfect bring useful infos. The aim
> of
> > this new listener is:
> > 1/ Allow generation in gui and non gui mode
> > 2/ Have a "sexy" (not yet ) report readable in browser with dynamic
> > behaviour (zoom, select some samples...)
> > 3/ Allow generation at end of load test
> > 4/ Make it easily customizable as FTL will be a property
> > 5/ Make it extensible in the future
> >
> >
> >
> >
> >> >
> >> >
> >> >
> >> >
> >> >> > @Felix thanks for review of initial patch.
> >> >> > Regards
> >> >> >
> >> >> > On Sunday, February 1, 2015, sebb <sebbaz@gmail.com
> <javascript:;>>
> >> >> wrote:
> >> >> >
> >> >> >> On 31 January 2015 at 13:16, Philippe Mouawad
> >> >> >> <philippe.mouawad@gmail.com <javascript:;> <javascript:;>> wrote:
> >> >> >> > On Sat, Jan 31, 2015 at 12:30 PM, Felix Schumacher <
> >> >> >> > felix.schumacher@internetallee.de <javascript:;>
> <javascript:;>>
> >> >> wrote:
> >> >> >> >
> >> >> >> >> Am 30.01.2015 um 13:31 schrieb Philippe Mouawad:
> >> >> >> >>
> >> >> >> >>> Hi,
> >> >> >> >>> I intend to commit a BackendListener client implementation
> that
> >> >> >> computes
> >> >> >> >>> APDEX at end of Load Test.
> >> >> >> >>> It will take:
> >> >> >> >>>
> >> >> >> >>> - Acceptable Response Time (http://www.apdex.fr/) T
> >> >> >> >>>
> >> >> >> >> The link was in french (my french is not very good (really non
> >> >> existant
> >> >> >> >> :)), but luckily wikipedia had an article about apdex, which I
> >> could
> >> >> >> read.
> >> >> >> >>
> >> >> >> >
> >> >> >> > Sorry for the french link some references:
> >> >> >> > http://www.apdex.org/overview.html
> >> >> >> >
> >> >> >> > - Compute F as 4xT but allow customization
> >> >> >> >>> - List of samples taken into account
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>> This listener will compute it during load test and generate an
> >> HTML
> >> >> >> file
> >> >> >> >>> at
> >> >> >> >>> end of test containing it.
> >> >> >> >>>
> >> >> >> >>> This listener could be later enhanced to :
> >> >> >> >>>
> >> >> >> >>> - Generate equivalent of Aggregate Report
> >> >> >> >>> - Possibly graphs based on chartjs or some other graphing
> JS
> >> >> >> Library
> >> >> >> >>>
> >> >> >> >> I understand, that apdex is a simple mean to have one metric.
> As
> >> >> such I
> >> >> >> >> wonder why we would want to graph it.
> >> >> >> >
> >> >> >> > In fact my proposal was larger than just APDEX.
> >> >> >> > What I am proposing is to implement a report generator that at
> >> end of
> >> >> >> Test
> >> >> >> > would compute HTML page with:
> >> >> >> > - APDEX
> >> >> >> > - Some graphs like:
> >> >> >> > - Plot of OK/KO (
> http://www.chartjs.org/docs/#doughnut-pie-chart
> >> )
> >> >> >> > - Request Summary (what's currently in Aggregate Graph)
> >> >> >> > http://www.chartjs.org/docs/#bar-chart
> >> >> >> > - Response Time over time (
> >> http://www.chartjs.org/docs/#line-chart)
> >> >> >> >
> >> >> >> > I think all this should in fact be computed at end of load test
> >> >> instead
> >> >> >> of
> >> >> >> > during it.
> >> >> >> > The input of this listener would be the generated CSV file, and
> at
> >> >> end of
> >> >> >> > Test it would read it an compute all this.
> >> >> >>
> >> >> >> In which case it should be a stand-alone app.
> >> >> >>
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> >> If we want to display a graph of apdex values for different
> >> tolerated
> >> >> >> >> times, than we would have to store all values, which would make
> >> the
> >> >> >> >> listener quite heavy on the memory side.
> >> >> >> >>
> >> >> >> >>>
> >> >> >> >>> Thoughts ?
> >> >> >> >>>
> >> >> >> >> How will errors be counted? Sometimes I can cope with errors,
> as
> >> >> long as
> >> >> >> >> they are reported fast and sometimes an error would result in a
> >> very
> >> >> >> >> unhappy consumer.
> >> >> >> >>
> >> >> >> > Good catch, I think only successful samples must be taken into
> >> >> account.
> >> >> >> > Error will count only toward total requests (unsatisfied).
> >> >> >> >
> >> >> >> >>
> >> >> >> >> Regards
> >> >> >> >> Felix
> >> >> >> >>
> >> >> >> >>
> >> >> >> >
> >> >> >> >
> >> >> >> > --
> >> >> >> > Cordialement.
> >> >> >> > Philippe Mouawad.
> >> >> >>
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Cordialement.
> >> >> > Philippe Mouawad.
> >> >>
> >> >
> >> >
> >> > --
> >> > Cordialement.
> >> > Philippe Mouawad.
> >>
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
> >
> >
> >
> >
>
> --
> Cordialement.
> Philippe Mouawad.
>
Re: New Listener to generate HTML report at end of load test Was
APDEX Computing and reporting
Posted by sebb <se...@gmail.com>.
On 3 February 2015 at 12:38, Philippe Mouawad
<ph...@gmail.com> wrote:
> Hi,
> Any additional opinions of dev team or users ?
> I 'd like to know if I should continue efforts on this or stop it.
I am not saying it should not be included with JMeter.
I just don't think it belongs in the main application, for the reasons
already given.
> Thanks
> Regards
>
> On Monday, February 2, 2015, Philippe Mouawad <ph...@gmail.com>
> wrote:
>
>>
>>
>> On Monday, February 2, 2015, sebb <sebbaz@gmail.com
>> <javascript:_e(%7B%7D,'cvml','sebbaz@gmail.com');>> wrote:
>>
>>> On 1 February 2015 at 22:41, Philippe Mouawad
>>> <ph...@gmail.com> wrote:
>>> > On Sunday, February 1, 2015, sebb <se...@gmail.com> wrote:
>>> >
>>> >> On 1 February 2015 at 19:31, Philippe Mouawad
>>> >> <philippe.mouawad@gmail.com <javascript:;>> wrote:
>>> >> > Hi ,
>>> >> > I uploaded a screenshot and new patch code showing what I meant.
>>> >> > I think including it in core should be considered seriously.
>>> >>
>>> >> If the code reads the CSV file at the end of a test to produce the
>>> >> HTML file, there is no need for it to be included in the JMeter code
>>> >> which is used to run tests.
>>> >> It should be a separate application.
>>> >>
>>> >> I don't share your opinion sebb.
>>> > Having a report ready at end of load test is nowadays a standard, look
>>> at
>>> > market tools wether open source or not.
>>> > I don't see the problem here, can you give more details ?
>>>
>>> The problem is that it adds extra code to the JMeter application which
>>> takes extra memory.
>>
>>
>> My last implementation only uses memory at end of test not morz during it.
>>
>>
>>> Also, if the user wants to process existing files, it will be easier
>>
>> to have a standalone application rather than having to start the
>>> JMeter test application and then find the reporting tool.
>>
>>
>> It is possible with it, just make a plan with 1 debug sampler and put
>> listener. Report will be generated.
>>
>>>
>>> It would be useful if the tool could be used in batch mode to process
>>> a set of CSV files.
>>
>>
>> Maybe, but I think it's an additional feature. Also having another tool
>> may lead to what happened to the report package, it seems it was never
>> used. But if you implement it it will be ok for me.
>>
>>
>>>
>>> It's also likely to be easier for others to contribute additional
>>> reporting tools if they are part of a separate application.
>>
>> I don't understand this ? how is it easier if it's in jmeter core. If it's
>> apart I don't support this option.
>>
>>
>>
>>> It should make the coding requirements simpler.
>>
>> how ?
>>
>>>
>>> > Having another application( by the way do you mean provided as a tool in
>>> > jmeter or third party) means in the first case you need to setup another
>>> > tool and in the latter case need to develop your own, or use or pay one.
>>>
>>> Of course the code should be included with JMeter.
>>> But it should be a separate tool.
>>>
>>> > This is the major drawback of JMeter I hear from customers and read
>>> around
>>> > the net.
>>>
>>> > I first though it would not be that easy but it appears we are able to
>>> > reuse existing code.
>>>
>>> I don't follow.
>>>
>>> > If you disagree, I think we should wait for other team members their
>>> > opinion and/or setup a vote for this.
>>>
>>> The main JMeter code should be reserved for setting up and running tests.
>>>
>> IMHO reporting must be part of JMeter, and anyway it is already through
>> some Listeners , which although not perfect bring useful infos. The aim of
>> this new listener is:
>> 1/ Allow generation in gui and non gui mode
>> 2/ Have a "sexy" (not yet ) report readable in browser with dynamic
>> behaviour (zoom, select some samples...)
>> 3/ Allow generation at end of load test
>> 4/ Make it easily customizable as FTL will be a property
>> 5/ Make it extensible in the future
>>
>>
>>
>>
>>> >
>>> >
>>> >
>>> >
>>> >> > @Felix thanks for review of initial patch.
>>> >> > Regards
>>> >> >
>>> >> > On Sunday, February 1, 2015, sebb <sebbaz@gmail.com <javascript:;>>
>>> >> wrote:
>>> >> >
>>> >> >> On 31 January 2015 at 13:16, Philippe Mouawad
>>> >> >> <philippe.mouawad@gmail.com <javascript:;> <javascript:;>> wrote:
>>> >> >> > On Sat, Jan 31, 2015 at 12:30 PM, Felix Schumacher <
>>> >> >> > felix.schumacher@internetallee.de <javascript:;> <javascript:;>>
>>> >> wrote:
>>> >> >> >
>>> >> >> >> Am 30.01.2015 um 13:31 schrieb Philippe Mouawad:
>>> >> >> >>
>>> >> >> >>> Hi,
>>> >> >> >>> I intend to commit a BackendListener client implementation that
>>> >> >> computes
>>> >> >> >>> APDEX at end of Load Test.
>>> >> >> >>> It will take:
>>> >> >> >>>
>>> >> >> >>> - Acceptable Response Time (http://www.apdex.fr/) T
>>> >> >> >>>
>>> >> >> >> The link was in french (my french is not very good (really non
>>> >> existant
>>> >> >> >> :)), but luckily wikipedia had an article about apdex, which I
>>> could
>>> >> >> read.
>>> >> >> >>
>>> >> >> >
>>> >> >> > Sorry for the french link some references:
>>> >> >> > http://www.apdex.org/overview.html
>>> >> >> >
>>> >> >> > - Compute F as 4xT but allow customization
>>> >> >> >>> - List of samples taken into account
>>> >> >> >>>
>>> >> >> >>>
>>> >> >> >>> This listener will compute it during load test and generate an
>>> HTML
>>> >> >> file
>>> >> >> >>> at
>>> >> >> >>> end of test containing it.
>>> >> >> >>>
>>> >> >> >>> This listener could be later enhanced to :
>>> >> >> >>>
>>> >> >> >>> - Generate equivalent of Aggregate Report
>>> >> >> >>> - Possibly graphs based on chartjs or some other graphing JS
>>> >> >> Library
>>> >> >> >>>
>>> >> >> >> I understand, that apdex is a simple mean to have one metric. As
>>> >> such I
>>> >> >> >> wonder why we would want to graph it.
>>> >> >> >
>>> >> >> > In fact my proposal was larger than just APDEX.
>>> >> >> > What I am proposing is to implement a report generator that at
>>> end of
>>> >> >> Test
>>> >> >> > would compute HTML page with:
>>> >> >> > - APDEX
>>> >> >> > - Some graphs like:
>>> >> >> > - Plot of OK/KO (http://www.chartjs.org/docs/#doughnut-pie-chart
>>> )
>>> >> >> > - Request Summary (what's currently in Aggregate Graph)
>>> >> >> > http://www.chartjs.org/docs/#bar-chart
>>> >> >> > - Response Time over time (
>>> http://www.chartjs.org/docs/#line-chart)
>>> >> >> >
>>> >> >> > I think all this should in fact be computed at end of load test
>>> >> instead
>>> >> >> of
>>> >> >> > during it.
>>> >> >> > The input of this listener would be the generated CSV file, and at
>>> >> end of
>>> >> >> > Test it would read it an compute all this.
>>> >> >>
>>> >> >> In which case it should be a stand-alone app.
>>> >> >>
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >> If we want to display a graph of apdex values for different
>>> tolerated
>>> >> >> >> times, than we would have to store all values, which would make
>>> the
>>> >> >> >> listener quite heavy on the memory side.
>>> >> >> >>
>>> >> >> >>>
>>> >> >> >>> Thoughts ?
>>> >> >> >>>
>>> >> >> >> How will errors be counted? Sometimes I can cope with errors, as
>>> >> long as
>>> >> >> >> they are reported fast and sometimes an error would result in a
>>> very
>>> >> >> >> unhappy consumer.
>>> >> >> >>
>>> >> >> > Good catch, I think only successful samples must be taken into
>>> >> account.
>>> >> >> > Error will count only toward total requests (unsatisfied).
>>> >> >> >
>>> >> >> >>
>>> >> >> >> Regards
>>> >> >> >> Felix
>>> >> >> >>
>>> >> >> >>
>>> >> >> >
>>> >> >> >
>>> >> >> > --
>>> >> >> > Cordialement.
>>> >> >> > Philippe Mouawad.
>>> >> >>
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Cordialement.
>>> >> > Philippe Mouawad.
>>> >>
>>> >
>>> >
>>> > --
>>> > Cordialement.
>>> > Philippe Mouawad.
>>>
>>
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>>
>>
>>
>>
>
> --
> Cordialement.
> Philippe Mouawad.