Re: Issues in JMETER Dashboard Latency Percentile Calculations

[Malith Jayasinghe <ma...@gmail.com>]

A detailed discussion of this issue can be found in here
https://dzone.com/articles/how-to-achieve-better-accuracy-in-latency-percenti

On Wed, Nov 29, 2017 at 5:43 AM, Malith Jayasinghe <
malith.jayasinghe@gmail.com> wrote:

> .Hi Phiilppe,
>
> No, I did not create this issue.   There is a significant increase in the
> accuracy when using "large" window sizes.
>
> Thanks
>
> Malith
>
>
> On Wed, Nov 29, 2017 at 3:53 AM, Philippe Mouawad <
> philippe.mouawad@gmail.com> wrote:
>
>> Hello,
>> Any feedback on this ?
>> Are you the opener of this ?:
>>
>> - https://bz.apache.org/bugzilla/show_bug.cgi?id=61725
>>
>> Thanks
>>
>> On Thu, Nov 23, 2017 at 11:32 PM, Philippe Mouawad <
>> philippe.mouawad@gmail.com> wrote:
>>
>> > Hello,
>> > You can adjust accuracy by setting in user.properties:
>> >
>> >    - jmeter.reportgenerator.statistic_window = 20000
>> >
>> > Caution : higher value provides a better accuracy but needs more memory.
>> >
>> > This is not a bug.
>> >
>> > We use DescriptiveStatistics from commons-math with a sliding window.
>> >
>> > So if after modifying this value you still face issue, please report and
>> > open a bug at commons-math.
>> >
>> > Regards
>> >
>> >
>> > On Thu, Nov 23, 2017 at 5:04 AM, Malith Jayasinghe <
>> > malith.jayasinghe@gmail.com> wrote:
>> >
>> >> Dear All,
>> >>
>> >> We have been using JMETER extensively for performance testing.
>> >>
>> >> All our performance reports are generated using the JMETER Dashboard.
>> >>
>> >> Recently, we have found out that there is an issue in
>> >> the JMETER dashboard's *latency percentile calculation. *
>> >>
>> >> There is a significant difference in the actual percentile values and
>> the
>> >> values shown in the dashboard. We have verified this using "R" and in
>> >> fact,
>> >> the values shown the JMETER aggregate report are correct. The values
>> that
>> >> appear in the JMETER Dashboard are not.
>> >>
>> >> We understand that JMETER Dashboard may be using a different algorithm
>> >> (approximation) for percentile calculation. However, the problem is
>> that
>> >> that latency percentiles values calculated using this algorithm are not
>> >> accurate (at least for certain scenarios).
>> >>
>> >> We have had a case where the Dashboard showing a 95% percentile value
>> 1000
>> >> ms. However, the correct value was 321 ms.
>> >>
>> >> The accuracy of percentile values are of utmost importance to us and we
>> >> would kindly ask you to look into this issue and release a patch if
>> >> possible.
>> >>
>> >> If you can implement the same algorithm used in the aggregate report
>> >> within
>> >> the dashboard that will resolve this issue.
>> >>
>> >>
>> >> Thanks
>> >>
>> >> Malith Jayasinghe
>> >>
>> >
>> >
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>> >
>> >
>> >
>>
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>>
>
>

Re: Issues in JMETER Dashboard Latency Percentile Calculations

[Philippe Mouawad <ph...@gmail.com>]

Hello,
What about contributing a documentation PR based on your blog in this doc:

- http://jmeter.apache.org/usermanual/generating-dashboard.html

Which is generated from :

xdocs/usermanual/generating-dashboard.xml


It would be nice.


Thanks

On Monday, December 11, 2017, Malith Jayasinghe <ma...@gmail.com>
wrote:

> A detailed discussion of this issue can be found in here
> https://dzone.com/articles/how-to-achieve-better-accuracy-in-latency-
> percenti
>
> On Wed, Nov 29, 2017 at 5:43 AM, Malith Jayasinghe <
> malith.jayasinghe@gmail.com> wrote:
>
>> .Hi Phiilppe,
>>
>> No, I did not create this issue.   There is a significant increase in
>> the accuracy when using "large" window sizes.
>>
>> Thanks
>>
>> Malith
>>
>>
>> On Wed, Nov 29, 2017 at 3:53 AM, Philippe Mouawad <
>> philippe.mouawad@gmail.com> wrote:
>>
>>> Hello,
>>> Any feedback on this ?
>>> Are you the opener of this ?:
>>>
>>> - https://bz.apache.org/bugzilla/show_bug.cgi?id=61725
>>>
>>> Thanks
>>>
>>> On Thu, Nov 23, 2017 at 11:32 PM, Philippe Mouawad <
>>> philippe.mouawad@gmail.com> wrote:
>>>
>>> > Hello,
>>> > You can adjust accuracy by setting in user.properties:
>>> >
>>> >    - jmeter.reportgenerator.statistic_window = 20000
>>> >
>>> > Caution : higher value provides a better accuracy but needs more
>>> memory.
>>> >
>>> > This is not a bug.
>>> >
>>> > We use DescriptiveStatistics from commons-math with a sliding window.
>>> >
>>> > So if after modifying this value you still face issue, please report
>>> and
>>> > open a bug at commons-math.
>>> >
>>> > Regards
>>> >
>>> >
>>> > On Thu, Nov 23, 2017 at 5:04 AM, Malith Jayasinghe <
>>> > malith.jayasinghe@gmail.com> wrote:
>>> >
>>> >> Dear All,
>>> >>
>>> >> We have been using JMETER extensively for performance testing.
>>> >>
>>> >> All our performance reports are generated using the JMETER Dashboard.
>>> >>
>>> >> Recently, we have found out that there is an issue in
>>> >> the JMETER dashboard's *latency percentile calculation. *
>>> >>
>>> >> There is a significant difference in the actual percentile values and
>>> the
>>> >> values shown in the dashboard. We have verified this using "R" and in
>>> >> fact,
>>> >> the values shown the JMETER aggregate report are correct. The values
>>> that
>>> >> appear in the JMETER Dashboard are not.
>>> >>
>>> >> We understand that JMETER Dashboard may be using a different algorithm
>>> >> (approximation) for percentile calculation. However, the problem is
>>> that
>>> >> that latency percentiles values calculated using this algorithm are
>>> not
>>> >> accurate (at least for certain scenarios).
>>> >>
>>> >> We have had a case where the Dashboard showing a 95% percentile value
>>> 1000
>>> >> ms. However, the correct value was 321 ms.
>>> >>
>>> >> The accuracy of percentile values are of utmost importance to us and
>>> we
>>> >> would kindly ask you to look into this issue and release a patch if
>>> >> possible.
>>> >>
>>> >> If you can implement the same algorithm used in the aggregate report
>>> >> within
>>> >> the dashboard that will resolve this issue.
>>> >>
>>> >>
>>> >> Thanks
>>> >>
>>> >> Malith Jayasinghe
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > Cordialement.
>>> > Philippe Mouawad.
>>> >
>>> >
>>> >
>>>
>>>
>>> --
>>> Cordialement.
>>> Philippe Mouawad.
>>>
>>
>>
>

-- 
Cordialement.
Philippe Mouawad.