You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Kamil Wasilewski <ka...@polidea.com> on 2019/08/23 10:22:25 UTC
[PROPOSAL] Storing, displaying and detecting anomalies in test results
Hi all,
Recently we did some research on how to visualize IO performance tests,
Nexmark and Load test results better and how to detect regressions
automatically in an easy way using tools dedicated for the job.
We'd like to share a proposal with you:
<https://docs.google.com/document/d/1dtHzcp2K3rZihEgNSjFmwN9CF6NbspWhFRaDBdEVNyo/edit?usp=sharing>
https://s.apache.org/test-metrics-storage
Any comments are highly appreciated.
Thanks,
Kamil
Re: [PROPOSAL] Storing, displaying and detecting anomalies in test results
Posted by Kamil Wasilewski <ka...@polidea.com>.
Thanks for spotting this! It should be working fine now.
On Thu, Nov 7, 2019 at 5:40 PM Dan Gazineu <ga...@google.com> wrote:
> Thank you for the update Kamil! Please fix the sharing options in the new
> doc.
>
> On Thu, Nov 7, 2019 at 7:22 AM Kamil Wasilewski <
> kamil.wasilewski@polidea.com> wrote:
>
>> Hi all,
>>
>> For a while we have been working on the implementation of proposal
>> presented in this thread. Just to remind what it was about in short words —
>> we wanted to use Prometheus and Grafana to visualize Beam performance tests
>> results in addition to detect regressions automatically. A couple of
>> obstacles forced us to change our approach and that's why I want to share
>> with you a major proposal revision which you can find here:
>> https://s.apache.org/test-metrics-storage-corrected.
>>
>> As usual, we are open to your suggestions and thoughts.
>>
>> Thanks,
>> Kamil
>>
>> On Thu, Sep 5, 2019 at 5:11 PM Kamil Wasilewski <
>> kamil.wasilewski@polidea.com> wrote:
>>
>>> We've just started working on this and here comes the first PR:
>>> https://github.com/apache/beam/pull/9482.
>>>
>>> Feel free to share your thoughts and ideas.
>>>
>>> Kamil
>>>
>>> On Tue, Aug 27, 2019 at 1:19 AM Pablo Estrada <pa...@google.com>
>>> wrote:
>>>
>>>> Thanks Kamil for bringing this!
>>>> +Manisha Bhardwaj <ma...@google.com> +Mark Liu <ma...@google.com> have
>>>> worked on internal benchmarking at Google - would you take a look please?
>>>>
>>>> On Fri, Aug 23, 2019 at 3:22 AM Kamil Wasilewski <
>>>> kamil.wasilewski@polidea.com> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> Recently we did some research on how to visualize IO performance
>>>>> tests, Nexmark and Load test results better and how to detect regressions
>>>>> automatically in an easy way using tools dedicated for the job.
>>>>>
>>>>> We'd like to share a proposal with you:
>>>>> <https://docs.google.com/document/d/1dtHzcp2K3rZihEgNSjFmwN9CF6NbspWhFRaDBdEVNyo/edit?usp=sharing>
>>>>> https://s.apache.org/test-metrics-storage
>>>>>
>>>>>
>>>>>
>>>>> Any comments are highly appreciated.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Kamil
>>>>>
>>>>
Re: [PROPOSAL] Storing, displaying and detecting anomalies in test results
Posted by Dan Gazineu <ga...@google.com>.
Thank you for the update Kamil! Please fix the sharing options in the new
doc.
On Thu, Nov 7, 2019 at 7:22 AM Kamil Wasilewski <
kamil.wasilewski@polidea.com> wrote:
> Hi all,
>
> For a while we have been working on the implementation of proposal
> presented in this thread. Just to remind what it was about in short words —
> we wanted to use Prometheus and Grafana to visualize Beam performance tests
> results in addition to detect regressions automatically. A couple of
> obstacles forced us to change our approach and that's why I want to share
> with you a major proposal revision which you can find here:
> https://s.apache.org/test-metrics-storage-corrected.
>
> As usual, we are open to your suggestions and thoughts.
>
> Thanks,
> Kamil
>
> On Thu, Sep 5, 2019 at 5:11 PM Kamil Wasilewski <
> kamil.wasilewski@polidea.com> wrote:
>
>> We've just started working on this and here comes the first PR:
>> https://github.com/apache/beam/pull/9482.
>>
>> Feel free to share your thoughts and ideas.
>>
>> Kamil
>>
>> On Tue, Aug 27, 2019 at 1:19 AM Pablo Estrada <pa...@google.com> wrote:
>>
>>> Thanks Kamil for bringing this!
>>> +Manisha Bhardwaj <ma...@google.com> +Mark Liu <ma...@google.com> have
>>> worked on internal benchmarking at Google - would you take a look please?
>>>
>>> On Fri, Aug 23, 2019 at 3:22 AM Kamil Wasilewski <
>>> kamil.wasilewski@polidea.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Recently we did some research on how to visualize IO performance tests,
>>>> Nexmark and Load test results better and how to detect regressions
>>>> automatically in an easy way using tools dedicated for the job.
>>>>
>>>> We'd like to share a proposal with you:
>>>> <https://docs.google.com/document/d/1dtHzcp2K3rZihEgNSjFmwN9CF6NbspWhFRaDBdEVNyo/edit?usp=sharing>
>>>> https://s.apache.org/test-metrics-storage
>>>>
>>>>
>>>>
>>>> Any comments are highly appreciated.
>>>>
>>>> Thanks,
>>>>
>>>> Kamil
>>>>
>>>
Re: [PROPOSAL] Storing, displaying and detecting anomalies in test results
Posted by Kamil Wasilewski <ka...@polidea.com>.
Hi all,
For a while we have been working on the implementation of proposal
presented in this thread. Just to remind what it was about in short words —
we wanted to use Prometheus and Grafana to visualize Beam performance tests
results in addition to detect regressions automatically. A couple of
obstacles forced us to change our approach and that's why I want to share
with you a major proposal revision which you can find here:
https://s.apache.org/test-metrics-storage-corrected.
As usual, we are open to your suggestions and thoughts.
Thanks,
Kamil
On Thu, Sep 5, 2019 at 5:11 PM Kamil Wasilewski <
kamil.wasilewski@polidea.com> wrote:
> We've just started working on this and here comes the first PR:
> https://github.com/apache/beam/pull/9482.
>
> Feel free to share your thoughts and ideas.
>
> Kamil
>
> On Tue, Aug 27, 2019 at 1:19 AM Pablo Estrada <pa...@google.com> wrote:
>
>> Thanks Kamil for bringing this!
>> +Manisha Bhardwaj <ma...@google.com> +Mark Liu <ma...@google.com> have
>> worked on internal benchmarking at Google - would you take a look please?
>>
>> On Fri, Aug 23, 2019 at 3:22 AM Kamil Wasilewski <
>> kamil.wasilewski@polidea.com> wrote:
>>
>>> Hi all,
>>>
>>> Recently we did some research on how to visualize IO performance tests,
>>> Nexmark and Load test results better and how to detect regressions
>>> automatically in an easy way using tools dedicated for the job.
>>>
>>> We'd like to share a proposal with you:
>>> <https://docs.google.com/document/d/1dtHzcp2K3rZihEgNSjFmwN9CF6NbspWhFRaDBdEVNyo/edit?usp=sharing>
>>> https://s.apache.org/test-metrics-storage
>>>
>>>
>>>
>>> Any comments are highly appreciated.
>>>
>>> Thanks,
>>>
>>> Kamil
>>>
>>
Re: [PROPOSAL] Storing, displaying and detecting anomalies in test results
Posted by Kamil Wasilewski <ka...@polidea.com>.
We've just started working on this and here comes the first PR:
https://github.com/apache/beam/pull/9482.
Feel free to share your thoughts and ideas.
Kamil
On Tue, Aug 27, 2019 at 1:19 AM Pablo Estrada <pa...@google.com> wrote:
> Thanks Kamil for bringing this!
> +Manisha Bhardwaj <ma...@google.com> +Mark Liu <ma...@google.com> have
> worked on internal benchmarking at Google - would you take a look please?
>
> On Fri, Aug 23, 2019 at 3:22 AM Kamil Wasilewski <
> kamil.wasilewski@polidea.com> wrote:
>
>> Hi all,
>>
>> Recently we did some research on how to visualize IO performance tests,
>> Nexmark and Load test results better and how to detect regressions
>> automatically in an easy way using tools dedicated for the job.
>>
>> We'd like to share a proposal with you:
>> <https://docs.google.com/document/d/1dtHzcp2K3rZihEgNSjFmwN9CF6NbspWhFRaDBdEVNyo/edit?usp=sharing>
>> https://s.apache.org/test-metrics-storage
>>
>>
>>
>> Any comments are highly appreciated.
>>
>> Thanks,
>>
>> Kamil
>>
>
Re: [PROPOSAL] Storing, displaying and detecting anomalies in test results
Posted by Pablo Estrada <pa...@google.com>.
Thanks Kamil for bringing this!
+Manisha Bhardwaj <ma...@google.com> +Mark Liu <ma...@google.com> have
worked on internal benchmarking at Google - would you take a look please?
On Fri, Aug 23, 2019 at 3:22 AM Kamil Wasilewski <
kamil.wasilewski@polidea.com> wrote:
> Hi all,
>
> Recently we did some research on how to visualize IO performance tests,
> Nexmark and Load test results better and how to detect regressions
> automatically in an easy way using tools dedicated for the job.
>
> We'd like to share a proposal with you:
> <https://docs.google.com/document/d/1dtHzcp2K3rZihEgNSjFmwN9CF6NbspWhFRaDBdEVNyo/edit?usp=sharing>
> https://s.apache.org/test-metrics-storage
>
>
>
> Any comments are highly appreciated.
>
> Thanks,
>
> Kamil
>