You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Mikhail Gryzykhin <mi...@google.com> on 2018/09/13 18:11:56 UTC
Beam dashboards
Hi everyone,
Huygaa and me are working on creating dashboards for Beam.
So far I created a POC dashboard for post-commit greenness
<http://104.154.241.245/d/D81lW0pmk/post-commit-tests> that includes a list
of latest failed jobs and dashboard for pre-commits
<http://104.154.241.245/d/_TNndF2iz/pre-commit-tests> duration, that one
contains only duration so far.
We have work ongoing for adding github statistics.
Please, check those out. Any feedback is welcome.
Small hints:
* You can change time range you're looking at in to-right corner.
* You can choose dashboard by clicking dashboard name on top-left corner.
As a short insight:
Thank you everyone who helps fixing post-commit tests flakes. We moved from
~30% to 75-85% successful runs.
Best regards,
--Mikhail
Have feedback <http://go/migryz-feedback>?
Re: Beam dashboards
Posted by Łukasz Gajowy <lg...@apache.org>.
Thanks for the hint Mikhail! I'll take a look at that. :)
Best,
Łukasz
śr., 22 maj 2019 o 00:19 Mikhail Gryzykhin <mi...@google.com> napisał(a):
> @Łukasz Gajowy <lu...@gmail.com>
>
> Reviving old thread:
> Grafana doesn't support BigQuery officially, but recently there were news
> with unofficial BQ plugin:
> *
> https://blog.doit-intl.com/power-grafana-with-google-bigquery-6822443a7f99
> * https://github.com/doitintl/bigquery-grafana
>
> I didn't try it yet, but is looks promising.
>
> --Mikhail
>
> On Tue, Sep 25, 2018 at 4:37 AM Łukasz Gajowy <lu...@gmail.com>
> wrote:
>
>> Nice! The Grafana dashboards look great!
>>
>> Side question: do you think Grafana can be used to present results stored
>> in BigQuery? Maybe in the future, we could use Grafana to show every
>> testing dashboards (IO tests, nexmark, currently developed load tests) with
>> this tool leveraging its more advanced features such as dynamic data
>> ranges, alerts, ad-hoc filters etc[1]?
>>
>> [1] https://grafana.com/grafana
>>
>> pon., 17 wrz 2018 o 19:14 Mikhail Gryzykhin <mi...@google.com>
>> napisał(a):
>>
>>> Thank you for feedback.
>>>
>>> I had the idea of adding the list of tests with duration. Unfortunately,
>>> I'm not clear on how to represent such metric. One of the ideas I had is to
>>> show graph with top 10 slowest tests for each job run. However we have too
>>> many different test jobs. I'm open to brainstorming ideas in this area.
>>>
>>> Meanwhile I'll play around with different options and see what I
>>> can come up with.
>>>
>>> Regards,
>>> --Mikhail
>>>
>>> Have feedback <http://go/migryz-feedback>?
>>>
>>>
>>> On Sun, Sep 16, 2018 at 4:04 AM Maximilian Michels <mx...@apache.org>
>>> wrote:
>>>
>>>> Thanks Mikhail, that will help to identify flaky or slow tests. At the
>>>> size of the Beam code base, such statistics are extremely helpful.
>>>>
>>>> If we had a list of test cases ordered by test duration, that would be
>>>> a
>>>> great addition.
>>>>
>>>> On 14.09.18 00:30, Connell O'Callaghan wrote:
>>>> > Thank you Mikhail for sharing this and to everyone involved in these
>>>> > improvements!!! It will be great to hear about progress and any
>>>> blockers
>>>> > encountered with this work.
>>>> >
>>>> > On Thu, Sep 13, 2018 at 11:12 AM Mikhail Gryzykhin <migryz@google.com
>>>> > <ma...@google.com>> wrote:
>>>> >
>>>> > Hi everyone,
>>>> >
>>>> > Huygaa and me are working on creating dashboards for Beam.
>>>> >
>>>> > So far I created a POC dashboard for post-commit greenness
>>>> > <http://104.154.241.245/d/D81lW0pmk/post-commit-tests> that
>>>> includes
>>>> > a list of latest failed jobs and dashboard for pre-commits
>>>> > <http://104.154.241.245/d/_TNndF2iz/pre-commit-tests> duration,
>>>> that
>>>> > one contains only duration so far.
>>>> >
>>>> > We have work ongoing for adding github statistics.
>>>> >
>>>> > Please, check those out. Any feedback is welcome.
>>>> >
>>>> > Small hints:
>>>> > * You can change time range you're looking at in to-right corner.
>>>> > * You can choose dashboard by clicking dashboard name on top-left
>>>> > corner.
>>>> >
>>>> > As a short insight:
>>>> > Thank you everyone who helps fixing post-commit tests flakes. We
>>>> > moved from ~30% to 75-85% successful runs.
>>>> >
>>>> > Best regards,
>>>> > --Mikhail
>>>> >
>>>> > Have feedback <http://go/migryz-feedback>?
>>>> >
>>>>
>>>
Re: Beam dashboards
Posted by Mikhail Gryzykhin <mi...@google.com>.
@Łukasz Gajowy <lu...@gmail.com>
Reviving old thread:
Grafana doesn't support BigQuery officially, but recently there were news
with unofficial BQ plugin:
* https://blog.doit-intl.com/power-grafana-with-google-bigquery-6822443a7f99
* https://github.com/doitintl/bigquery-grafana
I didn't try it yet, but is looks promising.
--Mikhail
On Tue, Sep 25, 2018 at 4:37 AM Łukasz Gajowy <lu...@gmail.com>
wrote:
> Nice! The Grafana dashboards look great!
>
> Side question: do you think Grafana can be used to present results stored
> in BigQuery? Maybe in the future, we could use Grafana to show every
> testing dashboards (IO tests, nexmark, currently developed load tests) with
> this tool leveraging its more advanced features such as dynamic data
> ranges, alerts, ad-hoc filters etc[1]?
>
> [1] https://grafana.com/grafana
>
> pon., 17 wrz 2018 o 19:14 Mikhail Gryzykhin <mi...@google.com>
> napisał(a):
>
>> Thank you for feedback.
>>
>> I had the idea of adding the list of tests with duration. Unfortunately,
>> I'm not clear on how to represent such metric. One of the ideas I had is to
>> show graph with top 10 slowest tests for each job run. However we have too
>> many different test jobs. I'm open to brainstorming ideas in this area.
>>
>> Meanwhile I'll play around with different options and see what I can come
>> up with.
>>
>> Regards,
>> --Mikhail
>>
>> Have feedback <http://go/migryz-feedback>?
>>
>>
>> On Sun, Sep 16, 2018 at 4:04 AM Maximilian Michels <mx...@apache.org>
>> wrote:
>>
>>> Thanks Mikhail, that will help to identify flaky or slow tests. At the
>>> size of the Beam code base, such statistics are extremely helpful.
>>>
>>> If we had a list of test cases ordered by test duration, that would be a
>>> great addition.
>>>
>>> On 14.09.18 00:30, Connell O'Callaghan wrote:
>>> > Thank you Mikhail for sharing this and to everyone involved in these
>>> > improvements!!! It will be great to hear about progress and any
>>> blockers
>>> > encountered with this work.
>>> >
>>> > On Thu, Sep 13, 2018 at 11:12 AM Mikhail Gryzykhin <migryz@google.com
>>> > <ma...@google.com>> wrote:
>>> >
>>> > Hi everyone,
>>> >
>>> > Huygaa and me are working on creating dashboards for Beam.
>>> >
>>> > So far I created a POC dashboard for post-commit greenness
>>> > <http://104.154.241.245/d/D81lW0pmk/post-commit-tests> that
>>> includes
>>> > a list of latest failed jobs and dashboard for pre-commits
>>> > <http://104.154.241.245/d/_TNndF2iz/pre-commit-tests> duration,
>>> that
>>> > one contains only duration so far.
>>> >
>>> > We have work ongoing for adding github statistics.
>>> >
>>> > Please, check those out. Any feedback is welcome.
>>> >
>>> > Small hints:
>>> > * You can change time range you're looking at in to-right corner.
>>> > * You can choose dashboard by clicking dashboard name on top-left
>>> > corner.
>>> >
>>> > As a short insight:
>>> > Thank you everyone who helps fixing post-commit tests flakes. We
>>> > moved from ~30% to 75-85% successful runs.
>>> >
>>> > Best regards,
>>> > --Mikhail
>>> >
>>> > Have feedback <http://go/migryz-feedback>?
>>> >
>>>
>>
Re: Beam dashboards
Posted by Łukasz Gajowy <lu...@gmail.com>.
Nice! The Grafana dashboards look great!
Side question: do you think Grafana can be used to present results stored
in BigQuery? Maybe in the future, we could use Grafana to show every
testing dashboards (IO tests, nexmark, currently developed load tests) with
this tool leveraging its more advanced features such as dynamic data
ranges, alerts, ad-hoc filters etc[1]?
[1] https://grafana.com/grafana
pon., 17 wrz 2018 o 19:14 Mikhail Gryzykhin <mi...@google.com> napisał(a):
> Thank you for feedback.
>
> I had the idea of adding the list of tests with duration. Unfortunately,
> I'm not clear on how to represent such metric. One of the ideas I had is to
> show graph with top 10 slowest tests for each job run. However we have too
> many different test jobs. I'm open to brainstorming ideas in this area.
>
> Meanwhile I'll play around with different options and see what I can come
> up with.
>
> Regards,
> --Mikhail
>
> Have feedback <http://go/migryz-feedback>?
>
>
> On Sun, Sep 16, 2018 at 4:04 AM Maximilian Michels <mx...@apache.org> wrote:
>
>> Thanks Mikhail, that will help to identify flaky or slow tests. At the
>> size of the Beam code base, such statistics are extremely helpful.
>>
>> If we had a list of test cases ordered by test duration, that would be a
>> great addition.
>>
>> On 14.09.18 00:30, Connell O'Callaghan wrote:
>> > Thank you Mikhail for sharing this and to everyone involved in these
>> > improvements!!! It will be great to hear about progress and any
>> blockers
>> > encountered with this work.
>> >
>> > On Thu, Sep 13, 2018 at 11:12 AM Mikhail Gryzykhin <migryz@google.com
>> > <ma...@google.com>> wrote:
>> >
>> > Hi everyone,
>> >
>> > Huygaa and me are working on creating dashboards for Beam.
>> >
>> > So far I created a POC dashboard for post-commit greenness
>> > <http://104.154.241.245/d/D81lW0pmk/post-commit-tests> that
>> includes
>> > a list of latest failed jobs and dashboard for pre-commits
>> > <http://104.154.241.245/d/_TNndF2iz/pre-commit-tests> duration,
>> that
>> > one contains only duration so far.
>> >
>> > We have work ongoing for adding github statistics.
>> >
>> > Please, check those out. Any feedback is welcome.
>> >
>> > Small hints:
>> > * You can change time range you're looking at in to-right corner.
>> > * You can choose dashboard by clicking dashboard name on top-left
>> > corner.
>> >
>> > As a short insight:
>> > Thank you everyone who helps fixing post-commit tests flakes. We
>> > moved from ~30% to 75-85% successful runs.
>> >
>> > Best regards,
>> > --Mikhail
>> >
>> > Have feedback <http://go/migryz-feedback>?
>> >
>>
>
Re: Beam dashboards
Posted by Mikhail Gryzykhin <mi...@google.com>.
Thank you for feedback.
I had the idea of adding the list of tests with duration. Unfortunately,
I'm not clear on how to represent such metric. One of the ideas I had is to
show graph with top 10 slowest tests for each job run. However we have too
many different test jobs. I'm open to brainstorming ideas in this area.
Meanwhile I'll play around with different options and see what I can come
up with.
Regards,
--Mikhail
Have feedback <http://go/migryz-feedback>?
On Sun, Sep 16, 2018 at 4:04 AM Maximilian Michels <mx...@apache.org> wrote:
> Thanks Mikhail, that will help to identify flaky or slow tests. At the
> size of the Beam code base, such statistics are extremely helpful.
>
> If we had a list of test cases ordered by test duration, that would be a
> great addition.
>
> On 14.09.18 00:30, Connell O'Callaghan wrote:
> > Thank you Mikhail for sharing this and to everyone involved in these
> > improvements!!! It will be great to hear about progress and any blockers
> > encountered with this work.
> >
> > On Thu, Sep 13, 2018 at 11:12 AM Mikhail Gryzykhin <migryz@google.com
> > <ma...@google.com>> wrote:
> >
> > Hi everyone,
> >
> > Huygaa and me are working on creating dashboards for Beam.
> >
> > So far I created a POC dashboard for post-commit greenness
> > <http://104.154.241.245/d/D81lW0pmk/post-commit-tests> that includes
> > a list of latest failed jobs and dashboard for pre-commits
> > <http://104.154.241.245/d/_TNndF2iz/pre-commit-tests> duration, that
> > one contains only duration so far.
> >
> > We have work ongoing for adding github statistics.
> >
> > Please, check those out. Any feedback is welcome.
> >
> > Small hints:
> > * You can change time range you're looking at in to-right corner.
> > * You can choose dashboard by clicking dashboard name on top-left
> > corner.
> >
> > As a short insight:
> > Thank you everyone who helps fixing post-commit tests flakes. We
> > moved from ~30% to 75-85% successful runs.
> >
> > Best regards,
> > --Mikhail
> >
> > Have feedback <http://go/migryz-feedback>?
> >
>
Re: Beam dashboards
Posted by Maximilian Michels <mx...@apache.org>.
Thanks Mikhail, that will help to identify flaky or slow tests. At the
size of the Beam code base, such statistics are extremely helpful.
If we had a list of test cases ordered by test duration, that would be a
great addition.
On 14.09.18 00:30, Connell O'Callaghan wrote:
> Thank you Mikhail for sharing this and to everyone involved in these
> improvements!!! It will be great to hear about progress and any blockers
> encountered with this work.
>
> On Thu, Sep 13, 2018 at 11:12 AM Mikhail Gryzykhin <migryz@google.com
> <ma...@google.com>> wrote:
>
> Hi everyone,
>
> Huygaa and me are working on creating dashboards for Beam.
>
> So far I created a POC dashboard for post-commit greenness
> <http://104.154.241.245/d/D81lW0pmk/post-commit-tests> that includes
> a list of latest failed jobs and dashboard for pre-commits
> <http://104.154.241.245/d/_TNndF2iz/pre-commit-tests> duration, that
> one contains only duration so far.
>
> We have work ongoing for adding github statistics.
>
> Please, check those out. Any feedback is welcome.
>
> Small hints:
> * You can change time range you're looking at in to-right corner.
> * You can choose dashboard by clicking dashboard name on top-left
> corner.
>
> As a short insight:
> Thank you everyone who helps fixing post-commit tests flakes. We
> moved from ~30% to 75-85% successful runs.
>
> Best regards,
> --Mikhail
>
> Have feedback <http://go/migryz-feedback>?
>
Re: Beam dashboards
Posted by Connell O'Callaghan <co...@google.com>.
Thank you Mikhail for sharing this and to everyone involved in these
improvements!!! It will be great to hear about progress and any blockers
encountered with this work.
On Thu, Sep 13, 2018 at 11:12 AM Mikhail Gryzykhin <mi...@google.com>
wrote:
> Hi everyone,
>
> Huygaa and me are working on creating dashboards for Beam.
>
> So far I created a POC dashboard for post-commit greenness
> <http://104.154.241.245/d/D81lW0pmk/post-commit-tests> that includes a
> list of latest failed jobs and dashboard for pre-commits
> <http://104.154.241.245/d/_TNndF2iz/pre-commit-tests> duration, that one
> contains only duration so far.
>
> We have work ongoing for adding github statistics.
>
> Please, check those out. Any feedback is welcome.
>
> Small hints:
> * You can change time range you're looking at in to-right corner.
> * You can choose dashboard by clicking dashboard name on top-left corner.
>
> As a short insight:
> Thank you everyone who helps fixing post-commit tests flakes. We moved
> from ~30% to 75-85% successful runs.
>
> Best regards,
> --Mikhail
>
> Have feedback <http://go/migryz-feedback>?
>