You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Huygaa Batsaikhan <ba...@google.com> on 2018/08/06 22:43:50 UTC
[DISCUSSION] Tracking & Visualizing various metrics of the Beam community
Continuing the discussion
<https://lists.apache.org/thread.html/6138d08c551e254b5f13b26c6ba06579a49a4694f4d13ad6d164689a@%3Cdev.beam.apache.org%3E>
about improving Beam code review, I am looking into visualizing various
helpful Beam community metrics such as code velocity, reviewer load, and
new contributor's engagement.
So far, I found DevStats
<https://k8s.devstats.cncf.io/d/12/dashboards?refresh=15m&orgId=1>, an open
source (github <https://github.com/cncf/devstats>) dashboarding tool used
by Kubernetes, seems to provide almost everything we need. For example,
they have dashboards for metrics such as:
-
Time to approve or merge
<https://k8s.devstats.cncf.io/d/44/pr-time-to-approve-and-merge?orgId=1>
-
PR Time to engagement
<https://k8s.devstats.cncf.io/d/14/pr-time-to-engagment?orgId=1>
-
New and Episodic PR contributors
<https://prometheus.devstats.cncf.io/d/14/new-and-episodic-pr-contributors?orgId=1>
-
PR reviews by contributor
<https://k8s.devstats.cncf.io/d/46/pr-reviews-by-contributor?orgId=1>
-
Company statistics
It would be really cool if we can try it out for Beam. I don't have much
experience using open source projects. From what I understand: DevStats is
developed by CNCF <https://www.cncf.io/> and they manage their incubator
projects' dashboard. Since Beam is not part of the CNCF, in order to use
DevStats, we have to fork the project and maintain it ourselves.
1. What do you think about using DevStats for Beam? Do you know how it is
usually done?
2. If you are not sure about DevStats, do you know any other tool which
could help us track & visualize Beam metrics?
Thanks, Huygaa
Re: [DISCUSSION] Tracking & Visualizing various metrics of the Beam community
Posted by Mikhail Gryzykhin <mi...@google.com>.
Hi,
The UI looks cool and powerful. I'm not sure about using the devstats
service, since it only has github statistics as I can understand. I would
prefer to have single entry point for all of our dashboards: build stats,
eng metrics, tests, project health metrics, etc.
However, devstats uses grafana underneath. If we can utilize grafana for UI
and host all of our dashboards there, it would be great. If we can extend
devstats to show our project-specific metrics as well, that would be good.
If not, we can look into hosting grafana or utilizing their cloud solution
if they provide support for opensource.
Alternatively, we can look into utilizing Kibana, but I'm not clear if it
is best option for visualizing metrics. Grafana seem to be more oriented
towards our usecase.
--Mikhail
Have feedback <http://go/migryz-feedback>?
On Tue, Aug 7, 2018 at 1:00 PM Huygaa Batsaikhan <ba...@google.com> wrote:
> Thanks Udi, I am working on spinning up a test instance.
>
> On Tue, Aug 7, 2018 at 11:31 AM Udi Meiri <eh...@google.com> wrote:
>
>> These tables look very cool!
>>
>> I certainly don't object to using tools made by another organization. The
>> only issue might be compatibility with our process.
>>
>> From my understanding of the Kubernetes review process, they use Github's
>> tagging feature to specify PR statuses such as LGTM and approval.
>> Random example: https://github.com/kubernetes/kubernetes/pull/67089
>> I'm not sure that statistics such as pr-time-to-approve-and-merge would
>> be captured at all on apache/beam since we don't use these tags.
>>
>> I would definitely like to see how useful this tool is regardless, and
>> since it doesn't seem to require any special permissions it should be
>> simple to set up a test instance.
>>
>>
>> On Tue, Aug 7, 2018 at 11:24 AM Huygaa Batsaikhan <ba...@google.com>
>> wrote:
>>
>>> tl;dr - is there any objections to trying out DevStats tool created by
>>> CNCF?
>>>
>>> On Mon, Aug 6, 2018 at 3:43 PM Huygaa Batsaikhan <ba...@google.com>
>>> wrote:
>>>
>>>> Continuing the discussion
>>>> <https://lists.apache.org/thread.html/6138d08c551e254b5f13b26c6ba06579a49a4694f4d13ad6d164689a@%3Cdev.beam.apache.org%3E>
>>>> about improving Beam code review, I am looking into visualizing various
>>>> helpful Beam community metrics such as code velocity, reviewer load, and
>>>> new contributor's engagement.
>>>>
>>>> So far, I found DevStats
>>>> <https://k8s.devstats.cncf.io/d/12/dashboards?refresh=15m&orgId=1>, an
>>>> open source (github <https://github.com/cncf/devstats>) dashboarding
>>>> tool used by Kubernetes, seems to provide almost everything we need. For
>>>> example, they have dashboards for metrics such as:
>>>>
>>>> -
>>>>
>>>> Time to approve or merge
>>>> <https://k8s.devstats.cncf.io/d/44/pr-time-to-approve-and-merge?orgId=1>
>>>> -
>>>>
>>>> PR Time to engagement
>>>> <https://k8s.devstats.cncf.io/d/14/pr-time-to-engagment?orgId=1>
>>>> -
>>>>
>>>> New and Episodic PR contributors
>>>> <https://prometheus.devstats.cncf.io/d/14/new-and-episodic-pr-contributors?orgId=1>
>>>> -
>>>>
>>>> PR reviews by contributor
>>>> <https://k8s.devstats.cncf.io/d/46/pr-reviews-by-contributor?orgId=1>
>>>> -
>>>>
>>>> Company statistics
>>>>
>>>> It would be really cool if we can try it out for Beam. I don't have
>>>> much experience using open source projects. From what I understand:
>>>> DevStats is developed by CNCF <https://www.cncf.io/> and they manage
>>>> their incubator projects' dashboard. Since Beam is not part of the CNCF, in
>>>> order to use DevStats, we have to fork the project and maintain it
>>>> ourselves.
>>>>
>>>> 1. What do you think about using DevStats for Beam? Do you know how it
>>>> is usually done?
>>>> 2. If you are not sure about DevStats, do you know any other tool which
>>>> could help us track & visualize Beam metrics?
>>>>
>>>> Thanks, Huygaa
>>>>
>>>
Re: [DISCUSSION] Tracking & Visualizing various metrics of the Beam community
Posted by Huygaa Batsaikhan <ba...@google.com>.
Thanks Udi, I am working on spinning up a test instance.
On Tue, Aug 7, 2018 at 11:31 AM Udi Meiri <eh...@google.com> wrote:
> These tables look very cool!
>
> I certainly don't object to using tools made by another organization. The
> only issue might be compatibility with our process.
>
> From my understanding of the Kubernetes review process, they use Github's
> tagging feature to specify PR statuses such as LGTM and approval.
> Random example: https://github.com/kubernetes/kubernetes/pull/67089
> I'm not sure that statistics such as pr-time-to-approve-and-merge would be
> captured at all on apache/beam since we don't use these tags.
>
> I would definitely like to see how useful this tool is regardless, and
> since it doesn't seem to require any special permissions it should be
> simple to set up a test instance.
>
>
> On Tue, Aug 7, 2018 at 11:24 AM Huygaa Batsaikhan <ba...@google.com>
> wrote:
>
>> tl;dr - is there any objections to trying out DevStats tool created by
>> CNCF?
>>
>> On Mon, Aug 6, 2018 at 3:43 PM Huygaa Batsaikhan <ba...@google.com>
>> wrote:
>>
>>> Continuing the discussion
>>> <https://lists.apache.org/thread.html/6138d08c551e254b5f13b26c6ba06579a49a4694f4d13ad6d164689a@%3Cdev.beam.apache.org%3E>
>>> about improving Beam code review, I am looking into visualizing various
>>> helpful Beam community metrics such as code velocity, reviewer load, and
>>> new contributor's engagement.
>>>
>>> So far, I found DevStats
>>> <https://k8s.devstats.cncf.io/d/12/dashboards?refresh=15m&orgId=1>, an
>>> open source (github <https://github.com/cncf/devstats>) dashboarding
>>> tool used by Kubernetes, seems to provide almost everything we need. For
>>> example, they have dashboards for metrics such as:
>>>
>>> -
>>>
>>> Time to approve or merge
>>> <https://k8s.devstats.cncf.io/d/44/pr-time-to-approve-and-merge?orgId=1>
>>> -
>>>
>>> PR Time to engagement
>>> <https://k8s.devstats.cncf.io/d/14/pr-time-to-engagment?orgId=1>
>>> -
>>>
>>> New and Episodic PR contributors
>>> <https://prometheus.devstats.cncf.io/d/14/new-and-episodic-pr-contributors?orgId=1>
>>> -
>>>
>>> PR reviews by contributor
>>> <https://k8s.devstats.cncf.io/d/46/pr-reviews-by-contributor?orgId=1>
>>> -
>>>
>>> Company statistics
>>>
>>> It would be really cool if we can try it out for Beam. I don't have
>>> much experience using open source projects. From what I understand:
>>> DevStats is developed by CNCF <https://www.cncf.io/> and they manage
>>> their incubator projects' dashboard. Since Beam is not part of the CNCF, in
>>> order to use DevStats, we have to fork the project and maintain it
>>> ourselves.
>>>
>>> 1. What do you think about using DevStats for Beam? Do you know how it
>>> is usually done?
>>> 2. If you are not sure about DevStats, do you know any other tool which
>>> could help us track & visualize Beam metrics?
>>>
>>> Thanks, Huygaa
>>>
>>
Re: [DISCUSSION] Tracking & Visualizing various metrics of the Beam community
Posted by Udi Meiri <eh...@google.com>.
These tables look very cool!
I certainly don't object to using tools made by another organization. The
only issue might be compatibility with our process.
From my understanding of the Kubernetes review process, they use Github's
tagging feature to specify PR statuses such as LGTM and approval.
Random example: https://github.com/kubernetes/kubernetes/pull/67089
I'm not sure that statistics such as pr-time-to-approve-and-merge would be
captured at all on apache/beam since we don't use these tags.
I would definitely like to see how useful this tool is regardless, and
since it doesn't seem to require any special permissions it should be
simple to set up a test instance.
On Tue, Aug 7, 2018 at 11:24 AM Huygaa Batsaikhan <ba...@google.com> wrote:
> tl;dr - is there any objections to trying out DevStats tool created by
> CNCF?
>
> On Mon, Aug 6, 2018 at 3:43 PM Huygaa Batsaikhan <ba...@google.com>
> wrote:
>
>> Continuing the discussion
>> <https://lists.apache.org/thread.html/6138d08c551e254b5f13b26c6ba06579a49a4694f4d13ad6d164689a@%3Cdev.beam.apache.org%3E>
>> about improving Beam code review, I am looking into visualizing various
>> helpful Beam community metrics such as code velocity, reviewer load, and
>> new contributor's engagement.
>>
>> So far, I found DevStats
>> <https://k8s.devstats.cncf.io/d/12/dashboards?refresh=15m&orgId=1>, an
>> open source (github <https://github.com/cncf/devstats>) dashboarding
>> tool used by Kubernetes, seems to provide almost everything we need. For
>> example, they have dashboards for metrics such as:
>>
>> -
>>
>> Time to approve or merge
>> <https://k8s.devstats.cncf.io/d/44/pr-time-to-approve-and-merge?orgId=1>
>> -
>>
>> PR Time to engagement
>> <https://k8s.devstats.cncf.io/d/14/pr-time-to-engagment?orgId=1>
>> -
>>
>> New and Episodic PR contributors
>> <https://prometheus.devstats.cncf.io/d/14/new-and-episodic-pr-contributors?orgId=1>
>> -
>>
>> PR reviews by contributor
>> <https://k8s.devstats.cncf.io/d/46/pr-reviews-by-contributor?orgId=1>
>> -
>>
>> Company statistics
>>
>> It would be really cool if we can try it out for Beam. I don't have much
>> experience using open source projects. From what I understand: DevStats is
>> developed by CNCF <https://www.cncf.io/> and they manage their incubator
>> projects' dashboard. Since Beam is not part of the CNCF, in order to use
>> DevStats, we have to fork the project and maintain it ourselves.
>>
>> 1. What do you think about using DevStats for Beam? Do you know how it is
>> usually done?
>> 2. If you are not sure about DevStats, do you know any other tool which
>> could help us track & visualize Beam metrics?
>>
>> Thanks, Huygaa
>>
>
Re: [DISCUSSION] Tracking & Visualizing various metrics of the Beam community
Posted by Huygaa Batsaikhan <ba...@google.com>.
tl;dr - is there any objections to trying out DevStats tool created by CNCF?
On Mon, Aug 6, 2018 at 3:43 PM Huygaa Batsaikhan <ba...@google.com> wrote:
> Continuing the discussion
> <https://lists.apache.org/thread.html/6138d08c551e254b5f13b26c6ba06579a49a4694f4d13ad6d164689a@%3Cdev.beam.apache.org%3E>
> about improving Beam code review, I am looking into visualizing various
> helpful Beam community metrics such as code velocity, reviewer load, and
> new contributor's engagement.
>
> So far, I found DevStats
> <https://k8s.devstats.cncf.io/d/12/dashboards?refresh=15m&orgId=1>, an
> open source (github <https://github.com/cncf/devstats>) dashboarding
> tool used by Kubernetes, seems to provide almost everything we need. For
> example, they have dashboards for metrics such as:
>
> -
>
> Time to approve or merge
> <https://k8s.devstats.cncf.io/d/44/pr-time-to-approve-and-merge?orgId=1>
> -
>
> PR Time to engagement
> <https://k8s.devstats.cncf.io/d/14/pr-time-to-engagment?orgId=1>
> -
>
> New and Episodic PR contributors
> <https://prometheus.devstats.cncf.io/d/14/new-and-episodic-pr-contributors?orgId=1>
> -
>
> PR reviews by contributor
> <https://k8s.devstats.cncf.io/d/46/pr-reviews-by-contributor?orgId=1>
> -
>
> Company statistics
>
> It would be really cool if we can try it out for Beam. I don't have much
> experience using open source projects. From what I understand: DevStats is
> developed by CNCF <https://www.cncf.io/> and they manage their incubator
> projects' dashboard. Since Beam is not part of the CNCF, in order to use
> DevStats, we have to fork the project and maintain it ourselves.
>
> 1. What do you think about using DevStats for Beam? Do you know how it is
> usually done?
> 2. If you are not sure about DevStats, do you know any other tool which
> could help us track & visualize Beam metrics?
>
> Thanks, Huygaa
>