You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@apache.org by Tobiasz Kędzierski <to...@polidea.com> on 2020/11/20 11:56:18 UTC
GitHub Actions statistics for ASF
Hi all,
There was discussion recently about usage of GitHub Actions [GA] within ASF.
There are more than 200 ASF projects which use GitHub Actions and the limit
for concurrent jobs is only 180 ( [1], enterprise plan).
There may be a situation where one or a few projects exhaust this limit
significantly which could lead to extensive queues in the rest of the
projects.
Since GitHub does not provide statistics for GIthub Actions I created
repository [2] which collects snapshots of workflow runs in "queued" and
"in_progress" states for every ASF repo which uses GA. It is done within
the mentioned repository by scheduled github action. Data is uploaded as
artifacts, uploaded to Google Cloud Storage Bucket and pushed to the
BigQuery table.
Repo: https://github.com/TobKed/fetch-apache-ga-stats/
Screenshot of the simple report in DataStudio based on the mentioned data.
[image: image.png]
This kind of statistics may help to identify projects which may require
some optimizations in workflows (e.g. workflow cancelling like it is done
in Airflow [3]).
Let me know what do you think about it.
[1] -
https://docs.github.com/en/free-pro-team@latest/actions/reference/usage-limits-billing-and-administration#usage-limits
<https://docs.github.com/en/free-pro-team@latest/actions/reference/usage-limits-billing-and-administration#usage-limits>
[2] - https://github.com/TobKed/fetch-apache-ga-stats/
[3] -
https://github.com/apache/airflow/blob/c38dadb526f7104df7a1a6feda72ce1b65557bd9/.github/workflows/build-images-workflow-run.yml#L45
BR
Tobiasz Kędzierski
Re: GitHub Actions statistics for ASF
Posted by Tobiasz Kędzierski <to...@polidea.com>.
Thanks Zach.
I tried to attach a screenshot in this email.
I case it is not working I also exported data from simple report to
spreadsheet:
https://docs.google.com/spreadsheets/d/1SE9HIHBPmTZuW1WAgdVbEcGouGesiyrnXDIZxx25RSE/edit#gid=0
BR
Tobiasz
On Fri, Nov 20, 2020 at 5:17 PM Zach Hoffman <zr...@apache.org> wrote:
> > Screenshot of the simple report in DataStudio based on the mentioned
> data.
> > [image: image.png]
>
> If there was supposed to be a screenshot, it didn't make it.
>
> -Zach
>
> On 2020/11/20 11:56:18, Tobiasz Kędzierski <to...@polidea.com>
> wrote:
> > Hi all,
> >
> > There was discussion recently about usage of GitHub Actions [GA] within
> ASF.
> > There are more than 200 ASF projects which use GitHub Actions and the
> limit
> > for concurrent jobs is only 180 ( [1], enterprise plan).
> >
> > There may be a situation where one or a few projects exhaust this limit
> > significantly which could lead to extensive queues in the rest of the
> > projects.
> >
> > Since GitHub does not provide statistics for GIthub Actions I created
> > repository [2] which collects snapshots of workflow runs in "queued" and
> > "in_progress" states for every ASF repo which uses GA. It is done within
> > the mentioned repository by scheduled github action. Data is uploaded as
> > artifacts, uploaded to Google Cloud Storage Bucket and pushed to the
> > BigQuery table.
> >
> > Repo: https://github.com/TobKed/fetch-apache-ga-stats/
> >
> > Screenshot of the simple report in DataStudio based on the mentioned
> data.
> > [image: image.png]
> >
> > This kind of statistics may help to identify projects which may require
> > some optimizations in workflows (e.g. workflow cancelling like it is done
> > in Airflow [3]).
> >
> > Let me know what do you think about it.
> >
> > [1] -
> >
> https://docs.github.com/en/free-pro-team@latest/actions/reference/usage-limits-billing-and-administration#usage-limits
> > <
> https://docs.github.com/en/free-pro-team@latest/actions/reference/usage-limits-billing-and-administration#usage-limits
> >
> > [2] - https://github.com/TobKed/fetch-apache-ga-stats/
> > [3] -
> >
> https://github.com/apache/airflow/blob/c38dadb526f7104df7a1a6feda72ce1b65557bd9/.github/workflows/build-images-workflow-run.yml#L45
> >
> > BR
> > Tobiasz Kędzierski
> >
>
--
Tobiasz Kędzierski
Polidea <https://www.polidea.com/> | Software Engineer
E: tobiasz.kedzierski@polidea.com
[image: Polidea] <https://www.polidea.com/>
Check out our projects! <https://www.polidea.com/our-work>
[image: Github] <https://github.com/Polidea> [image: Facebook]
<https://www.facebook.com/Polidea.Software> [image: Twitter]
<https://twitter.com/polidea> [image: Linkedin]
<https://www.linkedin.com/company/polidea> [image: Instagram]
<https://instagram.com/polidea> [image: Behance]
<https://www.behance.net/polidea> [image: dribbble]
<https://dribbble.com/polideadesign>
Re: GitHub Actions statistics for ASF
Posted by Zach Hoffman <zr...@apache.org>.
> Screenshot of the simple report in DataStudio based on the mentioned data.
> [image: image.png]
If there was supposed to be a screenshot, it didn't make it.
-Zach
On 2020/11/20 11:56:18, Tobiasz Kędzierski <to...@polidea.com> wrote:
> Hi all,
>
> There was discussion recently about usage of GitHub Actions [GA] within ASF.
> There are more than 200 ASF projects which use GitHub Actions and the limit
> for concurrent jobs is only 180 ( [1], enterprise plan).
>
> There may be a situation where one or a few projects exhaust this limit
> significantly which could lead to extensive queues in the rest of the
> projects.
>
> Since GitHub does not provide statistics for GIthub Actions I created
> repository [2] which collects snapshots of workflow runs in "queued" and
> "in_progress" states for every ASF repo which uses GA. It is done within
> the mentioned repository by scheduled github action. Data is uploaded as
> artifacts, uploaded to Google Cloud Storage Bucket and pushed to the
> BigQuery table.
>
> Repo: https://github.com/TobKed/fetch-apache-ga-stats/
>
> Screenshot of the simple report in DataStudio based on the mentioned data.
> [image: image.png]
>
> This kind of statistics may help to identify projects which may require
> some optimizations in workflows (e.g. workflow cancelling like it is done
> in Airflow [3]).
>
> Let me know what do you think about it.
>
> [1] -
> https://docs.github.com/en/free-pro-team@latest/actions/reference/usage-limits-billing-and-administration#usage-limits
> <https://docs.github.com/en/free-pro-team@latest/actions/reference/usage-limits-billing-and-administration#usage-limits>
> [2] - https://github.com/TobKed/fetch-apache-ga-stats/
> [3] -
> https://github.com/apache/airflow/blob/c38dadb526f7104df7a1a6feda72ce1b65557bd9/.github/workflows/build-images-workflow-run.yml#L45
>
> BR
> Tobiasz Kędzierski
>