You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gravitino.apache.org by Robert Thomson <bt...@apache.org> on 2026/10/23 09:02:32 UTC
URGENT ACTION NEEDED - Excessive Consumption of GitHub hosted Action Runners Resources
Hi Gravitino project,
In 2024, the ASF introduced the policy for GitHub Actions usage across
the foundation[1]. As you have likely experienced, the ASF's Github
shared pool of Github-hosted runners has been at the maximum limit of
the number of jobs that can run for most of the day for some time now.
At the moment we are seeing your project 2nd in the list for consumption of
GitHub hosted Actions runners for the past week, therefore we require you to
review your usage and the workflows that are consuming those Runner resources
and to make any improvements that can be made to reduce this usage.
If the usage remains high your access to GitHub Actions may be removed per
the policy quoted above.
I have listed some documentation resources that may be useful to assist
you in reducing this consumption below and I have also included
some suggestions from prompting Claude to examine your project repo's
workflow code thus, "Analyze the GitHub Actions workflows in this repo
and suggest changes that would be beneficial for the goal of a)
Reducing the number of concurrent workflow jobs started and b) The
duration of the jobs that do start https://github.com/apache/gravitino "
The suggested changes are listed below, of course generated by GenAI,
so use your discretion and own reading there and I am no GitHub
Actions SME, so have not curated them, merely listed them verbatim.
Note, there is also now a skill available that has been developed by
the magpie project which can be used to give more detailed feedback on
CI using a Gen AI tool of your choice:
https://github.com/apache/magpie/blob/main/docs/repo-health/README.md#workflow-optimization-experimental
Make every effort to optimise your usage ASAP as the near constant
100% utilisation of our available GitHub hosted runner jobs is making
it practically impossible for all projects to get their CI jobs to
start and complete in a reasonable length of time, and it is incumbent
on all projects to do their part to help other projects get their fair
share of this limited resource.
Thanks.
Kind regards,
-Bob Thomson,
on behalf of ASF Infrastructure
Resources
You can use the infra reporting tool [2] to monitor your GHA usage as you
work on stream-lining and to locate any bottlenecks in the workflows.
There is now on-going community discussion on a Slack channel,
“project-workflow-optimisations” about the topic of optimising GitHub
Actions workflows so as to minimise both the number of concurrent jobs
running on our limited allocation, and the number of long-running
jobs.
To share best practices between projects on this area we encourage
projects to contribute to this cwiki page:
https://cwiki.apache.org/confluence/display/INFRA/GitHub+Actions+Recommended+Practices
It is also possible for a project to maintain their own self-hosted
runners if they can provide a suitable VM/on-prem hardware to execute
the runner:
https://cwiki.apache.org/confluence/display/INFRA/GitHub+self-hosted+runners
Claude generated suggestions:
Apache Gravitino
Fewer concurrent jobs
Compute every path filter once in eligibility (multiple named filters
in one step), using the compare API instead of a checkout. Gate suites
in fork-ci.yml with if:, and delete the 16 per-suite changes jobs.
Replace the fail_fast job, which holds an ubuntu-latest runner for up
to 6 hours, with a step in each suite:
yaml
- if: failure()
run: gh run cancel "$GITHUB_RUN_ID" # needs actions: write
In update-fork-ci.cjs, exit the polling loop early when nothing is
pending, and lengthen the */5 cron.
If compile-check only exists to satisfy a required check, make it a
no-op instead of running ./gradlew assemble.
Shorter jobs
In checkout-and-sync, replace fetch-depth: 0 with filter: blob:none,
or upload one source bundle for the other jobs to download. First
confirm the build doesn’t need git history (spotless ratchet, version
stamping).
Remove docker/setup-qemu-action from the 13 workflows that only build
linux/amd64.
Add gradle/actions/setup-gradle with the build cache, or use
Develocity’s remote cache if available. The current cache: 'gradle'
caches dependencies only, so every suite recompiles.
Check that the coverage job doesn’t re-run Gradle after build and core-test.
Move backend and idp-basic off ubuntu-22.04, which is mid-deprecation.
Re: URGENT ACTION NEEDED - Excessive Consumption of GitHub hosted Action Runners Resources
Posted by Qi Yu <yu...@apache.org>.
Hi Bob,
Thanks for bringing this to our attention. We are already taking action to
reduce Gravitino’s consumption of the ASF’s shared GitHub Actions runner
resources. We’ll share more details on the changes and progress shortly.
Best regards,
The Gravitino community
Robert Thomson <bt...@apache.org> 于2026年10月23日周五 17:02写道:
> Hi Gravitino project,
>
> In 2024, the ASF introduced the policy for GitHub Actions usage across
> the foundation[1]. As you have likely experienced, the ASF's Github
> shared pool of Github-hosted runners has been at the maximum limit of
> the number of jobs that can run for most of the day for some time now.
> At the moment we are seeing your project 2nd in the list for consumption of
> GitHub hosted Actions runners for the past week, therefore we require you
> to
> review your usage and the workflows that are consuming those Runner
> resources
> and to make any improvements that can be made to reduce this usage.
> If the usage remains high your access to GitHub Actions may be removed per
> the policy quoted above.
>
> I have listed some documentation resources that may be useful to assist
> you in reducing this consumption below and I have also included
> some suggestions from prompting Claude to examine your project repo's
> workflow code thus, "Analyze the GitHub Actions workflows in this repo
> and suggest changes that would be beneficial for the goal of a)
> Reducing the number of concurrent workflow jobs started and b) The
> duration of the jobs that do start https://github.com/apache/gravitino "
>
> The suggested changes are listed below, of course generated by GenAI,
> so use your discretion and own reading there and I am no GitHub
> Actions SME, so have not curated them, merely listed them verbatim.
>
> Note, there is also now a skill available that has been developed by
> the magpie project which can be used to give more detailed feedback on
> CI using a Gen AI tool of your choice:
>
> https://github.com/apache/magpie/blob/main/docs/repo-health/README.md#workflow-optimization-experimental
>
> Make every effort to optimise your usage ASAP as the near constant
> 100% utilisation of our available GitHub hosted runner jobs is making
> it practically impossible for all projects to get their CI jobs to
> start and complete in a reasonable length of time, and it is incumbent
> on all projects to do their part to help other projects get their fair
> share of this limited resource.
>
> Thanks.
> Kind regards,
> -Bob Thomson,
> on behalf of ASF Infrastructure
>
> Resources
>
> You can use the infra reporting tool [2] to monitor your GHA usage as you
> work on stream-lining and to locate any bottlenecks in the workflows.
>
> There is now on-going community discussion on a Slack channel,
> “project-workflow-optimisations” about the topic of optimising GitHub
> Actions workflows so as to minimise both the number of concurrent jobs
> running on our limited allocation, and the number of long-running
> jobs.
>
> To share best practices between projects on this area we encourage
> projects to contribute to this cwiki page:
>
>
> https://cwiki.apache.org/confluence/display/INFRA/GitHub+Actions+Recommended+Practices
>
> It is also possible for a project to maintain their own self-hosted
> runners if they can provide a suitable VM/on-prem hardware to execute
> the runner:
>
>
> https://cwiki.apache.org/confluence/display/INFRA/GitHub+self-hosted+runners
>
> Claude generated suggestions:
>
> Apache Gravitino
>
> Fewer concurrent jobs
>
> Compute every path filter once in eligibility (multiple named filters
> in one step), using the compare API instead of a checkout. Gate suites
> in fork-ci.yml with if:, and delete the 16 per-suite changes jobs.
> Replace the fail_fast job, which holds an ubuntu-latest runner for up
> to 6 hours, with a step in each suite:
>
> yaml
>
> - if: failure()
> run: gh run cancel "$GITHUB_RUN_ID" # needs actions: write
>
> In update-fork-ci.cjs, exit the polling loop early when nothing is
> pending, and lengthen the */5 cron.
> If compile-check only exists to satisfy a required check, make it a
> no-op instead of running ./gradlew assemble.
>
> Shorter jobs
>
> In checkout-and-sync, replace fetch-depth: 0 with filter: blob:none,
> or upload one source bundle for the other jobs to download. First
> confirm the build doesn’t need git history (spotless ratchet, version
> stamping).
> Remove docker/setup-qemu-action from the 13 workflows that only build
> linux/amd64.
> Add gradle/actions/setup-gradle with the build cache, or use
> Develocity’s remote cache if available. The current cache: 'gradle'
> caches dependencies only, so every suite recompiles.
> Check that the coverage job doesn’t re-run Gradle after build and
> core-test.
> Move backend and idp-basic off ubuntu-22.04, which is mid-deprecation.
>