You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@datafusion.apache.org by Robert Thomson <bt...@apache.org> on 2026/05/22 11:44:16 UTC
Datafusion Consumption of ASF Shared GitHub-hosted Runners
Hello, Datafusion PMC.
In 2024, the ASF introduced the policy for GitHub Actions usage
across the foundation[1]. The ASF Github shared pool of
Github-hosted runners has been at, or very close to the limit of
900 jobs most of the time in the past few weeks and this is the
case again today.
Your project has been identified as being among the top 5 consumers of
build time over the past 7 days and we request that you bring your
usage down by stream-lining long-running builds. Contact Infra for
a consultation if you are unable to streamline your builds further.
You can use the infra reporting tool[2] to monitor your GHA usage as you
work on stream-lining, as well as locate any bottlenecks in the workflows.
Infra will allow you two weeks time (till the 8th of June, 2026) to
progress this, but should you still be above the limits by then,
without a viable path forward, we will be limiting your GHA usage.
Kind regards,
Bob Thomson, on behalf of ASF Infrastructure.
[1] https://infra.apache.org/github-actions-policy.html
[2]
https://infra-reports.apache.org/#ghactions&project=datafusion&hours=24&limit=15&group=name
Re: Datafusion Consumption of ASF Shared GitHub-hosted Runners
Posted by "Oleks V." <co...@apache.org>.
Also pinged Bob in ASF slack
On 2026/05/30 00:22:46 Andrew Lamb wrote:
> Adding Bob back on cc in case he is not monitoring the dev list
>
> Andrew
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: Datafusion Consumption of ASF Shared GitHub-hosted Runners
Posted by "Oleks V." <co...@apache.org>.
Thanks Bob for confirmation and assistance
On 2026/06/11 08:48:46 Bob Thomson wrote:
> Hi,
>
> Reviewing the picture today, the GitHub hosted runners utilisation overall is still maxxing out daily. WIth respect to datafusion we can see over the last 7 days that the project has dropped out of the top 5 to 7 which is good progress. Your continued work and vigilence on this area helps all projects and is appreciated.
>
> Please note that there is now a page for sharing tips and best practice for optimising workflows with respect to utilisation:
>
> https://cwiki.apache.org/confluence/display/INFRA/GitHub+Actions+Recommended+Practices
>
> There is also a Slack channel for community discussion and sharing ideas: project-workflow-optimisations
>
> Thanks.
>
> Kind regards,
> -Bob Thomson,
> ASF Infrastructure
>
>
> On 2026/06/02 06:41:09 Robert Thomson wrote:
> > Hi,
> >
> > Yes, I can see Datafusion down to number 2 in the list of top 5 consumers
> > now. The top 5 consumers still account for nearly 1/3 of the total
> > consumption, so continued consideration and vigilance to keep it to a
> > minimum is appreciated, as such effort benefits all ASF projects of course:
> > 885/900 jobs in use as I write this.
> >
> > Thanks.
> >
> > Kind regards,
> > -Bob Thomson,
> > ASF Infrastructure
> >
> >
> > On Sat, May 30, 2026 at 2:23 AM Andrew Lamb <an...@gmail.com> wrote:
> >
> > > Adding Bob back on cc in case he is not monitoring the dev list
> > >
> > > Andrew
> > >
> > > On Fri, May 29, 2026 at 1:53 PM Oleks V. <co...@apache.org> wrote:
> > >
> > >> Thank you, Bob, for raising this concern regarding the consumption of ASF
> > >> shared resources.
> > >>
> > >> The Apache DataFusion team has already addressed several low-hanging
> > >> optimizations, and as a result, daily CI resource consumption has been
> > >> reduced by approximately 50%. 24Hr usage decreased from 113K minutes a week
> > >> ago to 55K minutes over the last 24 hours.
> > >>
> > >> These changes were introduced over the past few days, and the most recent
> > >> 5-day report is already approaching the threshold defined in the ASF policy:
> > >> https://infra.apache.org/github-actions-policy.html
> > >>
> > >> Specifically, the latest 5-day report shows 248K minutes compared to the
> > >> 216K threshold, with a clear downward trend continuing as the recent
> > >> optimizations fully propagate through the reporting window.
> > >>
> > >> In parallel, we are actively working on longer-term improvements that
> > >> will further reduce CI overhead while maintaining project quality and
> > >> contributor experience.
> > >>
> > >> Some of the options currently being evaluated include:
> > >>
> > >> * Offloading long-running CI pipelines to contributor forks, similar to
> > >> the approach used by Apache Spark:
> > >> https://github.com/apache/spark/tree/master/.github
> > >> This would shift execution of certain workflows from ASF-hosted
> > >> resources to contributor-owned resources.
> > >> * Refining test matrices to reduce redundant workflow combinations.
> > >> * Introducing a gated execution model where selected CI pipelines are
> > >> triggered manually instead of automatically for every commit.
> > >>
> > >> We would also appreciate any additional recommendations you may have
> > >> based on your experience managing CI resource utilization across ASF
> > >> projects.
> > >>
> > >>
> > >> Given the significant reduction already achieved and the continuing
> > >> downward trend, would it be possible to extend the current deadline to
> > >> allow the team sufficient time to implement the longer-term optimizations
> > >> without negatively impacting project quality, contributor confidence and
> > >> user trust?
> > >>
> > >> We appreciate your support and look forward to your feedback.
> > >>
> > >> Oleks V
> > >>
> > >> On 2026/05/22 11:44:16 Robert Thomson wrote:
> > >> > Hello, Datafusion PMC.
> > >> >
> > >> > In 2024, the ASF introduced the policy for GitHub Actions usage
> > >> > across the foundation[1]. The ASF Github shared pool of
> > >> > Github-hosted runners has been at, or very close to the limit of
> > >> > 900 jobs most of the time in the past few weeks and this is the
> > >> > case again today.
> > >> >
> > >> > Your project has been identified as being among the top 5 consumers of
> > >> > build time over the past 7 days and we request that you bring your
> > >> > usage down by stream-lining long-running builds. Contact Infra for
> > >> > a consultation if you are unable to streamline your builds further.
> > >> >
> > >> > You can use the infra reporting tool[2] to monitor your GHA usage as you
> > >> > work on stream-lining, as well as locate any bottlenecks in the
> > >> workflows.
> > >> >
> > >> > Infra will allow you two weeks time (till the 8th of June, 2026) to
> > >> > progress this, but should you still be above the limits by then,
> > >> > without a viable path forward, we will be limiting your GHA usage.
> > >> >
> > >> > Kind regards,
> > >> > Bob Thomson, on behalf of ASF Infrastructure.
> > >> >
> > >> >
> > >> > [1] https://infra.apache.org/github-actions-policy.html
> > >> > [2]
> > >> >
> > >> https://infra-reports.apache.org/#ghactions&project=datafusion&hours=24&limit=15&group=name
> > >> >
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
> > >> For additional commands, e-mail: dev-help@datafusion.apache.org
> > >>
> > >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
> For additional commands, e-mail: dev-help@datafusion.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: Datafusion Consumption of ASF Shared GitHub-hosted Runners
Posted by Bob Thomson <bt...@apache.org>.
Hi,
Reviewing the picture today, the GitHub hosted runners utilisation overall is still maxxing out daily. WIth respect to datafusion we can see over the last 7 days that the project has dropped out of the top 5 to 7 which is good progress. Your continued work and vigilence on this area helps all projects and is appreciated.
Please note that there is now a page for sharing tips and best practice for optimising workflows with respect to utilisation:
https://cwiki.apache.org/confluence/display/INFRA/GitHub+Actions+Recommended+Practices
There is also a Slack channel for community discussion and sharing ideas: project-workflow-optimisations
Thanks.
Kind regards,
-Bob Thomson,
ASF Infrastructure
On 2026/06/02 06:41:09 Robert Thomson wrote:
> Hi,
>
> Yes, I can see Datafusion down to number 2 in the list of top 5 consumers
> now. The top 5 consumers still account for nearly 1/3 of the total
> consumption, so continued consideration and vigilance to keep it to a
> minimum is appreciated, as such effort benefits all ASF projects of course:
> 885/900 jobs in use as I write this.
>
> Thanks.
>
> Kind regards,
> -Bob Thomson,
> ASF Infrastructure
>
>
> On Sat, May 30, 2026 at 2:23 AM Andrew Lamb <an...@gmail.com> wrote:
>
> > Adding Bob back on cc in case he is not monitoring the dev list
> >
> > Andrew
> >
> > On Fri, May 29, 2026 at 1:53 PM Oleks V. <co...@apache.org> wrote:
> >
> >> Thank you, Bob, for raising this concern regarding the consumption of ASF
> >> shared resources.
> >>
> >> The Apache DataFusion team has already addressed several low-hanging
> >> optimizations, and as a result, daily CI resource consumption has been
> >> reduced by approximately 50%. 24Hr usage decreased from 113K minutes a week
> >> ago to 55K minutes over the last 24 hours.
> >>
> >> These changes were introduced over the past few days, and the most recent
> >> 5-day report is already approaching the threshold defined in the ASF policy:
> >> https://infra.apache.org/github-actions-policy.html
> >>
> >> Specifically, the latest 5-day report shows 248K minutes compared to the
> >> 216K threshold, with a clear downward trend continuing as the recent
> >> optimizations fully propagate through the reporting window.
> >>
> >> In parallel, we are actively working on longer-term improvements that
> >> will further reduce CI overhead while maintaining project quality and
> >> contributor experience.
> >>
> >> Some of the options currently being evaluated include:
> >>
> >> * Offloading long-running CI pipelines to contributor forks, similar to
> >> the approach used by Apache Spark:
> >> https://github.com/apache/spark/tree/master/.github
> >> This would shift execution of certain workflows from ASF-hosted
> >> resources to contributor-owned resources.
> >> * Refining test matrices to reduce redundant workflow combinations.
> >> * Introducing a gated execution model where selected CI pipelines are
> >> triggered manually instead of automatically for every commit.
> >>
> >> We would also appreciate any additional recommendations you may have
> >> based on your experience managing CI resource utilization across ASF
> >> projects.
> >>
> >>
> >> Given the significant reduction already achieved and the continuing
> >> downward trend, would it be possible to extend the current deadline to
> >> allow the team sufficient time to implement the longer-term optimizations
> >> without negatively impacting project quality, contributor confidence and
> >> user trust?
> >>
> >> We appreciate your support and look forward to your feedback.
> >>
> >> Oleks V
> >>
> >> On 2026/05/22 11:44:16 Robert Thomson wrote:
> >> > Hello, Datafusion PMC.
> >> >
> >> > In 2024, the ASF introduced the policy for GitHub Actions usage
> >> > across the foundation[1]. The ASF Github shared pool of
> >> > Github-hosted runners has been at, or very close to the limit of
> >> > 900 jobs most of the time in the past few weeks and this is the
> >> > case again today.
> >> >
> >> > Your project has been identified as being among the top 5 consumers of
> >> > build time over the past 7 days and we request that you bring your
> >> > usage down by stream-lining long-running builds. Contact Infra for
> >> > a consultation if you are unable to streamline your builds further.
> >> >
> >> > You can use the infra reporting tool[2] to monitor your GHA usage as you
> >> > work on stream-lining, as well as locate any bottlenecks in the
> >> workflows.
> >> >
> >> > Infra will allow you two weeks time (till the 8th of June, 2026) to
> >> > progress this, but should you still be above the limits by then,
> >> > without a viable path forward, we will be limiting your GHA usage.
> >> >
> >> > Kind regards,
> >> > Bob Thomson, on behalf of ASF Infrastructure.
> >> >
> >> >
> >> > [1] https://infra.apache.org/github-actions-policy.html
> >> > [2]
> >> >
> >> https://infra-reports.apache.org/#ghactions&project=datafusion&hours=24&limit=15&group=name
> >> >
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
> >> For additional commands, e-mail: dev-help@datafusion.apache.org
> >>
> >>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: Datafusion Consumption of ASF Shared GitHub-hosted Runners
Posted by Robert Thomson <bt...@apache.org>.
Hi,
Yes, I can see Datafusion down to number 2 in the list of top 5 consumers
now. The top 5 consumers still account for nearly 1/3 of the total
consumption, so continued consideration and vigilance to keep it to a
minimum is appreciated, as such effort benefits all ASF projects of course:
885/900 jobs in use as I write this.
Thanks.
Kind regards,
-Bob Thomson,
ASF Infrastructure
On Sat, May 30, 2026 at 2:23 AM Andrew Lamb <an...@gmail.com> wrote:
> Adding Bob back on cc in case he is not monitoring the dev list
>
> Andrew
>
> On Fri, May 29, 2026 at 1:53 PM Oleks V. <co...@apache.org> wrote:
>
>> Thank you, Bob, for raising this concern regarding the consumption of ASF
>> shared resources.
>>
>> The Apache DataFusion team has already addressed several low-hanging
>> optimizations, and as a result, daily CI resource consumption has been
>> reduced by approximately 50%. 24Hr usage decreased from 113K minutes a week
>> ago to 55K minutes over the last 24 hours.
>>
>> These changes were introduced over the past few days, and the most recent
>> 5-day report is already approaching the threshold defined in the ASF policy:
>> https://infra.apache.org/github-actions-policy.html
>>
>> Specifically, the latest 5-day report shows 248K minutes compared to the
>> 216K threshold, with a clear downward trend continuing as the recent
>> optimizations fully propagate through the reporting window.
>>
>> In parallel, we are actively working on longer-term improvements that
>> will further reduce CI overhead while maintaining project quality and
>> contributor experience.
>>
>> Some of the options currently being evaluated include:
>>
>> * Offloading long-running CI pipelines to contributor forks, similar to
>> the approach used by Apache Spark:
>> https://github.com/apache/spark/tree/master/.github
>> This would shift execution of certain workflows from ASF-hosted
>> resources to contributor-owned resources.
>> * Refining test matrices to reduce redundant workflow combinations.
>> * Introducing a gated execution model where selected CI pipelines are
>> triggered manually instead of automatically for every commit.
>>
>> We would also appreciate any additional recommendations you may have
>> based on your experience managing CI resource utilization across ASF
>> projects.
>>
>>
>> Given the significant reduction already achieved and the continuing
>> downward trend, would it be possible to extend the current deadline to
>> allow the team sufficient time to implement the longer-term optimizations
>> without negatively impacting project quality, contributor confidence and
>> user trust?
>>
>> We appreciate your support and look forward to your feedback.
>>
>> Oleks V
>>
>> On 2026/05/22 11:44:16 Robert Thomson wrote:
>> > Hello, Datafusion PMC.
>> >
>> > In 2024, the ASF introduced the policy for GitHub Actions usage
>> > across the foundation[1]. The ASF Github shared pool of
>> > Github-hosted runners has been at, or very close to the limit of
>> > 900 jobs most of the time in the past few weeks and this is the
>> > case again today.
>> >
>> > Your project has been identified as being among the top 5 consumers of
>> > build time over the past 7 days and we request that you bring your
>> > usage down by stream-lining long-running builds. Contact Infra for
>> > a consultation if you are unable to streamline your builds further.
>> >
>> > You can use the infra reporting tool[2] to monitor your GHA usage as you
>> > work on stream-lining, as well as locate any bottlenecks in the
>> workflows.
>> >
>> > Infra will allow you two weeks time (till the 8th of June, 2026) to
>> > progress this, but should you still be above the limits by then,
>> > without a viable path forward, we will be limiting your GHA usage.
>> >
>> > Kind regards,
>> > Bob Thomson, on behalf of ASF Infrastructure.
>> >
>> >
>> > [1] https://infra.apache.org/github-actions-policy.html
>> > [2]
>> >
>> https://infra-reports.apache.org/#ghactions&project=datafusion&hours=24&limit=15&group=name
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
>> For additional commands, e-mail: dev-help@datafusion.apache.org
>>
>>
Re: Datafusion Consumption of ASF Shared GitHub-hosted Runners
Posted by Andrew Lamb <an...@gmail.com>.
Adding Bob back on cc in case he is not monitoring the dev list
Andrew
On Fri, May 29, 2026 at 1:53 PM Oleks V. <co...@apache.org> wrote:
> Thank you, Bob, for raising this concern regarding the consumption of ASF
> shared resources.
>
> The Apache DataFusion team has already addressed several low-hanging
> optimizations, and as a result, daily CI resource consumption has been
> reduced by approximately 50%. 24Hr usage decreased from 113K minutes a week
> ago to 55K minutes over the last 24 hours.
>
> These changes were introduced over the past few days, and the most recent
> 5-day report is already approaching the threshold defined in the ASF policy:
> https://infra.apache.org/github-actions-policy.html
>
> Specifically, the latest 5-day report shows 248K minutes compared to the
> 216K threshold, with a clear downward trend continuing as the recent
> optimizations fully propagate through the reporting window.
>
> In parallel, we are actively working on longer-term improvements that will
> further reduce CI overhead while maintaining project quality and
> contributor experience.
>
> Some of the options currently being evaluated include:
>
> * Offloading long-running CI pipelines to contributor forks, similar to
> the approach used by Apache Spark:
> https://github.com/apache/spark/tree/master/.github
> This would shift execution of certain workflows from ASF-hosted
> resources to contributor-owned resources.
> * Refining test matrices to reduce redundant workflow combinations.
> * Introducing a gated execution model where selected CI pipelines are
> triggered manually instead of automatically for every commit.
>
> We would also appreciate any additional recommendations you may have based
> on your experience managing CI resource utilization across ASF projects.
>
>
> Given the significant reduction already achieved and the continuing
> downward trend, would it be possible to extend the current deadline to
> allow the team sufficient time to implement the longer-term optimizations
> without negatively impacting project quality, contributor confidence and
> user trust?
>
> We appreciate your support and look forward to your feedback.
>
> Oleks V
>
> On 2026/05/22 11:44:16 Robert Thomson wrote:
> > Hello, Datafusion PMC.
> >
> > In 2024, the ASF introduced the policy for GitHub Actions usage
> > across the foundation[1]. The ASF Github shared pool of
> > Github-hosted runners has been at, or very close to the limit of
> > 900 jobs most of the time in the past few weeks and this is the
> > case again today.
> >
> > Your project has been identified as being among the top 5 consumers of
> > build time over the past 7 days and we request that you bring your
> > usage down by stream-lining long-running builds. Contact Infra for
> > a consultation if you are unable to streamline your builds further.
> >
> > You can use the infra reporting tool[2] to monitor your GHA usage as you
> > work on stream-lining, as well as locate any bottlenecks in the
> workflows.
> >
> > Infra will allow you two weeks time (till the 8th of June, 2026) to
> > progress this, but should you still be above the limits by then,
> > without a viable path forward, we will be limiting your GHA usage.
> >
> > Kind regards,
> > Bob Thomson, on behalf of ASF Infrastructure.
> >
> >
> > [1] https://infra.apache.org/github-actions-policy.html
> > [2]
> >
> https://infra-reports.apache.org/#ghactions&project=datafusion&hours=24&limit=15&group=name
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
> For additional commands, e-mail: dev-help@datafusion.apache.org
>
>
Re: Datafusion Consumption of ASF Shared GitHub-hosted Runners
Posted by "Oleks V." <co...@apache.org>.
Thank you, Bob, for raising this concern regarding the consumption of ASF shared resources.
The Apache DataFusion team has already addressed several low-hanging optimizations, and as a result, daily CI resource consumption has been reduced by approximately 50%. 24Hr usage decreased from 113K minutes a week ago to 55K minutes over the last 24 hours.
These changes were introduced over the past few days, and the most recent 5-day report is already approaching the threshold defined in the ASF policy:
https://infra.apache.org/github-actions-policy.html
Specifically, the latest 5-day report shows 248K minutes compared to the 216K threshold, with a clear downward trend continuing as the recent optimizations fully propagate through the reporting window.
In parallel, we are actively working on longer-term improvements that will further reduce CI overhead while maintaining project quality and contributor experience.
Some of the options currently being evaluated include:
* Offloading long-running CI pipelines to contributor forks, similar to the approach used by Apache Spark:
https://github.com/apache/spark/tree/master/.github
This would shift execution of certain workflows from ASF-hosted resources to contributor-owned resources.
* Refining test matrices to reduce redundant workflow combinations.
* Introducing a gated execution model where selected CI pipelines are triggered manually instead of automatically for every commit.
We would also appreciate any additional recommendations you may have based on your experience managing CI resource utilization across ASF projects.
Given the significant reduction already achieved and the continuing downward trend, would it be possible to extend the current deadline to allow the team sufficient time to implement the longer-term optimizations without negatively impacting project quality, contributor confidence and user trust?
We appreciate your support and look forward to your feedback.
Oleks V
On 2026/05/22 11:44:16 Robert Thomson wrote:
> Hello, Datafusion PMC.
>
> In 2024, the ASF introduced the policy for GitHub Actions usage
> across the foundation[1]. The ASF Github shared pool of
> Github-hosted runners has been at, or very close to the limit of
> 900 jobs most of the time in the past few weeks and this is the
> case again today.
>
> Your project has been identified as being among the top 5 consumers of
> build time over the past 7 days and we request that you bring your
> usage down by stream-lining long-running builds. Contact Infra for
> a consultation if you are unable to streamline your builds further.
>
> You can use the infra reporting tool[2] to monitor your GHA usage as you
> work on stream-lining, as well as locate any bottlenecks in the workflows.
>
> Infra will allow you two weeks time (till the 8th of June, 2026) to
> progress this, but should you still be above the limits by then,
> without a viable path forward, we will be limiting your GHA usage.
>
> Kind regards,
> Bob Thomson, on behalf of ASF Infrastructure.
>
>
> [1] https://infra.apache.org/github-actions-policy.html
> [2]
> https://infra-reports.apache.org/#ghactions&project=datafusion&hours=24&limit=15&group=name
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: Datafusion Consumption of ASF Shared GitHub-hosted Runners
Posted by Andrew Lamb <an...@gmail.com>.
Thank you -- I have filed a ticket in our issue tracker to address this
consumption[1]
[1]: https://github.com/apache/datafusion/issues/22455
On Fri, May 22, 2026 at 7:44 AM Robert Thomson <bt...@apache.org> wrote:
> Hello, Datafusion PMC.
>
> In 2024, the ASF introduced the policy for GitHub Actions usage
> across the foundation[1]. The ASF Github shared pool of
> Github-hosted runners has been at, or very close to the limit of
> 900 jobs most of the time in the past few weeks and this is the
> case again today.
>
> Your project has been identified as being among the top 5 consumers of
> build time over the past 7 days and we request that you bring your
> usage down by stream-lining long-running builds. Contact Infra for
> a consultation if you are unable to streamline your builds further.
>
> You can use the infra reporting tool[2] to monitor your GHA usage as you
> work on stream-lining, as well as locate any bottlenecks in the workflows.
>
> Infra will allow you two weeks time (till the 8th of June, 2026) to
> progress this, but should you still be above the limits by then,
> without a viable path forward, we will be limiting your GHA usage.
>
> Kind regards,
> Bob Thomson, on behalf of ASF Infrastructure.
>
>
> [1] https://infra.apache.org/github-actions-policy.html
> [2]
>
> https://infra-reports.apache.org/#ghactions&project=datafusion&hours=24&limit=15&group=name
>