You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/01/08 19:31:26 UTC
[GitHub] [pulsar] vlsi opened a new issue #9154: Please reduce GitHub Actions matrix or improve the build time
vlsi opened a new issue #9154:
URL: https://github.com/apache/pulsar/issues/9154
GitHub Actions queue is very high for Apache projects, and it looks like a significant number of executors are occupied by Pulsar jobs :-(
Note: all Apache projects share the same limit of shared GitHub Actions runners, and based on the chart below Pulsar is consuming 30+ runners while the total limit (for all ASF projects) is 180.
See https://lists.apache.org/thread.html/r5303eec41cc1dfc51c15dbe44770e37369330f9644ef09813f649120%40%3Cbuilds.apache.org%3E
>number of GA workflows in/progress/queued per project and they clearly show the situation is getting worse by day:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie closed issue #9154: Please reduce GitHub Actions matrix or improve the build time
Posted by GitBox <gi...@apache.org>.
sijie closed issue #9154:
URL: https://github.com/apache/pulsar/issues/9154
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #9154: Please reduce GitHub Actions matrix or improve the build time
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #9154:
URL: https://github.com/apache/pulsar/issues/9154#issuecomment-757088852
Thanks, we will start as soon as possible.
Sijie Guo <no...@github.com>于2021年1月9日 周六04:39写道：
>
>
> @vlsi <https://github.com/vlsi> Thank you for your feedback. I will let
> @codelipenghui <https://github.com/codelipenghui> @zymap
> <https://github.com/zymap> take the actions from here.
>
>
>
>
> —
> You are receiving this because you were mentioned.
>
>
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/pulsar/issues/9154#issuecomment-756987607>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ADACIBNCGQRXRCPJ65JSCDTSY5UQ7ANCNFSM4V23YJBQ>
> .
>
>
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] vlsi edited a comment on issue #9154: Please reduce GitHub Actions matrix or improve the build time
Posted by GitBox <gi...@apache.org>.
vlsi edited a comment on issue #9154:
URL: https://github.com/apache/pulsar/issues/9154#issuecomment-756984731
I see you've removed https://github.com/apache/pulsar/pull/9069 `styfle/cancel-workflow-action` recently.
However, there's a workaround to use third-party actions (provided you have reviewed them):
https://github.com/apache/calcite/pull/2318/files
The idea is that you clone the action via SHA identifier during the build and use `locally cloned action`, so the action can't silently change after your review.
An alternative option is to raise INFRA ticket to allow `styfle/cancel-workflow-action` (and still use SHA-based action ids like `styfle/cancel-workflow-action@123123123...123123`)
WDYT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on issue #9154: Please reduce GitHub Actions matrix or improve the build time
Posted by GitBox <gi...@apache.org>.
eolivelli commented on issue #9154:
URL: https://github.com/apache/pulsar/issues/9154#issuecomment-766908675
Probably the fix here is not good
https://github.com/apache/pulsar/pull/9159
see my comments
https://github.com/apache/pulsar/actions?page=4&query=is%3Aqueued
we are still spawning too many job requests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on issue #9154: Please reduce GitHub Actions matrix or improve the build time
Posted by GitBox <gi...@apache.org>.
eolivelli commented on issue #9154:
URL: https://github.com/apache/pulsar/issues/9154#issuecomment-766908675
Probably the fix here is not good
https://github.com/apache/pulsar/pull/9159
see my comments
https://github.com/apache/pulsar/actions?page=4&query=is%3Aqueued
we are still spawning too many job requests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #9154: Please reduce GitHub Actions matrix or improve the build time
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #9154:
URL: https://github.com/apache/pulsar/issues/9154#issuecomment-756980166
@vlsi will work on this.
@codelipenghui @zymap Can you work on adding cancel-previous-run action back to our Github Actions?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #9154: Please reduce GitHub Actions matrix or improve the build time
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #9154:
URL: https://github.com/apache/pulsar/issues/9154#issuecomment-758416749
This is fixed by #9159
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] vlsi commented on issue #9154: Please reduce GitHub Actions matrix or improve the build time
Posted by GitBox <gi...@apache.org>.
vlsi commented on issue #9154:
URL: https://github.com/apache/pulsar/issues/9154#issuecomment-756983070
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] vlsi commented on issue #9154: Please reduce GitHub Actions matrix or improve the build time
Posted by GitBox <gi...@apache.org>.
vlsi commented on issue #9154:
URL: https://github.com/apache/pulsar/issues/9154#issuecomment-756984731
I see you've removed https://github.com/apache/pulsar/pull/9069 `styfle/cancel-workflow-action` recently.
However, there's a workaround to use third-party actions (provided you have reviewed them):
https://github.com/apache/calcite/pull/2318/files
The idea is that you clone the action via SHA identifier, so the action can't silently change after your review.
An alternative option is to raise INFRA ticket to allow `styfle/cancel-workflow-action` (and still use SHA-based action ids like `styfle/cancel-workflow-action@123123123...123123`)
WDYT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] vlsi commented on issue #9154: Please reduce GitHub Actions matrix or improve the build time
Posted by GitBox <gi...@apache.org>.
vlsi commented on issue #9154:
URL: https://github.com/apache/pulsar/issues/9154#issuecomment-756983070
Just in case, there's update from Jarek:
> There is one problem with the charts, They are flawed. **They show
>'workflows' not 'jobs'** and one workflow might mean many jobs :(. For
>example the big number of workflows you can see in Airflow yesterday come
>from "Label when reviewed" workflows - each of which has 1 job that
>takes 10 seconds or so. One workflow can be 20/30 times more important
>than another.
> We cannot easily drill down to jobs, because we are using Github API to get
>the information, but there are limits (max num requests/hr) and we are
>already close to hitting it with the current setup.
In any case, it would be nice if you could reduce Actions workload ;-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #9154: Please reduce GitHub Actions matrix or improve the build time
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #9154:
URL: https://github.com/apache/pulsar/issues/9154#issuecomment-756987607
@vlsi Thank you for your feedback. I will let @codelipenghui @zymap take the actions from here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #9154: Please reduce GitHub Actions matrix or improve the build time
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #9154:
URL: https://github.com/apache/pulsar/issues/9154#issuecomment-756980166
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #9154: Please reduce GitHub Actions matrix or improve the build time
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #9154:
URL: https://github.com/apache/pulsar/issues/9154#issuecomment-757088852
Thanks, we will start as soon as possible.
Sijie Guo <no...@github.com>于2021年1月9日 周六04:39写道：
>
>
> @vlsi <https://github.com/vlsi> Thank you for your feedback. I will let
> @codelipenghui <https://github.com/codelipenghui> @zymap
> <https://github.com/zymap> take the actions from here.
>
>
>
>
> —
> You are receiving this because you were mentioned.
>
>
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/pulsar/issues/9154#issuecomment-756987607>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ADACIBNCGQRXRCPJ65JSCDTSY5UQ7ANCNFSM4V23YJBQ>
> .
>
>
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] vlsi edited a comment on issue #9154: Please reduce GitHub Actions matrix or improve the build time
Posted by GitBox <gi...@apache.org>.
vlsi edited a comment on issue #9154:
URL: https://github.com/apache/pulsar/issues/9154#issuecomment-756984731
I see you've removed https://github.com/apache/pulsar/pull/9069 `styfle/cancel-workflow-action` recently.
However, there's a workaround to use third-party actions (provided you have reviewed them):
https://github.com/apache/calcite/pull/2318/files
The idea is that you clone the action via SHA identifier during the build and use `locally cloned action`, so the action can't silently change after your review.
An alternative option is to raise INFRA ticket to allow `styfle/cancel-workflow-action` (and still use SHA-based action ids like `styfle/cancel-workflow-action@123123123...123123`)
WDYT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org