You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/09/26 08:56:23 UTC
[GitHub] [tvm] lhotari opened a new issue, #12902: [CI Problem] apache/tvm consumes too much of Apache org shared GitHub Actions resources
lhotari opened a new issue, #12902:
URL: https://github.com/apache/tvm/issues/12902
Since September 14th, 2022, tvm has been consuming about 75% of the shared Apache org GitHub Actions resources
GitHub provides Apache with poor visibility into the usage metrics. The above report is generated with a solution developed by @TobKed, https://github.com/TobKed/fetch-apache-ga-stats .
When looking into the changes in twm GitHub Actions, it seems that https://github.com/apache/tvm/pull/12695 is the source of problems. Please revert the change so that the usage gets under control.
All GitHub Actions workflow maintainers in twm project should join [builds@apache.org mailing list](https://lists.apache.org/list.html?builds@apache.org) to stay aware of possible updates in this matter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] potiuk commented on issue #12902: [CI Problem] apache/tvm consumes too much of Apache org shared GitHub Actions resources
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12902:
URL: https://github.com/apache/tvm/issues/12902#issuecomment-1258508500
Down to 64% for today :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] potiuk commented on issue #12902: [CI Problem] apache/tvm consumes too much of Apache org shared GitHub Actions resources
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12902:
URL: https://github.com/apache/tvm/issues/12902#issuecomment-1258872092
BTW. Special kudos to @lhotari for flagging it and to @tobked for the dashboard we have :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhotari closed issue #12902: [CI Problem] apache/tvm consumes too much of Apache org shared GitHub Actions resources
Posted by GitBox <gi...@apache.org>.
lhotari closed issue #12902: [CI Problem] apache/tvm consumes too much of Apache org shared GitHub Actions resources
URL: https://github.com/apache/tvm/issues/12902
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] potiuk commented on issue #12902: [CI Problem] apache/tvm consumes too much of Apache org shared GitHub Actions resources
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12902:
URL: https://github.com/apache/tvm/issues/12902#issuecomment-1257727684
Oh yeah. It's not nice neither for your users nor for other apache projects and seems it just went out of control for you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] potiuk commented on issue #12902: [CI Problem] apache/tvm consumes too much of Apache org shared GitHub Actions resources
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12902:
URL: https://github.com/apache/tvm/issues/12902#issuecomment-1257957063
Thanks. I see that it already started to drop slighlty - > I see that it's down from 87 % of today's workflows to 80%
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] potiuk commented on issue #12902: [CI Problem] apache/tvm consumes too much of Apache org shared GitHub Actions resources
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12902:
URL: https://github.com/apache/tvm/issues/12902#issuecomment-1257731705
For me it looks like your comment bot triggers itself in recursive fashion :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] potiuk commented on issue #12902: [CI Problem] apache/tvm consumes too much of Apache org shared GitHub Actions resources
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12902:
URL: https://github.com/apache/tvm/issues/12902#issuecomment-1259350109
Quite spectacular:
I think we can close this one :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Mousius commented on issue #12902: [CI Problem] apache/tvm consumes too much of Apache org shared GitHub Actions resources
Posted by GitBox <gi...@apache.org>.
Mousius commented on issue #12902:
URL: https://github.com/apache/tvm/issues/12902#issuecomment-1258009414
@gigiblender / @potiuk I've started to take care of the remaining jobs 😸
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] potiuk commented on issue #12902: [CI Problem] apache/tvm consumes too much of Apache org shared GitHub Actions resources
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12902:
URL: https://github.com/apache/tvm/issues/12902#issuecomment-1258871699
Thanks for super-quick reaction @Mousius @gigiblender !
I also asked INFRA to see if we can work out some kind of escape hatch in case similar thing happens in the future (not necessarily with TVM :) ) in https://issues.apache.org/jira/browse/INFRA-23723
Comments/suggestions/learnings from your side would be great to add there
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] potiuk commented on issue #12902: [CI Problem] apache/tvm consumes too much of Apache org shared GitHub Actions resources
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12902:
URL: https://github.com/apache/tvm/issues/12902#issuecomment-1258870347
We are back in business :)
<img width="1258" alt="Screenshot 2022-09-27 at 04 05 10" src="https://user-images.githubusercontent.com/595491/192414685-a085d44c-c6bb-4a91-a998-830a40c915e4.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhotari commented on issue #12902: [CI Problem] apache/tvm consumes too much of Apache org shared GitHub Actions resources
Posted by GitBox <gi...@apache.org>.
lhotari commented on issue #12902:
URL: https://github.com/apache/tvm/issues/12902#issuecomment-1257714804
There's over 1000 workflow runs in your queue: https://github.com/apache/tvm/actions?query=is%3Aqueued and there's also 135 in progress: https://github.com/apache/tvm/actions?query=is%3Ain_progress . It seems that a lot of the ones shown in progress have been cancelled and that's probably a GitHub Actions bug ( related report https://github.com/community/community/discussions/31690 ).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] nicoloboschi commented on issue #12902: [CI Problem] apache/tvm consumes too much of Apache org shared GitHub Actions resources
Posted by GitBox <gi...@apache.org>.
nicoloboschi commented on issue #12902:
URL: https://github.com/apache/tvm/issues/12902#issuecomment-1257719169
You could use this script to cleanup your pending actions: https://github.com/nicoloboschi/pulsar-validation-tool/blob/master/pulsar-scripts/pulsar-gha/cancel-workflows.js
We used it in the Apache Pulsar community.
The GH bug put those runs in a weird state
>run.status === "in_progress" && run.conclusion === "failure"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] driazati commented on issue #12902: [CI Problem] apache/tvm consumes too much of Apache org shared GitHub Actions resources
Posted by GitBox <gi...@apache.org>.
driazati commented on issue #12902:
URL: https://github.com/apache/tvm/issues/12902#issuecomment-1258284106
Thanks everyone for flagging this and disabling the job quickly!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] potiuk commented on issue #12902: [CI Problem] apache/tvm consumes too much of Apache org shared GitHub Actions resources
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12902:
URL: https://github.com/apache/tvm/issues/12902#issuecomment-1257796206
This one has started to heavily affect other problems (the "fairness" algorithms of GitHub likely get a very tough time with the project taking already almost 90% of the queueu @apache/tvm-committers (sorry for spam) -> i created the infra issue in the ASF asking to disable the workflow https://issues.apache.org/jira/browse/INFRA-23723 - not sure if they will/can do it but It would be great if you do it in a controlled way and cancel all the running workflow because it will start really nasty.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] gigiblender commented on issue #12902: [CI Problem] apache/tvm consumes too much of Apache org shared GitHub Actions resources
Posted by GitBox <gi...@apache.org>.
gigiblender commented on issue #12902:
URL: https://github.com/apache/tvm/issues/12902#issuecomment-1257857648
Thank you for raising this, and we sincerely apologize for causing this inconvenience.
#12903 should disable the bot. Regarding skipping the already queued/in-progress runs, I don't have the GH permissions to do this and will have to wait a couple of hours for someone else (possibly in a different time zone) to do it. @driazati @areusch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org