You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@devlake.apache.org by GitBox <gi...@apache.org> on 2022/11/02 03:14:44 UTC
[GitHub] [incubator-devlake] Startrekzky opened a new issue, #3642: [Feature][GitHub] Provide workflow as a filter to limit the GitHub `workflow runs` to collect and improve the speed
Startrekzky opened a new issue, #3642:
URL: https://github.com/apache/incubator-devlake/issues/3642
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar feature requirement.
### Description
DevLake v0.14 collects ALL GitHub Action `workflow runs` and `jobs` by default, which makes the collection speed very slow. For example, it took 4+ hours to collect data from 'apache/incubator-devlake' by 2 tokens. However, before collecting GitHub Action data, the collection speed was about 40min by 1 token.
Therefore, we can see that there's a bottleneck in GitHub Action data collection.
### Use case
DevLake can provide workflow as a filter to limit the size of GitHub `workflow runs` to collect, because:
- from users' perspective, they don't always want to collect all CI data. They might sacrifice integrity for speed.
- from DevLake's perspective, DevLake's DORA metrics are based on `deployments` in the production environment. Other CI jobs such as linter, checker and etc. are not taken into calculation, which means they're to some extent useless.
Take 'apache/incubator-devlake' for example, around 30,000 requests will be posted to collect all CI data; while production deployments are all in the runs of one workflow 'Build-Images-Push-Docker'. By letting users to select this one single workflow before collection, the number of requests will be decreased to 57.
Below is the prototype for how to add the workflow filter.
`TBD`
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on issue #3642: [Feature][GitHub] Provide a workflow filter to limit the GitHub `workflow runs` to collect and improve the speed
Posted by GitBox <gi...@apache.org>.
klesh commented on issue #3642:
URL: https://github.com/apache/incubator-devlake/issues/3642#issuecomment-1327191663
Replaced by `Limited History` feature, close for now, feel free to reopen.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh closed issue #3642: [Feature][GitHub] Provide a workflow filter to limit the GitHub `workflow runs` to collect and improve the speed
Posted by GitBox <gi...@apache.org>.
klesh closed issue #3642: [Feature][GitHub] Provide a workflow filter to limit the GitHub `workflow runs` to collect and improve the speed
URL: https://github.com/apache/incubator-devlake/issues/3642
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org