[GitHub] [incubator-devlake] hezyin opened a new issue, #4210: [Feature][GitLab] Apply different rate limits to subtasks

[GitBox <gi...@apache.org>]

hezyin opened a new issue, #4210:
URL: https://github.com/apache/incubator-devlake/issues/4210

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar feature requirement.
   
   
   ### Use case
   
   Right now the default rate limit for GitLab is capped at 10k requests per hour.
   
   The reason is that GitLab Cloud’s most APIs are fast and can take 100k+ requests per hour, but GitLab’s pipeline API has some performance issues and can only take 10k requests per hour from our tests. If we set the default value higher than 10k, then the GitLab task might fail for users who would like to collect pipelines data from GitLab. So we decided to sacrifice the collection speed to avoid exceeding pipeline API’s rate limit.
   
   Ideally, we'd want to achieve 100k requests per hour collection speed for all GitLab entities except pipelines and only slow down for pipelines.
   
   ### Description
   
   See above
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-devlake] klesh closed issue #4210: [Feature][GitLab] Apply different rate limits to subtasks

["klesh (via GitHub)" <gi...@apache.org>]

klesh closed issue #4210: [Feature][GitLab] Apply different rate limits to subtasks
URL: https://github.com/apache/incubator-devlake/issues/4210


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org