You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@devlake.apache.org by "cedriclecoz (via GitHub)" <gi...@apache.org> on 2026/09/15 11:41:51 UTC
[I] [Bug][Github] GitHub PR collection stops on 404 for stale cached pull request [devlake]
cedriclecoz opened a new issue, #9140:
URL: https://github.com/apache/devlake/issues/9140
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar issues.
### What happened
The GitHub Collect Pull Requests subtask fails completely when a previously collected pull request returns HTTP 404 from GitHub.
The collector retries `GET /repos/{org}/{repo}/pulls/{pr_number}` three times, then terminates the subtask. Subsequent scheduled collections retry the same unavailable pull request and fail again. Newer pull requests are therefore not collected, leaving repository dashboard data stale.
The affected pull request was previously present in DevLake's GitHub cache with an `open` state, but its GitHub pull-request endpoint now returns `404 Not Found`. The reason it became unavailable is unknown; it likely has been deleted or made inaccessible by GitHub on request of the author (cred leak for example).
As a workaround, after taking a DB snapshot, I deleted the verified affected row from `_tool_github_pull_requests` and its linked record in `_raw_github_api_pull_requests`. The next collection succeeded and newer pull requests appeared in dashboards.
### What do you expect to happen
A 404 for one previously known pull request should not permanently block collection of later pull requests in the repository.
The collector should handle an unavailable/stale pull request as a non-fatal condition, record or reconcile that state, continue collecting remaining pull requests, and avoid retrying the same known 404 on every scheduled run.
A supported mechanism to invalidate or reconcile one stale cached pull-request record would also be helpful.
### How to reproduce
1. Configure the GitHub connector for a repository and run collection so a pull request is cached by DevLake.
2. Make that previously collected pull request unavailable to the configured GitHub token, for example by deleting it or otherwise making GitHub return 404 for `GET /repos/{org}/{repo}/pulls/{pr_number}`. Raising a support ticket with GitHub is likely needed.
3. Run the `Collect Pull Requests` subtask.
4. Observe that DevLake retries the individual pull-request endpoint and fails the entire subtask after the retry limit.
5. Run the scheduled collection again.
6. Observe that the same pull request is retried and collection fails again, so later pull requests are not ingested.
### Anything else
Sanitized error:
subtask Collect Pull Requests ended unexpectedly
caused by: Retry exceeded 3 times calling
repos/[ORGANIZATION]/[REPOSITORY]/pulls/[PR_NUMBER].
Response:
{
"message": "Not Found",
"documentation_url": "https://docs.github.com/rest/pulls/pulls#get-a-pull-request",
"status": "404"
}
The stack trace reaches `plugins/github/tasks.CollectApiPullRequests` in `plugins/github/tasks/pr_collector.go`.
The workaround removed exactly one verified cached pull-request record and its linked raw API payload in a transaction, after an RDS snapshot. This is not a general solution and should not be required for a single unavailable pull request.
This has happened repeatedly since early July. It is unknown whether the pull request was deleted, made private/inaccessible, or removed by GitHub for another reason.
```text
attached stack trace
github.com/apache/devlake/core/runner.RunPluginSubTasks
/app/core/runner/run_task.go:341
github.com/apache/devlake/helpers/pluginhelper/api.(*WorkerScheduler).WaitAsync
/app/helpers/pluginhelper/api/worker_scheduler.go:173
github.com/apache/devlake/helpers/pluginhelper/api.(*ApiCollector).Execute
/app/helpers/pluginhelper/api/api_collector.go:206
github.com/apache/devlake/helpers/pluginhelper/api.(*StatefulApiCollector).Execute
/app/helpers/pluginhelper/api/api_collector_stateful.go:97
github.com/apache/devlake/plugins/github/tasks.CollectApiPullRequests
/app/plugins/github/tasks/pr_collector.go:140
github.com/apache/devlake/core/runner.runSubtask
/app/core/runner/run_task.go:433
github.com/apache/devlake/core/runner.RunPluginSubTasks
/app/core/runner/run_task.go:338
github.com/apache/devlake/core/runner.RunPluginTask
/app/core/runner/run_task.go:173
github.com/apache/devlake/core/runner.RunTask
/app/core/runner/run_task.go:147
github.com/apache/devlake/server/services.runTaskStandalone
/app/server/services/task_runner.go:114
github.com/apache/devlake/server/services.RunTasksStandalone.func1
/app/server/services/task.go:187
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1771
```
### Version
v1.0.3-beta17
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][Github] GitHub PR collection stops on 404 for stale cached pull request [devlake]
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh closed issue #9140: [Bug][Github] GitHub PR collection stops on 404 for stale cached pull request
URL: https://github.com/apache/devlake/issues/9140
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][Github] GitHub PR collection stops on 404 for stale cached pull request [devlake]
Posted by "klesh (via GitHub)" <gi...@apache.org>.
klesh commented on issue #9140:
URL: https://github.com/apache/devlake/issues/9140#issuecomment-5700507835
> [@klesh](https://github.com/klesh) Hey so if you allow, i would like to take this issue up, basically i have scoped the problem currently we are not handling the failure causing it to crash and retry for 3 times, in my proposed solution when we first time receive the 404 error we pass it through a custom helper which would basically tell the collector skip this item do not crash and move ahead, i would not like to mark anything in the db as closed or inactive because, metrics accuracy and i have noticed this how 404s are handled in other places like cicd_job_collector. If you think there is some other way i am happy to follow along!
I agree with you, actually the same fix had been applied to other places for more than one time.
I am happy that you are willing to pick it up. Looking forward to your PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][Github] GitHub PR collection stops on 404 for stale cached pull request [devlake]
Posted by "Thundercloud12 (via GitHub)" <gi...@apache.org>.
Thundercloud12 commented on issue #9140:
URL: https://github.com/apache/devlake/issues/9140#issuecomment-5697896119
@klesh Hey so if you allow, i would like to take this issue up, basically i have scoped the problem currently we are not handling the failure causing it to crash and retry for 3 times, in my proposed solution when we first time receive the 404 error we pass it through a custom helper which would basically tell the collector skip this item do not crash and move ahead, i would not like to mark anything in the db as closed or inactive because, metrics accuracy and i have noticed this how 404s are handled in other places like cicd_job_collector.
If you think there is some other way i am happy to follow along!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org