You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@trafficserver.apache.org by "bryancall (via GitHub)" <gi...@apache.org> on 2026/09/30 21:12:55 UTC
[PR] Time out a branch build after 24 hours [trafficserver-ci]
bryancall opened a new pull request, #472:
URL: https://github.com/apache/trafficserver-ci/pull/472
Adds `options { timeout(time: 24, unit: 'HOURS') }` to `jenkins/branch/branch_build.pipeline`, so a hung branch build aborts itself after a day.
## Why
`branch_build` starts the child jobs (in_tree, the OS builds, autest, docs and so on) and waits for each to report back. When a child's completion never reaches the parent, as happens across a Jenkins controller restart, the parent stays "in progress" forever. `branch_build` runs one build at a time, so every later build for that branch sits in the queue with "Build #N is already in progress", and the branch gets no post-merge CI until someone notices and aborts it by hand. The pipeline has no timeout anywhere, so nothing stops it.
The two builds that were stuck until today:
| Build | Started (UTC) | Stuck for | Blocked |
|---|---|---|---|
| [master/branch_build #1403](https://ci.trafficserver.apache.org/job/master/job/branch_build/1403/) | 2026-06-29 13:54 | 93 days (2,239 h) | every master branch build since 6/29 |
| [9.2.x/branch_build #192](https://ci.trafficserver.apache.org/job/9.2.x/job/branch_build/192/) | 2026-07-27 22:06 | 65 days (1,559 h) | every 9.2.x branch build since 7/28 |
Both were aborted by hand on 9/30. Their consoles show the pipeline being resumed with "Ready to run at …" after each controller restart (7/27, 8/3, 8/14, 8/20, 9/17), with nothing after the last one. The children had long finished (for example [master/in_tree #1403](https://ci.trafficserver.apache.org/job/master/job/in_tree/1403/) and [master/cmake #944](https://ci.trafficserver.apache.org/job/master/job/cmake/944/) succeeded on 6/29), but the parent never registered it.
It is not a one-off. Earlier runs that hung until they were aborted or failed:
| Build | Ran for |
|---|---|
| [master #1401](https://ci.trafficserver.apache.org/job/master/job/branch_build/1401/) | 52 days |
| [master #1400](https://ci.trafficserver.apache.org/job/master/job/branch_build/1400/) | 48 days |
| [10.0.x #105](https://ci.trafficserver.apache.org/job/10.0.x/job/branch_build/105/) | 30 days |
| [10.2.x #1](https://ci.trafficserver.apache.org/job/10.2.x/job/branch_build/1/) | 21 days |
| [10.1.x #6](https://ci.trafficserver.apache.org/job/10.1.x/job/branch_build/6/) | 5 days |
## Why 24 hours
Runs that finish normally take 2 to 8 hours, for example [master #503](https://ci.trafficserver.apache.org/job/master/job/branch_build/503/) (2 h) and [9.2.x #163](https://ci.trafficserver.apache.org/job/9.2.x/job/branch_build/163/) (3 h). A few finished runs took longer, [9.2.x #190](https://ci.trafficserver.apache.org/job/9.2.x/job/branch_build/190/) (26 h) and [10.2.x #2](https://ci.trafficserver.apache.org/job/10.2.x/job/branch_build/2/) (69 h), probably from waiting for a `branch` executor, which a pipeline timeout also counts. 24 hours leaves room for a slow day and still caps a hang at a day instead of months. It is one number, easy to raise.
## Testing
The Jenkins declarative linter on ci.trafficserver.apache.org (`pipeline-model-converter/validate`) accepts the changed file, as it does the original.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Time out a branch build after 12 hours [trafficserver-ci]
Posted by "bneradt (via GitHub)" <gi...@apache.org>.
bneradt merged PR #472:
URL: https://github.com/apache/trafficserver-ci/pull/472
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org