You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2026/09/17 15:03:40 UTC

[PR] docs: how to check the scheduled CI runs are actually running [datafusion-comet]

andygrove opened a new pull request, #6000:
URL: https://github.com/apache/datafusion-comet/pull/6000

## Which issue does this PR close?

Relates to #5999. It does not close it: that issue is about fixing the Miri job, this documents how
to notice a job in that state in the first place.

- [Rendered page — CI guide](https://github.com/andygrove/datafusion-comet/blob/docs-scheduled-ci-health/docs/source/contributor-guide/ci.md#checking-that-the-scheduled-runs-are-healthy)
- [Rendered page — release process](https://github.com/andygrove/datafusion-comet/blob/docs-scheduled-ci-health/docs/source/contributor-guide/release_process.md#release-preparation)

## Rationale for this change

After #5939 and #5963, most of our coverage of the non-default Spark and Iceberg versions runs in
the nightly tier rather than on pull requests. That was the right trade for cost, but it moves a
lot of signal onto a schedule, and a scheduled run has no pull request to turn red. `ci.yml` has a
`nightly_report` job that opens a `ci-nightly-failure` issue, so that tier is covered. `miri.yml`
and `publish_snapshot.yml` have no such step — and both are currently broken without anyone having
noticed. Miri has failed 79 nights running (#5999) and the nightly SNAPSHOT publish has never
succeeded since it landed in #5902, failing with a 401 against repository.apache.org.

There is also a subtler failure I could not find documented anywhere. For `ci.yml`, a green
`Required Checks` does not mean the nightly suites ran. The path filters and the diff base are both
allowed to select nothing, which is correct on a documentation-only day, but a run that tested
nothing is indistinguishable from a run that tested everything if you only look at the verdict. If
the diff base ever drifts, the nightly goes green forever while testing nothing.

I checked the nightly tier itself and it is working — the 2026-09-16 and 2026-09-17 runs each
completed about 40 Spark SQL and Iceberg suite jobs, with shards running 25 to 67 minutes, and the
second run correctly resolved its diff base to the first run's head. So this PR is documentation
only; there is nothing to fix in the tier.

## What changes are included in this PR?

A new "Checking that the scheduled runs are healthy" section in the CI contributor guide, listing
the four schedules and which of them report their own failures, with a `gh api` loop over the three
daily ones and guidance on reading the output for the two distinct failures — a schedule that
stopped firing versus a streak of red nights. It also gives a second command that counts the
nightly's suite jobs by conclusion, so "green because everything passed" can be told apart from
"green because nothing ran."

The release process gets a checklist item and a paragraph under Release Preparation pointing at
that section, so this gets looked at once per release rather than only when someone happens to
wonder.

The documented commands use `gh api` rather than `gh run list --event=schedule`. The latter is not
reliable: while investigating this it omitted that morning's `ci.yml` scheduled run entirely, and
on another call returned three-week-old `miri.yml` runs as the latest three.

## How are these changes tested?

Documentation only, so no tests. Every command in the new section was run against the live
repository and produced the output the surrounding text describes. `prettier --check` and
`python3 dev/ci/check-ci-config.py` both pass.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org