(magpie) branch main updated: ci(prek): give the prek job four cores, drop the dead lychee cache (#1214)

[po...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/magpie.git


The following commit(s) were added to refs/heads/main by this push:
     new b36e96be ci(prek): give the prek job four cores, drop the dead lychee cache (#1214)
b36e96be is described below

commit b36e96beb434698a4121eb5803a42c0c8c1aa9a2
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Mon Sep 14 01:28:09 2026 +0630

    ci(prek): give the prek job four cores, drop the dead lychee cache (#1214)
    
    #1190 moved every job here to `ubuntu-slim` on the argument that
    nothing this repository runs in CI is heavy. That holds for ten of the
    eleven jobs, which land between 3s and 99s. It does not hold for
    `prek`, and this restores four cores for that one job only.
    
    prek splits each hook's file list across roughly `cpu_count`
    invocations, so a single core runs all ~40 hooks over ~4.9k files
    strictly serial. Measured, `Run prek` is 131s on the 4-vCPU image
    (#1190's own figure) and 183s / 211s in the two slim runs sampled; it
    is ~80% of the job's wall clock either way. The trade is three cores
    against ~60-80s on the job that every push and every pull request
    waits for.
    
    `ubuntu-slim`'s 15-minute container kill does not apply on
    `ubuntu-latest`, and no workflow here sets `timeout-minutes`, so moving
    the job off slim would silently swap that ceiling for the 6-hour
    default. `timeout-minutes: 15` keeps the property rather than dropping
    it as a side effect — roughly 4x the slowest run observed, including a
    674s outlier on a throttled host.
    
    Separately, the `Cache lychee link-check results` step was dead weight.
    `.lychee.toml` sets `offline = true`, so the hook resolves in-repo
    paths and anchors and never fetches a URL; there are no results to
    cache. The step restored 185 bytes and saved 188 on every run, and its
    `cache-lychee-${{ github.sha }}` key wrote a fresh entry per commit
    into the repository's cache budget, evicting the 251 MB prek hook-env
    cache sooner than necessary. Three comments claimed otherwise — in the
    workflow, in `.lychee.toml`, and on the lychee hook in
    `.pre-commit-config.yaml` — and now describe what actually happens.
    
    Touching `.pre-commit-config.yaml` busts the prek hook-env cache key.
    The `prek-Linux-` restore-key prefix still restores the previous
    archive and no hook definition changed, so no env is rebuilt; the next
    run re-uploads the archive once.
    
    Generated-by: Claude Opus 5
    Claude-Session: https://claude.ai/code/session_019rmycnoL6JcVWPgpyimD7S
---
 .github/workflows/pre-commit.yml | 41 ++++++++++++++++++++++++++++++----------
 .lychee.toml                     |  5 +++--
 .pre-commit-config.yaml          | 21 ++++++++++----------
 3 files changed, 45 insertions(+), 22 deletions(-)

diff --git a/.github/workflows/pre-commit.yml b/.github/workflows/pre-commit.yml
index eb3b4586..3a58b3d6 100644
--- a/.github/workflows/pre-commit.yml
+++ b/.github/workflows/pre-commit.yml
@@ -27,7 +27,29 @@ permissions: {}
 
 jobs:
   prek:
-    runs-on: ubuntu-slim
+    # The one job that runs on a 4-vCPU runner. #1190 moved every job
+    # here to `ubuntu-slim` (1 vCPU, container-based, 5 GB, killed at
+    # 15 minutes) on the argument that nothing this repo runs in CI is
+    # heavy — correct for the other ten, which land between 3s and 99s.
+    # prek is the exception: it splits each hook's file list across
+    # roughly `cpu_count` invocations, so on one core all ~40 hooks over
+    # ~4.9k files run strictly serial. Measured, `Run prek` is 131s on
+    # 4 vCPUs and 183-211s on slim, and it is ~80% of the job either
+    # way. That is the whole trade — three cores for ~60-80s on the job
+    # every push and every PR waits for. The rest of #1190's reasoning
+    # still holds and the other ten jobs stay put.
+    #
+    # Nothing in this job needs a Docker daemon, so the constraint that
+    # keeps zizmor off the `zizmor-action` (see `pyproject.toml`) is not
+    # what pins it here. Note that `ubuntu-slim`'s 15-minute container
+    # kill does not apply on `ubuntu-latest`, and no workflow here sets
+    # `timeout-minutes`, so leaving this job bare would swap that
+    # implicit ceiling for the 6-hour default. The explicit
+    # `timeout-minutes` below keeps the property #1190 relied on rather
+    # than dropping it as a side effect: 15 minutes is ~4x the slowest
+    # run observed, including a 674s outlier on a throttled host.
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
     permissions:
       contents: read
     steps:
@@ -74,15 +96,14 @@ jobs:
           key: prek-${{ runner.os }}-${{ hashFiles('.pre-commit-config.yaml') }}
           restore-keys: |
             prek-${{ runner.os }}-
-      # Restore the lychee result cache so external-URL checks reuse
-      # results across runs (`.lychee.toml` sets `max_cache_age = "7d"`).
-      - name: Cache lychee link-check results
-        uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9  # v6.1.0
-        with:
-          path: .lycheecache
-          key: cache-lychee-${{ github.sha }}
-          restore-keys: |
-            cache-lychee-
+      # There is deliberately no lychee result cache here. `.lychee.toml`
+      # sets `offline = true`, so the hook resolves only in-repo paths and
+      # anchors and never fetches a URL — there are no results to cache.
+      # The `actions/cache` step that used to sit here restored and saved
+      # a 185-byte `.lycheecache` on every run, and because its key was
+      # `cache-lychee-${{ github.sha }}` it wrote a fresh entry per commit,
+      # evicting the 251 MB prek hook-env cache above that much sooner.
+      # Reinstate it only together with turning `offline` back off.
       # Install prek via uv (rather than via the `j178/prek-action`
       # action) so the `[tool.uv] exclude-newer` cooldown in the
       # root `pyproject.toml` applies to the prek install as well.
diff --git a/.lychee.toml b/.lychee.toml
index 2abf8f5c..de1a8eb0 100644
--- a/.lychee.toml
+++ b/.lychee.toml
@@ -70,8 +70,9 @@ retry_wait_time = 2
 max_retries = 3
 
 # Cache successful results for 7 days. Mirrors the framework-wide
-# 7-day `[tool.uv] exclude-newer` cooldown convention. CI restores
-# this cache between runs to keep the link check fast.
+# 7-day `[tool.uv] exclude-newer` cooldown convention. Dormant while
+# `offline = true` — nothing is fetched, so nothing is cached, and CI
+# carries no `.lycheecache` cache step. Both would come back together.
 cache = true
 max_cache_age = "7d"
 
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index fa3444ca..4c2a4ce0 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -151,14 +151,14 @@ repos:
   # as the `lychee` hook below. That is a memory requirement, not a
   # preference: typos' peak RSS scales with the number of paths passed
   # to one invocation (~1.2 MB per path), and prek splits the file list
-  # across roughly `cpu_count` invocations. On a 4-vCPU runner the ~4.9k
-  # tracked files arrive as ~1.2k-path chunks (~925 MB each); on the
-  # 1-vCPU `ubuntu-slim` runner they arrive as a single invocation that
-  # peaks at ~5.7 GB and is OOM-killed. prek reports that as a bare
-  # `exit code: 1` with no output — a signal-killed process has no exit
-  # code of its own — which reads like a hook failure with a missing
-  # message rather than the kill it is. Walking costs ~21 MB flat,
-  # whatever the core count.
+  # across roughly `cpu_count` invocations. On the 4-vCPU runner the
+  # prek workflow now uses, the ~4.9k tracked files arrive as ~1.2k-path
+  # chunks (~925 MB each); on a 1-vCPU runner they arrive as a single
+  # invocation that peaks at ~5.7 GB and is OOM-killed. prek reports
+  # that as a bare `exit code: 1` with no output — a signal-killed
+  # process has no exit code of its own — which reads like a hook
+  # failure with a missing message rather than the kill it is. Walking
+  # costs ~21 MB flat, whatever the core count.
   #
   # typos honours .gitignore while walking, and uv writes a `*`
   # .gitignore inside `.venv`, so a synced workspace stays out of the
@@ -189,8 +189,9 @@ repos:
   # whole repo (so a renamed link target is caught no matter which
   # file references it); `files:` only gates *whether* the whole-repo
   # scan fires, i.e. it runs when any doc file changes and always on
-  # `prek run --all-files` (CI). External-URL results are cached for
-  # 7 days (`.lycheecache`, gitignored).
+  # `prek run --all-files` (CI). `.lychee.toml` sets `offline = true`, so
+  # no external URL is fetched and the `.lycheecache` result cache stays
+  # empty — CI carries no cache step for it.
   - repo: local
     hooks:
       - id: lychee