You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tooling.apache.org by gs...@apache.org on 2026/10/16 05:46:24 UTC
(tooling-llmao) 02/02: hosting/runpod: an image that provisions once, then runs supervisord
This is an automated email from the ASF dual-hosted git repository.
gstein pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tooling-llmao.git
commit 0920aed604b713b257d63ab578e200c291346055
Author: Greg Stein <gs...@gmail.com>
AuthorDate: Fri Oct 16 00:45:49 2026 -0500
hosting/runpod: an image that provisions once, then runs supervisord
RunPod pods run our image: vllm/vllm-openai:v0.25.1 plus supervisor,
with runpod/provision.sh as the entrypoint. The installer is not baked
in. provision.sh fetches the shared install_set.py on first boot, so a
change to it needs no new image.
A RunPod restart keeps only the volume, so the provisioned state lives
there: /workspace/.llmao holds the units (conf.d), the installer that
ran, and a `provisioned` marker written only after the units are in.
- Until provisioned, each boot fetches the installer and runs it with
--no-reload, retrying with backoff (15s doubling to 5 min). The usual
first failure is a pod created before its fleet.hosts row; retrying
means adding the row is enough.
- Once provisioned, a boot fetches nothing, not GitHub and not
/vllm/config. To change the set, delete the marker and restart.
- supervisord.conf, its socket and its pidfile are rewritten on every
boot off the volume, so a stale socket from an unclean stop cannot
trip the next start. supervisord then runs as PID 1 and restarts a
vLLM server that dies without touching the others.
- LLMAO_HOST comes from RUNPOD_PUBLIC_IP; a pod without a public IP
refuses to start.
Tests run the real script under bash with curl and supervisord stubbed:
first boot, a provisioned boot fetching nothing, retries after failed
fetches and a failing installer, and a pod with no public IP.
Not yet built or run on a pod. runpod/README.md, now a runbook, lists
what the first pod has to confirm.
Co-Authored-By: Claude Opus 5.5 <no...@anthropic.com>
---
README.md | 2 +-
hosting/README.md | 24 ++++--
hosting/runpod/Dockerfile | 41 +++++++++
hosting/runpod/README.md | 185 ++++++++++++++++++++++++-----------------
hosting/runpod/provision.sh | 115 +++++++++++++++++++++++++
tests/test_runpod_provision.py | 152 +++++++++++++++++++++++++++++++++
6 files changed, 432 insertions(+), 87 deletions(-)
diff --git a/README.md b/README.md
index 3a70e5e..84f1af1 100644
--- a/README.md
+++ b/README.md
@@ -232,6 +232,6 @@ certs/ mkcert PEMs + README
llmao/ seam, auth, models, fleet, proxy/ (LiteLLM client)
hosting/install_set.py GPU-box installer, shared by every provider
hosting/vast/ provision.sh (Vast on-create)
-hosting/runpod/ RunPod template notes (image not built yet)
+hosting/runpod/ Dockerfile + provision.sh (RunPod entrypoint)
tests/ offline seam, fleet, hosting installer
```
diff --git a/hosting/README.md b/hosting/README.md
index 52616ad..091f554 100644
--- a/hosting/README.md
+++ b/hosting/README.md
@@ -1,7 +1,7 @@
# GPU-box hosting
-Artifacts that run **on** a GPU instance (Vast.ai now; RunPod next). They
-are not part of the asfquart process.
+Artifacts that run **on** a GPU instance (Vast.ai and RunPod). They are
+not part of the asfquart process.
**Design (why):** [`docs/vllm-fleet-design.md`](../docs/vllm-fleet-design.md).
@@ -20,7 +20,9 @@ write Supervisor units; there is no long-lived Python launcher.
| `install_set.py` | Shared by every provider: GET JSON, write a `vllm-*.conf` per server, `supervisorctl update` (or not, with `--no-reload`) |
| `vast/provision.sh` | Vast on-create: set `LLMAO_HOST`, curl the installer from GitHub, run it |
| `vast/README.md` | Vast.ai runbook |
-| `runpod/README.md` | RunPod runbook (image not built yet) |
+| `runpod/Dockerfile` | RunPod image: stock vLLM plus supervisord and `runpod/provision.sh` |
+| `runpod/provision.sh` | RunPod entrypoint: provision once (retrying), then `exec supervisord` |
+| `runpod/README.md` | RunPod runbook |
## Box environment
@@ -32,19 +34,22 @@ Required:
and logs are `$DATA_DIRECTORY/hf-cache` and `.../logs`, not in the config JSON.
- `LLMAO_HOST` — the box's public IP, sent as `X-LLMAO-Host`. Each provider's
`provision.sh` sets it from that provider's own variable (Vast:
- `PUBLIC_IPADDR`), so `install_set.py` knows no provider's names
+ `PUBLIC_IPADDR`; RunPod: `RUNPOD_PUBLIC_IP`), so `install_set.py` knows
+ no provider's names
- `PROVISIONING_SCRIPT` — Vast only: raw URL of `provision.sh`
## Boot
```text
Vast: PROVISIONING_SCRIPT → provision.sh → curl install_set.py → GET JSON → supervisor units → vllm serve
-RunPod: image entrypoint → COPY'd installer → GET JSON → supervisor units → vllm serve
+RunPod: image entrypoint provision.sh → curl install_set.py → GET JSON → units on the volume → exec supervisord → vllm serve
+ (once provisioned, a boot skips straight to exec supervisord)
```
-Vast still curls `install_set.py` from `refs/heads/main`. Pin that URL to a
-**commit SHA** before treating instance-create as production-safe. RunPod
-pins by **image digest**.
+Both curl `install_set.py` from `refs/heads/main`. Pin that URL to a
+**commit SHA** before treating instance-create as production-safe: a broken
+`main` would stop every new box from provisioning. The RunPod image itself
+is pinned by its tag or digest in the template.
### Vast `PROVISIONING_SCRIPT` vs `install_set.py`
@@ -66,7 +71,8 @@ throwaway instance proves execute is shebang (`./provisioning.sh`) rather
than `bash /provisioning.sh`. Docs and the saved `.sh` name assume bash.
`install_set.py` has no shebang today (ASF license first).
-RunPod’s image COPY’s the installer; it does not use `provision.sh`.
+RunPod has its own `runpod/provision.sh`, baked into the image as the
+entrypoint; see [`runpod/README.md`](runpod/README.md).
## Fit check
diff --git a/hosting/runpod/Dockerfile b/hosting/runpod/Dockerfile
new file mode 100644
index 0000000..77386ce
--- /dev/null
+++ b/hosting/runpod/Dockerfile
@@ -0,0 +1,41 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# The RunPod box: stock vLLM, plus supervisord and the boot script.
+#
+# Deliberately thin. The installer is NOT here: provision.sh fetches it on
+# first boot, so a change to it needs no new image. Nor are weights; they
+# go to the volume. This image changes only when provision.sh or the vLLM
+# version does.
+#
+# docker build -t <registry>/llmao-vllm-box:<tag> hosting/runpod
+
+# A pinned version, not :latest -- every pod created from the template
+# would otherwise get whatever vLLM was newest that day.
+FROM vllm/vllm-openai:v0.25.1
+
+# supervisord is the one thing the stock image lacks: it restarts a vLLM
+# server that dies without disturbing the others on the box.
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends supervisor \
+ && rm -rf /var/lib/apt/lists/*
+
+COPY provision.sh /opt/llmao/provision.sh
+
+# Replaces the stock ENTRYPOINT ["vllm", "serve"]. The template's start
+# command must be left empty: anything there arrives as arguments to this.
+ENTRYPOINT ["/opt/llmao/provision.sh"]
diff --git a/hosting/runpod/README.md b/hosting/runpod/README.md
index 7c18f8e..fcbba01 100644
--- a/hosting/runpod/README.md
+++ b/hosting/runpod/README.md
@@ -1,80 +1,111 @@
-# RunPod runbook (knowledge; image not built yet)
-
-RunPod templates pull a **Docker image**. Vast uses a stock vLLM template
-and curls `install_set.py` at create. Same control plane, different
-artifact.
-
-Nothing in this directory is a running image yet. Registry location is a
-placeholder until Infra says where to push. The notes below are what the
-image and `fleet.hosts` row must satisfy.
-
-## Same as Vast
-
-- `GET /vllm/config` by **client IP**, Bearer `FLEET_KEY`.
-- JSON listen ports only. The box never sees the public mapping.
-- Template env: `FLEET_KEY`, `ASFQUART_URL=https://llm.apache.org`,
- `DATA_DIRECTORY` (typically `/workspace`). TLS verify is on; there is
- no skip flag.
-- **Supervisord** runs one `vllm serve` per `models.yaml` row (several listens
- in one pod). That is the usual shape for a GPU *pod*, not a one-process
- web container, and it matches Vast.
-- Weights and logs live on the volume (`$DATA_DIRECTORY/hf-cache`,
- `.../logs`), not in the image.
-- Pre-pull fit check: `models.yaml` `vram_gb` / `disk_gb` against the card and
- `statvfs`. That is **not** vLLM: vLLM only sizes KV after the weights
- are on disk (`torch` / HIP `mem_get_info`). NVIDIA probe today is
- `nvidia-smi`; missing probe is not a failure. AMD (`rocm-smi`) when we
- have a ROCm image. VRAM sums over the pod's servers; disk counts once per
- distinct model.
-- The same installer divides the card between the pod's servers; see
- [`../README.md`](../README.md) (sizing).
-
-## Different from Vast
-
-| | Vast | RunPod |
-|--|------|--------|
-| Image | stock `vastai/vllm` (or similar) | **our** image (`IMAGE_REGISTRY_PLACEHOLDER/llmao-vllm-box`) |
-| Installer | curl `install_set.py` from GitHub **main** | **COPY** in the image (boot must not depend on GitHub) |
-| Pin | later: curl a **commit SHA** | image **digest** |
-| Public port | Vast API HostPort (`vast_client`) | see Ports below |
-
-Pin the Vast curl before create-instance is treated as production-safe: a
-broken `main` would brick every new box. The image digest is the same
-kind of pin for RunPod.
-
-## Ports (open; 4th host-row slot is a stopgap)
-
-`fleet.hosts` already accepts `[model, listen, name|null, public]`. That
-was added so a RunPod box could get an `api_base` at all: RunPod does not
-expose a Vast-style HostPort to us today, and it **reassigns** the TCP
-public port on every pod recreate.
-
-We do **not** want operators editing that fourth number forever. Once
-self-deploy is formalized (create from template, then read the mapping),
-prefer one of:
-
-1. **HTTP proxy (stable URL).** Template exposes `N/http`. Public origin
- is `https://<podId>-<listen>.proxy.runpod.net`. The listen port is in
- the hostname, so recreate does not need a new fourth slot. Only use
- this as `api_base` after we prove LiteLLM and long generations survive
- that proxy (timeouts, TLS, HTTP/1.1).
-2. **Direct TCP + API.** Template exposes `N/tcp`. Public port changes.
- RunPod **REST v2** `GET /pods/{id}` → `portMappings` can fill
- `public_port` the way `vast_client` fills HostPort. GraphQL exists but
- is deprecated (~early 2027); new code should be REST.
-
-Until one of those is wired, keep the fourth slot. It may go away.
-
-The template must still **expose every listen port** the set uses (max 10
-HTTP ports on RunPod). Control plane JSON stays listen-only.
-
-## Image sketch (next, not this note)
-
-`FROM vllm/vllm-openai:…` (pin a digest later), install supervisord,
-COPY the shared installer, `CMD`/`ENTRYPOINT` that fetches config, writes
-units, `exec supervisord -n`. No baked weights.
-
-## Smoke (once a pod exists)
+# RunPod runbook
+
+RunPod pods run **our** image: stock vLLM plus supervisord and
+[`provision.sh`](provision.sh) as the entrypoint. The installer is the shared
+[`../install_set.py`](../install_set.py), fetched on first boot rather than
+baked in, so changing it needs no new image.
+
+Not yet run on a real pod. What is untested is listed at the end.
+
+## Image
+
+```bash
+docker build -t <registry>/llmao-vllm-box:<tag> hosting/runpod
+docker push <registry>/llmao-vllm-box:<tag>
+```
+
+The registry is still to be settled with Infra. Rebuild only when
+`provision.sh` or the vLLM version in the `Dockerfile` changes. The vLLM
+version is pinned (`v0.25.1`), not `:latest`, so every pod created from the
+template runs the same engine.
+
+## Template
+
+- **Image:** `<registry>/llmao-vllm-box:<tag>`
+- **Start command: empty.** The image's entrypoint is `provision.sh`; anything
+ here arrives as its arguments, which it ignores with a warning.
+- **Volume** mounted at `/workspace`, large enough for the HF cache.
+- **Public IP** required: `provision.sh` refuses to start without
+ `RUNPOD_PUBLIC_IP`, because it is how the box says which host it is.
+- **Expose** every listen port the box's `fleet.hosts` rows use (RunPod
+ allows at most 10 HTTP ports); see Ports below.
+- **Environment:**
+
+ ```bash
+ FLEET_KEY=<shared-secret>
+ ASFQUART_URL=https://llm.apache.org
+ # Optional:
+ # DATA_DIRECTORY=/workspace (default)
+ # LLMAO_INSTALLER_URL=<raw URL> pin install_set.py to a commit SHA
+ ```
+
+Add the pod's `RUNPOD_PUBLIC_IP` to `fleet.hosts` in `config.yaml`. Order does
+not matter: a pod created before its row retries until llmao knows it.
+
+## Boot
+
+On every start, [`provision.sh`](provision.sh):
+
+1. If `$DATA_DIRECTORY/.llmao/provisioned` is missing: clears
+ `.llmao/conf.d`, fetches `install_set.py`, and runs it with `--no-reload`
+ to write one Supervisor unit per server into `.llmao/conf.d`. On any
+ failure it retries, waiting 15s and doubling to at most 5 min. It writes
+ `provisioned` only after the units are in place.
+2. Writes `/etc/supervisor/supervisord.conf` (socket and pidfile under
+ `/run/llmao`), including `.llmao/conf.d/*.conf`.
+3. `exec supervisord -n`: supervisord is PID 1 and restarts any `vllm serve`
+ that dies, without touching the others.
+
+A restart keeps only the volume, so everything that must survive is there:
+the units, the marker, the installer that ran, weights (`hf-cache`) and logs.
+Once provisioned, a boot fetches nothing: not GitHub, not `/vllm/config`.
+
+Retrying covers a "will not fit" refusal too. It will not succeed until
+`models.yaml` or `fleet.hosts` changes, and the pod log says why on each try.
+
+## Changing a pod's set
+
+The set is frozen at provisioning. To change it:
+
+```bash
+rm /workspace/.llmao/provisioned
+```
+
+then restart the pod. The next boot clears the old units and provisions
+afresh. Stopping the servers first is not needed: a restart stops them.
+
+## Ports (open)
+
+`fleet.hosts` accepts `[model, listen, name|null, public]`. The fourth slot
+exists because RunPod does not give us a Vast-style HostPort lookup, and it
+**reassigns** the public TCP port on every pod recreate. Operators should not
+have to keep editing that number. Once creating pods is automated, prefer one
+of:
+
+1. **HTTP proxy (stable URL).** Expose `N/http`; the public origin is
+ `https://<podId>-<listen>.proxy.runpod.net`. Recreate does not change it.
+ Use it as `api_base` only after proving LiteLLM and long generations
+ survive that proxy (timeouts, TLS, HTTP/1.1).
+2. **Direct TCP + API.** Expose `N/tcp`; RunPod REST v2 `GET /pods/{id}` →
+ `portMappings` can fill `public_port` the way `vast_client` fills HostPort.
+ GraphQL is deprecated; new code should use REST.
+
+Until one of those is wired, keep the fourth slot.
+
+## Smoke
1. `supervisorctl status`
2. `curl -sS -H "Authorization: Bearer <api_key>" http://127.0.0.1:<listen>/v1/models`
+3. `$DATA_DIRECTORY/logs/<name>.log`: `Model loading took … GiB` (compare with
+ `vram_gb`). The boot itself logs to the pod's container log, prefixed
+ `llmao-provision:`.
+
+## Untested until the first pod
+
+- The image has not been built. The `Dockerfile` assumes the vLLM image runs
+ as root and has `curl` and `python3`, as its upstream Dockerfile shows.
+- That the IP the pod reaches llm.apache.org **from** matches
+ `RUNPOD_PUBLIC_IP`. llmao looks the box up by `X-LLMAO-Host`, so it need not,
+ but a proxy that strips headers would break the lookup.
+- RunPod's handling of a container whose start command is empty when the
+ image sets its own `ENTRYPOINT`.
diff --git a/hosting/runpod/provision.sh b/hosting/runpod/provision.sh
new file mode 100755
index 0000000..5ab2fcb
--- /dev/null
+++ b/hosting/runpod/provision.sh
@@ -0,0 +1,115 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# RunPod container entrypoint: provision once, then run supervisord as PID 1.
+#
+# A RunPod restart keeps only the volume ($DATA_DIRECTORY, /workspace by
+# default); everything else in the container is reset. So the provisioned
+# state lives on the volume:
+#
+# $DATA_DIRECTORY/.llmao/install_set.py the installer that ran
+# $DATA_DIRECTORY/.llmao/conf.d/*.conf one Supervisor unit per server
+# $DATA_DIRECTORY/.llmao/provisioned written last, once the units are in
+#
+# Until `provisioned` exists, every boot fetches the installer and runs it,
+# retrying until it succeeds. The usual first failure is not GitHub: it is a
+# pod created before its fleet.hosts row, which llmao refuses until the row
+# is added. Once provisioned, a boot fetches nothing -- not the installer,
+# not /vllm/config. To change the set, delete `provisioned` and restart.
+#
+# supervisord's own config, socket and pidfile are NOT on the volume. They
+# are rewritten on every boot, so a stale socket left by an unclean stop
+# cannot trip the next start.
+
+set -euo pipefail
+
+log() { echo "llmao-provision: $*" >&2; }
+
+if [ "$#" -gt 0 ]; then
+ log "ignoring start-command arguments ($*); leave the template's start command empty"
+fi
+
+: "${FLEET_KEY:?FLEET_KEY must be set by the RunPod template}"
+: "${ASFQUART_URL:?ASFQUART_URL must be set by the RunPod template}"
+: "${RUNPOD_PUBLIC_IP:?RUNPOD_PUBLIC_IP is unset: the pod has no public IP, so it cannot say which host it is}"
+
+export DATA_DIRECTORY="${DATA_DIRECTORY:-/workspace}"
+# The installer is shared between providers and knows none of their names.
+export LLMAO_HOST="$RUNPOD_PUBLIC_IP"
+
+INSTALLER_URL="${LLMAO_INSTALLER_URL:-https://raw.githubusercontent.com/apache/tooling-llmao/refs/heads/main/hosting/install_set.py}"
+STATE="$DATA_DIRECTORY/.llmao"
+CONF_D="$STATE/conf.d"
+SUPERVISORD_CONF="${LLMAO_SUPERVISORD_CONF:-/etc/supervisor/supervisord.conf}"
+RUN_DIR="${LLMAO_RUN_DIR:-/run/llmao}"
+RETRY_FIRST_S="${LLMAO_RETRY_FIRST_S:-15}"
+RETRY_MAX_S="${LLMAO_RETRY_MAX_S:-300}"
+
+provision_once() {
+ # A half-finished earlier attempt leaves units that may not match the set;
+ # start from nothing so a server dropped from the set does not linger.
+ rm -rf "$CONF_D"
+ mkdir -p "$CONF_D"
+ curl -fsSL --max-time 60 -o "$STATE/install_set.py.new" "$INSTALLER_URL" || return 1
+ mv "$STATE/install_set.py.new" "$STATE/install_set.py"
+ SUPERVISOR_CONF_DIR="$CONF_D" VLLM_BIN="$(command -v vllm || echo vllm)" \
+ python3 "$STATE/install_set.py" --no-reload || return 1
+}
+
+mkdir -p "$STATE"
+if [ -f "$STATE/provisioned" ]; then
+ log "already provisioned; fetching nothing"
+else
+ delay="$RETRY_FIRST_S"
+ until provision_once; do
+ log "provisioning failed; retrying in ${delay}s"
+ sleep "$delay"
+ delay=$(( delay * 2 ))
+ if [ "$delay" -gt "$RETRY_MAX_S" ]; then
+ delay="$RETRY_MAX_S"
+ fi
+ done
+ date -u +%Y-%m-%dT%H:%M:%SZ > "$STATE/provisioned"
+ log "provisioned"
+fi
+
+mkdir -p "$RUN_DIR" "$(dirname "$SUPERVISORD_CONF")"
+cat > "$SUPERVISORD_CONF" <<EOF
+; Written by llmao provision.sh on every boot; edits do not survive a restart.
+[unix_http_server]
+file=$RUN_DIR/supervisor.sock
+chmod=0700
+
+[supervisord]
+nodaemon=true
+user=root
+pidfile=$RUN_DIR/supervisord.pid
+logfile=$DATA_DIRECTORY/logs/supervisord.log
+
+[rpcinterface:supervisor]
+supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
+
+[supervisorctl]
+serverurl=unix://$RUN_DIR/supervisor.sock
+
+[include]
+files = $CONF_D/*.conf
+EOF
+mkdir -p "$DATA_DIRECTORY/logs"
+
+exec supervisord -n -c "$SUPERVISORD_CONF"
diff --git a/tests/test_runpod_provision.py b/tests/test_runpod_provision.py
new file mode 100644
index 0000000..4baad3b
--- /dev/null
+++ b/tests/test_runpod_provision.py
@@ -0,0 +1,152 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+
+"""hosting/runpod/provision.sh under bash, with curl and supervisord stubbed.
+
+The installer it fetches is a stand-in that writes one unit, so these test
+the boot script's own job: provision once, retry until it works, and fetch
+nothing once provisioned.
+"""
+
+import os
+import shutil
+import subprocess
+from pathlib import Path
+
+import pytest
+
+SCRIPT = Path(__file__).resolve().parents[1] / "hosting" / "runpod" / "provision.sh"
+
+# Stands in for install_set.py: proves what it was handed, writes one unit.
+FAKE_INSTALLER = """\
+import os, sys
+assert sys.argv[1:] == ["--no-reload"], sys.argv
+assert os.environ["LLMAO_HOST"] == "198.51.100.8"
+conf = os.environ["SUPERVISOR_CONF_DIR"]
+open(os.path.join(conf, "vllm-model-a.conf"), "w").write("[program:vllm-model-a]\\n")
+"""
+
+# Fails the first $STUB_CURL_FAILS calls, then "downloads" the installer.
+CURL = """\
+#!/bin/bash
+n=$(( $(cat "$STUB_DIR/curl.calls" 2>/dev/null || echo 0) + 1 ))
+echo "$n" > "$STUB_DIR/curl.calls"
+[ "$n" -le "${STUB_CURL_FAILS:-0}" ] && exit 22
+while [ $# -gt 0 ]; do [ "$1" = -o ] && out="$2"; shift; done
+cp "$STUB_DIR/installer.py" "$out"
+"""
+
+SUPERVISORD = """\
+#!/bin/bash
+echo "$*" > "$STUB_DIR/supervisord.args"
+"""
+
+
+@pytest.fixture
+def box(tmp_path):
+ if shutil.which("bash") is None:
+ pytest.skip("no bash")
+ stubs = tmp_path / "stubs"
+ stubs.mkdir()
+ for name, text in (("curl", CURL), ("supervisord", SUPERVISORD)):
+ (stubs / name).write_text(text)
+ (stubs / name).chmod(0o755)
+ (stubs / "installer.py").write_text(FAKE_INSTALLER)
+ workspace = tmp_path / "workspace"
+ env = {
+ "PATH": f"{stubs}:{os.environ['PATH']}",
+ "STUB_DIR": str(stubs),
+ "FLEET_KEY": "fleet-secret",
+ "ASFQUART_URL": "https://llm.example.org",
+ "RUNPOD_PUBLIC_IP": "198.51.100.8",
+ "DATA_DIRECTORY": str(workspace),
+ "LLMAO_SUPERVISORD_CONF": str(tmp_path / "etc" / "supervisord.conf"),
+ "LLMAO_RUN_DIR": str(tmp_path / "run"),
+ "LLMAO_RETRY_FIRST_S": "0",
+ }
+
+ def boot(**extra):
+ return subprocess.run(["bash", str(SCRIPT)], env={**env, **extra}, capture_output=True, text=True, timeout=30)
+
+ return boot, stubs, workspace, tmp_path
+
+
+def _calls(stubs):
+ return int((stubs / "curl.calls").read_text()) if (stubs / "curl.calls").exists() else 0
+
+
+def test_first_boot_provisions_and_execs_supervisord(box):
+ boot, stubs, workspace, tmp_path = box
+ result = boot()
+ assert result.returncode == 0, result.stderr
+ state = workspace / ".llmao"
+ assert (state / "provisioned").exists()
+ assert (state / "conf.d" / "vllm-model-a.conf").exists()
+ conf = tmp_path / "etc" / "supervisord.conf"
+ assert (stubs / "supervisord.args").read_text().split() == ["-n", "-c", str(conf)]
+ text = conf.read_text()
+ assert f"files = {state / 'conf.d'}/*.conf" in text
+ # The socket is on the container disk, never the volume.
+ assert f"file={tmp_path / 'run'}/supervisor.sock" in text
+
+
+def test_provisioned_boot_fetches_nothing(box):
+ boot, stubs, workspace, _ = box
+ assert boot().returncode == 0
+ assert _calls(stubs) == 1
+ result = boot()
+ assert result.returncode == 0, result.stderr
+ assert _calls(stubs) == 1
+ assert "already provisioned" in result.stderr
+ # The units survive: a restart serves the same set.
+ assert (workspace / ".llmao" / "conf.d" / "vllm-model-a.conf").exists()
+
+
+def test_retries_until_the_fetch_succeeds(box):
+ boot, stubs, workspace, _ = box
+ result = boot(STUB_CURL_FAILS="2")
+ assert result.returncode == 0, result.stderr
+ assert _calls(stubs) == 3
+ assert result.stderr.count("provisioning failed") == 2
+ assert (workspace / ".llmao" / "provisioned").exists()
+
+
+def test_a_failing_installer_is_retried_and_leaves_no_marker_until_it_works(box):
+ boot, stubs, workspace, _ = box
+ good = (stubs / "installer.py").read_text()
+ # Fails on its first run, as /vllm/config does before the fleet.hosts row exists.
+ (stubs / "installer.py").write_text(
+ "import os, sys\n"
+ "flag = os.path.join(os.environ['STUB_DIR'], 'ran')\n"
+ "if not os.path.exists(flag):\n"
+ " open(flag, 'w').close()\n"
+ " sys.exit('config fetch HTTP 403')\n" + good
+ )
+ result = boot()
+ assert result.returncode == 0, result.stderr
+ assert result.stderr.count("provisioning failed") == 1
+ assert (workspace / ".llmao" / "provisioned").exists()
+
+
+def test_no_public_ip_refuses_to_start(box):
+ boot, stubs, workspace, _ = box
+ result = boot(RUNPOD_PUBLIC_IP="")
+ assert result.returncode != 0
+ assert "RUNPOD_PUBLIC_IP" in result.stderr
+ assert not (workspace / ".llmao" / "provisioned").exists()
+ assert _calls(stubs) == 0
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For additional commands, e-mail: commits-help@tooling.apache.org