(tooling-llmao) branch main updated: docs: Vast PROVISIONING_SCRIPT vs install_set.py lives in hosting/

[gs...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gstein pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tooling-llmao.git


The following commit(s) were added to refs/heads/main by this push:
     new fc59bf8  docs: Vast PROVISIONING_SCRIPT vs install_set.py lives in hosting/
fc59bf8 is described below

commit fc59bf876333ae627e94dcbafad1d6fea4f71fa7
Author: Greg Stein <gs...@gmail.com>
AuthorDate: Sun Sep 20 05:17:20 2026 -0500

    docs: Vast PROVISIONING_SCRIPT vs install_set.py lives in hosting/
    
    The template fetches provision.sh (saved as /provisioning.sh, documented
    as a shell script). The stock container has no repo, so that wrapper
    curls install_set.py. mkdir could be Python; that is not why the .sh
    exists. Pointing PROVISIONING_SCRIPT at the .py is unverified (shebang
    vs bash). Record that on hosting/README.md, not as a novel in
    ARCHITECTURE.
    
    ARCHITECTURE GPU fleet is the control plane plus a pointer at hosting/.
    vast/README: set PROVISIONING_SCRIPT to provision.sh, not install_set.py.
    
    Co-authored-by: Grok <gr...@x.ai>
---
 docs/ARCHITECTURE.md   | 30 +++++++++++++++++-------------
 hosting/README.md      | 38 ++++++++++++++++++++++++++++++++------
 hosting/vast/README.md |  9 ++++++---
 3 files changed, 55 insertions(+), 22 deletions(-)

diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
index 8da12fe..91bb5b8 100644
--- a/docs/ARCHITECTURE.md
+++ b/docs/ARCHITECTURE.md
@@ -57,22 +57,26 @@ keys are team-scoped exceptions (who may create them is an **open RAI
 policy** question — see design + `docs/STATUS.md`). Secrets shown once;
 metadata in LiteLLM.
 
-**GPU fleet (vLLM on Vast; RunPod next):** `docs/vllm-fleet-design.md`.
-`hosting/runpod/README.md` is the template/image contract (not built yet).
-`APP.fleet` (`llmao/fleet.py`) is built at startup. `GET /vllm/config` (Bearer
+**GPU fleet (control plane):** `docs/vllm-fleet-design.md`. `APP.fleet`
+(`llmao/fleet.py`) is built at startup. `GET /vllm/config` (Bearer
 `fleet.key`) maps `X-LLMAO-Host` (Vast `PUBLIC_IPADDR`) or
 `X-Forwarded-For` / peer to `fleet.hosts`.
 JSON `{host, servers[]}` — `servers[].port` is the **container listen** port
-(not Vast's public HostPort). No `hf_home`/`log_dir`. Vast
-`hosting/vast/install_set.py` writes Supervisor units. Lifecycle/skew:
-`app.add_runner` (`fleet-lifecycle`, `litellm-skew`). After probes,
-`Fleet.after_probe` POSTs `/model/new` / `/model/delete`. `Fleet.models` is
-the YAML recipe. `FleetDeployment` is one intended LiteLLM backend (self-host
-or commercial). `VllmServer` states: `pending` / `starting` / `serving` /
-`down`. `/fleet` shows serving only when vLLM is up **and** LiteLLM has a
-deployment. Host:port and LiteLLM UI are site-admin.
-JSON handlers use `@api` in `api.py`. Do not wrap Quart `asgi_app` with
-Werkzeug ProxyFix.
+(not Vast's public HostPort). Lifecycle/skew: `app.add_runner`
+(`fleet-lifecycle`, `litellm-skew`). After probes, `Fleet.after_probe`
+POSTs `/model/new` / `/model/delete`. `Fleet.models` is the YAML recipe.
+`FleetDeployment` is one intended LiteLLM backend. `VllmServer` states:
+`pending` / `starting` / `serving` / `down`. `/fleet` shows serving only
+when vLLM is up **and** LiteLLM has a deployment. Host:port and LiteLLM UI
+are site-admin. JSON handlers use `@api` in `api.py`. Do not wrap Quart
+`asgi_app` with Werkzeug ProxyFix.
+
+**GPU boxes are not this process.** Vast: custom template,
+`PROVISIONING_SCRIPT` → `provision.sh` → curl `install_set.py` (stock
+container). RunPod: later, COPY in our image. Detail:
+[`hosting/README.md`](../hosting/README.md). Runbook:
+[`hosting/vast/README.md`](../hosting/vast/README.md);
+[`hosting/runpod/README.md`](../hosting/runpod/README.md) (image not built).
 
 Build status and backlog: **`docs/STATUS.md`**.
 
diff --git a/hosting/README.md b/hosting/README.md
index 108ef1f..2b0073b 100644
--- a/hosting/README.md
+++ b/hosting/README.md
@@ -6,8 +6,9 @@ are not part of the asfquart process.
 **Design (why):** [`docs/vllm-fleet-design.md`](../docs/vllm-fleet-design.md).
 
 **Control plane (all providers):** asfquart serves `GET /vllm/config`
-as JSON from `config.yaml` `fleet.hosts` (keyed by client IP) plus the catalog.
-Bearer fleet key (template).
+as JSON from `config.yaml` `fleet.hosts` plus `models.yaml`. Bearer fleet
+key (template). Vast boxes send `X-LLMAO-Host: $PUBLIC_IPADDR` (the TCP
+peer is often a transparent proxy).
 
 Provider-specific: how that JSON becomes running vLLM. Both providers
 write Supervisor units; there is no long-lived Python launcher.
@@ -29,20 +30,45 @@ Required:
 - `ASFQUART_URL` — origin only, `https://llm.apache.org`
 - `DATA_DIRECTORY` — volume workspace (typically `/workspace`). HF cache
   and logs are `$DATA_DIRECTORY/hf-cache` and `.../logs`, not in the config JSON.
+- `PUBLIC_IPADDR` — Vast sets this; `install_set.py` requires it (`X-LLMAO-Host`)
+- `PROVISIONING_SCRIPT` — Vast only: raw URL of `provision.sh`
 
 ## Boot
 
 ```text
-Vast:   provision.sh → curl install_set.py → GET JSON → supervisor units → vllm serve
+Vast:   PROVISIONING_SCRIPT → provision.sh → curl install_set.py → GET JSON → supervisor units → vllm serve
 RunPod: image entrypoint → COPY'd installer → GET JSON → supervisor units → vllm serve
 ```
 
-Vast still curls `refs/heads/main`. Pin that URL to a **commit SHA** before
-treating instance-create as production-safe. RunPod pins by **image digest**.
+Vast still curls `install_set.py` from `refs/heads/main`. Pin that URL to a
+**commit SHA** before treating instance-create as production-safe. RunPod
+pins by **image digest**.
+
+### Vast `PROVISIONING_SCRIPT` vs `install_set.py`
+
+The template sets
+
+`PROVISIONING_SCRIPT=https://raw.githubusercontent.com/apache/tooling-llmao/refs/heads/main/hosting/vast/provision.sh`
+
+The stock vLLM **container has none of our files**. Vast’s base-image
+provisioner downloads that URL to **`/provisioning.sh`**, `dos2unix`,
+`chmod +x`, and executes it. Their docs call it a **shell script**.
+
+`provision.sh` mkdirs `$DATA_DIRECTORY`, curls `install_set.py`, and runs
+`python3`. The `.sh` exists so Vast has something it will run from a URL.
+`DATA_DIRECTORY` mkdir could live in Python; that is not why the wrapper
+exists.
+
+Do **not** point `PROVISIONING_SCRIPT` at `install_set.py` until a
+throwaway instance proves execute is shebang (`./provisioning.sh`) rather
+than `bash /provisioning.sh`. Docs and the saved `.sh` name assume bash.
+`install_set.py` has no shebang today (ASF license first).
+
+RunPod’s image COPY’s the installer; it does not use `provision.sh`.
 
 ## Fit check
 
-Catalog `vram_gb` / `disk_gb` vs the card and `statvfs`, **before** the
+`models.yaml` `vram_gb` / `disk_gb` vs the card and `statvfs`, **before** the
 weights pull. vLLM's own allocator runs later (engine init) and does not
 replace this. NVIDIA: `nvidia-smi`. AMD: `rocm-smi` when we have a ROCm
 image. Silent if the probe is missing.
diff --git a/hosting/vast/README.md b/hosting/vast/README.md
index ce55f2c..bf6c7fb 100644
--- a/hosting/vast/README.md
+++ b/hosting/vast/README.md
@@ -35,9 +35,12 @@ That value must match a `fleet.hosts` key. There is no `?host=` query.
 
 ## On-create
 
-Point Vast on-start at `hosting/vast/provision.sh`. Today it pulls
-`install_set.py` from `main`; pin a commit SHA when this is no longer a
-moving target (safety: a broken main would brick every box on next create).
+Set template env `PROVISIONING_SCRIPT` to the raw GitHub URL of
+`hosting/vast/provision.sh` (not `install_set.py`). Vast fetches that into
+`/provisioning.sh` and runs it as a **shell** script; see
+[`../README.md`](../README.md) (boot). `provision.sh` then curls
+`install_set.py` from `main`; pin that second URL to a commit SHA when this
+is no longer a moving target (a broken `main` would brick every new box).
 
 ## Smoke
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For additional commands, e-mail: commits-help@tooling.apache.org