(tooling-llmao) 02/05: hosting/vast: how to re-provision a box

[gs...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gstein pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tooling-llmao.git

commit be13d437f0fe8cfd42a160ac11875d9c47a38e52
Author: Greg Stein <gs...@gmail.com>
AuthorDate: Fri Oct 16 12:27:40 2026 -0500

    hosting/vast: how to re-provision a box
    
    A box must be re-provisioned whenever what it was told has changed: its
    fleet.hosts rows, a model's recipe, or what /vllm/config sends (such as
    served_model_name). The runbook said to stop the servers and re-run the
    provisioning script.
    
    Stopping them is where that goes wrong. `supervisorctl stop all` stops
    every program the image's supervisord runs, and on Vast that includes
    the Jupyter terminal it was typed into, with no shell left to finish
    in.
    
    The runbook now says what Vast actually does: it runs
    PROVISIONING_SCRIPT once and records that in /.provisioning_complete
    and /.provisioner_state/. Remove both (`rm -r /.provision*`) and reboot
    the instance from the Vast console; on boot it provisions again. It
    warns against `supervisorctl stop all` and says why.
    
    hosting/README.md no longer says "stop the servers first" for a re-run
    on a live box; it points to the re-provisioning steps for Vast and
    RunPod.
    
    Co-Authored-By: Claude Opus 5.5 <no...@anthropic.com>
---
 hosting/README.md      |  6 ++++--
 hosting/vast/README.md | 27 ++++++++++++++++++++++++---
 2 files changed, 28 insertions(+), 5 deletions(-)

diff --git a/hosting/README.md b/hosting/README.md
index 7a0aa90..443925c 100644
--- a/hosting/README.md
+++ b/hosting/README.md
@@ -123,8 +123,10 @@ What this does not handle:
 - **Sizing happens once, at install.** Adding a server to a box means
   re-running the install so the existing servers restart with smaller shares.
 - **Re-running on a live box.** With the servers still up there is little
-  free memory, so the fit check may refuse and no fraction is set. Stop the
-  servers first.
+  free memory, so the fit check may refuse and no fraction is set.
+  Re-provision instead, which starts from a restart: Vast in
+  [`vast/README.md`](vast/README.md#re-provisioning-a-box), RunPod in
+  [`runpod/README.md`](runpod/README.md#changing-a-pods-set).
 - **Servers that load at the same time.** vLLM charges each one for the
   drop in free memory while it loads, which includes the other's weights.
   Two 15 GiB servers on a GB10 each ended with 17 GiB of KV cache where
diff --git a/hosting/vast/README.md b/hosting/vast/README.md
index e20bd3f..76b9c10 100644
--- a/hosting/vast/README.md
+++ b/hosting/vast/README.md
@@ -50,9 +50,30 @@ the card's memory between them and prints each server's fraction; see
 [`../README.md`](../README.md) (sizing) for the rule and its limits. Two
 servers of the same model need the disk for one copy of the weights.
 
-To change the set on a running box, stop the servers and re-run the
-provisioning script. A re-run with the old servers still up finds no free
-memory.
+To change the set on a running box, re-provision it; see below.
+
+## Re-provisioning a box
+
+Needed whenever what the box was told has changed: its rows in
+`fleet.hosts`, a model's recipe in `models.yaml`, or what llmao puts in
+`/vllm/config` (for example `served_model_name`).
+
+Vast runs `PROVISIONING_SCRIPT` once, and records that it has in two places
+at the root of the container: the file `/.provisioning_complete` and the
+directory `/.provisioner_state/`. While either is there, a reboot provisions
+nothing. To provision again, remove both:
+
+```bash
+rm -r /.provision*
+```
+
+then reboot the instance from the Vast console. On boot it finds no marker
+and provisions again: it fetches the installer and this box's config, and
+writes new Supervisor units.
+
+**Do not `supervisorctl stop all`.** Vast's image runs its own services under
+the same supervisord, including the Jupyter terminal you are typing in; it
+stops that too, and leaves you no shell to finish with.
 
 ## Smoke
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For additional commands, e-mail: commits-help@tooling.apache.org