You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tooling.apache.org by tn...@apache.org on 2026/09/21 16:56:23 UTC
(tooling-llmao) branch dependabot/uv/cryptography-50.0.0 updated (36c3c25 -> 11d11e3)
This is an automated email from the ASF dual-hosted git repository.
netomi pushed a change to branch dependabot/uv/cryptography-50.0.0
in repository https://gitbox.apache.org/repos/asf/tooling-llmao.git
from 36c3c25 Bump cryptography from 46.0.7 to 50.0.0
add d9241b4 Add PAT management UX against live LiteLLM virtual keys.
add 2431e06 Quiet noisy HTTP/async loggers in standalone mode.
add c76bc93 Document build status, edge cases, and high-level backlog.
add 718cf60 Serve site favicon at /favicon.ico (STeVe-style).
add 68af023 Note planned Models and Reports top-nav UX.
add 5a6fce2 Add team status email for post-import progress (4–9 Aug).
add f1d67b1 Gate seam methods with require_* decorators.
add 3afbeb3 Remove litellm mock/proxy app mode; mock is tests-only.
add 3c1be3b Use metadata.project for key project labels (required).
add dac5ab7 Align KeyInfo with design PAT vocabulary (project, user, purpose).
add b1aee8b Record created_by on automation keys (who saw the secret).
add fbff4b1 Tweak suppression of DEBUG messages
add f93a1a4 Split PAT UX into My Keys and Other Keys.
add b51fff0 Use in-memory dict for MockBackend test state.
add 4cc30bf Speak project in Backend API; hide LiteLLM team_id from Seam.
add 5be037b In-process team_id cache with fail-fast warm; drop StateStore.
add 83c68ad Store purpose in metadata.purpose, not key_alias.
add 46c32bb Add Models page with inventory table and detail modal.
add 59d4b08 Fix Models detail modal empty fields from multiline notes.
add a568cba Document project-centered product path and doc split.
add 9c64a49 Add P0.1 project budget seam (list + overview, no HTML).
add e8cb1c2 update the favicon
add a6cac0c Use brand logo in navbar with dark grey bar.
add d6b7f38 source to create favicon.ico
add d6f7fe1 add clone commit hash into the footer, using dunamai
add 31c3ba7 Add Projects list page and member-gated stub.
add b767ddd Replace image in navbar with llmao-logo.svg
add 30eef4c Use STeVe-style POST /do-* actions with 303 redirects.
add 2f278e6 Pass EZT booleans via ezt.boolean() so if-any works.
add 763bfb9 Improve the flash box when a key is created. Focus on the key itself, move details to lower addenda. Remove the "Create Another" since we now use a Flash on a page where "Create" is present. Add a [Copy] button, with auto-dismiss popup text.
add 29a76e2 Show GET page errors as flashes via page_errors.
add 467bd3d Raise AuthzError from current_identity when unsigned.
add 9b06fd1 Fold GET chrome into @page(title=) and inject result.
add bef7b36 Replace invented budget_type with team grantor.
add aa2b127 Initial draft of multi-vLLM architecture.
add a04fecf Add GPU-box hosting tree for Vast vLLM fleet.
add 32309b3 Clarify mkcert CA import steps for Chrome
add e33f7dd Update mkcert browser trust instructions
add 33f36a5 Provisioning script for vast.ai instances.
add 2739776 Fetch the launcher
add c666b88 the /workspace dir does not exist at creation. make it. then echo all our work.
add afea4a2 tweak the provisioning
add 732e61f Provision should not run servers; config them.
add eb477b5 Fetch vLLM set config as JSON when the launcher starts.
add d9be7c3 Replace the Python launcher with Supervisor units at Vast box-start.
add 797ecfb Pass the vLLM API key via VLLM_API_KEY, not argv.
add 7016e97 correct the path where "vllm" is found
add 2ba2222 Add an @api decorator so JSON handlers return dicts.
add 4e0d157 Log JSON API outcomes from the @api decorator.
add d8c900e Add a Vast operator tool to list, show, and set instance env.
add cb2f38d Put FLEET_KEY and VLLM_SET in Docker create options, not extra_env.
add 7a607c2 Move vLLM set placement into config.yaml fleet.sets.
add a489e68 Validate fleet config at process start, not on each request.
add 87753cd Use catalog / model / server consistently in fleet code and docs.
add e149588 Add Fleet/Server health probes and LiteLLM skew runners.
add 76ff54d Keep HF cache and log paths on the Vast box, not in set JSON.
add 4921c2d Key GPU boxes by public IP instead of a VLLM_SET env var.
add 7777b21 Stop wrapping Quart asgi_app with WSGI ProxyFix.
add 976fa5a Refresh docs for the live IP-keyed vLLM fleet.
add 3d8569c docs: split fleet membership out of config.yaml
add 53d6c34 Adding .asf.yaml to activate Issues
add 4a3ff4a Removing fleet.yaml after discussion
add b6e67d1 New version of example model_list config
add 276069c docs: LiteLLM is the source of truth for fleet state
add 47eb9a5 normalise api_base, and stop requiring api_key in the catalog
add 97f3ae8 Split vLLM listen vs public ports and rename fleet lifecycle states.
add 43242bb Repair fleet docs to match listen/public split and name the path ahead.
add c78882f Discover Vast public HostPorts in the fleet lifecycle tick.
add 0443744 Make the Models table catalog-first: id, status, available.
add f500e8f rename config.yaml to llmao.yaml, add the pipservice unit
add 3902cf8 keep config.yaml; revert the rename
add 45f8fdf fall back to a fleet-wide vLLM key when the catalog has none
add 6ef0c65 Adding selfhost api key to config example
add 02d4bfb pipservice-llmao.service: drop User and Group
add 5ac82dc pipservice-llmao.service: start through uv
add 16b35bc Adding ignore for uv-cache for deployment
add 0c548f8 point uv's cache inside the app directory
add db67bd9 redirect HOME instead of chasing cache variables
add c947db2 Show fleet status to signed-in users; gate host:port to admins.
add a394bc5 Show Models Available as Yes/No; drop the Status column.
add 27de0ef Stop including the catalog in LiteLLM; require self_hosted.
add ceaeff2 Adding trailing slash on vast instances to avoid dead-end redirects
add fe83b80 Call a vLLM process VllmServer; a LiteLLM backend a deployment.
add 1b43f62 Add FleetDeployment so commercial models show up on /fleet.
add 99ee646 redirect instead of rendering after a caught exception
add f469879 pin the fleet health state machine edges
add a46dea1 Register LiteLLM deployments when vLLM is actually serving.
add e7f5385 api_base should end with /v1
add 7b70b7f Remove unused Docker packaging.
add e6badd6 Catch docs up to mix, deployments, and no Docker.
add d69a4ed Add Aug–Sep status email: Models, fleet, and LiteLLM mix.
add ffd02af check that a placement fits before pulling weights
add f61cb88 scrape the measured KV cache and show it against the context window
add 45914bd enable tool calling for gemma4-26b
add 2e8c780 Adding qwen3-8b and qwen3.8-27b
add 0acc0e2 let a host row pin its public port
add ce656bb Raising max_model_len and context_window for qwen3.8
add ad2e14a qwen3-8b: use the hermes tool parser, and raise the scheduler budget
add 51070f4 Adding smoke and saturation tests for fleet
add 22072ce Lowering max batch tokens for qwen3-8b
add b73460e Updated readme with current status
add 3dd0a67 Cleaning up params and updating commentary in model_list example
add 9f60776 llmao-saturate: size prompts per model, and stop misreading the verdict
add c162005 Updating status doc
add f31f99d catalog: route self-hosted models via hosted_vllm/, not openai/
add efd5f3d catalog: per-model reasoning defaults
add 88f00be More benchmarking script work
add 766b97c Adding --effort flag to llmao-bench
add 4e4d7f0 dev tooling: ruff, prek, CI workflows, license headers
add e952c78 pre-commit: exclude static/ (vendored bootstrap assets)
add 740983d tests: regression test for root-vs-api_base scrape
add 5f0965f docs: STATUS metadata layout, drop fixed KV-scrape backlog item
add 88bcabc Merge pull request #38 from netomi/lint-and-ci
add dcd80fb test: preserve chat_template_kwargs through proxy (#31)
add f3c0fd1 Merge pull request #36 from KHARSHAVARDHAN-eng/test-preserve-chat-template-kwargs-31
add 31d0466 Linting fixes after @netomi's additions
add 2765db6 hosting: drop SSL_VERIFY; boxes talk to llm.apache.org on :443
add e90d29e docs: RunPod hosting contract (image not built; ports still open)
add 967f59b models.yaml: admin definitions, not a LiteLLM include
add 33a4073 Quiet down DEBUG messages in ASGI mode.
add ed91408 fleet: identify Vast boxes by X-LLMAO-Host (PUBLIC_IPADDR)
add fc59bf8 docs: Vast PROVISIONING_SCRIPT vs install_set.py lives in hosting/
add fd111ba docs: drop leftover catalog wording
add 2b94066 docs+cli: HMAC vLLM api_key (not wired to /vllm/config yet)
add 11d11e3 Merge branch 'main' into dependabot/uv/cryptography-50.0.0
No new revisions were added by this update.
Summary of changes:
.asf.yaml | 26 +
.github/workflows/analyze.yml | 45 ++
.github/workflows/build.yml | 41 +
.gitignore | 13 +-
.pre-commit-config.yaml | 61 ++
Dockerfile | 18 -
LICENSE-template.txt | 16 +
Makefile | 19 +-
README.md | 282 ++++---
api.py | 138 +++-
bin/README.md | 101 +++
bin/_discover.py | 124 ++++
bin/llmao-bench | 826 +++++++++++++++++++++
bin/llmao-saturate | 361 +++++++++
bin/llmao-smoke | 442 +++++++++++
bin/llmao-vllm-api-key | 71 ++
bin/setup_litellm_db.py | 16 +-
certs/README.md | 15 +-
config.yaml.example | 58 +-
docs/ARCHITECTURE.md | 75 +-
docs/STATUS.md | 277 +++++++
docs/favicon-source.svg | 23 +
docs/fleet-state.md | 333 +++++++++
docs/status-email-2026-08-09.txt | 70 ++
docs/status-email-2026-09-10.txt | 75 ++
docs/vllm-fleet-design.md | 303 ++++++++
hosting/README.md | 74 ++
hosting/runpod/README.md | 77 ++
hosting/vast/README.md | 48 ++
hosting/vast/install_set.py | 307 ++++++++
hosting/vast/provision.sh | 16 +
infra/docker/docker-compose.yml | 148 ----
litellm.yaml.example | 17 +-
llmao/__init__.py | 17 +
llmao/auth.py | 39 +-
llmao/fleet.py | 736 ++++++++++++++++++
llmao/litellm_client.py | 763 +++++++++++++++----
llmao/models.py | 183 ++++-
llmao/seam.py | 289 ++++++-
llmao/store.py | 46 --
llmao/vast_client.py | 104 +++
llmao/vllm_api_key.py | 52 ++
main.py | 122 +--
model_list.yaml.example | 53 --
models.yaml | 251 +++++++
pages.py | 493 +++++++++++-
pipservice-llmao.service | 58 ++
pyproject.toml | 61 ++
static/css/llmao.css | 15 +
static/favicon.ico | Bin 0 -> 24870 bytes
static/image.png | Bin 0 -> 659813 bytes
static/js/llmao.js | 90 ++-
static/llmao-logo.svg | 51 ++
templates/flash_key_created.ezt | 34 +
templates/flashes.ezt | 9 +-
templates/fleet.ezt | 101 +++
templates/footer.ezt | 5 +-
templates/header.ezt | 27 +-
templates/home.ezt | 3 +-
templates/key_create.ezt | 42 ++
templates/key_create_other.ezt | 37 +
templates/keys.ezt | 80 ++
templates/keys_other.ezt | 90 +++
templates/models.ezt | 127 ++++
templates/project.ezt | 14 +
templates/projects.ezt | 54 ++
tests/mock_backend.py | 157 ++++
tests/test_chat_template_kwargs.py | 159 ++++
tests/test_fleet.py | 274 +++++++
tests/test_fleet_deployments.py | 67 ++
tests/test_fleet_health.py | 425 +++++++++++
tests/test_hosting.py | 250 +++++++
tests/test_mix.py | 153 ++++
tests/test_models.py | 107 ++-
tests/test_phase1.py | 219 ++++--
tests/test_projects.py | 193 +++++
tests/test_vast_client.py | 167 +++++
.../test_vllm_api_key.py | 28 +-
uv.lock | 73 +-
79 files changed, 10045 insertions(+), 789 deletions(-)
create mode 100644 .asf.yaml
create mode 100644 .github/workflows/analyze.yml
create mode 100644 .github/workflows/build.yml
create mode 100644 .pre-commit-config.yaml
delete mode 100644 Dockerfile
create mode 100644 LICENSE-template.txt
create mode 100644 bin/README.md
create mode 100644 bin/_discover.py
create mode 100755 bin/llmao-bench
create mode 100755 bin/llmao-saturate
create mode 100755 bin/llmao-smoke
create mode 100755 bin/llmao-vllm-api-key
create mode 100644 docs/STATUS.md
create mode 100644 docs/favicon-source.svg
create mode 100644 docs/fleet-state.md
create mode 100644 docs/status-email-2026-08-09.txt
create mode 100644 docs/status-email-2026-09-10.txt
create mode 100644 docs/vllm-fleet-design.md
create mode 100644 hosting/README.md
create mode 100644 hosting/runpod/README.md
create mode 100644 hosting/vast/README.md
create mode 100755 hosting/vast/install_set.py
create mode 100755 hosting/vast/provision.sh
delete mode 100644 infra/docker/docker-compose.yml
create mode 100644 llmao/fleet.py
delete mode 100644 llmao/store.py
create mode 100644 llmao/vast_client.py
create mode 100644 llmao/vllm_api_key.py
delete mode 100644 model_list.yaml.example
create mode 100644 models.yaml
create mode 100644 pipservice-llmao.service
create mode 100644 static/favicon.ico
create mode 100644 static/image.png
create mode 100644 static/llmao-logo.svg
create mode 100644 templates/flash_key_created.ezt
create mode 100644 templates/fleet.ezt
create mode 100644 templates/key_create.ezt
create mode 100644 templates/key_create_other.ezt
create mode 100644 templates/keys.ezt
create mode 100644 templates/keys_other.ezt
create mode 100644 templates/models.ezt
create mode 100644 templates/project.ezt
create mode 100644 templates/projects.ezt
create mode 100644 tests/mock_backend.py
create mode 100644 tests/test_chat_template_kwargs.py
create mode 100644 tests/test_fleet.py
create mode 100644 tests/test_fleet_deployments.py
create mode 100644 tests/test_fleet_health.py
create mode 100644 tests/test_hosting.py
create mode 100644 tests/test_mix.py
create mode 100644 tests/test_projects.py
create mode 100644 tests/test_vast_client.py
copy bin/gen-litellm-master-key.sh => tests/test_vllm_api_key.py (56%)
mode change 100755 => 100644
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For additional commands, e-mail: commits-help@tooling.apache.org