You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tooling.apache.org by gs...@apache.org on 2026/08/09 12:52:09 UTC
(tooling-llmao) branch main updated: Document build status, edge cases, and high-level backlog.
This is an automated email from the ASF dual-hosted git repository.
gstein pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tooling-llmao.git
The following commit(s) were added to refs/heads/main by this push:
new c76bc93 Document build status, edge cases, and high-level backlog.
c76bc93 is described below
commit c76bc93f54b040e8d6eb508e2d48b6cef4c66280
Author: Greg Stein <gs...@gmail.com>
AuthorDate: Sun Aug 9 07:52:03 2026 -0500
Document build status, edge cases, and high-level backlog.
Add docs/STATUS.md as the detailed status of record, slim the README to a
summary plus links, and point ARCHITECTURE at STATUS and open PAT policy.
Co-authored-by: Grok <gr...@x.ai>
---
README.md | 38 ++++++++++++-----------
docs/ARCHITECTURE.md | 11 +++++--
docs/STATUS.md | 85 ++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 115 insertions(+), 19 deletions(-)
diff --git a/README.md b/README.md
index cb2d3c6..4199658 100644
--- a/README.md
+++ b/README.md
@@ -102,21 +102,25 @@ make test # offline seam + model_list tests (no OAuth session automatio
---
-## What this includes today
-
-| Capability | Where it lives | Notes |
-|---|---|---|
-| ASF login + PMC authz | asfquart always + `auth.py` | `@asfquart.auth.require`; project scope in seam |
-| Per-PMC budgets & spend (read) | litellm teams / `MockBackend` | one litellm *team* per ASF project |
-| Project ↔ team mapping | `seam.py` | provision team on first use |
-| Model inventory | `model_list.yaml` | LiteLLM `include` + llmao loader; metadata in `model_info` |
-| Per-project activity view | `api.py` | PMC admins / site admins only |
-| HTML shell | `pages.py` + EZT + `static/` | Bootstrap; PAT UI next |
-| Local TLS + configs | `main.py`, `config.yaml`, `litellm.yaml` | examples committed; secrets gitignored |
-
-**PATs (proxy mode):** open **API keys** in the nav — create personal keys
-(project + purpose), one-time secret display, list spend/limit, revoke with
-confirm modal. Automation keys are admin/PMC-only with required purpose.
+## What works today (summary)
+
+ASF login (asfquart), project membership from LDAP, LiteLLM admin via async
+httpx, **`model_list.yaml`** inventory (include + UX), system Postgres + Prisma
+setup, and **PAT UX** in proxy mode (personal keys; provisional automation keys).
+
+Full status, edge cases, and backlog: **[`docs/STATUS.md`](docs/STATUS.md)**.
+Architecture: **[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)**.
+Product design: **rai-private** `services/llmao/README.md`.
+Ops plans: Infra **`p6/modules/llmao/README.md`**.
+
+### Planned (not yet done) — high level only
+
+- Harden PAT flows against live LiteLLM edge cases
+- **Who may create team-scoped / automation PATs** (RAI policy; may be RAI-only, Chair/VP, or PMC)
+- Site-admin via `rai` PMC; PMC notification email; budgets/capacity UX
+- Production Puppet/systemd (p6); cleanup of container-oriented paths
+
+Details: [`docs/STATUS.md`](docs/STATUS.md).
---
@@ -150,8 +154,8 @@ the handler still runs.
No production env-var secret channel.
3. **LiteLLM** with Postgres (`database_url` in `litellm.yaml`) and
- `litellm --config litellm.yaml`. Model routes and provider keys live in
- that file (inventory design still open).
+ `litellm --config litellm.yaml`. Model routes live in **`model_list.yaml`**
+ (included); provider API keys via eyaml in production.
4. **Serve** llmao (`main.py` or Hypercorn). Point client tools at the
**LiteLLM** base URL with PATs, not at llmao for chat.
diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
index 941826e..fecc5c3 100644
--- a/docs/ARCHITECTURE.md
+++ b/docs/ARCHITECTURE.md
@@ -47,8 +47,15 @@ file are secrets (eyaml); `api_base` is cleartext. Restart LiteLLM after
inventory changes (Puppet/systemd later).
**LiteLLM virtual keys / teams** need Postgres + Prisma (`litellm[proxy,extra-proxy]`).
-Developers: system PostgreSQL + `make db` (`bin/setup_litellm_db.py`). Production:
-Puppet creates the DB and deploys on-disk `database_url`.
+Developers: system PostgreSQL + `make db`. Production: Puppet + on-disk
+`database_url`.
+
+**PATs:** personal keys bind ASF uid + project team + purpose; automation
+keys are team-scoped exceptions (who may create them is an **open RAI
+policy** question — see design + `docs/STATUS.md`). Secrets shown once;
+metadata in LiteLLM.
+
+Build status and backlog: **`docs/STATUS.md`**.
`litellm.mode: mock` in config is only an offline stand-in for team/usage
storage in tests and laptop work without a proxy—not a second auth system.
diff --git a/docs/STATUS.md b/docs/STATUS.md
new file mode 100644
index 0000000..14b6ca8
--- /dev/null
+++ b/docs/STATUS.md
@@ -0,0 +1,85 @@
+# Build status and backlog
+
+**As of:** 2026-08-09
+**Repo:** `apache/tooling-llmao`
+**Product design:** `apache/rai-private` → `services/llmao/README.md`
+**Ops plans:** Infra `p6/modules/llmao/README.md`
+
+Detailed “where we are.” Top-level README stays short and links here.
+
+---
+
+## Done
+
+| Area | Notes |
+|------|--------|
+| asfquart always | OAuth + LDAP session; no auth-mode split |
+| Run | `main.py` (TLS/`certs`) + ASGI `llmao_app` (Hypercorn) |
+| Config | `APP.cfg` EasyDict from `config.yaml` (dotted access) |
+| LiteLLM admin client | Async **httpx** |
+| Model inventory SoT | **`model_list.yaml`** + LiteLLM `include`; flat `model_info`; **no** `STORE_MODEL_IN_DB` for inventory |
+| Secrets | On-disk YAML; prod **hiera/eyaml**; same `master_key` in `config.yaml` + `litellm.yaml`; provider **API keys** in `model_list.yaml`; **api_base** cleartext |
+| Postgres / Prisma | `litellm[proxy,extra-proxy]`; system PostgreSQL; `make db` |
+| **PAT UX (proxy)** | `/keys` list/create/revoke; personal + admin automation; secret once; revoke modal |
+
+### Credential rules (as implemented)
+
+| Kind | Who creates (today) | Binding |
+|------|---------------------|---------|
+| **Personal** | Committer (self), member of project | `user_id` + `team_id` + purpose (`key_alias`) |
+| **Automation (team-scoped)** | **Any PMC member of project or site admin** (provisional) | `team_id` only; purpose required |
+
+- Team ensure → LiteLLM **team** (`team_id`) only; **not** a shared product team PAT.
+- Secret `sk-…` shown **once**; not kept in llmao for product PATs.
+- Project names = LDAP session names (no rename map).
+
+### Open policy: who may create automation PATs?
+
+**RAI decision pending.** Options: (A) RAI / proxy admins only, (B) PMC Chair/VP (or nominated role), (C) any PMC member (**current code**). Documented in design; tooling may tighten after RAI decides.
+
+### Local proxy path
+
+Copy three YAMLs from `*.example` → system Postgres + `make db` → matching master keys → `litellm.mode: proxy` → `make proxy` + `make run`.
+
+---
+
+## Edge cases and gaps
+
+| Issue | Notes |
+|-------|--------|
+| mock mode | PAT pages refuse; need proxy + DB |
+| Last used | Best-effort; may be `—` or `updated_at` |
+| Model cost $0 | LiteLLM default without pricing metadata |
+| Site admin = `rai` PMC | Not automatic; `site_admins` + isRoot |
+| Automation list | PMC committees / site admin project scan, not global all-teams |
+| List pagination | First page (~100 keys) |
+| Team alias / revoke id | Depends on LiteLLM list payload |
+| Master key drift | Two files must match (eyaml) |
+| Prisma path | site-packages schema; re-generate after upgrades |
+| Containers | Not the supported DB story |
+| OAuth HTTP tests | Not automated |
+| PMC email audit | Design §6.4; **not built** |
+| Member budgets / TPM | Design; **not built** |
+
+---
+
+## Next steps (detail)
+
+1. Harden PAT against live LiteLLM edge responses (pagination, team_alias, delete ids).
+2. **Decide automation creator role** (RAI) and tighten code if not (C).
+3. Site admin via **`rai` PMC** (optional keep cfg list).
+4. PMC notification email on key lifecycle.
+5. Budgets / capacity limits UX.
+6. **p6** Puppet: Postgres, eyaml→YAML, systemd, restart LiteLLM on config change.
+7. Cleanup container paths; optional un-package.
+8. Advisor / richer model UX.
+
+---
+
+## Multi-repo
+
+| Tree | Role |
+|------|------|
+| rai-private `services/llmao/` | Product design |
+| tooling-llmao | Software (this repo) |
+| p6 `modules/llmao` | Production deploy (planned) |
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For additional commands, e-mail: commits-help@tooling.apache.org