You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@casbin.apache.org by hs...@apache.org on 2026/10/22 13:54:45 UTC
(casbin-gateway) branch master updated: feat(providers): pool subscriptions with plan limits, strategies, sticky sessions and warm-up, and add GitHub Copilot sign-in
This is an automated email from the ASF dual-hosted git repository.
hsluoyz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/casbin-gateway.git
The following commit(s) were added to refs/heads/master by this push:
new 3cc9a06 feat(providers): pool subscriptions with plan limits, strategies, sticky sessions and warm-up, and add GitHub Copilot sign-in
3cc9a06 is described below
commit 3cc9a06cbeb242da288a9cc0f0ac58ba726dee2a
Author: Yang Luo <hs...@gmail.com>
AuthorDate: Thu Oct 22 21:54:37 2026 +0800
feat(providers): pool subscriptions with plan limits, strategies, sticky sessions and warm-up, and add GitHub Copilot sign-in
---
README.md | 20 ++
README_zh.md | 20 ++
agentusage/codex.go | 30 ++-
agentusage/subscription.go | 152 ++++++++++++
controllers/agent.go | 9 +-
controllers/model_route.go | 4 +-
controllers/provider_login.go | 19 ++
controllers/proxy.go | 28 ++-
controllers/tray.go | 4 +-
main.go | 1 +
object/agent.go | 31 ++-
object/model_route.go | 2 +-
object/provider.go | 13 +-
object/provider_limit.go | 321 ++++++++++++++++++++++++++
object/provider_pool.go | 273 ++++++++++++++++++++++
object/provider_quota.go | 17 +-
object/provider_subscription.go | 123 +++++++++-
object/provider_subscription_copilot.go | 344 ++++++++++++++++++++++++++++
object/provider_warmup.go | 312 +++++++++++++++++++++++++
object/snapshot.go | 4 +-
routers/router.go | 1 +
web/src/backend/AgentBackend.ts | 2 +
web/src/backend/ProviderBackend.ts | 7 +
web/src/components/ProviderCard.tsx | 25 ++
web/src/components/ProviderLoginField.tsx | 19 +-
web/src/components/ProviderQuota.tsx | 220 ++++++++++++++++--
web/src/components/ProviderSourcePicker.tsx | 7 +
web/src/lib/agents.ts | 2 +
web/src/lib/providers.ts | 25 ++
web/src/lib/quota.ts | 30 ++-
web/src/locales/en/data.json | 36 ++-
web/src/locales/zh/data.json | 36 ++-
web/src/pages/ProviderEditPage.tsx | 12 +-
web/src/pages/ProviderListPage.tsx | 2 +
web/src/types.ts | 42 ++++
35 files changed, 2118 insertions(+), 75 deletions(-)
diff --git a/README.md b/README.md
index 26dacbf..d6df754 100644
--- a/README.md
+++ b/README.md
@@ -198,6 +198,7 @@ The password behind that account is `admin` / `123`, and it only matters if you
- **[Is the API behind that key what it was sold as?](#the-killer-feature-is-the-api-behind-that-key-what-it-was-sold-as)** — a reseller can quietly swap in a cheaper model or fake a cache hit, and none of it shows up in the traffic. Authenticity asks the upstream directly and grades it A–F.
- **[Switch every agent's provider from one place](#send-an-agents-traffic-through-gateway)** — change an API key or base URL once, and every agent pointed at Gateway picks it up.
- **[Keep going when a provider runs out](#when-a-provider-cannot-answer-fallbacks-and-model-routing)** — a key out of credit, a rate limit or a relay that is down moves the request on to the next provider, across vendors and APIs, and model routing rules decide which model each request is actually sent and what it steps down to.
+- **[Pool several subscriptions](#pool-several-subscriptions)** — hold several Claude, ChatGPT or Copilot accounts, start each conversation on the plan whose quota resets soonest, keep it there while its prompt cache lasts, see what every plan has left, and start a 5-hour window on a schedule.
- **[Add a provider, an MCP server, a prompt or a skill from a link](#import-from-a-link)** — click a vendor's "add this" button on the web and Gateway opens with what the link carries, before any of it is written.
- **[Run several instances of one agent side by side](#what-to-do-next)** — e.g. multiple Claude Desktop instances, each signed in to a different account.
- **[Install, upgrade and roll back the agents themselves](#what-to-do-next)** — which build each agent is on against what its package manager publishes, and back to an older release when an update broke something. One click whichever way it was installed, with the command shown first, its progress on the row and its console output a click away.
@@ -305,6 +306,25 @@ A key runs out of credit, a relay goes down, a vendor rate-limits you halfway th
Requests that name no agent, on `/v1/chat/completions` or `/v1/messages`, go by model name to every enabled provider that serves it, lowest priority value first, and fail over the same way.
+### Pool several subscriptions
+
+Gateway can hold a subscription itself rather than borrow the agent's. On **Providers** → **New Provider**, pick **Claude subscription**, **ChatGPT subscription** or **GitHub Copilot subscription** and sign in once in the browser; Copilot shows a code to type on GitHub's page. The sign-in stays in Gateway, which renews it, and the agents use the provider like any other: Claude Code can spend a ChatGPT plan, and Codex runs on a ChatGPT plan through Gateway with no API key. Copilot names i [...]
+
+Bind one of them to an agent and pick the others as its fallbacks, and the agent's providers are a pool. Under the fallbacks, **Pool strategy** decides which one a request starts on:
+
+| Strategy | Starts on |
+| --- | --- |
+| In order | the bound provider, then the fallbacks in the order picked |
+| Soonest reset first | the plan whose 5-hour window resets soonest, since whatever it has left is lost at that reset; API keys come after every plan |
+| Rotate | the next provider, for each new conversation |
+| Most left first | the plan with the most left of its tightest window; API keys come after every plan |
+
+Whatever the strategy, a conversation stays on the provider that answered it until it has been idle for 15 minutes, so the prompt cache it built there is spent rather than paid for again elsewhere. Gateway knows a conversation by the session id the agent sends, which Claude Code and Codex do, and otherwise by how it began, which every later turn repeats. A plan that has run out of a window waits behind the others until that window resets, rather than for the few minutes a cooldown lasts, [...]
+
+**What each plan has left** is on its card on the Providers page: the 5-hour and the weekly window of a Claude or ChatGPT plan, the premium requests of a Copilot one, each with when it resets. Gateway reads it with the sign-in itself, and keeps it current from the headers of every answer it relays. When most of a window is about to be lost to its reset - half of it or more left, with an hour to go, or a day for the weekly one - the card says so.
+
+**Warm-up.** A Claude or ChatGPT plan counts its five hours from the first request. On a subscription provider's page, **Warm-up** sends the smallest request the plan takes at the times you set, unless the window is already running: send one at 7:00 and the window resets by noon, so the afternoon starts on a full one. It asks for the model the provider last relayed, or the one you name, and **Warm up now** does the same on the spot.
+
### Import from a link
Vendors put an "add this to my agent manager" button on their own pages, which opens a `ccswitch://` link rather than a page. Gateway reads that format instead of inventing a second one nobody would be given a button for, and one link carries any of four things: a provider with its base URL, key and model list; an MCP server, as the JSON block a server is written in; a set of instructions for an agent to read before every session; or a repository to install skills from.
diff --git a/README_zh.md b/README_zh.md
index 4205681..67d3b3d 100644
--- a/README_zh.md
+++ b/README_zh.md
@@ -196,6 +196,7 @@ Gateway 会在自己的窗口里打开 —— 不用登录：它只服务本机
- **[查一查那个 Key 背后的 API 是不是真的](#杀手锏那个-key-背后的-api真是卖给你的那个吗)** —— 中转商可以偷偷换成便宜模型、伪造缓存命中，流量里根本看不出来。Authenticity 直接问上游，测出 A 到 F 的等级。
- **[一个地方切换所有 Agent 的 API 供应商](#让-agent-的流量走-gateway)** —— 改一次 Key 或 base URL，接进 Gateway 的每个 Agent 都跟着换。
- **[一家额度用完，自动换下一家](#provider-答不上来时备用-provider-和模型路由)** —— Key 没余额、被限流、中转站挂了，请求自动转给下一个 Provider，可以跨厂商、跨 API；模型路由规则决定每个请求实际发给哪个模型，以及答不上来时降到哪个模型。
+- **[多个订阅组成账号池](#多个订阅组成账号池)** —— 保管多个 Claude、ChatGPT 或 Copilot 账号，每个对话先用额度最早重置的那个，提示词缓存还在时一直留在那里，看得到每个套餐还剩多少，还能定时开启 5 小时窗口。
- **[一条链接就能导入 Provider、MCP、提示词或技能](#从一条链接导入)** —— 在网页上点厂商的「添加到」按钮，Gateway 就带着链接里的东西打开，写进去之前先给你看。
- **[同一个 Agent 开多个实例](#接下来做什么)** —— 比如同时跑好几个 Claude Desktop，各自登录不同账号。
- **[安装、升级和回退 Agent 本身](#接下来做什么)** —— 本机装的是哪个版本，包管理器又发布到了哪个版本；升级坏了还能退回旧版。不管当初是怎么装的，都是一键完成：命令先摆出来，进度就在那一行上，控制台输出点一下就能看。
@@ -302,6 +303,25 @@ Key 没余额了、中转站挂了、厂商在会话中途开始限流 —— Ga
不带 Agent 的请求（`/v1/chat/completions`、`/v1/messages`）按模型名发给所有提供该模型的已启用 Provider，priority 值小的优先，切换规则相同。
+### 多个订阅组成账号池
+
+订阅也可以由 Gateway 自己保管，而不是借用 Agent 的登录。在 **Providers** → **New Provider** 里选 **Claude 订阅登录**、**ChatGPT 订阅**或 **GitHub Copilot 订阅登录**，在浏览器里登录一次；Copilot 会给出一个代码，在 GitHub 的页面上输入。登录留在 Gateway，由它续期，Agent 像用别的 Provider 一样用它：Claude Code 可以花 ChatGPT 的套餐，Codex 也能经 Gateway 用 ChatGPT 套餐，不需要 API Key。Copilot 的模型名有自己的写法（`claude-sonnet-4.5`），所以登录时会把这个账号能选的模型填进 Provider 的模型列表，Agent 点名别的名字时发给其中最接近的一个。换一个账号再登录一次，就给同一种套餐加了第二个 Provider。
+
+把其中一个绑定到 Agent，其余的选作备用，这个 Agent 的 Provider 就成了一个账号池。备用 Provider 下面的**账号池策略**决定请求先发给哪一个：
+
+| 策略 | 先发给 |
+| --- | --- |
+| 按顺序 | 绑定的 Provider，再按所选顺序用备用的 |
+| 先用快重置的 | 5 小时窗口最早重置的订阅，因为重置时没用完的额度就作废了；API Key 排在所有订阅之后 |
+| 轮流 | 下一个 Provider，每个新对话轮一次 |
+| 先用剩余多的 | 最紧的那个窗口剩得最多的订阅；API Key 排在所有订阅之后 |
+
+不管哪种策略，一个对话都会留在回答它的 Provider 上，直到空闲 15 分钟，这样在那里建起来的提示词缓存能接着用，而不是换一家重新付钱。Gateway 按 Agent 发来的会话 ID 认出一个对话（Claude Code 和 Codex 都会发），没有的话就看它是怎么开头的，后面每一轮都会重复开头。某个窗口额度用完的订阅会排到最后，直到这个窗口重置，而不是只冷却几分钟；前面说的切换规则对每个请求照样生效。
+
+**每个套餐还剩多少**，写在 Providers 页面上它的卡片里：Claude 和 ChatGPT 套餐的 5 小时窗口和每周窗口，Copilot 套餐的高级请求，以及各自什么时候重置。Gateway 用这个登录本身去查，转发的每次回答的响应头也会更新它。一个窗口的大部分额度快要因为重置作废时 —— 还剩一半以上，离重置不到一小时（每周窗口是一天）—— 卡片上会提示。
+
+**预热。** Claude 和 ChatGPT 套餐的 5 小时从第一次请求开始计时。在订阅 Provider 的页面上，**预热**会在你设定的时间点发一个套餐接受的最小请求，窗口已经在计时就不发：7:00 发一个，中午窗口就重置了，下午用的是满额度。它用这个 Provider 最近一次转发的模型，或者你指定的模型；**立即预热**马上做同样的事。
+
### 从一条链接导入
厂商会在自己的页面上放一个「添加到我的 Agent 管理器」按钮，点开的是一条 `ccswitch://` 链接，而不是一个网页。Gateway 直接读这个格式，而不是再造一个没人会给你按钮的格式。一条链接可以带四样东西之一：一个 Provider，连同它的 base URL、Key 和模型列表；一个 MCP 服务器，就是它平时写成的那段 JSON；一份 Agent 每次会话前要读的提示词；或者一个安装技能用的仓库。
diff --git a/agentusage/codex.go b/agentusage/codex.go
index ca30e91..cae93c6 100644
--- a/agentusage/codex.go
+++ b/agentusage/codex.go
@@ -49,14 +49,15 @@ type Window struct {
ResetTime string `json:"resetTime,omitempty"`
}
-// Usage is what one Codex account has left. Primary is the short window - five
-// hours on the plans today - and Secondary the weekly one.
+// Usage is what one account has left. Primary is the short window - five hours
+// on the Codex and Claude plans today - and Secondary the weekly one.
type Usage struct {
Primary *Window `json:"primary,omitempty"`
Secondary *Window `json:"secondary,omitempty"`
// ResetCredits is how many rate limit resets the account still holds. Each
// one empties a window before its own time.
- ResetCredits int `json:"resetCredits"`
+ ResetCredits int `json:"resetCredits"`
+ Plan string `json:"plan,omitempty"`
}
// codexWindow is one window as the endpoint reports it: what has been spent
@@ -83,17 +84,21 @@ func Codex(credential string) (*Usage, error) {
if auth.Tokens.AccessToken == "" {
return nil, errors.New("this account holds no ChatGPT sign-in, so it has no plan limits to report")
}
+ return CodexToken(auth.Tokens.AccessToken, auth.Tokens.AccountId)
+}
+// CodexToken is the same reading for a ChatGPT access token held anywhere.
+func CodexToken(accessToken string, accountId string) (*Usage, error) {
request, err := http.NewRequest(http.MethodGet, codexUsageUrl, nil)
if err != nil {
return nil, err
}
- request.Header.Set("Authorization", "Bearer "+auth.Tokens.AccessToken)
+ request.Header.Set("Authorization", "Bearer "+accessToken)
request.Header.Set("Accept", "application/json")
request.Header.Set("Originator", codexOriginator)
request.Header.Set("User-Agent", codexOriginator)
- if auth.Tokens.AccountId != "" {
- request.Header.Set("Chatgpt-Account-Id", auth.Tokens.AccountId)
+ if accountId != "" {
+ request.Header.Set("Chatgpt-Account-Id", accountId)
}
response, err := usageClient.Do(request)
@@ -111,6 +116,7 @@ func Codex(credential string) (*Usage, error) {
}
var answer struct {
+ PlanType string `json:"plan_type"`
RateLimit struct {
Primary *codexWindow `json:"primary_window"`
Secondary *codexWindow `json:"secondary_window"`
@@ -130,6 +136,7 @@ func Codex(credential string) (*Usage, error) {
Primary: codexWindowOf(answer.RateLimit.Primary),
Secondary: codexWindowOf(answer.RateLimit.Secondary),
ResetCredits: answer.ResetCredits.AvailableCount,
+ Plan: answer.PlanType,
}, nil
}
@@ -138,11 +145,16 @@ func codexWindowOf(window *codexWindow) *Window {
return nil
}
- // The moment beats the countdown, which the local clock can skew.
+ // A window nothing has been spent in has not started: the reset it reports
+ // is only where it would end if it started now, so none is given.
+ idle := window.UsedPercent == 0 && window.WindowSeconds > 0 && window.ResetAfter >= window.WindowSeconds
reset := ""
- if window.ResetAt > 0 {
+ switch {
+ case idle:
+ case window.ResetAt > 0:
+ // The moment beats the countdown, which the local clock can skew.
reset = time.Unix(window.ResetAt, 0).UTC().Format(time.RFC3339)
- } else if window.ResetAfter > 0 {
+ case window.ResetAfter > 0:
reset = time.Now().Add(time.Duration(window.ResetAfter) * time.Second).UTC().Format(time.RFC3339)
}
diff --git a/agentusage/subscription.go b/agentusage/subscription.go
new file mode 100644
index 0000000..eebc9f9
--- /dev/null
+++ b/agentusage/subscription.go
@@ -0,0 +1,152 @@
+// Copyright 2026 The casbin Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package agentusage
+
+import (
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "math"
+ "net/http"
+ "time"
+)
+
+var (
+ claudeUsageUrl = "https://api.anthropic.com/api/oauth/usage"
+ copilotUsageUrl = "https://api.github.com/copilot_internal/user"
+)
+
+// Claude reads what a Claude subscription has left, with the OAuth token the
+// sign-in was granted. The token needs the user:profile scope.
+func Claude(accessToken string) (*Usage, error) {
+ request, err := http.NewRequest(http.MethodGet, claudeUsageUrl, nil)
+ if err != nil {
+ return nil, err
+ }
+ request.Header.Set("Authorization", "Bearer "+accessToken)
+ request.Header.Set("Accept", "application/json")
+ request.Header.Set("Anthropic-Version", "2023-06-01")
+ request.Header.Set("Anthropic-Beta", "oauth-2025-04-20")
+
+ var answer struct {
+ FiveHour *claudeWindow `json:"five_hour"`
+ SevenDay *claudeWindow `json:"seven_day"`
+ }
+ if err := getUsage(request, "claude", &answer); err != nil {
+ return nil, err
+ }
+ if answer.FiveHour == nil && answer.SevenDay == nil {
+ return nil, errors.New("claude reported no limits for this account")
+ }
+ return &Usage{
+ Primary: answer.FiveHour.window(300),
+ Secondary: answer.SevenDay.window(10080),
+ }, nil
+}
+
+// claudeWindow reports what has been spent, in percent, and no reset at all for
+// a window that has not started.
+type claudeWindow struct {
+ Utilization float64 `json:"utilization"`
+ ResetsAt *string `json:"resets_at"`
+}
+
+func (window *claudeWindow) window(minutes int) *Window {
+ if window == nil {
+ return nil
+ }
+
+ reset := ""
+ if window.ResetsAt != nil {
+ if at, err := time.Parse(time.RFC3339, *window.ResetsAt); err == nil {
+ reset = at.UTC().Format(time.RFC3339)
+ }
+ }
+ return &Window{
+ RemainingPercent: clampPercent(100 - window.Utilization),
+ WindowMinutes: minutes,
+ ResetTime: reset,
+ }
+}
+
+// Copilot reads what a GitHub Copilot plan has left this month, with the GitHub
+// token the device sign-in was granted. Primary is the premium requests, and
+// Secondary the chat allowance of the free plan; a limit the plan does not have
+// is left out.
+func Copilot(githubToken string) (*Usage, error) {
+ request, err := http.NewRequest(http.MethodGet, copilotUsageUrl, nil)
+ if err != nil {
+ return nil, err
+ }
+ request.Header.Set("Authorization", "token "+githubToken)
+ request.Header.Set("Accept", "application/json")
+ request.Header.Set("X-Github-Api-Version", "2025-04-01")
+
+ var answer struct {
+ Plan string `json:"copilot_plan"`
+ ResetDate string `json:"quota_reset_date"`
+ Snapshots struct {
+ Premium *copilotSnapshot `json:"premium_interactions"`
+ Chat *copilotSnapshot `json:"chat"`
+ } `json:"quota_snapshots"`
+ }
+ if err := getUsage(request, "github", &answer); err != nil {
+ return nil, err
+ }
+
+ reset := ""
+ if at, err := time.Parse("2006-01-02", answer.ResetDate); err == nil {
+ reset = at.UTC().Format(time.RFC3339)
+ }
+ return &Usage{
+ Primary: answer.Snapshots.Premium.window(reset),
+ Secondary: answer.Snapshots.Chat.window(reset),
+ Plan: answer.Plan,
+ }, nil
+}
+
+type copilotSnapshot struct {
+ PercentRemaining float64 `json:"percent_remaining"`
+ Unlimited bool `json:"unlimited"`
+}
+
+func (snapshot *copilotSnapshot) window(reset string) *Window {
+ if snapshot == nil || snapshot.Unlimited {
+ return nil
+ }
+ return &Window{RemainingPercent: clampPercent(snapshot.PercentRemaining), ResetTime: reset}
+}
+
+// getUsage performs the GET and decodes the answer into target.
+func getUsage(request *http.Request, vendor string, target any) error {
+ response, err := usageClient.Do(request)
+ if err != nil {
+ return err
+ }
+ defer response.Body.Close()
+
+ if response.StatusCode == http.StatusUnauthorized || response.StatusCode == http.StatusForbidden {
+ return fmt.Errorf("the stored sign-in is no longer accepted (%s), sign in again", response.Status)
+ }
+ if response.StatusCode >= 400 {
+ return fmt.Errorf("%s answered %s", vendor, response.Status)
+ }
+ return json.NewDecoder(io.LimitReader(response.Body, maxUsageReply)).Decode(target)
+}
+
+func clampPercent(value float64) float64 {
+ return math.Min(100, math.Max(0, math.Round(value*10)/10))
+}
diff --git a/controllers/agent.go b/controllers/agent.go
index d1dcb53..01699e9 100644
--- a/controllers/agent.go
+++ b/controllers/agent.go
@@ -48,6 +48,7 @@ type discoveredAgent struct {
Provider string `json:"provider"`
Fallbacks []string `json:"fallbacks"`
Mode string `json:"mode"`
+ Strategy string `json:"strategy"`
// ProviderConfig is the state of the agent's own configuration file, which
// is what the config orchestrator writes.
ProviderConfig agentprovider.Status `json:"providerConfig"`
@@ -129,6 +130,7 @@ func (c *ApiController) GetAgents() {
if stored, ok := agents[installation.AgentId]; ok {
item.Provider = stored.Provider
item.Mode = stored.Mode
+ item.Strategy = stored.Strategy
if stored.Fallbacks != nil {
item.Fallbacks = stored.Fallbacks
}
@@ -180,6 +182,7 @@ func (c *ApiController) UpdateAgentRouting() {
Provider string `json:"provider"`
Fallbacks []string `json:"fallbacks"`
Mode string `json:"mode"`
+ Strategy string `json:"strategy"`
}
if err := json.Unmarshal(c.Ctx.Input.RequestBody, &form); err != nil {
c.ResponseError(err.Error())
@@ -190,7 +193,7 @@ func (c *ApiController) UpdateAgentRouting() {
return
}
- if err := saveAgentRouting(form.AgentId, form.Provider, form.Fallbacks, form.Mode); err != nil {
+ if err := saveAgentRouting(form.AgentId, form.Provider, form.Fallbacks, form.Mode, form.Strategy); err != nil {
c.ResponseError(err.Error())
return
}
@@ -201,12 +204,12 @@ func (c *ApiController) UpdateAgentRouting() {
// configuration of every installation Gateway can write. Both the routing form
// and the tray's one-click switch end here, so an agent reached from either
// place is left in the same state.
-func saveAgentRouting(agentId string, providerId string, fallbacks []string, mode string) error {
+func saveAgentRouting(agentId string, providerId string, fallbacks []string, mode string, strategy string) error {
if err := checkAgentProtocol(agentId, mode, append([]string{providerId}, fallbacks...)); err != nil {
return err
}
- if err := object.SetAgentRouting(agentId, providerId, fallbacks, mode); err != nil {
+ if err := object.SetAgentRouting(agentId, providerId, fallbacks, mode, strategy); err != nil {
return err
}
diff --git a/controllers/model_route.go b/controllers/model_route.go
index d30a462..2ab63ab 100644
--- a/controllers/model_route.go
+++ b/controllers/model_route.go
@@ -136,7 +136,7 @@ func (c *ApiController) PreviewModelRoute() {
DisplayName: attempt.Provider.DisplayName,
Model: attempt.Model,
Route: attempt.Route,
- Suspended: object.IsProviderSuspended(attempt.Provider.GetId()),
+ Suspended: object.IsProviderSuspended(attempt.Provider.GetId()) || object.IsSubscriptionSpent(attempt.Provider.GetId()),
})
}
c.ResponseOk(steps)
@@ -149,7 +149,7 @@ func (c *ApiController) planPreview(agentId string, model string) ([]object.Rout
return object.PlanModelRoute(model)
}
- providers, err := object.GetProvidersByAgent(agentId)
+ providers, err := object.GetAgentPool(agentId, false)
if err != nil {
return nil, err
}
diff --git a/controllers/provider_login.go b/controllers/provider_login.go
index 753661a..67c9cce 100644
--- a/controllers/provider_login.go
+++ b/controllers/provider_login.go
@@ -35,6 +35,25 @@ func (c *ApiController) SignInProvider() {
c.ResponseOk(session)
}
+// WarmUpProvider starts the short window of a subscription provider now, rather
+// than at the next time its schedule names.
+func (c *ApiController) WarmUpProvider() {
+ if c.RequireAdmin() {
+ return
+ }
+
+ provider, err := object.GetProvider(c.Input().Get("id"))
+ if err != nil {
+ c.ResponseError(err.Error())
+ return
+ }
+ if provider == nil {
+ c.ResponseError("the provider does not exist: " + c.Input().Get("id"))
+ return
+ }
+ c.ResponseOk(object.WarmUpProvider(provider))
+}
+
// GetProviderSignin reports how a sign-in that was started is getting on. The
// page polls it: approving one takes as long as whoever is at the browser.
func (c *ApiController) GetProviderSignin() {
diff --git a/controllers/proxy.go b/controllers/proxy.go
index 45840db..eda0c30 100644
--- a/controllers/proxy.go
+++ b/controllers/proxy.go
@@ -98,7 +98,10 @@ type proxyRoute struct {
// source describes how the providers were chosen, for the error a client
// sees when none of them can be used.
source string
- start time.Time
+ // conversation is the conversation this request continues, scoped to how
+ // its providers were chosen, see object.ConversationKey.
+ conversation string
+ start time.Time
// record accumulates what is written to the LLM record of this request. It
// is nil while recording is off.
record *object.LlmRecord
@@ -266,6 +269,7 @@ func (c *ApiController) proxyByModel(target proxyTarget) {
// somewhere other than the request body.
func (c *ApiController) forwardByModel(route *proxyRoute) {
route.source = "model: " + route.model
+ c.setConversation(route)
// Every way out of a proxy entry point ends the client request, which is
// what a record describes, so this is the only place one is written.
defer c.finishLlmRecord(route)
@@ -302,6 +306,7 @@ func (c *ApiController) proxyByAgent(target proxyTarget) {
func (c *ApiController) forwardByAgent(route *proxyRoute) {
agentId := c.Ctx.Input.Param(":agentId")
route.source = "agent: " + agentId
+ c.setConversation(route)
if route.record != nil {
route.record.Agent = agentId
}
@@ -321,8 +326,9 @@ func (c *ApiController) forwardByAgent(route *proxyRoute) {
}
// The whole chain is forwarded to, so a bound provider that is down fails
- // over to the agent's fallbacks instead of failing the request.
- providers, err := object.GetProvidersByAgent(agentId)
+ // over to the agent's fallbacks instead of failing the request. The agent's
+ // strategy decides which of them is tried first.
+ providers, err := object.GetAgentPool(agentId, true)
if err != nil {
if errors.Is(err, object.ErrAgentNoProvider) {
route.recordOutcome(http.StatusBadRequest, err.Error())
@@ -361,6 +367,14 @@ func (c *ApiController) forwardByAgent(route *proxyRoute) {
c.forwardAttempts(attempts, route)
}
+// setConversation names the conversation the request continues, so it is sent
+// where its earlier turns went while that provider still holds their cache.
+func (c *ApiController) setConversation(route *proxyRoute) {
+ if key := object.ConversationKey(c.Ctx.Request.Header, route.body); key != "" {
+ route.conversation = route.source + "|" + key
+ }
+}
+
// allowAgentRequest holds one relayed request to the agent's permissions: a
// model it may not ask for is refused outright, and the tools of a group it may
// not use are taken back out of the body before any provider sees it. A tool
@@ -467,6 +481,7 @@ func (c *ApiController) forwardAttempts(attempts []object.RouteAttempt, route *p
// A provider inside its failure cooldown goes last, so a dead upstream stops
// costing every request the time it takes to time out.
usable = object.SortAttemptsByHealth(usable)
+ usable = object.PreferSticky(route.conversation, usable)
// Fail over to the next attempt as long as nothing has been written to the
// client yet. The last one is never retried, so the client gets the real
@@ -520,7 +535,7 @@ func (c *ApiController) forwardToProvider(attempt object.RouteAttempt, route *pr
// the body is put in the shape that client would have sent.
requestBody = object.ShapeSubscriptionBody(provider, requestBody)
- upstreamUrl, err := object.BuildProviderUrl(provider.BaseUrl, upstream.Name(), route.upstreamEndpoint(upstream))
+ upstreamUrl, err := object.BuildProviderUrl(object.UpstreamBaseUrl(provider), upstream.Name(), route.upstreamEndpoint(upstream))
if err != nil {
object.ReportProviderFailure(provider.GetId(), err.Error())
return http.StatusBadGateway, err.Error(), false
@@ -551,6 +566,7 @@ func (c *ApiController) forwardToProvider(attempt object.RouteAttempt, route *pr
}
}
object.SetProviderAuth(upstreamReq.Header, provider)
+ object.SetSubscriptionRequestHeaders(upstreamReq.Header, provider, requestBody)
if object.UsesClientAuth(provider) {
c.copyClientAuthHeaders(upstreamReq.Header, upstream)
}
@@ -578,6 +594,10 @@ func (c *ApiController) forwardToProvider(attempt object.RouteAttempt, route *pr
defer upstreamResp.Body.Close()
reportProviderStatus(provider, upstreamResp.StatusCode)
+ object.ObserveSubscriptionLimits(provider, upstreamResp.StatusCode, upstreamResp.Header)
+ if isSuccessStatus(upstreamResp.StatusCode) {
+ object.RememberSticky(route.conversation, provider.GetId())
+ }
if !isLast && isRetryableStatus(upstreamResp.StatusCode) {
beego.Error("provider", provider.GetId(), "returned a retryable status:", upstreamResp.Status)
diff --git a/controllers/tray.go b/controllers/tray.go
index b2925c6..84eef95 100644
--- a/controllers/tray.go
+++ b/controllers/tray.go
@@ -122,6 +122,7 @@ func (c *ApiController) SetAgentProvider() {
fallbacks := []string{}
mode := object.ModeGateway
+ strategy := ""
routing, err := object.GetAgent(form.AgentId)
if err != nil {
c.ResponseError(err.Error())
@@ -129,12 +130,13 @@ func (c *ApiController) SetAgentProvider() {
}
if routing != nil {
fallbacks = routing.Fallbacks
+ strategy = routing.Strategy
if routing.Mode != "" {
mode = routing.Mode
}
}
- if err := saveAgentRouting(form.AgentId, form.Provider, fallbacks, mode); err != nil {
+ if err := saveAgentRouting(form.AgentId, form.Provider, fallbacks, mode, strategy); err != nil {
c.ResponseError(err.Error())
return
}
diff --git a/main.go b/main.go
index 672fe4c..de3d12a 100644
--- a/main.go
+++ b/main.go
@@ -115,6 +115,7 @@ func main() {
object.StartProviderProbeSweep()
// Prices are read over the same transport, and only when the sync is on.
object.StartModelsDevSync()
+ object.StartWarmupSchedule()
// A snapshot of the configuration, taken on a schedule and beside the
// database rather than inside it.
object.StartBackupSchedule()
diff --git a/object/agent.go b/object/agent.go
index f1577b9..ef6c94f 100644
--- a/object/agent.go
+++ b/object/agent.go
@@ -54,6 +54,9 @@ type Agent struct {
// the local proxy, ModeDirect writes the provider's own endpoint into the
// agent's configuration file.
Mode string `xorm:"varchar(20)" json:"mode"`
+ // Strategy is how the providers are taken in turn, see OrderPool. Empty
+ // tries them as listed.
+ Strategy string `xorm:"varchar(20)" json:"strategy"`
}
func (agent *Agent) GetId() string {
@@ -95,10 +98,13 @@ func GetAgents() (map[string]*Agent, error) {
// providers to fall over to when it cannot answer, and how the agent reaches
// them. Every provider is resolved here so a typo fails at the form rather than
// on the next relayed request.
-func SetAgentRouting(agentId string, providerId string, fallbacks []string, mode string) error {
+func SetAgentRouting(agentId string, providerId string, fallbacks []string, mode string, strategy string) error {
if agentId == "" {
return errors.New("the agent id is empty")
}
+ if err := validatePoolStrategy(strategy); err != nil {
+ return err
+ }
if mode == "" {
mode = ModeGateway
}
@@ -135,14 +141,15 @@ func SetAgentRouting(agentId string, providerId string, fallbacks []string, mode
Provider: providerId,
Fallbacks: fallbacks,
Mode: mode,
+ Strategy: strategy,
})
return err
}
// Cols() is what writes an empty provider: xorm skips zero values otherwise.
_, err = ormer.Engine.ID(core.PK{AgentOwner, agentId}).
- Cols("provider", "fallbacks", "mode", "updated_time").
- Update(&Agent{Provider: providerId, Fallbacks: fallbacks, Mode: mode, UpdatedTime: now})
+ Cols("provider", "fallbacks", "mode", "strategy", "updated_time").
+ Update(&Agent{Provider: providerId, Fallbacks: fallbacks, Mode: mode, Strategy: strategy, UpdatedTime: now})
return err
}
@@ -154,9 +161,9 @@ func SetAgentProvider(agentId string, providerId string) error {
return err
}
if stored == nil {
- return SetAgentRouting(agentId, providerId, nil, "")
+ return SetAgentRouting(agentId, providerId, nil, "", "")
}
- return SetAgentRouting(agentId, providerId, stored.Fallbacks, stored.Mode)
+ return SetAgentRouting(agentId, providerId, stored.Fallbacks, stored.Mode, stored.Strategy)
}
// normalizeFallbacks drops empty entries, duplicates and the primary provider,
@@ -223,6 +230,20 @@ func errNoProvider(agentId string) error {
ErrAgentNoProvider, agentId)
}
+// GetAgentPool is the chain in the order the agent's strategy tries it for the
+// request at hand.
+func GetAgentPool(agentId string, advance bool) ([]*Provider, error) {
+ providers, err := GetProvidersByAgent(agentId)
+ if err != nil {
+ return nil, err
+ }
+ agent, err := GetAgent(agentId)
+ if err != nil || agent == nil {
+ return providers, err
+ }
+ return OrderPool(agentId, agent.Strategy, providers, advance), nil
+}
+
// GetProviderByAgent resolves the first provider of an agent's chain.
func GetProviderByAgent(agentId string) (*Provider, error) {
providers, err := GetProvidersByAgent(agentId)
diff --git a/object/model_route.go b/object/model_route.go
index 624767d..e72b09a 100644
--- a/object/model_route.go
+++ b/object/model_route.go
@@ -424,7 +424,7 @@ func SortAttemptsByHealth(attempts []RouteAttempt) []RouteAttempt {
ready := make([]RouteAttempt, 0, len(attempts))
suspended := []RouteAttempt{}
for _, attempt := range attempts {
- if IsProviderSuspended(attempt.providerId()) {
+ if IsProviderSuspended(attempt.providerId()) || IsSubscriptionSpent(attempt.providerId()) {
suspended = append(suspended, attempt)
continue
}
diff --git a/object/provider.go b/object/provider.go
index 58c3ddf..cc23749 100644
--- a/object/provider.go
+++ b/object/provider.go
@@ -355,6 +355,8 @@ type Provider struct {
// Quota names the vendor's balance endpoint when there is no built-in one
// for it. Nil leaves the built-in table in provider_quota.go to answer.
Quota *QuotaConfig `xorm:"mediumtext json" json:"quota"`
+ // Warmup starts a subscription's short window on a schedule.
+ Warmup *WarmupConfig `xorm:"mediumtext json" json:"warmup"`
// TODO(1.2): Priority routing strategy will be defined in milestone 1.2.
Priority int `xorm:"int" json:"priority"`
Status string `xorm:"varchar(100)" json:"status"`
@@ -486,7 +488,7 @@ func validateProvider(provider *Provider) error {
}
}
- return nil
+ return normalizeWarmup(provider)
}
func validateBaseUrl(baseUrl string) error {
@@ -508,6 +510,13 @@ func validateBaseUrl(baseUrl string) error {
// /v1 a versionless base URL gets would miss it.
const ChatGptCodexPath = "/backend-api/codex"
+// servesApiAtRoot reports a host whose OpenAI-compatible API names no version,
+// which is every GitHub Copilot API host.
+func servesApiAtRoot(host string) bool {
+ host = strings.ToLower(host)
+ return host == "githubcopilot.com" || strings.HasSuffix(host, ".githubcopilot.com")
+}
+
// apiVersionSegment matches a path segment that names an API version, which is
// how a base URL says it is already the root of the API: v1, v3, v1beta.
var apiVersionSegment = regexp.MustCompile(`^v[0-9]`)
@@ -540,7 +549,7 @@ func BuildOpenAiUrl(baseUrl string, endpoint string) (string, error) {
}
path := strings.TrimSuffix(strings.TrimRight(u.Path, "/"), endpoint)
- if !namesApiVersion(path) && !strings.HasSuffix(path, ChatGptCodexPath) {
+ if !namesApiVersion(path) && !strings.HasSuffix(path, ChatGptCodexPath) && !servesApiAtRoot(u.Hostname()) {
path += "/v1"
}
diff --git a/object/provider_limit.go b/object/provider_limit.go
new file mode 100644
index 0000000..b9b7a41
--- /dev/null
+++ b/object/provider_limit.go
@@ -0,0 +1,321 @@
+// Copyright 2026 The casbin Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package object
+
+import (
+ "fmt"
+ "net/http"
+ "strconv"
+ "strings"
+ "sync"
+ "time"
+
+ "github.com/apache/casbin-gateway/agentusage"
+ "github.com/apache/casbin-gateway/util"
+)
+
+// Window names. The short and the weekly window are what a plan runs out of;
+// premium and chat are a month's allowance, which a subscription is not
+// refused for when it is spent.
+const (
+ WindowShort = "5h"
+ WindowWeek = "week"
+ WindowPremium = "premium"
+ WindowChat = "chat"
+)
+
+// QuotaWindow is one limit of a subscription: what is left of it, and when it
+// fills again. An empty ResetTime is a window that has not started.
+type QuotaWindow struct {
+ Name string `json:"name"`
+ RemainingPercent float64 `json:"remainingPercent"`
+ WindowMinutes int `json:"windowMinutes"`
+ ResetTime string `json:"resetTime,omitempty"`
+}
+
+func (window *QuotaWindow) reset() time.Time {
+ at, err := time.Parse(time.RFC3339, window.ResetTime)
+ if err != nil {
+ return time.Time{}
+ }
+ return at
+}
+
+// spent reports whether the plan refuses requests until this window resets.
+func (window *QuotaWindow) spent(now time.Time) bool {
+ if window.Name != WindowShort && window.Name != WindowWeek {
+ return false
+ }
+ return window.RemainingPercent <= 0 && window.reset().After(now)
+}
+
+func windowName(minutes int, fallback string) string {
+ switch {
+ case minutes == 300:
+ return WindowShort
+ case minutes == 10080:
+ return WindowWeek
+ case minutes > 0 && minutes%60 == 0:
+ return fmt.Sprintf("%dh", minutes/60)
+ }
+ return fallback
+}
+
+// usageWindows names the two windows a vendor's usage endpoint reports.
+func usageWindows(usage *agentusage.Usage, primary string, secondary string) []*QuotaWindow {
+ windows := []*QuotaWindow{}
+ for _, item := range []struct {
+ window *agentusage.Window
+ name string
+ }{{usage.Primary, primary}, {usage.Secondary, secondary}} {
+ if item.window == nil {
+ continue
+ }
+ name := item.name
+ if name == "" {
+ name = windowName(item.window.WindowMinutes, WindowShort)
+ }
+ windows = append(windows, &QuotaWindow{
+ Name: name,
+ RemainingPercent: item.window.RemainingPercent,
+ WindowMinutes: item.window.WindowMinutes,
+ ResetTime: item.window.ResetTime,
+ })
+ }
+ return windows
+}
+
+// subscriptionQuota asks the vendor what the plan behind a held sign-in has
+// left, with that sign-in's own token.
+func subscriptionQuota(provider *Provider, quota *ProviderQuota) *ProviderQuota {
+ vendor := vendorOfProvider(provider)
+ if vendor == nil {
+ quota.Error = "this provider is not signed in"
+ return quota
+ }
+ if vendor.usage == nil {
+ return quota
+ }
+ quota.Supported = true
+
+ if err := EnsureSubscription(provider); err != nil {
+ quota.Error = err.Error()
+ return quota
+ }
+ subscription := subscriptionOf(provider)
+ windows, plan, err := vendor.usage(subscription)
+ if err != nil {
+ quota.Error = err.Error()
+ return quota
+ }
+ quota.Windows = windows
+ quota.Plan = plan
+ if quota.Plan == "" {
+ quota.Plan = subscription.Plan
+ }
+ return quota
+}
+
+// limitBlocks holds the providers a plan refused, until the moment it said it
+// would take requests again. It is what keeps a spent account out of the way
+// for the rest of its window rather than for the breaker's few minutes.
+var limitBlocks = struct {
+ sync.Mutex
+ until map[string]time.Time
+}{until: map[string]time.Time{}}
+
+// ObserveSubscriptionLimits reads the limits a subscription endpoint reports on
+// every answer, which keeps what is left current without asking for it.
+func ObserveSubscriptionLimits(provider *Provider, statusCode int, header http.Header) {
+ vendor := vendorOfProvider(provider)
+ if vendor == nil || vendor.limits == nil {
+ return
+ }
+
+ now := time.Now()
+ windows, plan := vendor.limits(header)
+ if len(windows) > 0 {
+ storeObservedWindows(provider.GetId(), windows, plan)
+ }
+
+ if statusCode != http.StatusTooManyRequests {
+ return
+ }
+ until := time.Time{}
+ for _, window := range windows {
+ if window.spent(now) && window.reset().After(until) {
+ until = window.reset()
+ }
+ }
+ if until.IsZero() {
+ if seconds, err := strconv.Atoi(strings.TrimSpace(header.Get("Retry-After"))); err == nil && seconds > 0 {
+ until = now.Add(time.Duration(seconds) * time.Second)
+ }
+ }
+ if until.IsZero() {
+ return
+ }
+ limitBlocks.Lock()
+ limitBlocks.until[provider.GetId()] = until
+ limitBlocks.Unlock()
+}
+
+// storeObservedWindows puts what an answer reported in the quota cache, beside
+// whatever the usage endpoint said about the windows the answer did not carry.
+func storeObservedWindows(providerId string, windows []*QuotaWindow, plan string) {
+ providerQuotaMutex.Lock()
+ defer providerQuotaMutex.Unlock()
+
+ quota := &ProviderQuota{Provider: providerId, Supported: true}
+ if cached, ok := providerQuotaMap[providerId]; ok {
+ copied := *cached.quota
+ quota = &copied
+ }
+ merged := append([]*QuotaWindow{}, windows...)
+ for _, existing := range quota.Windows {
+ found := false
+ for _, window := range windows {
+ found = found || window.Name == existing.Name
+ }
+ if !found {
+ merged = append(merged, existing)
+ }
+ }
+ quota.Windows = merged
+ quota.Supported = true
+ quota.Error = ""
+ if plan != "" {
+ quota.Plan = plan
+ }
+ quota.Time = util.GetCurrentTime()
+ providerQuotaMap[providerId] = &cachedQuota{quota: quota, time: time.Now()}
+}
+
+// IsSubscriptionSpent reports whether the plan behind a provider has nothing
+// left until one of its windows resets. Like a provider in cooldown it is tried
+// last rather than dropped.
+func IsSubscriptionSpent(providerId string) bool {
+ now := time.Now()
+ limitBlocks.Lock()
+ until, blocked := limitBlocks.until[providerId]
+ if blocked && !until.After(now) {
+ delete(limitBlocks.until, providerId)
+ blocked = false
+ }
+ limitBlocks.Unlock()
+ if blocked {
+ return true
+ }
+
+ for _, window := range cachedWindows(providerId) {
+ if window.spent(now) {
+ return true
+ }
+ }
+ return false
+}
+
+func cachedWindows(providerId string) []*QuotaWindow {
+ providerQuotaMutex.Lock()
+ defer providerQuotaMutex.Unlock()
+ if cached, ok := providerQuotaMap[providerId]; ok {
+ return cached.quota.Windows
+ }
+ return nil
+}
+
+// headerFloat reads a number a header carries, false when there is none.
+func headerFloat(header http.Header, name string) (float64, bool) {
+ value := strings.TrimSpace(header.Get(name))
+ if value == "" {
+ return 0, false
+ }
+ number, err := strconv.ParseFloat(value, 64)
+ return number, err == nil
+}
+
+func unixTime(seconds float64) string {
+ if seconds <= 0 {
+ return ""
+ }
+ return time.Unix(int64(seconds), 0).UTC().Format(time.RFC3339)
+}
+
+// claudeLimits reads the windows Anthropic reports on every answer to a
+// subscription token, as the fraction spent and the reset in Unix seconds.
+func claudeLimits(header http.Header) ([]*QuotaWindow, string) {
+ windows := []*QuotaWindow{}
+ for _, item := range []struct {
+ key string
+ name string
+ minutes int
+ }{{"5h", WindowShort, 300}, {"7d", WindowWeek, 10080}} {
+ prefix := "Anthropic-Ratelimit-Unified-" + item.key
+ used, hasUsed := headerFloat(header, prefix+"-Utilization")
+ reset, hasReset := headerFloat(header, prefix+"-Reset")
+ if !hasUsed && !hasReset {
+ continue
+ }
+ windows = append(windows, &QuotaWindow{
+ Name: item.name,
+ RemainingPercent: clampPercent(100 - used*100),
+ WindowMinutes: item.minutes,
+ ResetTime: unixTime(reset),
+ })
+ }
+ return windows, ""
+}
+
+// codexLimits reads the windows the Codex backend reports on every answer, in
+// the same terms as its usage endpoint.
+func codexLimits(header http.Header) ([]*QuotaWindow, string) {
+ windows := []*QuotaWindow{}
+ for _, item := range []struct {
+ key string
+ fallback string
+ }{{"primary", WindowShort}, {"secondary", WindowWeek}} {
+ prefix := "X-Codex-" + item.key
+ used, hasUsed := headerFloat(header, prefix+"-Used-Percent")
+ if !hasUsed {
+ continue
+ }
+ minutes, _ := headerFloat(header, prefix+"-Window-Minutes")
+ resetAt, _ := headerFloat(header, prefix+"-Reset-At")
+ resetAfter, _ := headerFloat(header, prefix+"-Reset-After-Seconds")
+
+ reset := unixTime(resetAt)
+ if used == 0 && minutes > 0 && resetAfter >= minutes*60 {
+ reset = ""
+ }
+ windows = append(windows, &QuotaWindow{
+ Name: windowName(int(minutes), item.fallback),
+ RemainingPercent: clampPercent(100 - used),
+ WindowMinutes: int(minutes),
+ ResetTime: reset,
+ })
+ }
+ return windows, header.Get("X-Codex-Plan-Type")
+}
+
+func clampPercent(value float64) float64 {
+ rounded := float64(int64(value*10+0.5)) / 10
+ if value < 0 {
+ rounded = 0
+ }
+ if rounded > 100 {
+ return 100
+ }
+ return rounded
+}
diff --git a/object/provider_pool.go b/object/provider_pool.go
new file mode 100644
index 0000000..37168f6
--- /dev/null
+++ b/object/provider_pool.go
@@ -0,0 +1,273 @@
+// Copyright 2026 The casbin Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package object
+
+import (
+ "crypto/sha256"
+ "encoding/hex"
+ "encoding/json"
+ "fmt"
+ "net/http"
+ "sort"
+ "strings"
+ "sync"
+ "time"
+)
+
+// How an agent's providers - the bound one and its fallbacks - are taken in
+// turn. With several accounts of one plan among them, they are a pool.
+const (
+ // PoolOrder tries them as listed, the bound provider first.
+ PoolOrder = "order"
+ // PoolSmart spends first the plan whose short window resets soonest, since
+ // what it has left is lost at that reset, and an API key only after every
+ // plan.
+ PoolSmart = "smart"
+ // PoolRotate starts each new conversation on the next provider.
+ PoolRotate = "rotate"
+ // PoolUsage starts on the plan with the most left.
+ PoolUsage = "usage"
+)
+
+var poolStrategies = []string{"", PoolOrder, PoolSmart, PoolRotate, PoolUsage}
+
+func validatePoolStrategy(strategy string) error {
+ if !containsString(poolStrategies, strategy) {
+ return fmt.Errorf("invalid pool strategy: %s", strategy)
+ }
+ return nil
+}
+
+var poolTurns = struct {
+ sync.Mutex
+ next map[string]int
+}{next: map[string]int{}}
+
+// OrderPool puts an agent's providers in the order its strategy tries them.
+// advance moves a rotation on to the next provider, which only a request that
+// is really sent should do.
+func OrderPool(agentId string, strategy string, providers []*Provider, advance bool) []*Provider {
+ if len(providers) < 2 {
+ return providers
+ }
+
+ ordered := append([]*Provider{}, providers...)
+ switch strategy {
+ case PoolRotate:
+ poolTurns.Lock()
+ turn := poolTurns.next[agentId] % len(ordered)
+ if advance {
+ poolTurns.next[agentId] = turn + 1
+ }
+ poolTurns.Unlock()
+ return append(ordered[turn:], ordered[:turn]...)
+ case PoolSmart, PoolUsage:
+ now := time.Now()
+ standings := make(map[string]poolStanding, len(ordered))
+ for _, provider := range ordered {
+ standings[provider.GetId()] = standingOf(provider, now)
+ }
+ sort.SliceStable(ordered, func(i, j int) bool {
+ left, right := standings[ordered[i].GetId()], standings[ordered[j].GetId()]
+ if left.class != right.class {
+ return left.class < right.class
+ }
+ if left.class != poolKnown {
+ return false
+ }
+ if strategy == PoolSmart && !left.reset.Equal(right.reset) {
+ return left.reset.Before(right.reset)
+ }
+ return left.remaining > right.remaining
+ })
+ }
+ return ordered
+}
+
+const (
+ poolKnown = iota
+ poolUnknown
+ poolKey
+)
+
+// poolStanding is what a strategy compares providers by: a plan whose limits
+// are known, a plan whose limits are not, or a key with no plan behind it.
+type poolStanding struct {
+ class int
+ // reset is when the short window resets, far off for one that has not
+ // started: its allowance is not lost by waiting.
+ reset time.Time
+ // remaining is the least left of any window, since the tightest one is
+ // what refuses the next request.
+ remaining float64
+}
+
+func standingOf(provider *Provider, now time.Time) poolStanding {
+ if !UsesSubscription(provider) {
+ return poolStanding{class: poolKey}
+ }
+
+ windows := cachedWindows(provider.GetId())
+ standing := poolStanding{class: poolUnknown, reset: now.Add(365 * 24 * time.Hour), remaining: 100}
+ for _, window := range windows {
+ if window.Name != WindowShort && window.Name != WindowWeek {
+ continue
+ }
+ standing.class = poolKnown
+ if window.RemainingPercent < standing.remaining {
+ standing.remaining = window.RemainingPercent
+ }
+ if at := window.reset(); window.Name == WindowShort && at.After(now) {
+ standing.reset = at
+ }
+ }
+ return standing
+}
+
+// stickyIdle is how long a conversation keeps the provider that answered it
+// after its last turn. Prompt caches outlive a short pause and not much more,
+// so past this a conversation is free to go where the strategy says.
+const stickyIdle = 15 * time.Minute
+
+var stickies = struct {
+ sync.Mutex
+ byKey map[string]stickyEntry
+}{byKey: map[string]stickyEntry{}}
+
+type stickyEntry struct {
+ provider string
+ seen time.Time
+}
+
+// ConversationKey names the conversation a request belongs to, empty when it
+// cannot be told. An agent that says which session it is in is taken at its
+// word; otherwise a conversation is known by how it began, which every later
+// turn repeats.
+func ConversationKey(header http.Header, body []byte) string {
+ for _, name := range []string{"X-Claude-Code-Session-Id", "Session_id", "Session-Id", "X-Session-Id", "Thread-Id", "Conversation_id"} {
+ if value := strings.TrimSpace(header.Get(name)); value != "" {
+ return "session:" + value
+ }
+ }
+
+ var fields struct {
+ PromptCacheKey string `json:"prompt_cache_key"`
+ Metadata struct {
+ UserId string `json:"user_id"`
+ } `json:"metadata"`
+ Messages []json.RawMessage `json:"messages"`
+ Input json.RawMessage `json:"input"`
+ Contents []json.RawMessage `json:"contents"`
+ System json.RawMessage `json:"system"`
+ Instructions string `json:"instructions"`
+ }
+ if json.Unmarshal(body, &fields) != nil {
+ return ""
+ }
+ if fields.PromptCacheKey != "" {
+ return "cache:" + fields.PromptCacheKey
+ }
+ // Claude Code puts its session in the user id; other clients put a user
+ // there, which is still one conversation stream per person.
+ if fields.Metadata.UserId != "" {
+ return "user:" + fields.Metadata.UserId
+ }
+
+ first := []byte(nil)
+ switch {
+ case len(fields.Messages) > 0:
+ first = firstNonSystem(fields.Messages)
+ case len(fields.Contents) > 0:
+ first = fields.Contents[0]
+ case len(fields.Input) > 0:
+ var items []json.RawMessage
+ if json.Unmarshal(fields.Input, &items) == nil && len(items) > 0 {
+ first = items[0]
+ } else {
+ first = fields.Input
+ }
+ }
+ if len(first) == 0 {
+ return ""
+ }
+
+ sum := sha256.New()
+ sum.Write(fields.System)
+ sum.Write([]byte(fields.Instructions))
+ sum.Write(first)
+ return "start:" + hex.EncodeToString(sum.Sum(nil)[:12])
+}
+
+func firstNonSystem(messages []json.RawMessage) []byte {
+ for _, message := range messages {
+ var role struct {
+ Role string `json:"role"`
+ }
+ if json.Unmarshal(message, &role) == nil && (role.Role == "system" || role.Role == "developer") {
+ continue
+ }
+ return message
+ }
+ return nil
+}
+
+// PreferSticky moves the provider that answered this conversation last to the
+// front, so its prompt cache is spent rather than paid for again elsewhere. A
+// provider that is cooling down or out of its plan is left where it is.
+func PreferSticky(key string, attempts []RouteAttempt) []RouteAttempt {
+ if key == "" || len(attempts) < 2 {
+ return attempts
+ }
+
+ stickies.Lock()
+ entry, ok := stickies.byKey[key]
+ stickies.Unlock()
+ if !ok || time.Since(entry.seen) > stickyIdle {
+ return attempts
+ }
+ if IsProviderSuspended(entry.provider) || IsSubscriptionSpent(entry.provider) {
+ return attempts
+ }
+
+ front := []RouteAttempt{}
+ rest := []RouteAttempt{}
+ for _, attempt := range attempts {
+ if attempt.providerId() == entry.provider {
+ front = append(front, attempt)
+ } else {
+ rest = append(rest, attempt)
+ }
+ }
+ return append(front, rest...)
+}
+
+// RememberSticky records which provider answered a conversation.
+func RememberSticky(key string, providerId string) {
+ if key == "" {
+ return
+ }
+
+ now := time.Now()
+ stickies.Lock()
+ defer stickies.Unlock()
+ stickies.byKey[key] = stickyEntry{provider: providerId, seen: now}
+ if len(stickies.byKey) > 4096 {
+ for name, entry := range stickies.byKey {
+ if now.Sub(entry.seen) > stickyIdle {
+ delete(stickies.byKey, name)
+ }
+ }
+ }
+}
diff --git a/object/provider_quota.go b/object/provider_quota.go
index 752a584..6a884a0 100644
--- a/object/provider_quota.go
+++ b/object/provider_quota.go
@@ -45,8 +45,14 @@ type ProviderQuota struct {
Used *float64 `json:"used"`
Total *float64 `json:"total"`
Unit string `json:"unit"`
- Error string `json:"error"`
- Time string `json:"time"`
+ // Windows is what a subscription has left of its plan limits, which is a
+ // share of each window rather than an amount of money.
+ Windows []*QuotaWindow `json:"windows,omitempty"`
+ Plan string `json:"plan,omitempty"`
+ // Warmup is how the last warm-up of a subscription went.
+ Warmup *WarmupResult `json:"warmup,omitempty"`
+ Error string `json:"error"`
+ Time string `json:"time"`
}
// QuotaConfig points the probe at a vendor's own balance endpoint, for the
@@ -295,6 +301,9 @@ func probeQuota(provider *Provider) *ProviderQuota {
Time: util.GetCurrentTime(),
}
+ if UsesSubscription(provider) {
+ return subscriptionQuota(provider, quota)
+ }
if UsesClientAuth(provider) {
quota.Error = "the caller's own login is forwarded, so there is no account here to ask about"
return quota
@@ -458,7 +467,9 @@ func GetProviderQuotas(providers []*Provider) []*ProviderQuota {
result := []*ProviderQuota{}
for _, provider := range providers {
if cached, ok := providerQuotaMap[provider.GetId()]; ok {
- result = append(result, cached.quota)
+ quota := *cached.quota
+ quota.Warmup = lastWarmup(provider.GetId())
+ result = append(result, "a)
}
}
return result
diff --git a/object/provider_subscription.go b/object/provider_subscription.go
index 5281bfc..a38e655 100644
--- a/object/provider_subscription.go
+++ b/object/provider_subscription.go
@@ -26,6 +26,7 @@ import (
"time"
"github.com/apache/casbin-gateway/agent"
+ "github.com/apache/casbin-gateway/agentusage"
"github.com/apache/casbin-gateway/proxy"
"github.com/xorm-io/core"
)
@@ -36,6 +37,7 @@ import (
const (
SubscriptionOpenAi = "openai"
SubscriptionAnthropic = "anthropic"
+ SubscriptionCopilot = "copilot"
)
// What a subscription's requests have to look like to be served: the vendor
@@ -84,6 +86,26 @@ type SubscriptionVendor struct {
// shape rewrites a request body the vendor's endpoint would otherwise
// refuse. Nil where nothing has to be changed.
shape func([]byte) []byte
+
+ // deviceCodeUrl marks a vendor signed in to with a code typed on its site
+ // rather than with a redirect back here.
+ deviceCodeUrl string
+ // renew turns the long-lived grant into a token requests are made with, for
+ // a vendor that does not renew through the OAuth refresh grant.
+ renew func(*ProviderSubscription) (*ProviderSubscription, error)
+ // headers adds what depends on the request itself, beside the sign-in.
+ headers func(http.Header, []byte)
+ // usage asks what the plan behind a sign-in has left, and limits reads the
+ // same out of the headers of an answer. Either is nil where the vendor
+ // reports nothing.
+ usage func(*ProviderSubscription) ([]*QuotaWindow, string, error)
+ limits func(http.Header) ([]*QuotaWindow, string)
+ // warmModel is what a warm-up asks for when the provider has relayed no
+ // request yet to take the model from.
+ warmModel string
+ // warms marks a plan whose short window starts at the first request, which
+ // is the only kind a warm-up does anything for.
+ warms bool
}
var subscriptionVendors = []*SubscriptionVendor{
@@ -114,6 +136,15 @@ var subscriptionVendors = []*SubscriptionVendor{
header.Set("Originator", codexOriginator)
}
},
+ usage: func(subscription *ProviderSubscription) ([]*QuotaWindow, string, error) {
+ usage, err := agentusage.CodexToken(subscription.AccessToken, subscription.AccountId)
+ if err != nil {
+ return nil, "", err
+ }
+ return usageWindows(usage, "", ""), usage.Plan, nil
+ },
+ limits: codexLimits,
+ warms: true,
},
{
Id: SubscriptionAnthropic,
@@ -142,6 +173,37 @@ var subscriptionVendors = []*SubscriptionVendor{
header.Set("User-Agent", claudeUserAgent)
},
shape: withClaudePreamble,
+ usage: func(subscription *ProviderSubscription) ([]*QuotaWindow, string, error) {
+ usage, err := agentusage.Claude(subscription.AccessToken)
+ if err != nil {
+ return nil, "", err
+ }
+ return usageWindows(usage, WindowShort, WindowWeek), "", nil
+ },
+ limits: claudeLimits,
+ warmModel: "claude-haiku-4-5",
+ warms: true,
+ },
+ {
+ Id: SubscriptionCopilot,
+ Label: "GitHub Copilot",
+ BaseUrl: copilotBaseUrl,
+ Protocol: ProtocolOpenAi,
+
+ clientId: copilotClientId,
+ deviceCodeUrl: "https://github.com/login/device/code",
+ tokenUrl: "https://github.com/login/oauth/access_token",
+ scope: "read:user",
+ renew: renewCopilot,
+ apply: applyCopilot,
+ headers: copilotRequestHeaders,
+ usage: func(subscription *ProviderSubscription) ([]*QuotaWindow, string, error) {
+ usage, err := agentusage.Copilot(subscription.RefreshToken)
+ if err != nil {
+ return nil, "", err
+ }
+ return usageWindows(usage, WindowPremium, WindowChat), usage.Plan, nil
+ },
},
}
@@ -175,6 +237,9 @@ type ProviderSubscription struct {
Account string `json:"account"`
Plan string `json:"plan"`
ExpiresAt string `json:"expiresAt"`
+ // Endpoint is where this account's requests go, for a vendor that hands
+ // each account an API host of its own. Empty uses the provider's base URL.
+ Endpoint string `json:"endpoint,omitempty"`
}
func (subscription *ProviderSubscription) expiry() time.Time {
@@ -260,7 +325,12 @@ func EnsureSubscription(provider *Provider) error {
if err != nil {
return err
}
- granted, err := refreshSubscription(vendor, subscription)
+ var granted *ProviderSubscription
+ if vendor.renew != nil {
+ granted, err = vendor.renew(subscription)
+ } else {
+ granted, err = refreshSubscription(vendor, subscription)
+ }
if err != nil {
return err
}
@@ -300,6 +370,23 @@ func SetSubscriptionAuth(header http.Header, provider *Provider) {
vendor.apply(header, subscription)
}
+// SetSubscriptionRequestHeaders adds what a subscription endpoint reads off the
+// request itself rather than off the sign-in.
+func SetSubscriptionRequestHeaders(header http.Header, provider *Provider, body []byte) {
+ if vendor := vendorOfProvider(provider); vendor != nil && vendor.headers != nil {
+ vendor.headers(header, body)
+ }
+}
+
+// UpstreamBaseUrl is where a provider's requests go: the base URL, or the API
+// host the vendor handed the account behind a held sign-in.
+func UpstreamBaseUrl(provider *Provider) string {
+ if subscription := subscriptionOf(provider); subscription != nil && subscription.Endpoint != "" {
+ return subscription.Endpoint
+ }
+ return provider.BaseUrl
+}
+
// SubscriptionQuery is what a subscription endpoint wants on the URL, empty for
// one that wants nothing.
func SubscriptionQuery(provider *Provider) string {
@@ -408,12 +495,18 @@ type SubscriptionLogin struct {
Id string `json:"id"`
Vendor string `json:"vendor"`
// Url is where the sign-in is approved. The page opens it.
- Url string `json:"url"`
- Running bool `json:"running"`
- Ok bool `json:"ok"`
- Account string `json:"account,omitempty"`
- Plan string `json:"plan,omitempty"`
- Error string `json:"error,omitempty"`
+ Url string `json:"url"`
+ // UserCode is what is typed on the vendor's page, for a sign-in that is
+ // approved there rather than redirected back here.
+ UserCode string `json:"userCode,omitempty"`
+ Running bool `json:"running"`
+ Ok bool `json:"ok"`
+ Account string `json:"account,omitempty"`
+ Plan string `json:"plan,omitempty"`
+ Error string `json:"error,omitempty"`
+ // Models is what the account may ask for, for a vendor whose model names
+ // are its own and not the ones an agent asks for.
+ Models []string `json:"models,omitempty"`
// granted is what the finished sign-in brought back, taken out of here by
// the save that stores it on a provider. It never reaches the browser.
@@ -441,6 +534,9 @@ func StartSubscriptionLogin(vendorId string) (*SubscriptionLogin, error) {
if err != nil {
return nil, err
}
+ if vendor.deviceCodeUrl != "" {
+ return startDeviceLogin(vendor)
+ }
id, err := randomString()
if err != nil {
@@ -552,11 +648,18 @@ func cancelRunningLogins(vendorId string) {
}
session.Unlock()
if running {
- _ = session.server.Close()
+ session.close()
}
}
}
+// close stops listening for the browser, for a sign-in that listens at all.
+func (session *loginSession) close() {
+ if session.server != nil {
+ _ = session.server.Close()
+ }
+}
+
// SubscriptionLoginSession is how a sign-in that was started is getting on.
func SubscriptionLoginSession(id string) (SubscriptionLogin, bool) {
logins.Lock()
@@ -615,9 +718,7 @@ func (session *loginSession) finish(granted *ProviderSubscription, err error) {
session.Plan = granted.Plan
}
session.Unlock()
- go func() {
- _ = session.server.Close()
- }()
+ go session.close()
}
func finishSubscriptionLogin(session *loginSession, vendor *SubscriptionVendor, query url.Values, writer http.ResponseWriter) {
diff --git a/object/provider_subscription_copilot.go b/object/provider_subscription_copilot.go
new file mode 100644
index 0000000..f849c80
--- /dev/null
+++ b/object/provider_subscription_copilot.go
@@ -0,0 +1,344 @@
+// Copyright 2026 The casbin Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package object
+
+import (
+ "encoding/json"
+ "fmt"
+ "io"
+ "net/http"
+ "net/url"
+ "strings"
+ "time"
+
+ "github.com/apache/casbin-gateway/agentusage"
+ "github.com/apache/casbin-gateway/proxy"
+)
+
+// A Copilot plan is signed in to with GitHub's device flow, under the client id
+// the Copilot editor plugins use: the GitHub token it grants is exchanged for a
+// short-lived Copilot token, which is what the requests carry.
+const (
+ copilotClientId = "Iv1.b507a08c87ecfe98"
+ copilotBaseUrl = "https://api.githubcopilot.com"
+ copilotTokenUrl = "https://api.github.com/copilot_internal/v2/token"
+ copilotUserUrl = "https://api.github.com/user"
+ copilotEditorVersion = "vscode/1.107.0"
+ copilotPluginVersion = "copilot-chat/0.35.0"
+)
+
+func startDeviceLogin(vendor *SubscriptionVendor) (*SubscriptionLogin, error) {
+ form := url.Values{}
+ form.Set("client_id", vendor.clientId)
+ form.Set("scope", vendor.scope)
+
+ var device struct {
+ DeviceCode string `json:"device_code"`
+ UserCode string `json:"user_code"`
+ VerificationUri string `json:"verification_uri"`
+ ExpiresIn int `json:"expires_in"`
+ Interval int `json:"interval"`
+ Error string `json:"error"`
+ ErrorText string `json:"error_description"`
+ }
+ if err := postGithubForm(vendor.deviceCodeUrl, form, &device); err != nil {
+ return nil, err
+ }
+ if device.Error != "" {
+ return nil, fmt.Errorf("github refused the sign-in: %s %s", device.Error, device.ErrorText)
+ }
+ if device.DeviceCode == "" || device.UserCode == "" {
+ return nil, fmt.Errorf("github returned no code to sign in with")
+ }
+
+ id, err := randomString()
+ if err != nil {
+ return nil, err
+ }
+ session := &loginSession{
+ SubscriptionLogin: SubscriptionLogin{
+ Id: id,
+ Vendor: vendor.Id,
+ Url: device.VerificationUri,
+ UserCode: device.UserCode,
+ Running: true,
+ },
+ started: time.Now(),
+ }
+
+ cancelRunningLogins(vendor.Id)
+ logins.Lock()
+ forgetFinishedLoginsLocked()
+ logins.byId[id] = session
+ logins.Unlock()
+
+ expires := time.Duration(device.ExpiresIn) * time.Second
+ if expires <= 0 || expires > pendingTtl {
+ expires = pendingTtl
+ }
+ go pollDeviceLogin(session, vendor, device.DeviceCode, device.Interval, time.Now().Add(expires))
+
+ answer := session.snapshot()
+ return &answer, nil
+}
+
+// pollDeviceLogin asks GitHub, at the pace it set, whether the code was entered.
+func pollDeviceLogin(session *loginSession, vendor *SubscriptionVendor, deviceCode string, interval int, deadline time.Time) {
+ if interval <= 0 {
+ interval = 5
+ }
+
+ for time.Now().Before(deadline) {
+ time.Sleep(time.Duration(interval) * time.Second)
+
+ session.Lock()
+ running := session.Running
+ session.Unlock()
+ if !running {
+ return
+ }
+
+ form := url.Values{}
+ form.Set("client_id", vendor.clientId)
+ form.Set("device_code", deviceCode)
+ form.Set("grant_type", "urn:ietf:params:oauth:grant-type:device_code")
+
+ var answer struct {
+ AccessToken string `json:"access_token"`
+ Error string `json:"error"`
+ ErrorText string `json:"error_description"`
+ }
+ if err := postGithubForm(vendor.tokenUrl, form, &answer); err != nil {
+ continue
+ }
+
+ switch answer.Error {
+ case "":
+ case "authorization_pending":
+ continue
+ case "slow_down":
+ interval += 5
+ continue
+ default:
+ session.finish(nil, fmt.Errorf("%s %s", answer.Error, answer.ErrorText))
+ return
+ }
+ if answer.AccessToken == "" {
+ continue
+ }
+
+ granted, err := renewCopilot(&ProviderSubscription{Vendor: vendor.Id, RefreshToken: answer.AccessToken})
+ if err == nil {
+ describeCopilot(granted)
+ models := copilotModels(granted)
+ session.Lock()
+ session.Models = models
+ session.Unlock()
+ }
+ session.finish(granted, err)
+ return
+ }
+
+ session.finish(nil, fmt.Errorf("the sign-in was not finished in time"))
+}
+
+func postGithubForm(endpoint string, form url.Values, target any) error {
+ request, err := http.NewRequest(http.MethodPost, endpoint, strings.NewReader(form.Encode()))
+ if err != nil {
+ return err
+ }
+ request.Header.Set("Content-Type", "application/x-www-form-urlencoded")
+ request.Header.Set("Accept", "application/json")
+
+ client := &http.Client{Timeout: exchangeTimeout, Transport: proxy.Transport()}
+ response, err := client.Do(request)
+ if err != nil {
+ return err
+ }
+ defer response.Body.Close()
+
+ body, err := io.ReadAll(io.LimitReader(response.Body, 1024*1024))
+ if err != nil {
+ return err
+ }
+ if err := json.Unmarshal(body, target); err != nil {
+ return fmt.Errorf("github answered %s: %s", response.Status, summarize(body))
+ }
+ return nil
+}
+
+// renewCopilot exchanges the GitHub token for a Copilot token, which lives for
+// about half an hour and names the API host this account is served from.
+func renewCopilot(subscription *ProviderSubscription) (*ProviderSubscription, error) {
+ request, err := http.NewRequest(http.MethodGet, copilotTokenUrl, nil)
+ if err != nil {
+ return nil, err
+ }
+ request.Header.Set("Authorization", "token "+subscription.RefreshToken)
+ request.Header.Set("Accept", "application/json")
+ request.Header.Set("Editor-Version", copilotEditorVersion)
+ request.Header.Set("Editor-Plugin-Version", copilotPluginVersion)
+
+ client := &http.Client{Timeout: exchangeTimeout, Transport: proxy.Transport()}
+ response, err := client.Do(request)
+ if err != nil {
+ return nil, err
+ }
+ defer response.Body.Close()
+
+ body, err := io.ReadAll(io.LimitReader(response.Body, 1024*1024))
+ if err != nil {
+ return nil, err
+ }
+ if response.StatusCode == http.StatusUnauthorized {
+ return nil, fmt.Errorf("github no longer accepts this sign-in, sign in again")
+ }
+ if response.StatusCode == http.StatusForbidden || response.StatusCode == http.StatusNotFound {
+ return nil, fmt.Errorf("this GitHub account has no Copilot plan: %s", summarize(body))
+ }
+ if response.StatusCode >= 400 {
+ return nil, fmt.Errorf("github answered %s: %s", response.Status, summarize(body))
+ }
+
+ var answer struct {
+ Token string `json:"token"`
+ ExpiresAt int64 `json:"expires_at"`
+ Endpoints struct {
+ Api string `json:"api"`
+ } `json:"endpoints"`
+ }
+ if err := json.Unmarshal(body, &answer); err != nil || answer.Token == "" {
+ return nil, fmt.Errorf("github returned no Copilot token: %s", summarize(body))
+ }
+
+ renewed := *subscription
+ renewed.AccessToken = answer.Token
+ renewed.Endpoint = strings.TrimRight(answer.Endpoints.Api, "/")
+ renewed.ExpiresAt = ""
+ if answer.ExpiresAt > 0 {
+ renewed.ExpiresAt = time.Unix(answer.ExpiresAt, 0).UTC().Format(time.RFC3339)
+ }
+ return &renewed, nil
+}
+
+// describeCopilot fills in whose account this is and which plan it is on. What
+// cannot be read is left empty rather than failing a sign-in that worked.
+func describeCopilot(subscription *ProviderSubscription) {
+ request, err := http.NewRequest(http.MethodGet, copilotUserUrl, nil)
+ if err == nil {
+ request.Header.Set("Authorization", "token "+subscription.RefreshToken)
+ request.Header.Set("Accept", "application/json")
+ client := &http.Client{Timeout: exchangeTimeout, Transport: proxy.Transport()}
+ if response, err := client.Do(request); err == nil {
+ var user struct {
+ Login string `json:"login"`
+ Email string `json:"email"`
+ }
+ if json.NewDecoder(io.LimitReader(response.Body, 1024*1024)).Decode(&user) == nil {
+ fillClaim(&subscription.Account, user.Login)
+ }
+ response.Body.Close()
+ }
+ }
+
+ if usage, err := agentusage.Copilot(subscription.RefreshToken); err == nil {
+ fillClaim(&subscription.Plan, usage.Plan)
+ }
+}
+
+// copilotModels lists the chat models the account may pick. A Copilot model is
+// named its own way (claude-sonnet-4.5), so the list is what lets an agent's
+// own model names be matched to one.
+func copilotModels(subscription *ProviderSubscription) []string {
+ base := subscription.Endpoint
+ if base == "" {
+ base = copilotBaseUrl
+ }
+ request, err := http.NewRequest(http.MethodGet, base+"/models", nil)
+ if err != nil {
+ return nil
+ }
+ applyCopilot(request.Header, subscription)
+
+ client := &http.Client{Timeout: exchangeTimeout, Transport: proxy.Transport()}
+ response, err := client.Do(request)
+ if err != nil {
+ return nil
+ }
+ defer response.Body.Close()
+ if response.StatusCode >= 400 {
+ return nil
+ }
+
+ var answer struct {
+ Data []struct {
+ Id string `json:"id"`
+ PickerOn bool `json:"model_picker_enabled"`
+ Capabilities struct {
+ Type string `json:"type"`
+ } `json:"capabilities"`
+ } `json:"data"`
+ }
+ if json.NewDecoder(io.LimitReader(response.Body, maxProbeBody)).Decode(&answer) != nil {
+ return nil
+ }
+
+ models := []string{}
+ seen := map[string]bool{}
+ for _, model := range answer.Data {
+ if !model.PickerOn || (model.Capabilities.Type != "" && model.Capabilities.Type != "chat") || seen[model.Id] {
+ continue
+ }
+ seen[model.Id] = true
+ models = append(models, model.Id)
+ }
+ return models
+}
+
+func applyCopilot(header http.Header, subscription *ProviderSubscription) {
+ header.Set("Authorization", "Bearer "+subscription.AccessToken)
+ header.Set("Copilot-Integration-Id", "vscode-chat")
+ header.Set("Editor-Version", copilotEditorVersion)
+ header.Set("Editor-Plugin-Version", copilotPluginVersion)
+ header.Set("Openai-Intent", "conversation-edits")
+}
+
+// copilotRequestHeaders says who started the turn. A turn the agent took on its
+// own - answering a tool result - is not a premium request the person made, and
+// Copilot bills it that way only when told.
+func copilotRequestHeaders(header http.Header, body []byte) {
+ var request struct {
+ Messages []struct {
+ Role string `json:"role"`
+ Content json.RawMessage `json:"content"`
+ } `json:"messages"`
+ }
+ if json.Unmarshal(body, &request) != nil || len(request.Messages) == 0 {
+ return
+ }
+
+ initiator := "agent"
+ if last := request.Messages[len(request.Messages)-1]; last.Role == "user" {
+ initiator = "user"
+ }
+ header.Set("X-Initiator", initiator)
+
+ for _, message := range request.Messages {
+ if strings.Contains(string(message.Content), `"image_url"`) {
+ header.Set("Copilot-Vision-Request", "true")
+ break
+ }
+ }
+}
diff --git a/object/provider_warmup.go b/object/provider_warmup.go
new file mode 100644
index 0000000..f5dcda0
--- /dev/null
+++ b/object/provider_warmup.go
@@ -0,0 +1,312 @@
+// Copyright 2026 The casbin Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package object
+
+import (
+ "bytes"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "net/http"
+ "regexp"
+ "strings"
+ "sync"
+ "time"
+
+ "github.com/apache/casbin-gateway/protocol"
+ "github.com/apache/casbin-gateway/proxy"
+ "github.com/apache/casbin-gateway/util"
+)
+
+// WarmupConfig starts a subscription's short window at set times of the day. A
+// Claude or ChatGPT plan counts its five hours from the first request, so one
+// sent early in the morning has the window reset by midday, and the afternoon
+// starts on a full one.
+type WarmupConfig struct {
+ Enabled bool `json:"enabled"`
+ // Times are local, "HH:MM".
+ Times []string `json:"times"`
+ // Model is what is asked for. Empty takes the model this provider last
+ // relayed, or the vendor's smallest.
+ Model string `json:"model"`
+}
+
+// WarmupResult is how one warm-up went.
+type WarmupResult struct {
+ Time string `json:"time"`
+ Ok bool `json:"ok"`
+ Model string `json:"model,omitempty"`
+ Message string `json:"message"`
+}
+
+var warmupTime = regexp.MustCompile(`^([01][0-9]|2[0-3]):[0-5][0-9]$`)
+
+// normalizeWarmup drops what a schedule cannot mean, and refuses a time that
+// is not one.
+func normalizeWarmup(provider *Provider) error {
+ config := provider.Warmup
+ if config == nil {
+ return nil
+ }
+ if !UsesSubscription(provider) {
+ provider.Warmup = nil
+ return nil
+ }
+
+ times := []string{}
+ seen := map[string]bool{}
+ for _, value := range config.Times {
+ value = strings.TrimSpace(value)
+ if value == "" || seen[value] {
+ continue
+ }
+ if !warmupTime.MatchString(value) {
+ return fmt.Errorf("the warm-up time %q is not a time of day written as HH:MM", value)
+ }
+ seen[value] = true
+ times = append(times, value)
+ }
+ config.Times = times
+ config.Model = strings.TrimSpace(config.Model)
+ return nil
+}
+
+var warmups = struct {
+ sync.Mutex
+ last map[string]*WarmupResult
+ fired map[string]bool
+}{last: map[string]*WarmupResult{}, fired: map[string]bool{}}
+
+func lastWarmup(providerId string) *WarmupResult {
+ warmups.Lock()
+ defer warmups.Unlock()
+ return warmups.last[providerId]
+}
+
+func rememberWarmup(providerId string, result *WarmupResult) *WarmupResult {
+ warmups.Lock()
+ warmups.last[providerId] = result
+ warmups.Unlock()
+ return result
+}
+
+// WarmUpProvider sends a subscription the smallest request its endpoint takes,
+// which is what starts the plan's short window.
+func WarmUpProvider(provider *Provider) *WarmupResult {
+ result := &WarmupResult{Time: util.GetCurrentTime()}
+ model, err := warmUp(provider)
+ result.Model = model
+ switch {
+ case err != nil:
+ result.Message = err.Error()
+ case runningShortWindow(provider.GetId(), time.Now()) != "":
+ result.Ok = true
+ result.Message = "the window runs until " + runningShortWindow(provider.GetId(), time.Now())
+ default:
+ result.Ok = true
+ result.Message = "the request was answered"
+ }
+ return rememberWarmup(provider.GetId(), result)
+}
+
+func warmUp(provider *Provider) (string, error) {
+ vendor := vendorOfProvider(provider)
+ if vendor == nil {
+ return "", errors.New("this provider holds no sign-in to warm up")
+ }
+ if !vendor.warms {
+ return "", fmt.Errorf("a %s plan has no window that a request starts", vendor.Label)
+ }
+ if err := EnsureSubscription(provider); err != nil {
+ return "", err
+ }
+
+ model := ""
+ if provider.Warmup != nil {
+ model = provider.Warmup.Model
+ }
+ if model == "" {
+ model = lastRelayedModel(provider.GetId())
+ }
+ if model == "" {
+ model = vendor.warmModel
+ }
+ if model == "" {
+ return "", errors.New("name the model to warm up with: this provider has not relayed a request yet")
+ }
+
+ name := ProviderProtocol(provider)
+ upstream, err := protocol.UpstreamOf(name)
+ if err != nil {
+ return model, err
+ }
+ target, err := BuildProviderUrl(UpstreamBaseUrl(provider), name, upstream.Endpoint())
+ if err != nil {
+ return model, err
+ }
+ target = AppendQuery(target, SubscriptionQuery(provider))
+
+ var body map[string]any
+ switch name {
+ case ProtocolAnthropic:
+ body = map[string]any{
+ "model": model,
+ "max_tokens": 1,
+ "messages": []any{map[string]any{"role": "user", "content": "hi"}},
+ }
+ case ProtocolResponses:
+ body = map[string]any{
+ "model": model,
+ "input": []any{map[string]any{
+ "role": "user",
+ "content": []any{map[string]any{"type": "input_text", "text": "hi"}},
+ }},
+ "store": false,
+ "stream": true,
+ }
+ default:
+ body = map[string]any{
+ "model": model,
+ "max_tokens": 1,
+ "messages": []any{map[string]any{"role": "user", "content": "hi"}},
+ }
+ }
+ encoded, err := json.Marshal(body)
+ if err != nil {
+ return model, err
+ }
+ encoded = ShapeSubscriptionBody(provider, encoded)
+
+ request, err := http.NewRequest(http.MethodPost, target, bytes.NewReader(encoded))
+ if err != nil {
+ return model, err
+ }
+ request.Header.Set("Content-Type", "application/json")
+ SetProviderAuth(request.Header, provider)
+
+ client := &http.Client{Timeout: time.Minute, Transport: proxy.Transport()}
+ response, err := client.Do(request)
+ if err != nil {
+ return model, err
+ }
+ defer response.Body.Close()
+ answer, _ := io.ReadAll(io.LimitReader(response.Body, maxProbeBody))
+
+ ObserveSubscriptionLimits(provider, response.StatusCode, response.Header)
+ if response.StatusCode >= 300 {
+ return model, fmt.Errorf("the vendor answered %s%s", response.Status, probeDetail(answer))
+ }
+ return model, nil
+}
+
+// lastRelayedModel is the model this provider last answered for, which is one
+// its plan certainly serves.
+func lastRelayedModel(providerId string) string {
+ if ormer == nil || ormer.Engine == nil {
+ return ""
+ }
+ record := &LlmRecord{}
+ existed, err := ormer.Engine.Where("provider = ?", providerId).
+ And("status >= 200 and status < 300").
+ Desc("created_time").Get(record)
+ if err != nil || !existed {
+ return ""
+ }
+ // A request relayed as it arrived names its model only once.
+ if record.UpstreamModel != "" {
+ return record.UpstreamModel
+ }
+ return record.Model
+}
+
+// StartWarmupSchedule checks every half minute for a warm-up that is due.
+func StartWarmupSchedule() {
+ go func() {
+ ticker := time.NewTicker(30 * time.Second)
+ defer ticker.Stop()
+ for now := range ticker.C {
+ runDueWarmups(now)
+ }
+ }()
+}
+
+func runDueWarmups(now time.Time) {
+ providers, err := listEnabledProviders()
+ if err != nil {
+ return
+ }
+ decryptProviders(providers)
+
+ clock := now.Format("15:04")
+ day := now.Format("2006-01-02")
+ for _, provider := range providers {
+ config := provider.Warmup
+ if !UsesSubscription(provider) || config == nil || !config.Enabled {
+ continue
+ }
+ for _, at := range config.Times {
+ if at != clock {
+ continue
+ }
+ key := provider.GetId() + "|" + day + "|" + at
+ warmups.Lock()
+ fired := warmups.fired[key]
+ warmups.fired[key] = true
+ warmups.Unlock()
+ if fired {
+ continue
+ }
+
+ go warmUpIfIdle(provider)
+ }
+ }
+ forgetOldWarmups(day)
+}
+
+// warmUpIfIdle warms a subscription up unless its short window is already
+// running, which another request would not start again. What is left is asked
+// of the vendor first when nothing is known, which costs none of the plan.
+func warmUpIfIdle(provider *Provider) {
+ RefreshProviderQuotas([]*Provider{provider}, false)
+ if running := runningShortWindow(provider.GetId(), time.Now()); running != "" {
+ rememberWarmup(provider.GetId(), &WarmupResult{
+ Time: util.GetCurrentTime(),
+ Ok: true,
+ Message: "the window was already running, until " + running,
+ })
+ return
+ }
+ WarmUpProvider(provider)
+}
+
+func runningShortWindow(providerId string, now time.Time) string {
+ for _, window := range cachedWindows(providerId) {
+ if window.Name == WindowShort && window.reset().After(now) {
+ return window.ResetTime
+ }
+ }
+ return ""
+}
+
+func forgetOldWarmups(day string) {
+ warmups.Lock()
+ defer warmups.Unlock()
+ for key := range warmups.fired {
+ if !strings.Contains(key, "|"+day+"|") {
+ delete(warmups.fired, key)
+ }
+ }
+}
diff --git a/object/snapshot.go b/object/snapshot.go
index 7e01191..4efb697 100644
--- a/object/snapshot.go
+++ b/object/snapshot.go
@@ -594,7 +594,7 @@ func importAgents(snapshot *Snapshot, mode string, dryRun bool, report *ImportRe
// A routing whose provider is not on this machine is refused by
// SetAgentRouting, which is what the report says rather than the import
// failing over one agent.
- if err := SetAgentRouting(agent.Name, agent.Provider, agent.Fallbacks, agent.Mode); err != nil {
+ if err := SetAgentRouting(agent.Name, agent.Provider, agent.Fallbacks, agent.Mode, agent.Strategy); err != nil {
report.record(SectionAgents, agent.Name, ChangeFail, err.Error())
continue
}
@@ -621,7 +621,7 @@ func importAgents(snapshot *Snapshot, mode string, dryRun bool, report *ImportRe
report.record(SectionAgents, name, ChangeDelete, "")
continue
}
- if err := SetAgentRouting(name, "", nil, ""); err != nil {
+ if err := SetAgentRouting(name, "", nil, "", ""); err != nil {
report.record(SectionAgents, name, ChangeFail, err.Error())
continue
}
diff --git a/routers/router.go b/routers/router.go
index 9cf221a..89ad69a 100644
--- a/routers/router.go
+++ b/routers/router.go
@@ -191,6 +191,7 @@ func initAPI() {
beego.Router("/api/get-provider-health", &controllers.ApiController{}, "GET:GetProviderHealth")
beego.Router("/api/get-provider-quotas", &controllers.ApiController{}, "GET:GetProviderQuotas")
beego.Router("/api/refresh-provider-quotas", &controllers.ApiController{}, "POST:RefreshProviderQuotas")
+ beego.Router("/api/warm-up-provider", &controllers.ApiController{}, "POST:WarmUpProvider")
beego.Router("/api/get-provider-probes", &controllers.ApiController{}, "GET:GetProviderProbes")
beego.Router("/api/get-provider-probe-history", &controllers.ApiController{}, "GET:GetProviderProbeHistory")
beego.Router("/api/probe-provider", &controllers.ApiController{}, "POST:ProbeProvider")
diff --git a/web/src/backend/AgentBackend.ts b/web/src/backend/AgentBackend.ts
index d376282..9218660 100644
--- a/web/src/backend/AgentBackend.ts
+++ b/web/src/backend/AgentBackend.ts
@@ -262,6 +262,8 @@ export interface AgentRouting {
provider: string;
fallbacks: string[];
mode: string;
+ /** How the providers are taken in turn. Empty tries them as listed. */
+ strategy?: string;
}
export function updateAgentRouting(agentId: string, routing: AgentRouting) {
diff --git a/web/src/backend/ProviderBackend.ts b/web/src/backend/ProviderBackend.ts
index 9c7ea5c..28e1339 100644
--- a/web/src/backend/ProviderBackend.ts
+++ b/web/src/backend/ProviderBackend.ts
@@ -21,6 +21,7 @@ import type {
ProviderQuota,
ProviderSignin,
ProviderTestResult,
+ WarmupResult,
} from "@/types";
export function getProviders(
@@ -81,6 +82,12 @@ export function signInProvider(vendor: string) {
return request<ProviderSignin>(`/api/sign-in-provider${query({vendor: vendor})}`, "POST");
}
+/** Starts a subscription's short window now, rather than at its next scheduled
+ * warm-up. */
+export function warmUpProvider(id: string) {
+ return request<WarmupResult>(`/api/warm-up-provider${query({id: id})}`, "POST");
+}
+
/** How a sign-in that was started is getting on. Approving one takes as long as
* whoever is at the browser, so the page polls. */
export function getProviderSignin(id: string) {
diff --git a/web/src/components/ProviderCard.tsx b/web/src/components/ProviderCard.tsx
index 8fec4c4..c8a3b60 100644
--- a/web/src/components/ProviderCard.tsx
+++ b/web/src/components/ProviderCard.tsx
@@ -23,6 +23,7 @@ import {AgentIcon} from "@/components/AgentIcon";
import {ProviderIcon} from "@/components/ProviderIcon";
import {ConfirmDialog} from "@/components/shared/confirm-dialog";
import {Loading} from "@/components/shared/loading";
+import {SimpleSelect} from "@/components/shared/simple-select";
import {MessageAlert} from "@/components/ui/alert";
import {Badge} from "@/components/ui/badge";
import {Button} from "@/components/ui/button";
@@ -235,6 +236,8 @@ export function ProviderCard({
const stranded =
agent.provider === "" && (providerConfig.current ?? "").includes("/v1/agents/");
+ const strategy = agent.strategy || "order";
+
const toggleFallback = (id: string) => {
onRouting({
fallbacks: fallbacks.includes(id)
@@ -330,6 +333,28 @@ export function ProviderCard({
)}
</div>
+ {/* With fallbacks the providers are a pool, and which of them a
+ request starts on is a choice of its own. */}
+ {fallbacks.length > 0 ? (
+ <div className="space-y-2">
+ <p className="text-sm font-medium">{i18next.t("agent:Pool strategy")}</p>
+ <SimpleSelect
+ className="max-w-xs"
+ value={strategy}
+ disabled={busy}
+ onChange={value => onRouting({strategy: value})}
+ options={[
+ {label: i18next.t("agent:Pool order"), value: "order"},
+ {label: i18next.t("agent:Pool smart"), value: "smart"},
+ {label: i18next.t("agent:Pool rotate"), value: "rotate"},
+ {label: i18next.t("agent:Pool usage"), value: "usage"},
+ ]}
+ />
+ <p className="text-sm text-muted-foreground">{i18next.t(`agent:Pool ${strategy} hint`)}</p>
+ <p className="text-sm text-muted-foreground">{i18next.t("agent:Pool sticky hint")}</p>
+ </div>
+ ) : null}
+
<label className="flex items-start gap-2 text-sm">
<Switch
className="mt-0.5"
diff --git a/web/src/components/ProviderLoginField.tsx b/web/src/components/ProviderLoginField.tsx
index 361e22a..c78a805 100644
--- a/web/src/components/ProviderLoginField.tsx
+++ b/web/src/components/ProviderLoginField.tsx
@@ -13,7 +13,7 @@
// limitations under the License.
import * as React from "react";
-import {CircleCheck, ExternalLink, LogIn} from "lucide-react";
+import {CircleCheck, Copy, ExternalLink, LogIn} from "lucide-react";
import i18next from "i18next";
import * as ProviderBackend from "@/backend/ProviderBackend";
@@ -125,6 +125,23 @@ export function ProviderLoginField({
</Badge>
)}
</div>
+ {/* A vendor that approves a sign-in on its own site asks for a code
+ typed there, which is the one thing this page has to show. */}
+ {waiting && session?.userCode ? (
+ <div className="flex flex-wrap items-center gap-2 text-xs">
+ <span className="text-muted-foreground">{i18next.t("provider:Enter this code on that page")}</span>
+ <code className="rounded border px-2 py-0.5 font-mono text-sm tracking-widest">{session.userCode}</code>
+ <Button
+ type="button"
+ size="xs"
+ variant="ghost"
+ onClick={() => void navigator.clipboard?.writeText(session.userCode ?? "")}
+ >
+ <Copy />
+ {i18next.t("general:Copy")}
+ </Button>
+ </div>
+ ) : null}
{waiting && session ? (
<p className="text-muted-foreground text-xs">
{i18next.t("provider:Finish the sign-in in the browser")}{" "}
diff --git a/web/src/components/ProviderQuota.tsx b/web/src/components/ProviderQuota.tsx
index f1756d4..379f503 100644
--- a/web/src/components/ProviderQuota.tsx
+++ b/web/src/components/ProviderQuota.tsx
@@ -13,7 +13,7 @@
// limitations under the License.
import * as React from "react";
-import {CircleX, RefreshCw, Wallet} from "lucide-react";
+import {AlarmClock, CircleX, Flame, RefreshCw, Wallet} from "lucide-react";
import i18next from "i18next";
import * as ProviderBackend from "@/backend/ProviderBackend";
@@ -27,12 +27,60 @@ import {SimpleSelect} from "@/components/shared/simple-select";
import {Badge} from "@/components/ui/badge";
import {Button} from "@/components/ui/button";
import {Input} from "@/components/ui/input";
+import {Switch} from "@/components/ui/switch";
import {Textarea} from "@/components/ui/textarea";
import {SimpleTooltip} from "@/components/ui/tooltip";
import {formatPairs, parsePairs} from "@/lib/pairs";
-import {emptyQuotaConfig, formatQuota, quotaPresets} from "@/lib/quota";
-import {usesClientAuth} from "@/lib/providers";
-import type {Provider, ProviderQuota, QuotaConfig} from "@/types";
+import {emptyQuotaConfig, formatQuota, minutesToUnusedReset, quotaPresets, windowLabel} from "@/lib/quota";
+import {usageResetLabel} from "@/lib/agents";
+import {usesClientAuth, usesSubscription} from "@/lib/providers";
+import type {Provider, ProviderQuota, QuotaConfig, QuotaWindow, WarmupConfig, WarmupResult} from "@/types";
+
+/** What one plan limit has left, with when it fills again beside it. */
+function WindowBadge({window, quota}: {window: QuotaWindow; quota: ProviderQuota}) {
+ const reset = usageResetLabel(window.resetTime);
+ const detail = [
+ reset === ""
+ ? i18next.t("provider:Not started yet")
+ : i18next.t("agent:Resets {time}").replace("{time}", reset),
+ quota.plan ? `${i18next.t("provider:Plan")} ${quota.plan}` : "",
+ `${i18next.t("provider:Asked at")} ${quota.time}`,
+ ]
+ .filter(part => part !== "")
+ .join(" · ");
+ const variant = window.remainingPercent <= 0 ? "destructive" : window.remainingPercent <= 20 ? "warning" : "success";
+
+ return (
+ <SimpleTooltip title={detail}>
+ <Badge variant={variant}>{`${windowLabel(window)} ${window.remainingPercent}%`}</Badge>
+ </SimpleTooltip>
+ );
+}
+
+/** The limits of a subscription plan, and a word when most of one is about to
+ * be lost to its reset. */
+function WindowBadges({quota}: {quota: ProviderQuota}) {
+ const windows = quota.windows ?? [];
+ return (
+ <span className="flex flex-wrap items-center gap-1">
+ {windows.map(window => (
+ <WindowBadge key={window.name} window={window} quota={quota} />
+ ))}
+ {windows.map(window => {
+ const minutes = minutesToUnusedReset(window);
+ return minutes === 0 ? null : (
+ <Badge key={`${window.name}-reminder`} variant="info">
+ <AlarmClock />
+ {i18next
+ .t("provider:{percent}% unused, resets in {minutes} min")
+ .replace("{percent}", String(window.remainingPercent))
+ .replace("{minutes}", String(minutes))}
+ </Badge>
+ );
+ })}
+ </span>
+ );
+}
/** The one line a table has room for: what is left, and why there is no number
* when there is none. */
@@ -55,6 +103,9 @@ export function QuotaBadge({quota}: {quota: ProviderQuota | undefined}) {
if (!quota.supported) {
return <span className="text-muted-foreground">{i18next.t("provider:No balance endpoint")}</span>;
}
+ if ((quota.windows ?? []).length > 0) {
+ return <WindowBadges quota={quota} />;
+ }
const amount = quota.remaining ?? quota.total ?? quota.used;
if (amount === null || amount === undefined) {
@@ -143,16 +194,22 @@ export function ProviderQuotaSection({
}
};
+ // A plan's limits are read with the sign-in itself, so there is nothing to
+ // point anywhere.
+ const subscription = usesSubscription(provider);
+
return (
<Section
columns={2}
title={
<span className="flex items-center gap-2">
<Wallet className="size-4" />
- {i18next.t("provider:Balance")}
+ {subscription ? i18next.t("provider:Plan limits") : i18next.t("provider:Balance")}
</span>
}
- description={i18next.t("provider:Balance description")}
+ description={
+ subscription ? i18next.t("provider:Plan limits description") : i18next.t("provider:Balance description")
+ }
>
<Field label={i18next.t("provider:Remaining")} className="md:col-span-2">
<div className="flex flex-wrap items-center gap-3">
@@ -164,24 +221,26 @@ export function ProviderQuotaSection({
</div>
</Field>
- <Field
- label={i18next.t("provider:Balance source")}
- hint={i18next.t("provider:Balance source hint")}
- className="md:col-span-2"
- >
- <SimpleSelect
- className="max-w-xs"
- value={mode}
- onChange={setMode}
- options={[
- {label: i18next.t("provider:Automatic"), value: "auto"},
- {label: i18next.t("provider:Custom endpoint"), value: "custom"},
- {label: i18next.t("provider:Manual amount"), value: "manual"},
- ]}
- />
- </Field>
+ {subscription ? null : (
+ <Field
+ label={i18next.t("provider:Balance source")}
+ hint={i18next.t("provider:Balance source hint")}
+ className="md:col-span-2"
+ >
+ <SimpleSelect
+ className="max-w-xs"
+ value={mode}
+ onChange={setMode}
+ options={[
+ {label: i18next.t("provider:Automatic"), value: "auto"},
+ {label: i18next.t("provider:Custom endpoint"), value: "custom"},
+ {label: i18next.t("provider:Manual amount"), value: "manual"},
+ ]}
+ />
+ </Field>
+ )}
- {config !== null && config.manual ? (
+ {!subscription && config !== null && config.manual ? (
<>
<Field label={i18next.t("provider:Initial amount")} hint={i18next.t("provider:Manual amount hint")}>
<NumberInput min={0} value={config.initial} onChange={value => setConfigField("initial", value)} />
@@ -211,7 +270,7 @@ export function ProviderQuotaSection({
</>
) : null}
- {config !== null && !config.manual ? (
+ {!subscription && config !== null && !config.manual ? (
<>
<Field label={i18next.t("provider:Preset")} hint={i18next.t("provider:Preset hint")}>
<SimpleSelect
@@ -294,3 +353,116 @@ export function ProviderQuotaSection({
</Section>
);
}
+
+function emptyWarmup(): WarmupConfig {
+ return {enabled: false, times: ["07:00"], model: ""};
+}
+
+/** Reads the times as typed, separated by commas or spaces. */
+function parseTimes(text: string) {
+ return text
+ .split(/[\s,，]+/)
+ .map(value => value.trim())
+ .filter(value => value !== "");
+}
+
+/**
+ * When a subscription's five-hour window is started on its own. The window
+ * counts from the first request, so starting it early in the day has it reset,
+ * full again, by the time the day's real work is under way.
+ */
+export function ProviderWarmupSection({
+ provider,
+ saved,
+ onChange,
+}: {
+ provider: Provider;
+ /** Whether the provider is stored, which is what a warm-up is sent with. */
+ saved: boolean;
+ onChange: (warmup: WarmupConfig) => void;
+}) {
+ const config = provider.warmup ?? emptyWarmup();
+ const [result, setResult] = React.useState<WarmupResult | null>(null);
+ const [running, setRunning] = React.useState(false);
+ const [times, setTimes] = React.useState(config.times.join(", "));
+ const id = `${provider.owner}/${provider.name}`;
+
+ React.useEffect(() => {
+ ProviderBackend.getProviderQuotas().then(res => {
+ if (res.status === "ok") {
+ setResult((res.data ?? []).find(quota => quota.provider === id)?.warmup ?? null);
+ }
+ });
+ }, [id]);
+
+ const warmUp = () => {
+ setRunning(true);
+ ProviderBackend.warmUpProvider(id)
+ .then(res => {
+ if (res.status !== "ok" || !res.data) {
+ Setting.showMessage("error", res.msg ?? "");
+ return;
+ }
+ setResult(res.data);
+ })
+ .catch(error => Setting.showMessage("error", String(error)))
+ .then(() => setRunning(false));
+ };
+
+ return (
+ <Section
+ columns={2}
+ title={
+ <span className="flex items-center gap-2">
+ <Flame className="size-4" />
+ {i18next.t("provider:Warm-up")}
+ </span>
+ }
+ description={i18next.t("provider:Warm-up description")}
+ >
+ <Field label={i18next.t("provider:Warm up on a schedule")} className="md:col-span-2">
+ <label className="flex items-center gap-2 text-sm">
+ <Switch checked={config.enabled} onCheckedChange={enabled => onChange({...config, enabled: enabled})} />
+ <span className="text-muted-foreground">{i18next.t("provider:Warm-up schedule hint")}</span>
+ </label>
+ </Field>
+ <Field label={i18next.t("provider:Times")} htmlFor="warmup-times" hint={i18next.t("provider:Warm-up times hint")}>
+ <Input
+ id="warmup-times"
+ placeholder="07:00, 12:30"
+ value={times}
+ onChange={event => {
+ setTimes(event.target.value);
+ onChange({...config, times: parseTimes(event.target.value)});
+ }}
+ />
+ </Field>
+ <Field label={i18next.t("provider:Model")} htmlFor="warmup-model" hint={i18next.t("provider:Warm-up model hint")}>
+ <Input
+ id="warmup-model"
+ value={config.model}
+ onChange={event => onChange({...config, model: event.target.value})}
+ />
+ </Field>
+ <Field label={i18next.t("provider:Last warm-up")} className="md:col-span-2">
+ <div className="flex flex-wrap items-center gap-3">
+ <Button type="button" size="sm" variant="outline" loading={running} disabled={!saved} onClick={warmUp}>
+ <Flame />
+ {i18next.t("provider:Warm up now")}
+ </Button>
+ {result === null ? (
+ <span className="text-muted-foreground text-sm">{i18next.t("provider:Not warmed up yet")}</span>
+ ) : (
+ <span className="flex flex-wrap items-center gap-2 text-sm">
+ <Badge variant={result.ok ? "success" : "destructive"}>{result.time}</Badge>
+ <span className="text-muted-foreground">
+ {result.model ? `${result.model}: ` : ""}
+ {result.message}
+ </span>
+ </span>
+ )}
+ </div>
+ </Field>
+ </Section>
+ );
+}
diff --git a/web/src/components/ProviderSourcePicker.tsx b/web/src/components/ProviderSourcePicker.tsx
index aab8a11..a5f9415 100644
--- a/web/src/components/ProviderSourcePicker.tsx
+++ b/web/src/components/ProviderSourcePicker.tsx
@@ -24,6 +24,7 @@ import {
chatgptSource,
customSource,
claudeSource,
+ copilotSource,
vendorCategories,
type ProviderSource,
type VendorCategory,
@@ -41,6 +42,9 @@ export function sourceTitle(source: ProviderSource) {
if (source.key === chatgptSource) {
return i18next.t("provider:ChatGPT subscription");
}
+ if (source.key === copilotSource) {
+ return i18next.t("provider:GitHub Copilot subscription");
+ }
if (source.key === customSource) {
return i18next.t("provider:Custom vendor");
}
@@ -54,6 +58,9 @@ function sourceDetail(source: ProviderSource) {
if (source.key === chatgptSource) {
return i18next.t("provider:ChatGPT subscription detail");
}
+ if (source.key === copilotSource) {
+ return i18next.t("provider:GitHub Copilot subscription detail");
+ }
if (source.key === customSource) {
return i18next.t("provider:Custom vendor detail");
}
diff --git a/web/src/lib/agents.ts b/web/src/lib/agents.ts
index 84e13a8..764b4f8 100644
--- a/web/src/lib/agents.ts
+++ b/web/src/lib/agents.ts
@@ -391,6 +391,7 @@ export function useAgents(enabled = true) {
provider: routing.provider ?? agent.provider,
fallbacks: routing.fallbacks ?? agent.fallbacks ?? [],
mode: routing.mode ?? agent.mode ?? gatewayMode,
+ strategy: routing.strategy ?? agent.strategy ?? "",
};
setBusyKey(agentKey(agent));
@@ -433,6 +434,7 @@ export function useAgents(enabled = true) {
provider: providerId,
fallbacks: [],
mode: agent.mode || gatewayMode,
+ strategy: agent.strategy ?? "",
};
const writes = providerId !== "" && (agent.providerConfig?.supported ?? false);
diff --git a/web/src/lib/providers.ts b/web/src/lib/providers.ts
index e2076e0..1cdabb1 100644
--- a/web/src/lib/providers.ts
+++ b/web/src/lib/providers.ts
@@ -77,6 +77,12 @@ export function usesClientAuth(provider: {authMode?: string} | undefined) {
return provider?.authMode === authClient;
}
+/** Whether a subscription's plan has a short window that starts at the first
+ * request, which is what a warm-up is for. A Copilot plan counts by month. */
+export function warmsUp(provider: {authMode?: string; subscriptionVendor?: string}) {
+ return usesSubscription(provider) && ["anthropic", "openai"].includes(provider.subscriptionVendor ?? "");
+}
+
/** Mirrors object.UsesSubscription: a sign-in Gateway holds and renews itself. */
export function usesSubscription(provider: {authMode?: string} | undefined) {
return provider?.authMode === authSubscription;
@@ -577,11 +583,15 @@ export interface ProviderSource {
export const claudeSource = "claude";
export const chatgptSource = "chatgpt";
+export const copilotSource = "copilot";
export const customSource = "custom";
/** Where a ChatGPT subscription's Codex requests go, which is not the API. */
export const chatgptCodexBaseUrl = "https://chatgpt.com/backend-api/codex";
+/** Where a Copilot plan's requests go, when the account names no host of its own. */
+export const copilotBaseUrl = "https://api.githubcopilot.com";
+
/**
* The sign-ins come first: they are the only sources that need nothing filled
* in, and the ones people with a subscription and no API key are here for. One
@@ -616,6 +626,21 @@ export const providerSources: ProviderSource[] = [
authMode: authSubscription,
},
},
+ {
+ key: copilotSource,
+ label: "",
+ vendor: "copilot",
+ // The API host has no icon of its own; the plan is GitHub's.
+ website: "https://github.com",
+ provider: {
+ type: "openai",
+ icon: "github.com",
+ baseUrl: copilotBaseUrl,
+ models: [],
+ apiKey: "",
+ authMode: authSubscription,
+ },
+ },
{
key: customSource,
label: "",
diff --git a/web/src/lib/quota.ts b/web/src/lib/quota.ts
index cbb40a0..ef70c4e 100644
--- a/web/src/lib/quota.ts
+++ b/web/src/lib/quota.ts
@@ -12,7 +12,9 @@
// See the License for the specific language governing permissions and
// limitations under the License.
-import type {QuotaConfig} from "@/types";
+import i18next from "i18next";
+
+import type {QuotaConfig, QuotaWindow} from "@/types";
const currencySymbols: Record<string, string> = {USD: "$", CNY: "¥", EUR: "€", GBP: "£", JPY: "¥"};
@@ -84,3 +86,29 @@ export const quotaPresets: QuotaPreset[] = [
},
},
];
+
+/** What a plan limit is called on the page. */
+export function windowLabel(window: QuotaWindow) {
+ switch (window.name) {
+ case "week":
+ return i18next.t("provider:Week");
+ case "premium":
+ return i18next.t("provider:Premium requests");
+ case "chat":
+ return i18next.t("provider:Chat");
+ default:
+ return window.name;
+ }
+}
+
+/** How long, in minutes, until a window that still has most of its allowance
+ * resets and loses it. Zero when that is not about to happen: the allowance a
+ * plan gives for one window is not carried into the next. */
+export function minutesToUnusedReset(window: QuotaWindow, now = Date.now()) {
+ if ((window.name !== "5h" && window.name !== "week") || window.remainingPercent < 50 || !window.resetTime) {
+ return 0;
+ }
+ const minutes = Math.ceil((new Date(window.resetTime).getTime() - now) / 60000);
+ const soon = window.name === "week" ? 24 * 60 : 60;
+ return minutes > 0 && minutes <= soon ? minutes : 0;
+}
diff --git a/web/src/locales/en/data.json b/web/src/locales/en/data.json
index 3e8c941..f999600 100644
--- a/web/src/locales/en/data.json
+++ b/web/src/locales/en/data.json
@@ -436,7 +436,17 @@
"{count} messages": "{count} messages",
"{count} resets left": "{count} rate limit resets left",
"{count} running": "{count} running",
- "{percent}% left": "{percent}% left"
+ "{percent}% left": "{percent}% left",
+ "Pool strategy": "Pool strategy",
+ "Pool order": "In order",
+ "Pool smart": "Soonest reset first",
+ "Pool rotate": "Rotate",
+ "Pool usage": "Most left first",
+ "Pool order hint": "The bound provider first, then the fallbacks in the order picked.",
+ "Pool smart hint": "Spends first the plan whose 5-hour window resets soonest, since whatever it has left is lost at that reset. API keys come after every plan.",
+ "Pool rotate hint": "Each new conversation starts on the next provider.",
+ "Pool usage hint": "Starts on the plan with the most left of its tightest window. API keys come after every plan.",
+ "Pool sticky hint": "A conversation stays on the provider that answered it until it has been idle for 15 minutes, so its prompt cache is reused, and a plan that has run out waits behind the others until its window resets."
},
"agentConfig": {
"1 file": "1 file",
@@ -1327,6 +1337,7 @@
"Bring it all over": "Bring it all over",
"Caller's own login": "The caller's own login",
"Change source": "Change",
+ "Chat": "Chat",
"ChatGPT subscription": "ChatGPT subscription",
"ChatGPT subscription detail": "Sign in to ChatGPT here. Gateway holds the subscription and spends it like a key, for any agent.",
"Claude subscription": "Claude subscription",
@@ -1356,6 +1367,7 @@
"Draft saved": "Draft saved",
"Edit Provider": "Edit Provider",
"Enabled": "Enabled",
+ "Enter this code on that page": "Enter this code on GitHub's page:",
"Failed to add": "Failed to add",
"Failed to delete": "Failed to delete",
"Failed to fetch models": "Failed to fetch models",
@@ -1369,6 +1381,8 @@
"Finish the sign-in in the browser": "Finish the sign-in in the browser tab that just opened.",
"From the type": "From the type",
"Get a key from {vendor}": "Get a key from {vendor}",
+ "GitHub Copilot subscription": "GitHub Copilot subscription",
+ "GitHub Copilot subscription detail": "Sign in to GitHub here. Gateway holds the Copilot plan and spends it like a key, for any agent.",
"Granted": "granted",
"Granted path": "Granted path",
"Health": "Health",
@@ -1378,9 +1392,11 @@
"Icon hint": "The site the vendor's icon is taken from, or an image URL. Leave it empty to take it from the base URL.",
"Inference platforms": "Inference platforms",
"Initial amount": "Initial amount",
+ "Last warm-up": "Last warm-up",
"Manual amount": "Manual amount",
"Manual amount hint": "The balance starts at this figure and drops by what Gateway records spending through this provider. Spend is priced from the model pricing table, so keep that table in the same currency.",
"Manual currency hint": "The label the amounts carry. It converts nothing: the recorded spend is subtracted as it stands.",
+ "Model": "Model",
"Model routing hint": "Requests that arrive here are routed by the model name they carry, to the enabled provider with the smallest priority value that serves it. To send one agent's requests to this provider whatever model they name, bind the provider on the agent's page instead.",
"Model vendors": "Model vendors",
"Models": "Models",
@@ -1394,7 +1410,9 @@
"No providers yet detail": "A provider is your own vendor account: a base URL, the models it serves, and either an API key or the caller's own login forwarded upstream. Add one, then bind it to an agent to route that agent's requests through it.",
"No vendor matches": "No vendor matches that. Pick Custom to enter one by hand.",
"Not asked yet": "Not asked yet",
+ "Not started yet": "Not started: the window starts at the first request",
"Not used yet": "Not used yet",
+ "Not warmed up yet": "Not warmed up since Gateway started",
"Notes": "Notes",
"Notes hint": "For whoever reads this later: which account the key belongs to, when it expires, who pays for it.",
"Nothing to test": "There is nothing to test yet",
@@ -1403,7 +1421,11 @@
"Paste a link or connection info": "Paste a link or connection info",
"Paste a link or connection info hint": "Vendors publish these as \"add to CC Switch\" buttons, and a New API site copies its connection info out as JSON. Gateway reads either one and fills the form in below, and stores nothing until you confirm.",
"Path hint": "A dotted path into the answer, where a number indexes an array: balance_infos.0.total_balance.",
+ "Plan": "Plan",
+ "Plan limits": "Plan limits",
+ "Plan limits description": "What the plan behind this sign-in has left of each window. It is read with the sign-in itself, and every answer this provider relays keeps it current.",
"Plan usage": "Plan usage",
+ "Premium requests": "Premium",
"Preset": "Preset",
"Preset hint": "Fills in the paths for a family of sites, so they do not have to be read out of the vendor's API documentation.",
"Priority": "Priority",
@@ -1442,6 +1464,7 @@
"Switched": "Switched",
"Test Connectivity": "Test Connectivity",
"This link cannot be read": "This link cannot be read",
+ "Times": "Times",
"Type": "Type",
"Unfinished draft": "Unfinished draft",
"Upstream API": "Upstream API",
@@ -1451,7 +1474,16 @@
"Used": "used",
"Used by": "Used by",
"Used by no agent": "No agent uses this provider yet",
- "Used path": "Used path"
+ "Used path": "Used path",
+ "Warm up now": "Warm up now",
+ "Warm up on a schedule": "Warm up on a schedule",
+ "Warm-up": "Warm-up",
+ "Warm-up description": "A Claude or ChatGPT plan counts its five hours from the first request. Send one early, at 7:00 say, and the window resets by noon, so the afternoon starts on a full one.",
+ "Warm-up model hint": "Empty uses the model this provider last relayed.",
+ "Warm-up schedule hint": "At each time below, unless the window is already running, Gateway sends the smallest request the plan takes.",
+ "Warm-up times hint": "Local times written as HH:MM, separated by commas.",
+ "Week": "Week",
+ "{percent}% unused, resets in {minutes} min": "{percent}% unused, resets in {minutes} min"
},
"setting": {
"A backup was taken first": "A backup was taken first, as {name}.",
diff --git a/web/src/locales/zh/data.json b/web/src/locales/zh/data.json
index 0bbd244..dd43aa8 100644
--- a/web/src/locales/zh/data.json
+++ b/web/src/locales/zh/data.json
@@ -436,7 +436,17 @@
"{count} messages": "{count}条消息",
"{count} resets left": "剩余{count}次重置额度",
"{count} running": "{count}个在运行",
- "{percent}% left": "剩余{percent}%"
+ "{percent}% left": "剩余{percent}%",
+ "Pool strategy": "账号池策略",
+ "Pool order": "按顺序",
+ "Pool smart": "先用快重置的",
+ "Pool rotate": "轮流",
+ "Pool usage": "先用剩余多的",
+ "Pool order hint": "先用绑定的供应商，再按所选顺序用备用的。",
+ "Pool smart hint": "先用5小时窗口最早重置的订阅，因为重置时没用完的额度就作废了。API Key排在所有订阅之后。",
+ "Pool rotate hint": "每个新对话从下一个供应商开始。",
+ "Pool usage hint": "先用剩余额度最多的订阅，按最紧的那个窗口算。API Key排在所有订阅之后。",
+ "Pool sticky hint": "一个对话会留在回答它的供应商上，直到空闲15分钟，这样能复用提示词缓存；额度用完的订阅在窗口重置前排到最后。"
},
"agentConfig": {
"1 file": "1个文件",
@@ -1513,6 +1523,7 @@
"Bring it all over": "全部导入",
"Caller's own login": "调用方自己的登录",
"Change source": "更换",
+ "Chat": "对话",
"ChatGPT subscription": "ChatGPT订阅",
"ChatGPT subscription detail": "在这里登录ChatGPT，Gateway保管这个订阅，像Key一样给任何Agent用。",
"Claude subscription": "Claude订阅登录",
@@ -1542,6 +1553,7 @@
"Draft saved": "草稿已保存",
"Edit Provider": "编辑供应商",
"Enabled": "已启用",
+ "Enter this code on that page": "在GitHub页面输入这个代码：",
"Failed to add": "添加失败",
"Failed to delete": "删除失败",
"Failed to fetch models": "获取模型列表失败",
@@ -1555,6 +1567,8 @@
"Finish the sign-in in the browser": "在刚打开的浏览器标签页里完成登录。",
"From the type": "按类型推断",
"Get a key from {vendor}": "去{vendor}获取密钥",
+ "GitHub Copilot subscription": "GitHub Copilot订阅登录",
+ "GitHub Copilot subscription detail": "在这里登录GitHub，Gateway保管Copilot订阅，像Key一样给任何Agent用。",
"Granted": "总额",
"Granted path": "总额字段路径",
"Health": "健康状态",
@@ -1564,9 +1578,11 @@
"Icon hint": "取图标的站点域名，或一个图片地址。留空则从Base URL推断。",
"Inference platforms": "推理平台",
"Initial amount": "初始金额",
+ "Last warm-up": "上次预热",
"Manual amount": "手动填写",
"Manual amount hint": "余额从这个金额开始，按Gateway记录的、经由该供应商的花费递减。花费依据模型价格表计算，请让该表保持同一币种。",
"Manual currency hint": "金额显示用的单位。它不做任何换算：记录的花费按原值扣减。",
+ "Model": "模型",
"Model routing hint": "到达这里的请求按自己带的模型名路由，转发到支持该模型、且priority值最小的已启用供应商。如果想让某个Agent的请求不管模型名一律走这个供应商，请到该Agent的页面上绑定。",
"Model vendors": "模型厂商",
"Models": "模型",
@@ -1580,7 +1596,9 @@
"No providers yet detail": "一个供应商就是你自己的厂商账号：base URL、它提供的模型，以及一个API Key，或者转发调用方自己的登录。先建一个，再把它绑定到Agent，该Agent的请求就会走它。",
"No vendor matches": "没有匹配的厂商。可以选「自定义」手动填写。",
"Not asked yet": "尚未查询",
+ "Not started yet": "还没开始：第一次请求时开始计时",
"Not used yet": "尚未使用",
+ "Not warmed up yet": "Gateway启动以来还没预热过",
"Notes": "备注",
"Notes hint": "写给以后的自己：这个密钥属于哪个账号、什么时候过期、由谁付费。",
"Nothing to test": "暂时没有可探测的内容",
@@ -1589,7 +1607,11 @@
"Paste a link or connection info": "粘贴链接或连接信息",
"Paste a link or connection info hint": "中转站通常把它做成“添加到CC Switch”的按钮，New API站点则是把连接信息复制成一段JSON。两种Gateway都认，读完只会把下面的表单填好，你确认之前不会保存任何东西。",
"Path hint": "返回内容里的点分路径，数字表示数组下标：balance_infos.0.total_balance。",
+ "Plan": "套餐",
+ "Plan limits": "套餐额度",
+ "Plan limits description": "这个订阅每个窗口还剩多少。用它自己的登录查询，经它转发的每次回答也会更新。",
"Plan usage": "套餐用量",
+ "Premium requests": "高级请求",
"Preset": "预设",
"Preset hint": "为某一类站点填好字段路径，省去翻厂商API文档。",
"Priority": "优先级",
@@ -1628,6 +1650,7 @@
"Switched": "已切换",
"Test Connectivity": "测试连接",
"This link cannot be read": "这个链接读不出来",
+ "Times": "时间",
"Type": "类型",
"Unfinished draft": "有未完成的草稿",
"Upstream API": "上游接口",
@@ -1637,7 +1660,16 @@
"Used": "已用",
"Used by": "正在使用：",
"Used by no agent": "还没有Agent客户端使用这个供应商",
- "Used path": "已用字段路径"
+ "Used path": "已用字段路径",
+ "Warm up now": "立即预热",
+ "Warm up on a schedule": "定时预热",
+ "Warm-up": "预热",
+ "Warm-up description": "Claude和ChatGPT订阅的5小时窗口从第一次请求开始计时。早上（比如7:00）先发一个请求，中午窗口就重置了，下午用的是满额度。",
+ "Warm-up model hint": "留空则用这个供应商最近一次转发的模型。",
+ "Warm-up schedule hint": "到下面每个时间点，如果窗口没在计时，Gateway就发一个最小的请求。",
+ "Warm-up times hint": "本地时间，写成HH:MM，用逗号分隔。",
+ "Week": "周",
+ "{percent}% unused, resets in {minutes} min": "还有{percent}%没用，{minutes}分钟后重置"
},
"setting": {
"A backup was taken first": "导入前已先备份为{name}。",
diff --git a/web/src/pages/ProviderEditPage.tsx b/web/src/pages/ProviderEditPage.tsx
index 80d8391..54b8329 100644
--- a/web/src/pages/ProviderEditPage.tsx
+++ b/web/src/pages/ProviderEditPage.tsx
@@ -22,7 +22,7 @@ import * as Setting from "@/Setting";
import {ProviderIcon, ProviderIconField} from "@/components/ProviderIcon";
import {ProviderLoginField} from "@/components/ProviderLoginField";
import {ProviderModelsField} from "@/components/ProviderModelsField";
-import {ProviderQuotaSection} from "@/components/ProviderQuota";
+import {ProviderQuotaSection, ProviderWarmupSection} from "@/components/ProviderQuota";
import {ProviderTestField, useProviderTest} from "@/components/ProviderTestField";
import {EnvSnippet} from "@/components/EnvSnippet";
import {Field} from "@/components/shared/form-dialog";
@@ -45,6 +45,7 @@ import {
localShell,
usesClientAuth,
usesSubscription,
+ warmsUp,
wireApiEndpoint,
} from "@/lib/providers";
import type {Provider, QuotaConfig} from "@/types";
@@ -70,7 +71,9 @@ function buildOpenAiUrl(baseUrl: string, endpoint: string) {
if (path.endsWith(endpoint)) {
path = path.slice(0, path.length - endpoint.length);
}
- if (!namesApiVersion(path)) {
+ const host = url.hostname.toLowerCase();
+ const atRoot = host === "githubcopilot.com" || host.endsWith(".githubcopilot.com");
+ if (!namesApiVersion(path) && !path.endsWith("/backend-api/codex") && !atRoot) {
path += "/v1";
}
@@ -256,6 +259,7 @@ export default function ProviderEditPage() {
loginId: session.id,
subscriptionAccount: session.account ?? "",
subscriptionPlan: session.plan ?? "",
+ models: current.models.length > 0 ? current.models : (session.models ?? []),
}
: current,
)
@@ -335,6 +339,10 @@ export default function ProviderEditPage() {
onChange={(quota: QuotaConfig | null) => setField("quota", quota)}
/>
+ {warmsUp(provider) ? (
+ <ProviderWarmupSection provider={provider} saved onChange={warmup => setField("warmup", warmup)} />
+ ) : null}
+
<Section title={i18next.t("provider:Usage")} description={i18next.t("provider:Usage hint")} columns={1}>
<EnvSnippet
protocol={providerProtocol(provider.type)}
diff --git a/web/src/pages/ProviderListPage.tsx b/web/src/pages/ProviderListPage.tsx
index 72cf917..e41da28 100644
--- a/web/src/pages/ProviderListPage.tsx
+++ b/web/src/pages/ProviderListPage.tsx
@@ -367,6 +367,7 @@ export default function ProviderListPage({account}: {account: Account}) {
provider: providerIdOf(provider),
fallbacks: [],
mode: agent.mode || "gateway",
+ strategy: agent.strategy ?? "",
})
.then(res => {
if (res.status === "ok") {
@@ -634,6 +635,7 @@ export default function ProviderListPage({account}: {account: Account}) {
loginId: session.id,
subscriptionAccount: session.account ?? "",
subscriptionPlan: session.plan ?? "",
+ models: prev.models.length > 0 ? prev.models : (session.models ?? []),
}))
}
/>
diff --git a/web/src/types.ts b/web/src/types.ts
index 3c0d0e3..d50b82e 100644
--- a/web/src/types.ts
+++ b/web/src/types.ts
@@ -80,6 +80,36 @@ export interface Provider {
notes: string;
/** Null leaves the built-in balance endpoints to answer. */
quota?: QuotaConfig | null;
+ /** When a subscription's short window is started on its own. */
+ warmup?: WarmupConfig | null;
+}
+
+/** Times of day a subscription is sent the smallest request, which starts its
+ * five-hour window. Mirrors object.WarmupConfig. */
+export interface WarmupConfig {
+ enabled: boolean;
+ /** Local times, "HH:MM". */
+ times: string[];
+ /** Empty takes the model the provider last relayed. */
+ model: string;
+}
+
+/** How the last warm-up of a subscription went. */
+export interface WarmupResult {
+ time: string;
+ ok: boolean;
+ model?: string;
+ message: string;
+}
+
+/** One limit of a subscription plan: what is left of it, and when it fills.
+ * No resetTime is a window that has not started. */
+export interface QuotaWindow {
+ /** "5h", "week", "premium" or "chat". */
+ name: string;
+ remainingPercent: number;
+ windowMinutes: number;
+ resetTime?: string;
}
/** Where a provider's balance is read from, when there is no built-in endpoint
@@ -113,6 +143,10 @@ export interface ProviderQuota {
used: number | null;
total: number | null;
unit: string;
+ /** What a subscription has left of its plan limits. */
+ windows?: QuotaWindow[];
+ plan?: string;
+ warmup?: WarmupResult;
error: string;
time: string;
}
@@ -123,11 +157,16 @@ export interface ProviderSignin {
vendor: string;
/** Where the sign-in is approved. The page opens it. */
url: string;
+ /** The code typed on that page, for a vendor approved there rather than by
+ * a redirect back to Gateway. */
+ userCode?: string;
running: boolean;
ok: boolean;
account?: string;
plan?: string;
error?: string;
+ /** The models the account may ask for, for a vendor with names of its own. */
+ models?: string[];
}
export interface ProviderTestResult {
@@ -208,6 +247,9 @@ export interface Agent {
fallbacks: string[];
/** "gateway" routes through the local proxy, "direct" writes the upstream. */
mode: string;
+ /** How the bound provider and its fallbacks are taken in turn: "order"
+ * (empty), "smart", "rotate" or "usage". */
+ strategy?: string;
providerConfig: AgentProviderConfig;
/** Where this agent reaches its bound provider, as the server resolved it. */
proxyBaseUrl?: string;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@casbin.apache.org
For additional commands, e-mail: commits-help@casbin.apache.org