You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@casbin.apache.org by "BeiBaiNian (via GitHub)" <gi...@apache.org> on 2026/08/20 05:02:40 UTC

[PR] feat: add OpenAI-compatible LLM request audit [casbin-gateway]

BeiBaiNian opened a new pull request, #217:
URL: https://github.com/apache/casbin-gateway/pull/217

   ## Why
   
   The gateway is the single choke point for /v1/chat/completions traffic, but operators have no visibility into what models are actually being asked to do. Debugging prompt injection, abuse, or unexpected model usage today means tapping every upstream or grepping logs that never capture request bodies. Prompt content is sensitive operational data, so any retention must be opt-in, sanitized, and strictly bounded.
   
   ## What changed
   
   Gateway can now record sanitized chat-completions requests in a new llm\_request\_audit table, viewable by administrators in the management UI.
   
   ### Backend:
   
   - object/llm\_request\_audit.go: new table (id, createdTime, model, channel, clientIp, stream, payload, truncated, originalBytes), created via Sync2 on startup.
   - Best-effort async writer with a bounded queue: the proxy goroutine only enqueues a metadata snapshot and never parses, sanitizes, or writes; a full queue drops records and logs the drop count at most once a minute; graceful shutdown drains accepted entries.
   - Sanitization runs only on the writer goroutine — the sole body-retention boundary. Headers (especially inbound authorization) are never copied into a record. Credential-bearing fields and embedded secrets are redacted via auditutil.SanitizeValue; bodies larger than 64 KiB are stored as safe metadata (truncated, originalBytes) instead of raw bytes.
   - Retention is bounded by age (llmRequestAuditRetentionDays) and count (llmRequestAuditMaxRecords), pruned once a minute.
   - controllers/proxy.go: forwardToChannel records an entry only after the upstream responds, so each failover attempt keeps its real channel.
   - New admin-only, paginated endpoint GET /api/get-llm-request-audits.
   - conf/conf.go + conf/app.conf: llmRequestAuditEnabled (opt-in; disabled installs retain nothing and pay no proxy-path cost), llmRequestAuditQueueCapacity, llmRequestAuditRetentionDays, llmRequestAuditMaxRecords.
     Frontend:
   
   New admin-only "LLM Request Audits" page (web/src/pages/LlmRequestAuditsPage.tsx) with pagination and a table of channel / model / mode / client IP / stored / truncated; full i18n (en + zh); route and sidebar entry added.
   
   ## Verification
   
   - go test ./object -run TestLlmRequestAudit -v: covers sanitization, credential redaction, oversized-body metadata, and invalid-JSON rejection.
   - go test ./controllers ./object: passes, including the updated forwardToChannel tests.
   - go build ./...: clean.
   - Manual: enable llmRequestAuditEnabled \= true, send requests through /v1/chat/completions, confirm records appear in the admin page, embedded credentials in messages/tools are redacted, oversized requests are truncated, and entries beyond the retention window are pruned.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@casbin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@casbin.apache.org
For additional commands, e-mail: commits-help@casbin.apache.org

Re: [PR] feat: add OpenAI-compatible LLM request audit [casbin-gateway]

Posted by "hsluoyz (via GitHub)" <gi...@apache.org>.
hsluoyz commented on PR #217:
URL: https://github.com/apache/casbin-gateway/pull/217#issuecomment-5359602325

   replaced by: https://github.com/apache/casbin-gateway/commit/0fdbdf5b5a581fcc867f2aee601fcaf46509c1d2


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@casbin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@casbin.apache.org
For additional commands, e-mail: commits-help@casbin.apache.org

Re: [PR] feat: add OpenAI-compatible LLM request audit [casbin-gateway]

Posted by "hsluoyz (via GitHub)" <gi...@apache.org>.
hsluoyz closed pull request #217: feat: add OpenAI-compatible LLM request audit
URL: https://github.com/apache/casbin-gateway/pull/217


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@casbin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@casbin.apache.org
For additional commands, e-mail: commits-help@casbin.apache.org