[PR] fix(ai-transport): stop internal LLM calls leaking client headers [apisix]

["AlinsRan (via GitHub)" <gi...@apache.org>]

AlinsRan opened a new pull request, #13699:
URL: https://github.com/apache/apisix/pull/13699

   ### Description
   
   `ai-transport`'s `construct_forward_headers` read the downstream request's headers straight from `ctx` (`core.request.headers(ctx)`) and **always** forwarded them to the LLM endpoint. That is correct for the transparent proxy path (`ai-proxy` / `ai-proxy-multi`), but wrong for plugins that make a **self-contained internal LLM call**:
   
   `ai-request-rewrite` calls an LLM to rewrite the request body. Because the shared transport always forwarded client headers, the client's `Authorization` / `Cookie` were sent to that third-party LLM endpoint — a header leak.
   
   **Fix — make the transport `ctx`-free:**
   
   - `construct_forward_headers(ext_opts_headers, client_headers)` now takes an explicit `client_headers` table (nil for internal requests) instead of reading `core.request.headers(ctx)`. The transport no longer has any downstream-request coupling.
   - `build_request` forwards `opts.client_headers`; only the proxy path (`ai-proxy/base.lua`) sets it from `core.request.headers(ctx)` (it legitimately holds `ctx` in the request phase).
   - Internal callers (`ai-request-rewrite`, and any future embeddings / RAG / moderation call) pass none, so their own credentials — not the client's — reach the LLM.
   
   No config or behavior change for the proxy path; the leak is closed for internal calls.
   
   ### Tests
   
   Added `t/plugin/ai-transport-header-forwarding.t` with self-contained mocks:
   - `ai-request-rewrite`'s internal rewrite call receives **no** client `Cookie` (leak fixed).
   - `ai-proxy-multi`'s transparent proxy path **still** forwards the client `Cookie`.
   
   Verified the test catches the leak: it fails on the pre-fix code (the mock LLM receives `session=super-secret`) and passes after.
   
   ### Checklist
   
   - [x] I have explained the need for this PR and the problem it solves
   - [x] I have explained the changes or the new features added to this PR
   - [x] I have added tests corresponding to this change
   - [x] I have updated the documentation to reflect this change (n/a — no user-facing config change)
   - [x] I have verified that this change is backward compatible
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] refactor(ai-providers): make the LLM request client ctx-free [apisix]

["AlinsRan (via GitHub)" <gi...@apache.org>]

AlinsRan merged PR #13699:
URL: https://github.com/apache/apisix/pull/13699


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] refactor(ai-providers): make the LLM request client ctx-free [apisix]

["membphis (via GitHub)" <gi...@apache.org>]

membphis commented on PR #13699:
URL: https://github.com/apache/apisix/pull/13699#issuecomment-5029480220

   [P2] GCP token cache key can alias different credentials and TTL policies
   
   The new cache key uses `ngx.crc32_long(service_account_json) .. "#" .. name`. Because it relies on a 32-bit CRC and omits `expire_early_secs` and `max_ttl`, separate configurations can alias: different credentials may reuse the wrong cached access token after a collision, while identical credentials and names with different TTL policies can bypass the latter policy.
   
   Please use a collision-resistant digest of the resolved credentials and all fields that affect cache reuse, without logging plaintext secrets, and add coverage for same-name configurations with different TTL policies.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] refactor(ai-providers): make the LLM request client ctx-free [apisix]

["AlinsRan (via GitHub)" <gi...@apache.org>]

AlinsRan commented on PR #13699:
URL: https://github.com/apache/apisix/pull/13699#issuecomment-5030389664

   Fixed on both this PR and the upstream apache/apisix#13699.
   
   The key is now `ngx.md5(service_account_json .. "\0" .. expire_early_secs .. "\0" .. max_ttl) .. "#" .. name`:
   
   - a 128-bit digest instead of the 32-bit CRC, so distinct credentials no longer collide onto the wrong cached token;
   - `expire_early_secs` and `max_ttl` are part of the key, so identical credentials under different TTL policies get separate entries and the second policy is no longer bypassed;
   - the raw JSON still never enters the key or the log (only the digest does).
   
   Used `ngx.md5` rather than SHA-256 to follow the existing cache-key convention in the codebase (e.g. `graphql-proxy-cache`); this is a non-adversarial cache key, and 128 bits removes the accidental-collision risk the CRC had. Happy to switch to SHA-256 if you prefer.
   
   Added a regression test (vertex-ai TEST 13): same name and credentials with different `max_ttl` must not share a token. Verified discriminating — with the old CRC key it reports `calls=1 distinct=false` (aliased), with the fix `calls=2 distinct=true cachehit=true`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org