[I] feat: `$secret://` support - manage APISIX `secrets` (secret managers) through the controller [apisix-ingress-controller]

["jens-skribble (via GitHub)" <gi...@apache.org>]

jens-skribble opened a new issue, #2831:
URL: https://github.com/apache/apisix-ingress-controller/issues/2831

   ### Description
   
   ## Summary
   
   Bring APISIX's native **`secrets`** secret-manager resource (Vault / AWS / GCP) into
   the controller's resource model, so a `secrets` manager can be **declared through the
   controller** and becomes part of the full-config snapshot ADC pushes. That makes
   `$secret://<manager>/<id>/...` references usable on controller-managed resources
   (routes, plugin configs, consumers, SSL, `GatewayProxy`), instead of the value having
   to exist as a plaintext Kubernetes Secret.
   
   Follow-up to #2821 (the standalone sync deadlock). As discussed there, the supported
   answer isn't out-of-band registration — it's making `secrets` a first-class resource
   the controller owns and syncs. This issue is the design discussion for that.
   
   ## Motivation / use case
   
   We want to reference secrets from plugin config via APISIX's native `$secret://`
   indirection rather than materializing the value into the config. Concrete case:
   central, HA-safe rotation of the `openid-connect` **session/cookie secret** via Vault —
   
   ```yaml
   # today (works, but the value lives in the data-plane env / a k8s Secret):
   session:
     secret: "$env://APISIX_SESSION_SECRET"
   
   # what we'd like — resolved from a Vault secret manager, rotated centrally:
   session:
     secret: "$secret://vault/oidc/oidc/session_secret"
   ```
   
   APISIX itself already resolves `$secret://vault/...` in API-driven standalone mode
   (verified in #2821). The only missing piece is a controller-side way to declare the
   backing `secrets` manager.
   
   ## Why not `secretRef`
   
   `secretRef` reads a Kubernetes Secret and injects the **plaintext value** into the
   pushed config — a different mechanism. It doesn't give APISIX's native `$secret://`
   indirection, where the data plane fetches from Vault/AWS/GCP at request time and the
   secret material never has to live in the k8s config or the ADC snapshot. Both are
   useful; this request is specifically about the native `secrets` managers.
   
   ## Proposed direction (for discussion)
   
   Add `secrets` to the control-plane resource model so it's part of the snapshot. Open
   questions we'd love maintainer input on:
   
   1. **Where to declare it.** Two shapes we can see:
      - a `secrets` list on **`GatewayProxy`** (alongside `pluginMetadata`) — simple, one
        place, naturally scoped to the gateway the controller drives; or
      - a dedicated **CRD** (e.g. `ApisixSecret` / `SecretManager`) translating to APISIX's
        `secrets` resource — more flexible, referenceable across namespaces, its own
        lifecycle/status.
   
   2. **Backends.** APISIX supports `vault`, `aws`, and `gcp` managers — mirror those
      fields (`uri`, `prefix`, auth, etc.).
   
   3. **Manager credentials should not be inline plaintext.** The Vault token / AWS / GCP
      credentials the manager needs should be sourceable from a Kubernetes Secret via a
      `valueFrom: { secretKeyRef: ... }` pattern — the same shape the chart already uses
      for `GatewayProxy` `adminKey.valueFrom`. (This is the one place `secretRef`-style
      k8s-Secret reading is still needed — to bootstrap the manager's own auth.)
   
   4. **Referencing.** Confirm the reference syntax works from the plugin configs we care
      about (`ApisixPluginConfig` / `ApisixRoute` plugins, `openid-connect` `session.secret`,
      and ideally `ApisixConsumer` / SSL).
   
   5. **Ownership semantics.** Once the controller owns `secrets`, a manager registered out
      of band is (correctly) removed on the next full sync — that's expected and fine; it's
      the flip side of #2821 and we're not asking for out-of-band to be supported.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] feat: `$secret://` support - manage APISIX `secrets` (secret managers) through the controller [apisix-ingress-controller]

["AlinsRan (via GitHub)" <gi...@apache.org>]

AlinsRan closed issue #2831: feat: `$secret://` support - manage APISIX `secrets` (secret managers) through the controller
URL: https://github.com/apache/apisix-ingress-controller/issues/2831


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] feat: `$secret://` support - manage APISIX `secrets` (secret managers) through the controller [apisix-ingress-controller]

["AlinsRan (via GitHub)" <gi...@apache.org>]

AlinsRan commented on issue #2831:
URL: https://github.com/apache/apisix-ingress-controller/issues/2831#issuecomment-6077777236

   @jens-skribble
   
   For now, the recommended approach is to sync the value from Vault into a Kubernetes Secret with Vault Secrets Operator or External Secrets Operator, and reference it from the plugin with `secretRef`. Secret keys can use dots for nested fields, so `session.secret` is written to `openid-connect`'s `session.secret`:
   
   ```yaml
   apiVersion: v1
   kind: Secret
   metadata:
     name: oidc-session        # synced from Vault by VSO / ESO
   stringData:
     session.secret: "..."
   ---
   plugins:
     - name: openid-connect
       enable: true
       secretRef: oidc-session
       config: { ... }
   ```
   
   The controller watches referenced Secrets, so when the value is rotated in Vault and the Secret is updated, the config is pushed again automatically.
   
   Why `$secret://` is not being implemented in the controller for now:
   
   - ADC does not support the `secrets` resource. It would first need to be added as a resource type in ADC, and then the controller would need a new resource model on top, so the change spans several components.
   - The secret manager's own credentials (Vault token, AWS/GCP credentials) would still have to live in a Kubernetes Secret, so sensitive data stays in Kubernetes either way.
   
   Limitations of the recommended approach: the plaintext value is present in the Kubernetes Secret and in the config pushed to APISIX, and global plugins on `GatewayProxy` do not support `secretRef`.
   
   Closing this as not planned for now. If ADC gains support for the `secrets` resource, or more users ask for the same thing, feel free to reopen.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org