[PR] fix: rotate ADC cacheKey nonce on leader acquisition [apisix-ingress-controller]

["AlinsRan (via GitHub)" <gi...@apache.org>]

AlinsRan opened a new pull request, #2785:
URL: https://github.com/apache/apisix-ingress-controller/pull/2785

   ### Problem
   
   Fixes #2774.
   
   In a multi-replica controller deployment, a leader switch **does not restart the pods**, so the long-lived ADC sidecar keeps its in-memory cache alive on the standby pod. ADC stores, per `cacheKey`, the last-synced content plus the `conf_version` it generated.
   
   Only the **leader** runs the sync loop, so a standby pod never refreshes its local ADC cache. Meanwhile the current leader pushes new config and bumps `*_conf_version` in APISIX. When leadership switches **back**, the controller rebuilds correct state from the API server, but the ADC cache on that pod is still frozen at its last-leader snapshot. ADC diffs the current state against that stale baseline, reuses an **older** `conf_version`, and APISIX rejects it:
   
   ```
   upstreams_conf_version must be greater than or equal to (1779434128737)
   ```
   
   ### Fix
   
   Append a per-leadership **nonce** to the `cacheKey` sent to ADC and rotate it whenever the provider starts (i.e. on leader acquisition). ADC then sees a fresh `cacheKey` with no baseline and regenerates `conf_version` from scratch on the first sync, which APISIX accepts.
   
   - `HTTPADCExecutor` holds an atomic nonce, applied only to the `cacheKey` field (the bare `config.Name` is still used for the store, metrics and error reporting).
   - `Client.RefreshCacheKeyNonce()` rotates it; `apisixProvider.Start` calls it on leadership acquisition.
   - An initial nonce is seeded in `client.New` so the cacheKey is isolated per process even before the first rotation.
   
   This is the controller-only fix discussed in #2774. The only trade-off is that abandoned `cacheKey`s remain in ADC memory until the sidecar restarts; a cleaner ADC-side cache-clear API can replace this later.
   
   ### Test
   
   - Unit test for the nonce-aware `cacheKey` and `RefreshCacheKeyNonce` rotation.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix: rebuild the ADC baseline on leader acquisition [apisix-ingress-controller]

["AlinsRan (via GitHub)" <gi...@apache.org>]

AlinsRan merged PR #2785:
URL: https://github.com/apache/apisix-ingress-controller/pull/2785


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix: rotate ADC cacheKey nonce on leader acquisition [apisix-ingress-controller]

["johannes-engler-mw (via GitHub)" <gi...@apache.org>]

johannes-engler-mw commented on PR #2785:
URL: https://github.com/apache/apisix-ingress-controller/pull/2785#issuecomment-4925311513

   @AlinsRan How is the status of this PR, its impacting us massively. Happy to contribute if you like.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix: rotate ADC cacheKey nonce on leader acquisition [apisix-ingress-controller]

["juzhiyuan (via GitHub)" <gi...@apache.org>]

juzhiyuan commented on PR #2785:
URL: https://github.com/apache/apisix-ingress-controller/pull/2785#issuecomment-4951130997

   @johannes-engler-mw I will follow up this PR next week.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org