Re: [I] Discussion：Verion: 2.0.1 tips [apisix-ingress-controller]

["janiussyafiq (via GitHub)" <gi...@apache.org>]

janiussyafiq commented on issue #2756:
URL: https://github.com/apache/apisix-ingress-controller/issues/2756#issuecomment-5290372276

   I looked into the `socket hang up` part and managed to reproduce it locally. Sharing the root cause since it explains all the reports in this thread and #2704.
   
   **Root cause:** it is a keep-alive idle-timeout race between the ADC sidecar and the APISIX Admin API. ADC (<= 0.24.0) pools admin connections with a hardcoded 60s `freeSocketTimeout`, and the Admin API's nginx also uses `keepalive_timeout 60s` (APISIX and Helm chart default). With `syncPeriod: 1m`, each sync reuses a pooled connection right at the 60s idle boundary, exactly when nginx closes it, so the request lands on a closed socket and fails with `socket hang up`. Whether it appears as `HTTP 500 {"message":"Error: socket hang up"}` (dump stage, #2704) or a `partial_failure` with `reason: "socket hang up"` (apply stage, the log above) is just where in the sync it strikes.
   
   **Why the workaround works:** raising `nginx.keepaliveTimeout` to 75s (or 120s) makes the server-side timeout longer than ADC's 60s, so ADC closes idle connections first and the race disappears.
   
   **This is already fixed:** api7/adc#407 + api7/adc#409 lowered ADC's default to 50s, effective from **adc 0.24.2**. However:
   
   - controller install manifests 2.0.0 / 2.0.1 / 2.1.0 pin adc **0.23.1** (affected)
   - Helm chart `apisix-ingress-controller-1.1.1` (pulled by the `apisix-2.14.0` umbrella chart) also pins **0.23.1** (affected)
   - controller 2.2.0 and chart 1.1.2+ ship adc >= 0.24.2 (fixed)
   
   **Remediation:** upgrade to controller 2.2.0 / a chart with adc >= 0.24.2, or simply override the adc sidecar image tag (`adcContainer.image.tag` in Helm, the `sidecar` image in kustomize) to >= 0.24.2. No nginx tuning needed after that.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org