[GitHub] [apisix] Yiyiyimu edited a comment on issue #3673: request help: A node in the K8S environment ETCD cluster died, causing Apisix to fail

[GitBox <gi...@apache.org>]

Yiyiyimu edited a comment on issue #3673:
URL: https://github.com/apache/apisix/issues/3673#issuecomment-787094518


   Hi @GBXing, I tried to reproduce the problem, and my reproduce steps are:
   
   1. Configure etcd host with domain name:
   
       ```
       DNS_IP=$(kubectl get svc -n kube-system -l k8s-app=kube-dns -o 'jsonpath={..spec.clusterIP}')
       echo "dns_resolver:
         - ${DNS_IP}
       etcd:
         host:
           - \\"<http://etcd-cluster-client.default.svc.cluster.local:2379>\\" " > ./conf/config.yaml
       ```
   
   2. Setup APISIX and everything works as expect
   3. Kill leader/follower pod of etcd ( gives me the same result ), and the error log would produce:
   
       ```
       # Multiple of
       2021/02/27 15:30:19 [error] 49#49: *114289 [lua] config_etcd.lua:566: failed to fetch data from etcd: /usr/local/apisix//deps/share/lua/5.1/resty/etcd/v3.lua:593: attempt to index field 'result' (a nil value)
       stack traceback:
       /usr/local/apisix//deps/share/lua/5.1/resty/etcd/v3.lua:593: in function 'res_func'
       /usr/local/apisix/apisix/core/config_etcd.lua:125: in function 'waitdir'
       /usr/local/apisix/apisix/core/config_etcd.lua:305: in function 'sync_data'
       /usr/local/apisix/apisix/core/config_etcd.lua:540: in function </usr/local/apisix/apisix/core/config_etcd.lua:530>
       [C]: in function 'xpcall'
       /usr/local/apisix/apisix/core/config_etcd.lua:530: in function </usr/local/apisix/apisix/core/config_etcd.lua:521>,  etcd key: /apisix/global_rules, context: ngx.timer
   
       # Multiple of
       2021/02/27 15:30:38 [error] 53#53: *113602 [lua] config_etcd.lua:544: failed to fetch data from etcd: connection refused, etcd key: /apisix/ssl, context: ngx.timer
       ```
   
       With etcd-operator, etcd got unreachable for seconds and returned back to normal.
   
   Is the error log the same with what you met


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org