You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by "maslow (via GitHub)" <gi...@apache.org> on 2023/10/27 03:16:56 UTC

[I] bug: Enabling the APISIX ingress controller can lead to a surge in CPU usage on the APISIX gateway. [apisix-ingress-controller]

maslow opened a new issue, #2028:
URL: https://github.com/apache/apisix-ingress-controller/issues/2028

   ### Current Behavior
   
   Enabling the APISIX ingress controller can lead to a surge in CPU usage on the APISIX gateway. 
   
   Turning off the APISIX ingress controller will resolve the issue.
   
   
   <img width="347" alt="image" src="https://github.com/apache/apisix-ingress-controller/assets/7122237/27b621ef-90b5-4957-8345-801af28c29d2">
   
   The above-mentioned issue can be consistently and reliably reproduced in my cluster. Each time, the memory usage increases for three minutes, then the CPU usage returns to a lower level for three minutes, and after three minutes, it will rise again, continuing for three minutes. It's almost unusable and severely affects operations, with the service delay increasing from 20ms to 2000ms.
   
   ### Expected Behavior
   
   _No response_
   
   ### Error Logs
   
   2023-10-27T11:11:29+08:00	error	ingress/ingress.go:148	failed to translate ingress	{"error": "endpoints: endpoints \"j3t46i\" not found", "ingress": {}}
   2023-10-27T11:11:29+08:00	warn	ingress/ingress.go:268	sync ingress failed, will retry	{"object": {"Type":1,"Object":{"Key":"j3t46i/j3t46i","GroupVersion":"networking/v1","OldObject":null},"OldObject":null,"Tombstone":null}, "error": "endpoints: endpoints \"j3t46i\" not found"}
   2023-10-27T11:11:29+08:00	error	ingress/ingress.go:472	failed to get APISIX gateway external IPs	{"error": "resource name may not be empty"}
   2023-10-27T11:11:29+08:00	error	translation/translator.go:158	failed to translate ingress backend to upstream	{"error": "endpoints: endpoints \"dz24qv\" not found", "ingress": "&Ingress{ObjectMeta:{dz24qv  dz24qv  37023cf1-b564-46cc-8779-77ed62cf901b 295270932 1 2023-10-19 20:31:45 +0800 HKT <nil> <nil> map[] map[k8s.apisix.apache.org/cors-allow-credential:false k8s.apisix.apache.org/cors-allow-headers:* k8s.apisix.apache.org/cors-allow-methods:* k8s.apisix.apache.org/cors-allow-origin:* k8s.apisix.apache.org/cors-expose-headers:* k8s.apisix.apache.org/enable-cors:true k8s.apisix.apache.org/enable-websocket:true k8s.apisix.apache.org/svc-namespace:dz24qv laf.dev/appid:dz24qv laf.dev/ingress.type:runtime nginx.ingress.kubernetes.io/cors-allow-credentials:false nginx.ingress.kubernetes.io/cors-allow-headers:DNT,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Authorization,x-laf-develop-token,x-laf-func-data nginx.ingress.kubernetes.io/cors-a
 llow-methods:* nginx.ingress.kubernetes.io/cors-allow-origin:* nginx.ingress.kubernetes.io/cors-expose-headers:* nginx.ingress.kubernetes.io/enable-cors:true nginx.ingress.kubernetes.io/server-snippet:client_header_buffer_size 4096k;\nlarge_client_header_buffers 8 512k;\n] [] [] [{unknown Update networking.k8s.io/v1 2023-10-19 20:31:45 +0800 HKT FieldsV1 {\"f:metadata\":{\"f:annotations\":{\".\":{},\"f:k8s.apisix.apache.org/cors-allow-credential\":{},\"f:k8s.apisix.apache.org/cors-allow-headers\":{},\"f:k8s.apisix.apache.org/cors-allow-methods\":{},\"f:k8s.apisix.apache.org/cors-allow-origin\":{},\"f:k8s.apisix.apache.org/cors-expose-headers\":{},\"f:k8s.apisix.apache.org/enable-cors\":{},\"f:k8s.apisix.apache.org/enable-websocket\":{},\"f:k8s.apisix.apache.org/svc-namespace\":{},\"f:laf.dev/appid\":{},\"f:laf.dev/ingress.type\":{},\"f:nginx.ingress.kubernetes.io/cors-allow-credentials\":{},\"f:nginx.ingress.kubernetes.io/cors-allow-headers\":{},\"f:nginx.ingress.kubernetes.io/cors-
 allow-methods\":{},\"f:nginx.ingress.kubernetes.io/cors-allow-origin\":{},\"f:nginx.ingress.kubernetes.io/cors-expose-headers\":{},\"f:nginx.ingress.kubernetes.io/enable-cors\":{},\"f:nginx.ingress.kubernetes.io/server-snippet\":{}}},\"f:spec\":{\"f:ingressClassName\":{},\"f:rules\":{}}} }]},Spec:IngressSpec{DefaultBackend:nil,TLS:[]IngressTLS{},Rules:[]IngressRule{IngressRule{Host:dz24qv.laf.dev,IngressRuleValue:IngressRuleValue{HTTP:&HTTPIngressRuleValue{Paths:[]HTTPIngressPath{HTTPIngressPath{Path:/,Backend:IngressBackend{Resource:nil,Service:&IngressServiceBackend{Name:dz24qv,Port:ServiceBackendPort{Name:,Number:8000,},},},PathType:*Prefix,},},},},},},IngressClassName:*apisix,},Status:IngressStatus{LoadBalancer:{[]},},}"}
   2023-10-27T11:11:29+08:00	error	ingress/ingress.go:148	failed to translate ingress	{"error": "endpoints: endpoints \"dz24qv\" not found", "ingress": {}}
   2023-10-27T11:11:29+08:00	warn	ingress/ingress.go:268	sync ingress failed, will retry	{"object": {"Type":1,"Object":{"Key":"dz24qv/dz24qv","GroupVersion":"networking/v1","OldObject":null},"OldObject":null,"Tombstone":null}, "error": "endpoints: endpoints \"dz24qv\" not found"}
   2023-10-27T11:11:29+08:00	error	ingress/ingress.go:472	failed to get APISIX gateway external IPs	{"error": "resource name may not be empty"}
   2023-10-27T11:11:30+08:00	error	ingress/ingress.go:472	failed to get APISIX gateway external IPs	{"error": "resource name may not be empty"}
   2023-10-27T11:11:31+08:00	error	ingress/ingress.go:472	failed to get APISIX gateway external IPs	{"error": "resource name may not be empty"}
   2023-10-27T11:11:31+08:00	error	ingress/ingress.go:472	failed to get APISIX gateway external IPs	{"error": "resource name may not be empty"}
   2023-10-27T11:11:31+08:00	error	translation/translator.go:158	failed to translate ingress backend to upstream	{"error": "service.spec.ports: port not defined", "ingress": "&Ingress{ObjectMeta:{6529093daa9e4f74966f5192  h7j5vb  5b043fd8-19fc-43b8-aa5c-be0d2b54933d 288674296 1 2023-10-13 17:09:18 +0800 HKT <nil> <nil> map[] map[k8s.apisix.apache.org/cors-allow-credential:false k8s.apisix.apache.org/cors-allow-headers:* k8s.apisix.apache.org/cors-allow-methods:* k8s.apisix.apache.org/cors-allow-origin:* k8s.apisix.apache.org/cors-expose-headers:* k8s.apisix.apache.org/enable-cors:true k8s.apisix.apache.org/svc-namespace:h7j5vb laf.dev/appid:h7j5vb laf.dev/bucket.name:h7j5vb-test laf.dev/ingress.type:bucket nginx.ingress.kubernetes.io/cors-allow-credentials:false nginx.ingress.kubernetes.io/cors-allow-headers:DNT,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Authorization,x-laf-develop-token,x-laf-func-data,x-amz-content-sha256,x-amz-securit
 y-token,x-amz-user-agent,x-amz-date nginx.ingress.kubernetes.io/cors-allow-methods:* nginx.ingress.kubernetes.io/cors-allow-origin:* nginx.ingress.kubernetes.io/cors-expose-headers:* nginx.ingress.kubernetes.io/enable-cors:true] [] [] [{unknown Update networking.k8s.io/v1 2023-10-13 17:09:18 +0800 HKT FieldsV1 {\"f:metadata\":{\"f:annotations\":{\".\":{},\"f:k8s.apisix.apache.org/cors-allow-credential\":{},\"f:k8s.apisix.apache.org/cors-allow-headers\":{},\"f:k8s.apisix.apache.org/cors-allow-methods\":{},\"f:k8s.apisix.apache.org/cors-allow-origin\":{},\"f:k8s.apisix.apache.org/cors-expose-headers\":{},\"f:k8s.apisix.apache.org/enable-cors\":{},\"f:k8s.apisix.apache.org/svc-namespace\":{},\"f:laf.dev/appid\":{},\"f:laf.dev/bucket.name\":{},\"f:laf.dev/ingress.type\":{},\"f:nginx.ingress.kubernetes.io/cors-allow-credentials\":{},\"f:nginx.ingress.kubernetes.io/cors-allow-headers\":{},\"f:nginx.ingress.kubernetes.io/cors-allow-methods\":{},\"f:nginx.ingress.kubernetes.io/cors-allow-or
 igin\":{},\"f:nginx.ingress.kubernetes.io/cors-expose-headers\":{},\"f:nginx.ingress.kubernetes.io/enable-cors\":{}}},\"f:spec\":{\"f:ingressClassName\":{},\"f:rules\":{}}} }]},Spec:IngressSpec{DefaultBackend:nil,TLS:[]IngressTLS{},Rules:[]IngressRule{IngressRule{Host:oss.laf.dev,IngressRuleValue:IngressRuleValue{HTTP:&HTTPIngressRuleValue{Paths:[]HTTPIngressPath{HTTPIngressPath{Path:/h7j5vb-test,Backend:IngressBackend{Resource:nil,Service:&IngressServiceBackend{Name:h7j5vb,Port:ServiceBackendPort{Name:,Number:9000,},},},PathType:*Prefix,},},},},},IngressRule{Host:h7j5vb-test.oss.laf.dev,IngressRuleValue:IngressRuleValue{HTTP:&HTTPIngressRuleValue{Paths:[]HTTPIngressPath{HTTPIngressPath{Path:/,Backend:IngressBackend{Resource:nil,Service:&IngressServiceBackend{Name:h7j5vb,Port:ServiceBackendPort{Name:,Number:9000,},},},PathType:*Prefix,},},},},},},IngressClassName:*apisix,},Status:IngressStatus{LoadBalancer:{[]},},}"}
   2023-10-27T11:11:31+08:00	error	ingress/ingress.go:148	failed to translate ingress	{"error": "service.spec.ports: port not defined", "ingress": {}}
   2023-10-27T11:11:31+08:00	warn	ingress/ingress.go:268	sync ingress failed, will retry	{"object": {"Type":1,"Object":{"Key":"h7j5vb/6529093daa9e4f74966f5192","GroupVersion":"networking/v1","OldObject":null},"OldObject":null,"Tombstone":null}, "error": "service.spec.ports: port not defined"}
   2023-10-27T11:11:31+08:00	error	ingress/ingress.go:472	failed to get APISIX gateway external IPs	{"error": "resource name may not be empty"}
   2023-10-27T11:11:32+08:00	error	ingress/ingress.go:472	failed to get APISIX gateway external IPs	{"error": "resource name may not be empty"}
   2023-10-27T11:11:32+08:00	error	ingress/ingress.go:472	failed to get APISIX gateway external IPs	{"error": "resource name may not be empty"}
   2023-10-27T11:11:33+08:00	error	ingress/ingress.go:472	failed to get APISIX gateway external IPs	{"error": "resource name may not be empty"}
   2023-10-27T11:11:34+08:00	error	ingress/ingress.go:472	failed to get APISIX gateway external IPs	{"error": "resource name may not be empty"}
   2023-10-27T11:11:34+08:00	error	ingress/ingress.go:472	failed to get APISIX gateway external IPs	{"error": "resource name may not be empty"}
   
   
   ### Steps to Reproduce
   
   Only my cluster.
   
   ### Environment
   
   - APISIX Ingress controller version (run `apisix-ingress-controller version --long`)
   image: apache/apisix-ingress-controller:1.6.0
   
   `apisix-ingress-controller` command not found in this docker image.
   <img width="463" alt="image" src="https://github.com/apache/apisix-ingress-controller/assets/7122237/9fe4fe8a-866d-4490-b7c8-4cee42b0ec63">
   
   
   - Kubernetes cluster version (run `kubectl version`)
   ```sh
   kubectl version
   WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
   Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.6", GitCommit:"ff2c119726cc1f8926fb0585c74b25921e866a28", GitTreeState:"clean", BuildDate:"2023-01-18T19:22:09Z", GoVersion:"go1.19.5", Compiler:"gc", Platform:"linux/amd64"}
   Kustomize Version: v4.5.7
   Server Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.6", GitCommit:"ff2c119726cc1f8926fb0585c74b25921e866a28", GitTreeState:"clean", BuildDate:"2023-01-18T19:15:26Z", GoVersion:"go1.19.5", Compiler:"gc", Platform:"linux/amd64"}
   ```
   - OS version if running APISIX Ingress controller in a bare-metal environment (run `uname -a`)
   ```sh
   uname -a
   Linux laf-canary-master001 5.4.0-137-generic #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bug: Enabling the APISIX ingress controller can lead to a surge in CPU usage on the APISIX gateway. [apisix-ingress-controller]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #2028:
URL: https://github.com/apache/apisix-ingress-controller/issues/2028#issuecomment-1942943599

   This issue has been marked as stale due to 90 days of inactivity. It will be closed in 30 days if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the dev@apisix.apache.org list. Thank you for your contributions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bug: Enabling the APISIX ingress controller can lead to a surge in CPU usage on the APISIX gateway. [apisix-ingress-controller]

Posted by "GhangZh (via GitHub)" <gi...@apache.org>.
GhangZh commented on issue #2028:
URL: https://github.com/apache/apisix-ingress-controller/issues/2028#issuecomment-2079459703

   We also found the same problem, apisix-ingress-controller when comparing apisix crd declarative configurations with etcd data, requesting apisix would cause apisix cpu resources to go up, and then go down again after a while
   ![image](https://github.com/apache/apisix-ingress-controller/assets/92301646/47de39f3-80fe-45b9-a109-96c5481ce1ac)
   
   ```bash
   2024-04-26T19:34:15+08:00	warn	ingress/compare.go:186	pluginConfig: 799f92e3 in APISIX but do not in declare yaml
   2024-04-26T19:34:15+08:00	warn	ingress/compare.go:186	pluginConfig: 1668e488 in APISIX but do not in declare yaml
   2024-04-26T19:34:15+08:00	warn	ingress/compare.go:186	pluginConfig: 2059c8b3 in APISIX but do not in declare yaml
   2024-04-26T19:34:15+08:00	warn	ingress/compare.go:186	pluginConfig: 97d60823 in APISIX but do not in declare yaml
   2024-04-26T19:34:15+08:00	warn	ingress/compare.go:186	pluginConfig: 6d49039a in APISIX but do not in declare yaml
   2024-04-26T19:34:15+08:00	warn	ingress/compare.go:186	pluginConfig: 47eee264 in APISIX but do not in declare yaml
   2024-04-26T19:34:15+08:00	warn	ingress/compare.go:186	pluginConfig: 16cc31 in APISIX but do not in declare yaml
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bug: Enabling the APISIX ingress controller can lead to a surge in CPU usage on the APISIX gateway. [apisix-ingress-controller]

Posted by "lingsamuel (via GitHub)" <gi...@apache.org>.
lingsamuel commented on issue #2028:
URL: https://github.com/apache/apisix-ingress-controller/issues/2028#issuecomment-1786360584

   `apisix-resource-sync-interval: 1h` is too short for hundreds, even thousands of resources. 
   I see hundreds of creation events in the 3-min log file. Since the events generated by full synchronization in 1.6 are also creation events, I can't tell if they are sync events or true resource creation events. If they are sync events, the resource sync interval should be increased. Otherwise, CP/DP isolated deployment mode is required.
   
   Also, some auto-renew resources are not excluded correctly. For example, `openebs.io-local` raises an update event every 2s from log.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bug: Enabling the APISIX ingress controller can lead to a surge in CPU usage on the APISIX gateway. [apisix-ingress-controller]

Posted by "GhangZh (via GitHub)" <gi...@apache.org>.
GhangZh commented on issue #2028:
URL: https://github.com/apache/apisix-ingress-controller/issues/2028#issuecomment-2081353991

   > We also found the same problem, apisix-ingress-controller when comparing apisix crd declarative configurations with etcd data, requesting apisix would cause apisix cpu resources to go up, and then go down again after a while ![image](https://private-user-images.githubusercontent.com/92301646/326003519-47de39f3-80fe-45b9-a109-96c5481ce1ac.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTQyODU3NTMsIm5iZiI6MTcxNDI4NTQ1MywicGF0aCI6Ii85MjMwMTY0Ni8zMjYwMDM1MTktNDdkZTM5ZjMtODBmZS00NWI5LWExMDktOTZjNTQ4MWNlMWFjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA0MjglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNDI4VDA2MjQxM1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWNkNmZkMmNkNzg2MDI2MGZlNGJjOGI3ZjEwZDMzMmE2OTg1M2ZiZWNjNzhhMjYwNzJlZTI0MTU5Y2M4NjUxM2MmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAm
 cmVwb19pZD0wIn0.S28LRJ05NxoZ7i7vyafd-glHu942NC5VlfSgsKPpTNE)
   > 
   > ```shell
   > 2024-04-26T19:34:15+08:00	warn	ingress/compare.go:186	pluginConfig: 799f92e3 in APISIX but do not in declare yaml
   > 2024-04-26T19:34:15+08:00	warn	ingress/compare.go:186	pluginConfig: 1668e488 in APISIX but do not in declare yaml
   > 2024-04-26T19:34:15+08:00	warn	ingress/compare.go:186	pluginConfig: 2059c8b3 in APISIX but do not in declare yaml
   > 2024-04-26T19:34:15+08:00	warn	ingress/compare.go:186	pluginConfig: 97d60823 in APISIX but do not in declare yaml
   > 2024-04-26T19:34:15+08:00	warn	ingress/compare.go:186	pluginConfig: 6d49039a in APISIX but do not in declare yaml
   > 2024-04-26T19:34:15+08:00	warn	ingress/compare.go:186	pluginConfig: 47eee264 in APISIX but do not in declare yaml
   > 2024-04-26T19:34:15+08:00	warn	ingress/compare.go:186	pluginConfig: 16cc31 in APISIX but do not in declare yaml
   > ```
   I'm experiencing the same scenario, please help look at it. Now as soon as I restart apisix-ingress-controller, apisix cpu goes up and then down again, about 5k+ apisixroutes and 5k+ upstreams @shreemaan-abhishek @Sn0rt 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bug: Enabling the APISIX ingress controller can lead to a surge in CPU usage on the APISIX gateway. [apisix-ingress-controller]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #2028:
URL: https://github.com/apache/apisix-ingress-controller/issues/2028#issuecomment-2311394209

   This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bug: Enabling the APISIX ingress controller can lead to a surge in CPU usage on the APISIX gateway. [apisix-ingress-controller]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #2028: bug: Enabling the APISIX ingress controller can lead to a surge in CPU usage on the APISIX gateway. 
URL: https://github.com/apache/apisix-ingress-controller/issues/2028


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bug: Enabling the APISIX ingress controller can lead to a surge in CPU usage on the APISIX gateway. [apisix-ingress-controller]

Posted by "Sn0rt (via GitHub)" <gi...@apache.org>.
Sn0rt commented on issue #2028:
URL: https://github.com/apache/apisix-ingress-controller/issues/2028#issuecomment-1788506318

   @lingsamuel Do we have a technical solution to reduce the requests to the CP side when endpoint resources are updated?
   
   For example, cache and other solutions?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bug: Enabling the APISIX ingress controller can lead to a surge in CPU usage on the APISIX gateway. [apisix-ingress-controller]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #2028:
URL: https://github.com/apache/apisix-ingress-controller/issues/2028#issuecomment-2254304338

   This issue has been marked as stale due to 90 days of inactivity. It will be closed in 30 days if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the dev@apisix.apache.org list. Thank you for your contributions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bug: Enabling the APISIX ingress controller can lead to a surge in CPU usage on the APISIX gateway. [apisix-ingress-controller]

Posted by "Sn0rt (via GitHub)" <gi...@apache.org>.
Sn0rt commented on issue #2028:
URL: https://github.com/apache/apisix-ingress-controller/issues/2028#issuecomment-1782354753

   thank your report 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bug: Enabling the APISIX ingress controller can lead to a surge in CPU usage on the APISIX gateway. [apisix-ingress-controller]

Posted by "lingsamuel (via GitHub)" <gi...@apache.org>.
lingsamuel commented on issue #2028:
URL: https://github.com/apache/apisix-ingress-controller/issues/2028#issuecomment-1784377371

   what's your apisix-ingress-controller config?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bug: Enabling the APISIX ingress controller can lead to a surge in CPU usage on the APISIX gateway. [apisix-ingress-controller]

Posted by "Sn0rt (via GitHub)" <gi...@apache.org>.
Sn0rt commented on issue #2028:
URL: https://github.com/apache/apisix-ingress-controller/issues/2028#issuecomment-1784524996

   ```yaml
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: apisix-ingress-controller
     namespace: ingress-apisix
     uid: c317229e-1f3a-4d85-b3ae-4be21d756ace
     resourceVersion: '216541530'
     generation: 1
     creationTimestamp: '2023-03-31T02:37:52Z'
     labels:
       app.kubernetes.io/instance: apisix
       app.kubernetes.io/managed-by: Helm
       app.kubernetes.io/name: ingress-controller
       app.kubernetes.io/version: 1.6.0
       helm.sh/chart: ingress-controller-0.11.4
     annotations:
       deployment.kubernetes.io/revision: '1'
       meta.helm.sh/release-name: apisix
       meta.helm.sh/release-namespace: ingress-apisix
   spec:
     replicas: 1
     selector:
       matchLabels:
         app.kubernetes.io/instance: apisix
         app.kubernetes.io/name: ingress-controller
     template:
       metadata:
         creationTimestamp: null
         labels:
           app.kubernetes.io/instance: apisix
           app.kubernetes.io/name: ingress-controller
         annotations:
           checksum/config: dc595ec92c5fdc9f40170836cf8831cff9c2aeb820a6d590c02912d518747607
       spec:
         volumes:
           - name: configuration
             configMap:
               name: apisix-configmap
               items:
                 - key: config.yaml
                   path: config.yaml
               defaultMode: 420
         initContainers:
           - name: wait-apisix-admin
             image: busybox:1.28
             command:
               - sh
               - '-c'
               - >-
                 until nc -z apisix-admin.ingress-apisix.svc.cluster.local 9180 ;
                 do echo waiting for apisix-admin; sleep 2; done;
             resources: {}
             terminationMessagePath: /dev/termination-log
             terminationMessagePolicy: File
             imagePullPolicy: IfNotPresent
             securityContext: {}
         containers:
           - name: ingress-controller
             image: apache/apisix-ingress-controller:1.6.0
             command:
               - /ingress-apisix/apisix-ingress-controller
               - ingress
               - '--config-path'
               - /ingress-apisix/conf/config.yaml
             ports:
               - name: http
                 containerPort: 8080
                 protocol: TCP
             env:
               - name: POD_NAMESPACE
                 valueFrom:
                   fieldRef:
                     apiVersion: v1
                     fieldPath: metadata.namespace
               - name: POD_NAME
                 valueFrom:
                   fieldRef:
                     apiVersion: v1
                     fieldPath: metadata.name
             resources: {}
             volumeMounts:
               - name: configuration
                 mountPath: /ingress-apisix/conf
             livenessProbe:
               httpGet:
                 path: /healthz
                 port: 8080
                 scheme: HTTP
               timeoutSeconds: 1
               periodSeconds: 10
               successThreshold: 1
               failureThreshold: 3
             readinessProbe:
               httpGet:
                 path: /healthz
                 port: 8080
                 scheme: HTTP
               timeoutSeconds: 1
               periodSeconds: 10
               successThreshold: 1
               failureThreshold: 3
             terminationMessagePath: /dev/termination-log
             terminationMessagePolicy: File
             imagePullPolicy: IfNotPresent
         restartPolicy: Always
         terminationGracePeriodSeconds: 30
         dnsPolicy: ClusterFirst
         serviceAccountName: apisix-ingress-controller
         serviceAccount: apisix-ingress-controller
         securityContext: {}
         schedulerName: default-scheduler
     strategy:
       type: RollingUpdate
       rollingUpdate:
         maxUnavailable: 25%
         maxSurge: 25%
     revisionHistoryLimit: 10
     progressDeadlineSeconds: 600
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bug: Enabling the APISIX ingress controller can lead to a surge in CPU usage on the APISIX gateway. [apisix-ingress-controller]

Posted by "Sn0rt (via GitHub)" <gi...@apache.org>.
Sn0rt commented on issue #2028:
URL: https://github.com/apache/apisix-ingress-controller/issues/2028#issuecomment-1792045747

   @maslow 
   
   I tried to reproduce this problem on the Alibaba Cloud ACK cluster, created 3500 APISIX-route, APISIX tls route and found no periodic high load.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bug: Enabling the APISIX ingress controller can lead to a surge in CPU usage on the APISIX gateway. [apisix-ingress-controller]

Posted by "shreemaan-abhishek (via GitHub)" <gi...@apache.org>.
shreemaan-abhishek commented on issue #2028:
URL: https://github.com/apache/apisix-ingress-controller/issues/2028#issuecomment-1811868160

   ping @maslow 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org