You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2022/04/07 02:13:28 UTC
[GitHub] [apisix] ltt1987 opened a new issue, #6798: Etcd CPU linear rise with an empty three node apisix cluster.
ltt1987 opened a new issue, #6798:
URL: https://github.com/apache/apisix/issues/6798
### Current Behavior
I created an empty three node apisix cluster, there is no anything routers. The three apisix node connected one new etcd node.
And I already **disabled** `server-info` plugin. Other plugins are configured by default.
After about 2 weeks of idle running, I got the following problem : Etcd server CPU linear rise .
<img width="1112" alt="WX20220406-134622@2x" src="https://user-images.githubusercontent.com/8955240/162104924-c67f9a38-3586-4529-b4dc-0d083e951ddc.png">
Until I stop the apisix service manually, etcd CPU is back to normal immediately.
### Expected Behavior
_No response_
### Error Logs
_No response_
### Steps to Reproduce
RUN with docker swarm.
etcd-compose.yaml
```
version: "3.8"
services:
etcd:
image: bitnami/etcd:3.5.2
networks:
- test_network
volumes:
- /etc/localtime:/etc/localtime
- "etcd:/bitnami/etcd"
environment:
ETCD_ROOT_PASSWORD: xxxxxx
ETCD_NAME: etcd
ETCD_INITIAL_ADVERTISE_PEER_URLS: http://etcd:2380
ETCD_LISTEN_PEER_URLS: http://0.0.0.0:2380
ETCD_LISTEN_CLIENT_URLS: http://0.0.0.0:2379
ETCD_ADVERTISE_CLIENT_URLS: http://etcd:2379
deploy:
replicas: 1
placement:
constraints:
- "node.hostname==test-etcd-01"
volumes:
etcd:
networks:
test_network:
external: true
```
apisix-compose.yaml
```
version: "3.8"
services:
apisix:
image: apache/apisix:2.12.1-alpine
volumes:
- /etc/localtime:/etc/localtime
- /app/apisix/apisix_log:/usr/local/apisix/logs
- /app/apisix/apisix.yaml:/usr/local/apisix/conf/config.yaml:ro
networks:
- test_network
environment:
HOSTNAME: ${HOSTNAME}_apisix
ports:
- "9080:9080"
- "9091:9091"
- "9043:9443"
- "9092:9092"
deploy:
mode: global
placement:
constraints: [node.role == manager]
networks:
test_network:
external: true
```
### Environment
- APISIX version: 2.12.1
- Operating system: Linux test-apisix-001 3.10.0-1160.42.2.el7.x86_64 #1 SMP Tue Sep 7 14:49:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
- etcd version, if relevant (run `curl http://127.0.0.1:9090/v1/server_info`): 3.5.2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] ltt1987 commented on issue #6798: Etcd CPU linear rise with an empty three node apisix cluster.
Posted by GitBox <gi...@apache.org>.
ltt1987 commented on issue #6798:
URL: https://github.com/apache/apisix/issues/6798#issuecomment-1098755250
This problem has been solved, It's a etcd problem because the etcd docker data volume is not empty.
Thanks~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] ltt1987 closed issue #6798: Etcd CPU linear rise with an empty three node apisix cluster.
Posted by GitBox <gi...@apache.org>.
ltt1987 closed issue #6798: Etcd CPU linear rise with an empty three node apisix cluster.
URL: https://github.com/apache/apisix/issues/6798
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] ltt1987 commented on issue #6798: Etcd CPU linear rise with an empty three node apisix cluster.
Posted by GitBox <gi...@apache.org>.
ltt1987 commented on issue #6798:
URL: https://github.com/apache/apisix/issues/6798#issuecomment-1094676992
> @ltt1987 Please refer to #6576 to see if it can help you, thanks!
Thanks for your reply, I checked related content and tried the following:
- updated apisix to 2.13 version
- create a new etcd node with `--auto-compaction-mode=periodic --auto-compaction-retention=1`
The problem is still there, hoping for more help.
<img width="1080" alt="WX20220411-155325@2x" src="https://user-images.githubusercontent.com/8955240/162690611-5064b134-f42f-4a2f-823a-97649c9d99c6.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tokers commented on issue #6798: Etcd CPU linear rise with an empty three node apisix cluster.
Posted by GitBox <gi...@apache.org>.
tokers commented on issue #6798:
URL: https://github.com/apache/apisix/issues/6798#issuecomment-1091426090
@ltt1987 Please refer to https://github.com/apache/apisix/issues/6576 to see if it can help you, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] ltt1987 commented on issue #6798: Etcd CPU linear rise with an empty three node apisix cluster.
Posted by GitBox <gi...@apache.org>.
ltt1987 commented on issue #6798:
URL: https://github.com/apache/apisix/issues/6798#issuecomment-1098755489
This problem has been solved, It's a etcd problem because the etcd docker data volume is not empty.
Thanks~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] leslie-tsang commented on issue #6798: Etcd CPU linear rise with an empty three node apisix cluster.
Posted by GitBox <gi...@apache.org>.
leslie-tsang commented on issue #6798:
URL: https://github.com/apache/apisix/issues/6798#issuecomment-1091012522
Hello there, Please check [this](https://github.com/apache/apisix/issues/5723) out if it can help you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tokers commented on issue #6798: Etcd CPU linear rise with an empty three node apisix cluster.
Posted by GitBox <gi...@apache.org>.
tokers commented on issue #6798:
URL: https://github.com/apache/apisix/issues/6798#issuecomment-1094769960
@ltt1987 Have you investigated the new rising trends for a long while to see if it'll reach the same level when you use APISIX `2.12.1`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org