You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2020/11/16 04:57:37 UTC
[GitHub] [apisix] Yiyiyimu opened a new issue #2757: feat: deploy ChaosMesh on APISIX, to simulate more faults
Yiyiyimu opened a new issue #2757:
URL: https://github.com/apache/apisix/issues/2757
Here to list a rough schedule of this feature:
### Basic Integration
- [ ] add a basic chaos to apisix
### Problems to simulate
- [ ] the connection with etcd is unstable
- [ ] etcd failure
- [ ] problems when cpu/memory/disk stressed out
### Correctness Proof
- [ ] error log of Nginx and APISIX
- [ ] whether cpu/memory use of APISIX is abnormally high
- [ ] whether `wrk` benchmarking would fail
### CI Deployment
- [ ] deploy on Github Actions
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Yiyiyimu edited a comment on issue #2757: feat: deploy ChaosMesh on APISIX, to simulate more faults
Posted by GitBox <gi...@apache.org>.
Yiyiyimu edited a comment on issue #2757:
URL: https://github.com/apache/apisix/issues/2757#issuecomment-740483891
Hi @moonming I made some basic tests, applying etcd network delay and etcd pods failed. And I got the following results for now.
1. Both of the chaos could take effect. For the network delay, I could see the delay when I put a new route. And for pods failure, I could see the new route setting would fail.
2. However a benchmark tool is somewhat not appropriate for testing the performance. I failed to found how to use `wrk` to PUT route, so I just [test with curl](https://dev.to/yuyatakeyama/how-i-measure-response-times-of-web-apis-using-curl-6nh). I could know if it works as expect and some error would indeed show up. But get to know the network delay or pod failure is still useless for us.
So I guess I could try to integrate it with the backend e2e test in dashboard, like running e2e test when certain chaos injected, so that could be more of a real user scenario and we could get more information. Hi @nic-chen do you think that's possible?
Some coarse todo:
- [ ] run APISIX on kubernetes on CI
- [ ] run e2e test on kubernetes
- [ ] add chaos
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] moonming commented on issue #2757: feat: deploy ChaosMesh on APISIX, to simulate more faults
Posted by GitBox <gi...@apache.org>.
moonming commented on issue #2757:
URL: https://github.com/apache/apisix/issues/2757#issuecomment-737784612
ok, take your time
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] moonming commented on issue #2757: feat: deploy ChaosMesh on APISIX, to simulate more faults
Posted by GitBox <gi...@apache.org>.
moonming commented on issue #2757:
URL: https://github.com/apache/apisix/issues/2757#issuecomment-737712859
@Yiyiyimu Any progress? Do you need help? ：）
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] nic-chen commented on issue #2757: feat: deploy ChaosMesh on APISIX, to simulate more faults
Posted by GitBox <gi...@apache.org>.
nic-chen commented on issue #2757:
URL: https://github.com/apache/apisix/issues/2757#issuecomment-740616155
> So I guess I could try to integrate it with the backend e2e test in dashboard, like running e2e test when certain chaos injected, so that could be more of a real user scenario and we could get more information. Hi @nic-chen do you think that's possible?
sure. E2E test and the main project are independent, you could run it directly on your test env.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Yiyiyimu edited a comment on issue #2757: feat: deploy ChaosMesh on APISIX, to simulate more faults
Posted by GitBox <gi...@apache.org>.
Yiyiyimu edited a comment on issue #2757:
URL: https://github.com/apache/apisix/issues/2757#issuecomment-740483891
Hi @moonming I made some basic tests, applying etcd network delay and etcd pods failed. And I got the following results for now.
1. Both of the chaos could take effect. For the network delay, I could see the delay when I put a new route. And for pods failure, I could see the new route setting would fail.
2. However a benchmark tool is somewhat not appropriate for testing the performance. I failed to found how to use `wrk` to PUT route, so I just [test with curl](https://dev.to/yuyatakeyama/how-i-measure-response-times-of-web-apis-using-curl-6nh). I could know if it works as expect and some error would indeed show up. But get to know the network delay or pod failure is still useless for us.
So I guess I could try to integrate it with the backend e2e test in dashboard, like running e2e test when certain chaos injected, so that could be more of a real user scenario and we could get more information. Hi @nic-chen do you think that's possible?
Some coarse todo:
- [ ] run e2e test on kubernetes on CI
- [ ] add chaos
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Yiyiyimu commented on issue #2757: feat: deploy ChaosMesh on APISIX, to simulate more faults
Posted by GitBox <gi...@apache.org>.
Yiyiyimu commented on issue #2757:
URL: https://github.com/apache/apisix/issues/2757#issuecomment-740483891
Hi @moonming I made some basic tests, applying etcd network delay and etcd pods failed. And I got the following results for now.
1. Both of the chaos could take effect. For the network delay, I could see the delay when I put a new route. And for pods failure, I could see the new route setting would fail.
2. However a benchmark tool is somewhat not appropriate for testing the performance. I failed to found how to use `wrk` to PUT route, so I just [test with curl](https://dev.to/yuyatakeyama/how-i-measure-response-times-of-web-apis-using-curl-6nh). I could know if it works as expect and some error would indeed show up. But get to know the network delay or pod failure is still useless for us.
So I guess I could try to integrate it with the backend e2e test in dashboard, like running e2e test when certain chaos injected, so that could be more of a real user scenario and we could get more information. Hi @nic-chen do you think that's possible?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] moonming commented on issue #2757: feat: deploy ChaosMesh on APISIX, to simulate more faults
Posted by GitBox <gi...@apache.org>.
moonming commented on issue #2757:
URL: https://github.com/apache/apisix/issues/2757#issuecomment-740712405
> So I guess I could try to integrate it with the backend e2e test in dashboard, like running e2e test when certain chaos injected, so that could be more of a real user scenario and we could get more information
I do not think so. Now the e2e test of dashboard is relatively simple and not enough to support chaos mesh. Most of the tests in the apisix project are also e2e tests, I think it is more appropriate to add a one new CI task for the apisix project
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Yiyiyimu commented on issue #2757: feat: deploy ChaosMesh on APISIX, to simulate more faults
Posted by GitBox <gi...@apache.org>.
Yiyiyimu commented on issue #2757:
URL: https://github.com/apache/apisix/issues/2757#issuecomment-737780448
Hi @moonming Sorry the progress is a bit slower than expected. This is the final week of my Community Bridge Program, so I think I would have more time working on this next week. I'll try to hurry up
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander closed issue #2757: feat: deploy ChaosMesh on APISIX, to simulate more faults
Posted by GitBox <gi...@apache.org>.
spacewander closed issue #2757:
URL: https://github.com/apache/apisix/issues/2757
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org