You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by "shreemaan-abhishek (via GitHub)" <gi...@apache.org> on 2024/01/17 09:12:51 UTC
[PR] fix: don't release healthchecker during etcd sync due to compaction [apisix]
shreemaan-abhishek opened a new pull request, #10837:
URL: https://github.com/apache/apisix/pull/10837
### Description
#### Root cause of the bug:
When APISIX tries to watch a revision that has been compacted by etcd compaction. This case is difficult to reproduce manually, so adding tests for this would be very unstable.
To reproduce this manually:
- Create an upstream where only one node is available:
```shell
curl http://127.0.0.1:9180/apisix/admin/upstreams/1 \
-H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"nodes": {
"127.0.0.1:1980": 1,
"127.0.0.1:1111": 1
},
"retries": 0,
"name": "测试心跳",
"type": "roundrobin",
"timeout": {
"connect": 60,
"read": 60,
"send": 60
},
"pass_host": "pass",
"scheme": "http",
"keepalive_pool": {
"idle_timeout": 60,
"requests": 1000,
"size": 320
},
"checks": {
"active": {
"concurrency": 10,
"healthy": {
"http_statuses": [
200,
302
],
"interval": 1,
"successes": 2
},
"http_path": "/firstwork/healthcheck",
"timeout": 5,
"type": "http",
"unhealthy": {
"http_failures": 5,
"http_statuses": [
429,
404,
500,
501,
502,
503,
504,
505
],
"interval": 10,
"tcp_failures": 2,
"timeouts": 2
}
}
}
}'
```
- Create a route that references the above upstream:
```shell
curl "http://127.0.0.1:9180/apisix/admin/routes/1" -X PUT \
-H "X-API-KEY: ${ADMIN_API_KEY}" \
-d '
{
"uri": "/*",
"upstream_id": "1",
"name": "测试心跳问题",
"methods": [
"GET",
"HEAD",
"POST",
"PUT",
"DELETE",
"OPTIONS",
"PATCH",
"CONNECT",
"TRACE"
],
"priority": 0,
"enable_websocket": false,
"status": 1
}'
```
- send a couple of requests
Fixes #10500
### Checklist
- [ ] I have explained the need for this PR and the problem it solves
- [ ] I have explained the changes or the new features added to this PR
- [ ] I have added tests corresponding to this change
- [ ] I have updated the documentation to reflect this change
- [ ] I have verified that this change is backward compatible (If not, please discuss on the [APISIX mailing list](https://github.com/apache/apisix/tree/master#community) first)
<!--
Note
1. Mark the PR as draft until it's ready to be reviewed.
2. Always add/update tests for any changes unless you have a good reason.
3. Always update the documentation to reflect the changes made in the PR.
4. Make a new commit to resolve conversations instead of `push -f`.
5. To resolve merge conflicts, merge master instead of rebasing.
6. Use "request review" to notify the reviewer after making changes.
7. Only a reviewer can mark a conversation as resolved.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: don't release healthchecker during etcd sync due to compaction [apisix]
Posted by "shreemaan-abhishek (via GitHub)" <gi...@apache.org>.
shreemaan-abhishek commented on PR #10837:
URL: https://github.com/apache/apisix/pull/10837#issuecomment-1898075786
closing in favour of: https://github.com/apache/apisix/issues/10500#issuecomment-1895436144
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: don't release healthchecker during etcd sync due to compaction [apisix]
Posted by "shreemaan-abhishek (via GitHub)" <gi...@apache.org>.
shreemaan-abhishek commented on code in PR #10837:
URL: https://github.com/apache/apisix/pull/10837#discussion_r1455116949
##########
apisix/upstream.lua:
##########
@@ -81,7 +81,11 @@ end
_M.set = set_directly
-local function release_checker(healthcheck_parent)
+local function release_checker(healthcheck_parent, is_compacted)
+ if is_compacted then
Review Comment:
could you clarify more? My changes will not release the checker only when the reason is compaction. There is still another case that can release the checkers properly:
https://github.com/shreemaan-abhishek/apisix/blob/105993f40f34768cd0a86035d01393346fff3a94/apisix/core/config_etcd.lua#L686
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: don't release healthchecker during etcd sync due to compaction [apisix]
Posted by "shreemaan-abhishek (via GitHub)" <gi...@apache.org>.
shreemaan-abhishek closed pull request #10837: fix: don't release healthchecker during etcd sync due to compaction
URL: https://github.com/apache/apisix/pull/10837
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: don't release healthchecker during etcd sync due to compaction [apisix]
Posted by "membphis (via GitHub)" <gi...@apache.org>.
membphis commented on code in PR #10837:
URL: https://github.com/apache/apisix/pull/10837#discussion_r1455081563
##########
apisix/upstream.lua:
##########
@@ -81,7 +81,11 @@ end
_M.set = set_directly
-local function release_checker(healthcheck_parent)
+local function release_checker(healthcheck_parent, is_compacted)
+ if is_compacted then
Review Comment:
This is a wrong solution and will cause the checker to never be released
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org