You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2021/10/09 09:06:06 UTC
[GitHub] [apisix] tzssangglass opened a new issue #5186: bug: confused watch keys lead to accidents
tzssangglass opened a new issue #5186:
URL: https://github.com/apache/apisix/issues/5186
### Issue description
with `enable_admin: false`, apisix will watch `/apisix/plugins`. see:https://github.com/apache/apisix/blob/03a1c760845e7666d924b4f817a42e8e4c0c7e0b/apisix/plugin.lua#L512-L513
If a new apisix instance connects to the same etcd cluster, the new instance will do something to `/apisix/plugins/` during the init etcd phase, causing the apisix which watching `/apisix/plugins` to get an incorrect response.
### Environment
- apisix version (cmd: `apisix version`): master branch
- OS (cmd: `uname -a`):Darwin Kernel Version 20.6.0
- OpenResty / Nginx version (cmd: `nginx -V` or `openresty -V`): apisix-base
- etcd version, if have (cmd: run `curl http://127.0.0.1:9090/v1/server_info` to get the info from server-info API):3.5.0
- apisix-dashboard version, if have:
- the plugin runner version, if the issue is about a plugin runner (cmd: depended on the kind of runner):
- luarocks version, if the issue is about installation (cmd: `luarocks --version`):3.4
### Steps to reproduce
1. prepare apisix instance A, change `enable_admin: false` in config.yaml, and start apisix instance A
2. prepare apisix instance B, change `enable_admin: false` in config.yaml, and start apisix instance B
3. look at the `error.log` for apisix instance A, there has some error level logs
### Actual result
has error logs
### Error log
I've added logging to apisix instance A the line below here:
https://github.com/apache/apisix/blob/03a1c760845e7666d924b4f817a42e8e4c0c7e0b/apisix/plugin.lua#L227 to print the `config`
there error logs as below:
```
2021/10/09 16:52:52 [warn] 29391#6632203: *79731 [lua] plugin.lua:228: load(): config : {
createdIndex = 1075,
key = "/apisix/plugins/",
modifiedIndex = 1239
}, context: ngx.timer
2021/10/09 16:52:52 [error] 29389#6632201: *79873 [lua] config_etcd.lua:591: failed to fetch data from etcd: /usr/local/Cellar/apisix/apisix/plugin.lua:249: bad argument #1 to 'ipairs' (table expected, got nil)
stack traceback:
[C]: in function 'ipairs'
/usr/local/Cellar/apisix/apisix/plugin.lua:249: in function 'load'
/usr/local/Cellar/apisix/apisix/plugin.lua:496: in function 'filter'
/usr/local/Cellar/apisix/apisix/core/config_etcd.lua:462: in function 'sync_data'
/usr/local/Cellar/apisix/apisix/core/config_etcd.lua:547: in function </usr/local/Cellar/apisix/apisix/core/config_etcd.lua:537>
[C]: in function 'xpcall'
/usr/local/Cellar/apisix/apisix/core/config_etcd.lua:537: in function </usr/local/Cellar/apisix/apisix/core/config_etcd.lua:516>, etcd key: /apisix/plugins, context: ngx.timer
2021/10/09 16:52:52 [error] 29387#6632199: *79736 [lua] config_etcd.lua:591: failed to fetch data from etcd: /usr/local/Cellar/apisix/apisix/plugin.lua:249: bad argument #1 to 'ipairs' (table expected, got nil)
```
### Expected result
The error log output for instance A is related to the start of instance B.
And as you can see from the error log output, instance A is listening to the key `/apisix/plugins/`, It should not actually listen to this key. It should be listening to `/apisix/plugins` and the corresponding value should be
```
{
clean_handlers = {},
createdIndex = 1091,
key = "/apisix/plugins",
modifiedIndex = 1091,
value = { {
name = "api-breaker"
}, {
name = "authz-keycloak"
}, {
name = "basic-auth"
}, {
name = "batch-requests"
}, {
name = "consumer-restriction"
}, {
name = "cors"
}, {
name = "echo"
}, {
name = "fault-injection"
}, {
name = "grpc-transcode"
}, {
name = "hmac-auth"
}, {
name = "http-logger"
}, {
name = "ip-restriction"
}, {
name = "jwt-auth"
}, {
name = "kafka-logger"
}, {
name = "key-auth"
}, {
name = "limit-conn"
}, {
name = "limit-count"
}, {
name = "limit-req"
}, {
name = "openid-connect"
}, {
name = "prometheus"
}, {
name = "proxy-cache"
}, {
name = "proxy-mirror"
}, {
name = "proxy-rewrite"
}, {
name = "redirect"
}, {
name = "referer-restriction"
}, {
name = "request-id"
}, {
name = "request-validation"
}, {
name = "response-rewrite"
}, {
name = "serverless-post-function"
}, {
name = "serverless-pre-function"
}, {
name = "sls-logger"
}, {
name = "syslog"
}, {
name = "tcp-logger"
}, {
name = "udp-logger"
}, {
name = "uri-blocker"
}, {
name = "wolf-rbac"
}, {
name = "zipkin"
}, {
name = "server-info"
}, {
name = "traffic-split"
}, {
name = "mqtt-proxy",
stream = true
} }
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander closed issue #5186: bug: confused watch keys lead to accidents
Posted by GitBox <gi...@apache.org>.
spacewander closed issue #5186:
URL: https://github.com/apache/apisix/issues/5186
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on issue #5186: bug: confused watch keys lead to accidents
Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #5186:
URL: https://github.com/apache/apisix/issues/5186#issuecomment-939262059
DP side needs to sync the plugin from the admin, if have. I am curious why instance B will write to the /apisix/plugins, as only Admin API can write to it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #5186: bug: confused watch keys lead to accidents
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #5186:
URL: https://github.com/apache/apisix/issues/5186#issuecomment-939262674
> DP side needs to sync the plugin from the admin, if have. I am curious why instance B will write to the /apisix/plugins, as only Admin API can write to it.
instance B doesn't write to `/apisix/plugins`, it writes to `/apisix/plugins/`, but instance A listens for changes to `/apisix/plugins/` when it listen `/apisix/plugins`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org