You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2021/05/27 00:09:08 UTC
[GitHub] [apisix] Yiyiyimu opened a new issue #4316: bug: etcd timeout in config is too large to be useful
Yiyiyimu opened a new issue #4316:
URL: https://github.com/apache/apisix/issues/4316
### Issue description
Just found adding 1s network delay between apisix and etcd could make all operation fails, although we set default etcd timeout to 30s. The reason is that there seems a default and non-configurable timeout context in the etcd, and it's around 5s. After that, it would just return the error `context deadline exceeded`.
So no matter how large we add network delay to etcd, the put operation with grpc (test with `etcdctl`) would timeout at 5s, and RESTful API (test with `curl`) would timeout at 7s. However, when set etcd timeout (used in `lua-resty-http`) as 30s, we need to wait more time (30s) to get the returned error when for example, add a route to APISIX. So I think reducing the default timeout to 10s is enough for us, and we could remind users greater timeout would be no use.
Here I list a `delay_time - process_time - error chart` for future reference
<!--StartFragment-->
network delay | grpc response time | restful response time | lua-resty-etcd response time | apisix response time | grpc error | restful error | lua-resty-etcd error | apisix error
-- | -- | -- | -- | -- | -- | -- | -- | --
0 | 0.026 | 0.014 | 0.004 | 0.008 | | | |
50ms | 0.128 | 0.112 | 0.255 | 0.154 | | | |
200ms | 0.427 | 0.412 | 1.00 | 1.00 | | | |
500ms | 1.03 | 1.02 | 1.50 | 3.00 | | | |
800ms | 5.02 | 7.01 | 7.80 | 8.60 | context deadline exceeded | etcdserver: request timed out | invalid response code: 500 | {"error_msg":"invalid response code: 503"}
2s | 5.02 | 7.02 | 9.00 | 11.0 | context deadline exceeded | context deadline exceeded | invalid response code: 500 | {"error_msg":"invalid response code: 503"}
10s | 5.02 | 7.01 | 30.0 | 38.0 | context deadline exceeded | context deadline exceeded | timeout | {"error_msg":"invalid response code: 503"} [from log] failed to fetch data from etcd: no route to host
30s | 5.05 | 7.01 | | | context deadline exceeded | context deadline exceeded | [can not create etcd client] timeout | {"error_msg":"no route to host"}/{"error_msg":"timeout"}
<!--EndFragment-->
### Environment
Bug report without environment information will be ignored or closed.
* apisix version (cmd: `apisix version`): docker latest version
### Minimal test code / Steps to reproduce the issue
Just like #4315
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tokers commented on issue #4316: bug: etcd timeout in config is too large to be useful
Posted by GitBox <gi...@apache.org>.
tokers commented on issue #4316:
URL: https://github.com/apache/apisix/issues/4316#issuecomment-849268550
Agree +1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on issue #4316: bug: etcd timeout in config is too large to be useful
Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #4316:
URL: https://github.com/apache/apisix/issues/4316#issuecomment-849219545
Agree
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Yiyiyimu commented on issue #4316: bug: etcd timeout in config is too large to be useful
Posted by GitBox <gi...@apache.org>.
Yiyiyimu commented on issue #4316:
URL: https://github.com/apache/apisix/issues/4316#issuecomment-849259498
Got it would fix it later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Firstsawyou commented on issue #4316: bug: etcd timeout in config is too large to be useful
Posted by GitBox <gi...@apache.org>.
Firstsawyou commented on issue #4316:
URL: https://github.com/apache/apisix/issues/4316#issuecomment-849337578
Agree +1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on issue #4316: bug: etcd timeout in config is too large to be useful
Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #4316:
URL: https://github.com/apache/apisix/issues/4316#issuecomment-921441515
Ping @Yiyiyimu?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] starsz commented on issue #4316: bug: etcd timeout in config is too large to be useful
Posted by GitBox <gi...@apache.org>.
starsz commented on issue #4316:
URL: https://github.com/apache/apisix/issues/4316#issuecomment-850299129
Agree +1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #4316: bug: etcd timeout in config is too large to be useful
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #4316:
URL: https://github.com/apache/apisix/issues/4316#issuecomment-849252972
agree
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on issue #4316: bug: etcd timeout in config is too large to be useful
Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #4316:
URL: https://github.com/apache/apisix/issues/4316#issuecomment-1007066527
I rethink this issue.
Is this timeout applied to the `watch` operation? The `watch` operation is implemented via HTTP long polling or gRPC streaming, which can't have such a short timeout.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org