You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2021/09/29 07:43:41 UTC
[GitHub] [apisix] dyrnq opened a new issue #5155: request help: configure etcd mTLS warn certificate host mismatch
dyrnq opened a new issue #5155:
URL: https://github.com/apache/apisix/issues/5155
### Issue description
Apisix use mTLS(self-certification) connect etcd report error when config etcd.tls.verify=true on apisix2.10.0 version，but the same config run ok on apisix2.9
## case 1
```yaml
apisix:
node_listen: 9080
enable_ipv6: false
enable_debug: true
allow_admin:
- 0.0.0.0/0
admin_key:
- name: "admin"
key: edd1c9f034335f136f87ad84b625c8f1
role: admin
ssl:
ssl_trusted_certificate: /opt/apisix/pki/etcd/ca.crt
etcd:
host:
- "https://192.168.27.11:2379"
- "https://192.168.27.12:2379"
- "https://192.168.27.13:2379"
prefix: "/apisix"
timeout: 30
tls:
cert: /opt/apisix/pki/etcd/etcd-client.crt
key: /opt/apisix/pki/etcd/etcd-client.key
verify: true
plugin_attr:
prometheus:
export_addr:
ip: "0.0.0.0"
port: 9091
```
## error.log
```bash
2021/09/29 07:15:32 [warn] 51#51: *55053 [lua] v3.lua:151: _request_uri(): https://192.168.27.11:2379: certificate host mismatch. Retrying, context: ngx.timer
2021/09/29 07:15:33 [warn] 51#51: *55053 [lua] v3.lua:151: _request_uri(): https://192.168.27.11:2379: certificate host mismatch. Retrying, context: ngx.timer
2021/09/29 07:15:33 [warn] 51#51: *55053 [lua] health_check.lua:63: report_failure(): update endpoint: https://192.168.27.11:2379 to unhealthy, context: ngx.timer
2021/09/29 07:15:33 [warn] 51#51: *55053 [lua] v3.lua:151: _request_uri(): https://192.168.27.11:2379: certificate host mismatch. Retrying, context: ngx.timer
2021/09/29 07:15:33 [warn] 51#51: *55053 [lua] v3.lua:151: _request_uri(): https://192.168.27.12:2379: certificate host mismatch. Retrying, context: ngx.timer
2021/09/29 07:15:33 [warn] 51#51: *55053 [lua] v3.lua:151: _request_uri(): https://192.168.27.12:2379: certificate host mismatch. Retrying, context: ngx.timer
2021/09/29 07:15:33 [warn] 51#51: *55053 [lua] health_check.lua:63: report_failure(): update endpoint: https://192.168.27.12:2379 to unhealthy, context: ngx.timer
2021/09/29 07:15:33 [warn] 51#51: *55053 [lua] v3.lua:151: _request_uri(): https://192.168.27.12:2379: certificate host mismatch. Retrying, context: ngx.timer
2021/09/29 07:15:34 [warn] 51#51: *55053 [lua] v3.lua:151: _request_uri(): https://192.168.27.13:2379: certificate host mismatch. Retrying, context: ngx.timer
2021/09/29 07:15:34 [error] 51#51: *55053 [lua] config_etcd.lua:591: failed to fetch data from etcd: /usr/local/openresty/lualib/resty/core/socket/tcp.lua:209: assertion failed!
stack traceback:
[C]: in function 'assert'
/usr/local/openresty/lualib/resty/core/socket/tcp.lua:209: in function 'tls_handshake'
.../local/apisix//deps/share/lua/5.1/resty/http_connect.lua:239: in function 'connect'
/usr/local/apisix//deps/share/lua/5.1/resty/http.lua:927: in function 'request_uri'
/usr/local/apisix//deps/share/lua/5.1/resty/etcd/v3.lua:72: in function 'http_request_uri'
/usr/local/apisix//deps/share/lua/5.1/resty/etcd/v3.lua:146: in function '_request_uri'
/usr/local/apisix//deps/share/lua/5.1/resty/etcd/v3.lua:493: in function 'readdir'
/usr/local/apisix/apisix/core/config_etcd.lua:100: in function 'readdir'
/usr/local/apisix/apisix/core/config_etcd.lua:296: in function 'sync_data'
/usr/local/apisix/apisix/core/config_etcd.lua:556: in function </usr/local/apisix/apisix/core/config_etcd.lua:537>
[C]: in function 'xpcall'
/usr/local/apisix/apisix/core/config_etcd.lua:537: in function </usr/local/apisix/apisix/core/config_etcd.lua:516>, etcd key: /apisix/proto, context: ngx.timer
```
when config etcd.tls.verify=false on apisix2.10.0 the error has gone，is this a bug，and etcd.tls.verify=true on apisix2.9 there is no such problem.
### Environment
- apisix version (cmd: `apisix version`): 2.10.0
- OS (cmd: `uname -a`):
```bash
Linux ef2357fe80f7 5.4.0-77-generic #86-Ubuntu SMP Thu Jun 17 02:35:03 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
```
- OpenResty / Nginx version (cmd: `nginx -V` or `openresty -V`):
```bash
nginx version: openresty/1.19.3.2
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC)
built with OpenSSL 1.1.1l 24 Aug 2021
TLS SNI support enabled
configure arguments: --prefix=/usr/local/openresty/nginx --with-cc-opt='-O2 -DAPISIX_BASE_VER=0.0.0 -DNGX_LUA_ABORT_AT_PANIC -I/usr/local/openresty/zlib/include -I/usr/local/openresty/pcre/include -I/usr/local/openresty/openssl111/include' --add-module=../ngx_devel_kit-0.3.1 --add-module=../echo-nginx-module-0.62 --add-module=../xss-nginx-module-0.06 --add-module=../ngx_coolkit-0.2 --add-module=../set-misc-nginx-module-0.32 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.08 --add-module=../srcache-nginx-module-0.32 --add-module=../ngx_lua-0.10.19 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.33 --add-module=../array-var-nginx-module-0.05 --add-module=../memc-nginx-module-0.19 --add-module=../redis2-nginx-module-0.15 --add-module=../redis-nginx-module-0.3.7 --add-module=../ngx_stream_lua-0.0.9 --with-ld-opt='-Wl,-rpath,/usr/local/openresty/luajit/lib -L/usr/local/openresty/zlib/lib -L/usr/local/openre
sty/pcre/lib -L/usr/local/openresty/openssl111/lib -Wl,-rpath,/usr/local/openresty/zlib/lib:/usr/local/openresty/pcre/lib:/usr/local/openresty/openssl111/lib' --add-module=/tmp/tmp.ROmvVHzfSe/openresty-1.19.3.2/../mod_dubbo --add-module=/tmp/tmp.ROmvVHzfSe/openresty-1.19.3.2/../ngx_multi_upstream_module --add-module=/tmp/tmp.ROmvVHzfSe/openresty-1.19.3.2/../apisix-nginx-module --add-module=/tmp/tmp.ROmvVHzfSe/openresty-1.19.3.2/../lua-var-nginx-module --with-poll_module --with-pcre-jit --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_v2_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module --with-http_realip_module --with-http_addition_module --with-http_auth_request_module --with-http_secure_link_module --with-http_random_index_module --with-http_gzip_static_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --
with-threads --with-compat --with-stream --with-http_ssl_module
```
- etcd version, if have (cmd: run `curl http://127.0.0.1:9090/v1/server_info` to get the info from server-info API): 3.5.0
- apisix-dashboard version, if have: 2.8
- the plugin runner version, if the issue is about a plugin runner (cmd: depended on the kind of runner):
- luarocks version, if the issue is about installation (cmd: `luarocks --version`):
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander edited a comment on issue #5155: request help: configure etcd mTLS warn certificate host mismatch
Posted by GitBox <gi...@apache.org>.
spacewander edited a comment on issue #5155:
URL: https://github.com/apache/apisix/issues/5155#issuecomment-929948164
We upgrade lua-resty-http in 2.10.0.
Previously, it doesn't send SNI and verify the common name part in the certificate: https://github.com/ledgetech/lua-resty-http/issues/236
This behavior is a bug and is fixed in the latest lua-resty-http.
You need to use `openssl x509 -text -noout -in your_domain_cert` to see what's your common name in the certificate and change the host in the etcd URI to match the common name.
For example, assume:
```
$ openssl x509 -text -noout -in t/certs/etcd.pem
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
8d:cc:7a:ef:e0:25:54:cb
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = CN, ST = Zhejiang, L = Hangzhou, O = test, OU = test, CN = blahblah
Validity
Not Before: Oct 28 03:33:02 2020 GMT
Not After : Oct 28 03:33:02 2021 GMT
Subject: C = CN, ST = Zhejiang, L = Hangzhou, O = test, OU = test, CN = etcd.cluster.local
```
You need to change:
```
etcd:
host:
- "https://192.168.27.11:2379"
```
to
```
etcd:
host:
- "https://etcd.cluster.local:2379"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on issue #5155: request help: configure etcd mTLS warn certificate host mismatch
Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #5155:
URL: https://github.com/apache/apisix/issues/5155#issuecomment-929956604
There is also an ongoing PR to let you configure a SNI name to match the common name in the certificate: https://github.com/api7/lua-resty-etcd/pull/145/files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] dyrnq closed issue #5155: request help: configure etcd mTLS warn certificate host mismatch
Posted by GitBox <gi...@apache.org>.
dyrnq closed issue #5155:
URL: https://github.com/apache/apisix/issues/5155
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] dyrnq closed issue #5155: request help: configure etcd mTLS warn certificate host mismatch
Posted by GitBox <gi...@apache.org>.
dyrnq closed issue #5155:
URL: https://github.com/apache/apisix/issues/5155
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] dyrnq commented on issue #5155: request help: configure etcd mTLS warn certificate host mismatch
Posted by GitBox <gi...@apache.org>.
dyrnq commented on issue #5155:
URL: https://github.com/apache/apisix/issues/5155#issuecomment-929966145
> There is also an ongoing PR to let you configure a SNI name to match the common name in the certificate: https://github.com/api7/lua-resty-etcd/pull/145/files
That was great !
Thank you for your reply.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander edited a comment on issue #5155: request help: configure etcd mTLS warn certificate host mismatch
Posted by GitBox <gi...@apache.org>.
spacewander edited a comment on issue #5155:
URL: https://github.com/apache/apisix/issues/5155#issuecomment-929948164
We upgrade lua-resty-http in 2.10.0.
Previously, it doesn't send SNI and verify the common name part in the certificate: https://github.com/ledgetech/lua-resty-http/issues/236
This behavior is a bug and is fixed in the latest lua-resty-http.
You need to use `openssl x509 -text -noout -in your_domain_cert` to see what's your common name in the certificate and change the host in the etcd URI to match the common name.
For example, assume:
```
$ openssl x509 -text -noout -in t/certs/etcd.pem
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
8d:cc:7a:ef:e0:25:54:cb
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = CN, ST = Zhejiang, L = Hangzhou, O = test, OU = test, CN = blahblah
Validity
Not Before: Oct 28 03:33:02 2020 GMT
Not After : Oct 28 03:33:02 2021 GMT
Subject: C = CN, ST = Zhejiang, L = Hangzhou, O = test, OU = test, CN = etcd.cluster.local
```
You need to change:
```
etcd:
host:
- "https://192.168.27.11:2379"
```
to
```
etcd:
host:
- "https://etcd.cluster.local:2379"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on issue #5155: request help: configure etcd mTLS warn certificate host mismatch
Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #5155:
URL: https://github.com/apache/apisix/issues/5155#issuecomment-929948164
We upgrade lua-resty-http in 2.10.0.
Previously, it doesn't send SNI and verify the common name part in the certificate: https://github.com/ledgetech/lua-resty-http/issues/236
This behavior is a bug and is fixed in the latest lua-resty-http.
You need to use `openssl x509 -text -noout -in your_domain_cert` and change the host in the etcd URI to the common name.
For example, assume:
```
$ openssl x509 -text -noout -in t/certs/etcd.pem
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
8d:cc:7a:ef:e0:25:54:cb
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = CN, ST = Zhejiang, L = Hangzhou, O = test, OU = test, CN = blahblah
Validity
Not Before: Oct 28 03:33:02 2020 GMT
Not After : Oct 28 03:33:02 2021 GMT
Subject: C = CN, ST = Zhejiang, L = Hangzhou, O = test, OU = test, CN = etcd.cluster.local
```
You need to change:
```
etcd:
host:
- "https://192.168.27.11:2379"
```
to
```
etcd:
host:
- "https://etcd.cluster.local:2379"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on issue #5155: request help: configure etcd mTLS warn certificate host mismatch
Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #5155:
URL: https://github.com/apache/apisix/issues/5155#issuecomment-929948164
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] dyrnq commented on issue #5155: request help: configure etcd mTLS warn certificate host mismatch
Posted by GitBox <gi...@apache.org>.
dyrnq commented on issue #5155:
URL: https://github.com/apache/apisix/issues/5155#issuecomment-929966145
> There is also an ongoing PR to let you configure a SNI name to match the common name in the certificate: https://github.com/api7/lua-resty-etcd/pull/145/files
That was great !
Thank you for your reply.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org