You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2021/06/03 09:53:31 UTC

[GitHub] [apisix] xyz2b opened a new issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

xyz2b opened a new issue #4370:
URL: https://github.com/apache/apisix/issues/4370


   ### Issue description
   An error is reported when the configuration is updated from etcd regularly.
   Apisix create ​​a timer in the init phase to update the configuration from etcd, but the `lua_ssl_trusted_certificate` configuration is configured in the server configuration block. The scope of `init_by_lua_block` is the http configuration block, So the `lua_ssl_trusted_certificate` configuration of apisix is ​​invalid in the http configuration block.
   I moved the configuration from the server configuration block to the http configuration block and solved it.
   
   error.log
   ```shell
   2021/06/03 15:46:39 [error] 22427#22427: *57 [lua] config_etcd.lua:551: failed to fetch data from etcd: 20: unable to get local issuer certificate,  etcd key: /apisix/routes, context: ngx.timer
   2021/06/03 15:46:39 [error] 22429#22429: *32 [lua] config_etcd.lua:551: failed to fetch data from etcd: 20: unable to get local issuer certificate,  etcd key: /apisix/routes, context: ngx.timer
   ```
   
   ### Environment
   
   Request help without environment information will be ignored or closed.
   
   * apisix version (cmd: `apisix version`):
   ```shell
   [app@VM_97_180_centos apisix]$ ./bin/apisix version
   /data/app/openresty/luajit/bin/luajit ./apisix/cli/apisix.lua version
   2.6
   ```
   * OS (cmd: `uname -a`):
   ```shell
   [app@VM_97_180_centos apisix]$ uname -a
   Linux VM_97_180_centos 3.10.0-1127.13.1.el7.x86_64 #1 SMP Tue Jun 23 15:46:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
   ```
   * OpenResty / Nginx version (cmd: `nginx -V` or `openresty -V`):
   ```shell
   [app@VM_97_180_centos apisix]$ openresty -V                                                               
   nginx version: openresty/1.19.3.1
   built by gcc 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) 
   built with OpenSSL 1.1.1k  25 Mar 2021
   TLS SNI support enabled
   configure arguments: --prefix=/data/app/openresty/nginx --with-debug --with-cc-opt='-DNGX_LUA_USE_ASSERT -DNGX_LUA_ABORT_AT_PANIC -O2' --add-module=../ngx_devel_kit-0.3.1 --add-module=../echo-nginx-module-0.62 --add-module=../xss-nginx-module-0.06 --add-module=../ngx_coolkit-0.2 --add-module=../set-misc-nginx-module-0.32 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.08 --add-module=../srcache-nginx-module-0.32 --add-module=../ngx_lua-0.10.19 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.33 --add-module=../array-var-nginx-module-0.05 --add-module=../memc-nginx-module-0.19 --add-module=../redis2-nginx-module-0.15 --add-module=../redis-nginx-module-0.3.7 --add-module=../ngx_stream_lua-0.0.9 --with-ld-opt=-Wl,-rpath,/data/app/openresty/luajit/lib --user=app --group=apps --add-module=/data/backup/openresty-1.19.3.1/../mod_dubbo --add-module=/data/backup/openresty-1.19.3.1/../ngx_multi_upstream_module --
 add-module=/data/backup/openresty-1.19.3.1/../apisix-nginx-module --with-poll_module --with-pcre-jit --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_v2_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module --with-http_realip_module --with-http_addition_module --with-http_auth_request_module --with-http_secure_link_module --with-http_random_index_module --with-http_gzip_static_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-threads --with-compat --with-openssl=/data/backup/openresty-1.19.3.1/../openssl-OpenSSL_1_1_1k --with-openssl-opt=-g --with-stream --with-http_ssl_module
   ```
   * etcd version, if have (cmd: run `curl http://127.0.0.1:9090/v1/server_info` to get the info from server-info API):
   ```shell
   [app@VM_97_180_centos apisix]$ curl --cert ./ssl/etcd.pem --key ./ssl/etcd-key.pem --cacert ./ssl/ca.pem -i https://etcd01.apisix.webank.com:2379/version
   
   {"etcdserver":"3.4.16","etcdcluster":"3.4.0"}
   ```
   * apisix-dashboard version, if have: ```2.6.1```
   * luarocks version, if the issue is about installation (cmd: `luarocks --version`):
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] xyz2b edited a comment on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
xyz2b edited a comment on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-854352089


   Reinstalled apisix openresty and apisix, the problem remains.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] spacewander closed issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
spacewander closed issue #4370:
URL: https://github.com/apache/apisix/issues/4370


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] xyz2b commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
xyz2b commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-854277066


   Yes. I used apisix openresty, please see openresty compilation parameters. At the same time I also turned on tls.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] spacewander removed a comment on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
spacewander removed a comment on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-856373053


   @tzssangglass 
   Could you provide a minimal reproduce case for the error?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] spacewander commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-854296181


   It is strange. The TLS handshake in the init phase doesn't use the `lua_ssl_trusted_certificate` directly.
   
   It just uses the configured CA:
   https://github.com/apache/apisix/blob/c61261af8d19557b77535c9c745f4f8182bb63b2/apisix/patch.lua#L207-L210
   
   I try with self-signed certificate locally and can't reproduce it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] spacewander commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-856373053


   @tzssangglass 
   Could you provide a minimal reproduce case for the error?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] tzssangglass commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-856485059


   > @tzssangglass Is the etcd server cert self-signed? Or you may try to put the whole cert chain into `mtls_server.crt` and see what happens.
   
   how to do it?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] tzssangglass edited a comment on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
tzssangglass edited a comment on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-856481940


   @spacewander @tokers , I use the certs in https://github.com/apache/apisix/tree/master/t/certs, I get this error when I verify these certificates
   
   ```shell
   $ /usr/local/openresty/openssl111/bin/openssl verify t/certs/mtls_client.crt
   C = cn, ST = GuangDong, O = api7, L = ZhuHai, CN = client.apisix.dev
   error 20 at 0 depth lookup: unable to get local issuer certificate
   error t/certs/mtls_client.crt: verification failed
   
   $ /usr/local/openresty/openssl111/bin/openssl verify t/certs/mtls_server.crt 
   C = cn, ST = GuangDong, O = api7, L = ZhuHai, CN = admin.apisix.dev
   error 20 at 0 depth lookup: unable to get local issuer certificate
   error t/certs/mtls_server.crt: verification failed
   
   $  /usr/local/openresty/openssl111/bin/openssl verify t/certs/mtls_server.crt -CAfile /usr/local/apisix/t/certs/mtls_ca.crt
   C = cn, ST = GuangDong, O = api7, L = ZhuHai, CN = admin.apisix.dev
   error 20 at 0 depth lookup: unable to get local issuer certificate
   error t/certs/mtls_server.crt: verification failed
   Can't open -CAfile for reading, No such file or directory
   281473499807792:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('-CAfile','r')
   281473499807792:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:
   unable to load certificate
   C = cn, ST = GuangDong, L = ZhuHai, O = api7, OU = ops, CN = ca.apisix.dev
   error 18 at 0 depth lookup: self signed certificate
   error /usr/local/apisix/t/certs/mtls_ca.crt: verification failed
   
   $  /usr/local/openresty/openssl111/bin/openssl verify t/certs/mtls_client.crt -CAfile /usr/local/apisix/t/certs/mtls_ca.crt 
   C = cn, ST = GuangDong, O = api7, L = ZhuHai, CN = client.apisix.dev
   error 20 at 0 depth lookup: unable to get local issuer certificate
   error t/certs/mtls_client.crt: verification failed
   Can't open -CAfile for reading, No such file or directory
   281473612107824:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('-CAfile','r')
   281473612107824:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:
   unable to load certificate
   C = cn, ST = GuangDong, L = ZhuHai, O = api7, OU = ops, CN = ca.apisix.dev
   error 18 at 0 depth lookup: self signed certificate
   error /usr/local/apisix/t/certs/mtls_ca.crt: verification failed
   
   ```
   
   I get the same error `unable to get local issuer certificate` as start apisix,  I think this problem is related to the certificate


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] xyz2b edited a comment on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
xyz2b edited a comment on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-854352089


   Reinstalled apisix openresty and apisix, the problem remains.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] spacewander commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-856541928


   > 
   > 
   > @spacewander @tokers , I use the certs in [https://github.com/apache/apisix/tree/master/t/certs](https://github.com/apache/apisix/tree/master/t/certs?rgh-link-date=2021-06-08T06%3A18%3A55Z), I get this error when I verify these certificates
   > 
   > ```shell
   > $ /usr/local/openresty/openssl111/bin/openssl verify t/certs/mtls_client.crt
   > C = cn, ST = GuangDong, O = api7, L = ZhuHai, CN = client.apisix.dev
   > error 20 at 0 depth lookup: unable to get local issuer certificate
   > error t/certs/mtls_client.crt: verification failed
   > 
   > $ /usr/local/openresty/openssl111/bin/openssl verify t/certs/mtls_server.crt 
   > C = cn, ST = GuangDong, O = api7, L = ZhuHai, CN = admin.apisix.dev
   > error 20 at 0 depth lookup: unable to get local issuer certificate
   > error t/certs/mtls_server.crt: verification failed
   > 
   > $  /usr/local/openresty/openssl111/bin/openssl verify t/certs/mtls_server.crt -CAfile /usr/local/apisix/t/certs/mtls_ca.crt
   > C = cn, ST = GuangDong, O = api7, L = ZhuHai, CN = admin.apisix.dev
   > error 20 at 0 depth lookup: unable to get local issuer certificate
   > error t/certs/mtls_server.crt: verification failed
   > Can't open -CAfile for reading, No such file or directory
   > 281473499807792:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('-CAfile','r')
   > 281473499807792:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:
   > unable to load certificate
   > C = cn, ST = GuangDong, L = ZhuHai, O = api7, OU = ops, CN = ca.apisix.dev
   > error 18 at 0 depth lookup: self signed certificate
   > error /usr/local/apisix/t/certs/mtls_ca.crt: verification failed
   > 
   > $  /usr/local/openresty/openssl111/bin/openssl verify t/certs/mtls_client.crt -CAfile /usr/local/apisix/t/certs/mtls_ca.crt 
   > C = cn, ST = GuangDong, O = api7, L = ZhuHai, CN = client.apisix.dev
   > error 20 at 0 depth lookup: unable to get local issuer certificate
   > error t/certs/mtls_client.crt: verification failed
   > Can't open -CAfile for reading, No such file or directory
   > 281473612107824:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('-CAfile','r')
   > 281473612107824:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:
   > unable to load certificate
   > C = cn, ST = GuangDong, L = ZhuHai, O = api7, OU = ops, CN = ca.apisix.dev
   > error 18 at 0 depth lookup: self signed certificate
   > error /usr/local/apisix/t/certs/mtls_ca.crt: verification failed
   > ```
   > 
   > a little strange, `/usr/local/apisix/t/certs/mtls_ca.crt` clearly has a certificate file
   > 
   > I get the same error `unable to get local issuer certificate` as start apisix, I think this problem is related to the certificate
   
   Need to use `openssl verify -CAfile $ca $crt`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] tzssangglass commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-856141083


   ok, I have the same problem……
   
   my apisix `config.yaml`
   ```yaml
   apisix:
     admin_key:
       - name: "admin"
         key: edd1c9f034335f136f87ad84b625c8f1
         role: admin
     ssl:
       ssl_trusted_certificate: /usr/local/apisix/t/certs/mtls_ca.crt
   etcd:
     host:
       - "https://admin.apisix.dev:22379"
     prefix: "/apisix"
     tls:
       cert: /usr/local/apisix/t/certs/mtls_client.crt
       key: /usr/local/apisix/t/certs/mtls_client.key
   ```
   
   my etcd cluster is same as https://github.com/apache/apisix/blob/a2d80b73ed1d61a9f3e02bf2ddd5c4f2ffc4ddac/.github/workflows/build.yml#L140-L153
   
   I executed
   ```shell
   echo "127.0.0.1 admin.apisix.dev" | sudo tee -a /etc/hosts
   ```
   to set local domain
   
   and then, I use curl to verify that it is a problem with etcd 
   
   ```shell
   $ curl --cert /usr/local/apisix/t/certs/mtls_client.crt --key /usr/local/apisix/t/certs/mtls_client.key --cacert /usr/local/apisix/t/certs/mtls_ca.crt https://admin.apisix.dev:22379/version
   {"etcdserver":"3.4.13","etcdcluster":"3.4.0"}#        
   ```
   tt doesn't look like.
   
   here is my error.log
   
   ```
   2021/06/08 01:50:33 [error] 283990#283990: *75 [lua] config_etcd.lua:550: failed to fetch data from etcd: 20: unable to get local issuer certificate,  etcd key: /apisix/routes, context: ngx.timer
   2021/06/08 01:50:33 [error] 283990#283990: *83 [lua] config_etcd.lua:550: failed to fetch data from etcd: 20: unable to get local issuer certificate,  etcd key: /apisix/plugin_configs, context: ngx.timer
   2021/06/08 01:50:38 [error] 283989#283989: *43 [lua] config_etcd.lua:572: failed to fetch data from etcd: /usr/local/openresty/lualib/resty/core/socket/tcp.lua:213: assertion failed!
   stack traceback:
   	[C]: in function 'assert'
   	/usr/local/openresty/lualib/resty/core/socket/tcp.lua:213: in function 'tls_handshake'
   	.../local/apisix//deps/share/lua/5.1/resty/http_connect.lua:239: in function 'connect'
   	/usr/local/apisix//deps/share/lua/5.1/resty/etcd/v3.lua:595: in function 'request_chunk'
   	/usr/local/apisix//deps/share/lua/5.1/resty/etcd/v3.lua:763: in function 'watchdir'
   	/usr/local/apisix/apisix/core/config_etcd.lua:122: in function 'waitdir'
   	/usr/local/apisix/apisix/core/config_etcd.lua:318: in function 'sync_data'
   	/usr/local/apisix/apisix/core/config_etcd.lua:546: in function </usr/local/apisix/apisix/core/config_etcd.lua:536>
   	[C]: in function 'xpcall'
   	/usr/local/apisix/apisix/core/config_etcd.lua:536: in function </usr/local/apisix/apisix/core/config_etcd.lua:527>,  etcd key: /apisix/plugin_configs, context: ngx.timer
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] tokers commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
tokers commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-853777484


   @xyz2b We need OpenResty for APISIX to enable  ETCD with mTLS .


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] xyz2b commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
xyz2b commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-854352089


   Reinstalled apisix, openresty and apisix, the problem remains.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] tokers commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
tokers commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-853777484


   @xyz2b We need OpenResty for APISIX to enable  ETCD with mTLS .


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] tzssangglass edited a comment on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
tzssangglass edited a comment on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-856481940


   @spacewander @tokers , I use the certs in https://github.com/apache/apisix/tree/master/t/certs, I get this error when I verify these certificates
   
   ```shell
   $ /usr/local/openresty/openssl111/bin/openssl verify t/certs/mtls_client.crt
   C = cn, ST = GuangDong, O = api7, L = ZhuHai, CN = client.apisix.dev
   error 20 at 0 depth lookup: unable to get local issuer certificate
   error t/certs/mtls_client.crt: verification failed
   
   $ /usr/local/openresty/openssl111/bin/openssl verify t/certs/mtls_server.crt 
   C = cn, ST = GuangDong, O = api7, L = ZhuHai, CN = admin.apisix.dev
   error 20 at 0 depth lookup: unable to get local issuer certificate
   error t/certs/mtls_server.crt: verification failed
   
   $  /usr/local/openresty/openssl111/bin/openssl verify t/certs/mtls_server.crt -CAfile /usr/local/apisix/t/certs/mtls_ca.crt
   C = cn, ST = GuangDong, O = api7, L = ZhuHai, CN = admin.apisix.dev
   error 20 at 0 depth lookup: unable to get local issuer certificate
   error t/certs/mtls_server.crt: verification failed
   Can't open -CAfile for reading, No such file or directory
   281473499807792:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('-CAfile','r')
   281473499807792:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:
   unable to load certificate
   C = cn, ST = GuangDong, L = ZhuHai, O = api7, OU = ops, CN = ca.apisix.dev
   error 18 at 0 depth lookup: self signed certificate
   error /usr/local/apisix/t/certs/mtls_ca.crt: verification failed
   
   $  /usr/local/openresty/openssl111/bin/openssl verify t/certs/mtls_client.crt -CAfile /usr/local/apisix/t/certs/mtls_ca.crt 
   C = cn, ST = GuangDong, O = api7, L = ZhuHai, CN = client.apisix.dev
   error 20 at 0 depth lookup: unable to get local issuer certificate
   error t/certs/mtls_client.crt: verification failed
   Can't open -CAfile for reading, No such file or directory
   281473612107824:error:02001002:system library:fopen:No such file or directory:crypto/bio/bss_file.c:69:fopen('-CAfile','r')
   281473612107824:error:2006D080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:76:
   unable to load certificate
   C = cn, ST = GuangDong, L = ZhuHai, O = api7, OU = ops, CN = ca.apisix.dev
   error 18 at 0 depth lookup: self signed certificate
   error /usr/local/apisix/t/certs/mtls_ca.crt: verification failed
   
   ```
   a little strange, `/usr/local/apisix/t/certs/mtls_ca.crt` clearly has a certificate file
   
   I get the same error `unable to get local issuer certificate` as start apisix,  I think this problem is related to the certificate


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] tzssangglass commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-856143398


   here is debug at /usr/local/openresty/lualib/resty/core/socket/tcp.lua:212, hope to provide a little help
   ![image](https://user-images.githubusercontent.com/30819887/121066423-73989e80-c7fc-11eb-8ad0-972c3c082f76.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] tokers commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
tokers commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-856359701


   @tzssangglass Is the etcd server cert self-signed? Or you may try to put the whole cert chain into `mtls_server.crt` and see what happens.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] spacewander commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-854296181


   It is strange. The TLS handshake in the init phase doesn't use the `lua_ssl_trusted_certificate` directly.
   
   It just uses the configured CA:
   https://github.com/apache/apisix/blob/c61261af8d19557b77535c9c745f4f8182bb63b2/apisix/patch.lua#L207-L210
   
   I try with self-signed certificate locally and can't reproduce it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] xyz2b commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
xyz2b commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-854277066






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] spacewander commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-856371769


   @tzssangglass 
   Could you provide a minimal reproduce case for the error?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] tzssangglass commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-856560604


   
   
   
   ```shell
   $ /usr/local/openresty/openssl111/bin/openssl verify -CAfile /usr/local/apisix/t/certs/mtls_ca.crt /usr/local/apisix/t/certs/mtls_client.crt 
   /usr/local/apisix/t/certs/mtls_client.crt: OK
   
   $ /usr/local/openresty/openssl111/bin/openssl verify -CAfile /usr/local/apisix/t/certs/mtls_ca.crt /usr/local/apisix/t/certs/mtls_server.crt 
   /usr/local/apisix/t/certs/mtls_server.crt: OK
   ```
   
   it' is ok.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] tokers commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
tokers commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-854619383


   @xyz2b You may also show us the APISIX config.yaml.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] tzssangglass commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-859379753


   @xyz2b hi, you are right!  the `lua_ssl_trusted_certificate` should be in the http configuration block, feel free to submit a PR to fix this!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [apisix] tzssangglass commented on issue #4370: request help: An error is reported when the configuration is updated from etcd regularly

Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #4370:
URL: https://github.com/apache/apisix/issues/4370#issuecomment-856481940


   @spacewander @tokers , I use the certs in https://github.com/apache/apisix/tree/master/t/certs, I get this error when I verify these certificates
   
   ```shell
   $ /usr/local/openresty/openssl111/bin/openssl verify t/certs/mtls_client.crt
   C = cn, ST = GuangDong, O = api7, L = ZhuHai, CN = client.apisix.dev
   error 20 at 0 depth lookup: unable to get local issuer certificate
   error t/certs/mtls_client.crt: verification failed
   
   $ /usr/local/openresty/openssl111/bin/openssl verify t/certs/mtls_server.crt 
   C = cn, ST = GuangDong, O = api7, L = ZhuHai, CN = admin.apisix.dev
   error 20 at 0 depth lookup: unable to get local issuer certificate
   error t/certs/mtls_server.crt: verification failed
   ```
   
   I get the same error `unable to get local issuer certificate` as start apisix,  I think this problem is related to the certificate


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org