You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by "hansedong (via GitHub)" <gi...@apache.org> on 2023/07/07 11:11:13 UTC
[GitHub] [apisix] hansedong opened a new issue, #9802: bug: APISIX executes reload, but the old process does not exit for a long time
hansedong opened a new issue, #9802:
URL: https://github.com/apache/apisix/issues/9802
### Current Behavior
As APISIX executes reload, but the old process does not exit for a long time.
```
root 885278 1 0 Jul05 ? 00:00:00 nginx: master process /usr/bin/openresty -p /usr/local/apisix -c /usr/local/apisix/conf/nginx.conf
nobody 885280 885278 0 Jul05 ? 00:09:03 nginx: worker process is shutting down
nobody 885281 885278 0 Jul05 ? 00:08:39 nginx: worker process is shutting down
nobody 885282 885278 0 Jul05 ? 00:08:24 nginx: worker process is shutting down
nobody 1288295 885278 0 18:09 ? 00:00:03 nginx: worker process is shutting down
nobody 1288517 885278 5 18:10 ? 00:03:10 nginx: worker process
nobody 1288518 885278 4 18:10 ? 00:02:16 nginx: worker process
nobody 1288519 885278 4 18:10 ? 00:02:15 nginx: worker process
nobody 1288520 885278 4 18:10 ? 00:02:12 nginx: worker process
nobody 1288521 885278 0 18:10 ? 00:00:01 nginx: cache manager process
root 1288522 885278 3 18:10 ? 00:01:41 nginx: privileged agent process
root 1296427 1280594 0 19:04 pts/0 00:00:00 grep --color=auto nginx
```
From above, it can be seen that the creation time of the new process is 18:10, but it is already 19:05 now
```
[root@knode10-72-73-177 logs]# date
Fri Jul 7 19:05:05 CST 2023
```
APISIX 配置文件中 worker_shutdown_timeout 的配置项的值为 240s
```
nginx_config: # config for render the template to generate nginx.conf
#user: root # specifies the execution user of the worker process.
# the "user" directive makes sense only if the master process runs with super-user privileges.
# if you're not root user,the default is current user.
error_log: /DATA1/apisix/apisix/logs/error.log
error_log_level: warn # warn,error
worker_processes: 4 # if you want use multiple cores in container, you can inject the number of cpu as environment variable "APISIX_WORKER_PROCESSES"
enable_cpu_affinity: false # disable CPU affinity by default, if APISIX is deployed on a physical machine, it can be enabled and work well.
worker_rlimit_nofile: 6553500 # the number of files a worker process can open, should be larger than worker_connections
worker_shutdown_timeout: 240s # timeout for a graceful shutdown of worker processes
```
### Expected Behavior
_No response_
### Error Logs
_No response_
### Steps to Reproduce
```
systemctl reload apisix
```
### Environment
- APISIX version (run `apisix version`): `3.4.0`
- Operating system (run `uname -a`): `Linux knode10-72-73-177 5.15.29-200.el7.x86_64 #1 SMP Thu Mar 31 14:09:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux`
- OpenResty / Nginx version (run `openresty -V` or `nginx -V`):
```
nginx version: openresty/1.21.4.1
built by gcc 9.3.1 20200408 (Red Hat 9.3.1-2) (GCC)
built with OpenSSL 1.1.1s 1 Nov 2022
TLS SNI support enabled
configure arguments: --prefix=/usr/local/openresty/nginx --with-cc-opt='-O2 -DAPISIX_BASE_VER=1.21.4.1.8 -DNGX_GRPC_CLI_ENGINE_PATH=/usr/local/openresty/libgrpc_engine.so -DNGX_HTTP_GRPC_CLI_ENGINE_PATH=/usr/local/openresty/libgrpc_engine.so -DNGX_LUA_ABORT_AT_PANIC -I/usr/local/openresty/zlib/include -I/usr/local/openresty/pcre/include -I/usr/local/openresty/openssl111/include' --add-module=../ngx_devel_kit-0.3.1 --add-module=../echo-nginx-module-0.62 --add-module=../xss-nginx-module-0.06 --add-module=../ngx_coolkit-0.2 --add-module=../set-misc-nginx-module-0.33 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.09 --add-module=../srcache-nginx-module-0.32 --add-module=../ngx_lua-0.10.21 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.33 --add-module=../array-var-nginx-module-0.05 --add-module=../memc-nginx-module-0.19 --add-module=../redis2-nginx-module-0.15 --add-module=../redis-nginx-module-0.3.9 --ad
d-module=../ngx_stream_lua-0.0.11 --with-ld-opt='-Wl,-rpath,/usr/local/openresty/luajit/lib -Wl,-rpath,/usr/local/openresty/wasmtime-c-api/lib -L/usr/local/openresty/zlib/lib -L/usr/local/openresty/pcre/lib -L/usr/local/openresty/openssl111/lib -Wl,-rpath,/usr/local/openresty/zlib/lib:/usr/local/openresty/pcre/lib:/usr/local/openresty/openssl111/lib' --add-module=/tmp/tmp.YzVafXtnkf/openresty-1.21.4.1/../mod_dubbo-1.0.2 --add-module=/tmp/tmp.YzVafXtnkf/openresty-1.21.4.1/../ngx_multi_upstream_module-1.1.1 --add-module=/tmp/tmp.YzVafXtnkf/openresty-1.21.4.1/../apisix-nginx-module-1.12.0 --add-module=/tmp/tmp.YzVafXtnkf/openresty-1.21.4.1/../apisix-nginx-module-1.12.0/src/stream --add-module=/tmp/tmp.YzVafXtnkf/openresty-1.21.4.1/../apisix-nginx-module-1.12.0/src/meta --add-module=/tmp/tmp.YzVafXtnkf/openresty-1.21.4.1/../wasm-nginx-module-0.6.4 --add-module=/tmp/tmp.YzVafXtnkf/openresty-1.21.4.1/../lua-var-nginx-module-v0.5.3 --add-module=/tmp/tmp.YzVafXtnkf/openresty-1.21.4.1/../grp
c-client-nginx-module-v0.4.2 --with-poll_module --with-pcre-jit --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_v2_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module --with-http_realip_module --with-http_addition_module --with-http_auth_request_module --with-http_secure_link_module --with-http_random_index_module --with-http_gzip_static_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-threads --with-compat --with-stream --with-http_ssl_module
```
- etcd version, if relevant (run `curl http://127.0.0.1:9090/v1/server_info`): `3.5.9`
- APISIX Dashboard version, if relevant: `3.0.1`
- Plugin runner version, for issues related to plugin runners:
- LuaRocks version, for installation issues (run `luarocks --version`):
```
/usr/local/bin/luarocks 3.8.0
LuaRocks main command-line interface
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] jiangfucheng commented on issue #9802: bug: APISIX executes reload, but the old process does not exit for a long time
Posted by "jiangfucheng (via GitHub)" <gi...@apache.org>.
jiangfucheng commented on issue #9802:
URL: https://github.com/apache/apisix/issues/9802#issuecomment-1627737586
I reproduced this bug. It seems cause by that the worker process not close connection with `config_list.sock` in woker exit phase. Let me investigate how to fix it, it seems a bit complicated to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] moonming closed issue #9802: bug: APISIX executes reload, but the old process does not exit for a long time
Posted by "moonming (via GitHub)" <gi...@apache.org>.
moonming closed issue #9802: bug: APISIX executes reload, but the old process does not exit for a long time
URL: https://github.com/apache/apisix/issues/9802
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org