You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2022/09/06 06:57:06 UTC
[GitHub] [apisix] monkeyDluffy6017 opened a new pull request, #7869: Reload once when log rotate
monkeyDluffy6017 opened a new pull request, #7869:
URL: https://github.com/apache/apisix/pull/7869
### Description
1. Waiting for the end of the old requests to avoid losing logs during compression
2. Refactor code to ensure that the signal is sent once at a time
### Checklist
- [x] I have explained the need for this PR and the problem it solves
- [x] I have explained the changes or the new features added to this PR
- [x] I have added tests corresponding to this change
- [ ] I have updated the documentation to reflect this change
- [x] I have verified that this change is backward compatible (If not, please discuss on the [APISIX mailing list](https://github.com/apache/apisix/tree/master#community) first)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a diff in pull request #7869: fix: reload once when log rotate
Posted by GitBox <gi...@apache.org>.
spacewander commented on code in PR #7869:
URL: https://github.com/apache/apisix/pull/7869#discussion_r965526130
##########
apisix/plugins/log-rotate.lua:
##########
@@ -218,25 +221,42 @@ local function file_size(file)
end
-local function rotate_file(files, now_time, max_kept)
+local function rotate_file(files, now_time, max_kept, wait_time)
+ if isempty(files) then
+ return
+ end
+
+ local new_files = new_tab(2, 0)
+ -- rename the log files
for _, file in ipairs(files) do
local now_date = os_date("%Y-%m-%d_%H-%M-%S", now_time)
local new_file = rename_file(default_logs[file], now_date)
if not new_file then
return
end
- local pid = process.get_master_pid()
- core.log.warn("send USR1 signal to master process [", pid, "] for reopening log file")
- local ok, err = signal.kill(pid, signal.signum("USR1"))
- if not ok then
- core.log.error("failed to send USR1 signal for reopening log file: ", err)
- end
+ tab_insert(new_files, new_file)
+ end
+
+ -- send signal to reopen log files
+ local pid = process.get_master_pid()
+ core.log.warn("send USR1 signal to master process [", pid, "] for reopening log file")
+ local ok, err = signal.kill(pid, signal.signum("USR1"))
+ if not ok then
+ core.log.error("failed to send USR1 signal for reopening log file: ", err)
+ end
+
+ if enable_compression then
+ -- Waiting for nginx reopen files
+ -- to avoid losing logs during compression
+ ngx_sleep(wait_time)
Review Comment:
If the wait time is only for hardcode, maybe we can add extra sleep in the test so we don't need to set a special value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a diff in pull request #7869: fix: reload once when log rotate
Posted by GitBox <gi...@apache.org>.
spacewander commented on code in PR #7869:
URL: https://github.com/apache/apisix/pull/7869#discussion_r964417079
##########
apisix/plugins/log-rotate.lua:
##########
@@ -218,25 +221,42 @@ local function file_size(file)
end
-local function rotate_file(files, now_time, max_kept)
+local function rotate_file(files, now_time, max_kept, wait_time)
+ if isempty(files) then
+ return
+ end
+
+ local new_files = new_tab(2, 0)
+ -- rename the log files
for _, file in ipairs(files) do
local now_date = os_date("%Y-%m-%d_%H-%M-%S", now_time)
local new_file = rename_file(default_logs[file], now_date)
if not new_file then
return
end
- local pid = process.get_master_pid()
- core.log.warn("send USR1 signal to master process [", pid, "] for reopening log file")
- local ok, err = signal.kill(pid, signal.signum("USR1"))
- if not ok then
- core.log.error("failed to send USR1 signal for reopening log file: ", err)
- end
+ tab_insert(new_files, new_file)
+ end
+
+ -- send signal to reopen log files
+ local pid = process.get_master_pid()
+ core.log.warn("send USR1 signal to master process [", pid, "] for reopening log file")
+ local ok, err = signal.kill(pid, signal.signum("USR1"))
+ if not ok then
+ core.log.error("failed to send USR1 signal for reopening log file: ", err)
+ end
+
+ if enable_compression then
+ -- Waiting for nginx reopen files
+ -- to avoid losing logs during compression
+ ngx_sleep(wait_time)
Review Comment:
Will the sleep block the timer and cause the next rotation time incorrect?
##########
apisix/plugins/log-rotate.lua:
##########
@@ -37,7 +37,9 @@ local str_format = string.format
local str_reverse = string.reverse
local tab_insert = table.insert
local tab_sort = table.sort
-
+local new_tab = require "table.new"
+local ngx_sleep = require("apisix.core.utils").sleep
+local isempty = require "table.isempty"
Review Comment:
Better to introduce it via `core.table `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #7869: fix: reload once when log rotate
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #7869:
URL: https://github.com/apache/apisix/pull/7869#discussion_r965526686
##########
apisix/plugins/log-rotate.lua:
##########
@@ -218,25 +221,42 @@ local function file_size(file)
end
-local function rotate_file(files, now_time, max_kept)
+local function rotate_file(files, now_time, max_kept, wait_time)
+ if isempty(files) then
+ return
+ end
+
+ local new_files = new_tab(2, 0)
+ -- rename the log files
for _, file in ipairs(files) do
local now_date = os_date("%Y-%m-%d_%H-%M-%S", now_time)
local new_file = rename_file(default_logs[file], now_date)
if not new_file then
return
end
- local pid = process.get_master_pid()
- core.log.warn("send USR1 signal to master process [", pid, "] for reopening log file")
- local ok, err = signal.kill(pid, signal.signum("USR1"))
- if not ok then
- core.log.error("failed to send USR1 signal for reopening log file: ", err)
- end
+ tab_insert(new_files, new_file)
+ end
+
+ -- send signal to reopen log files
+ local pid = process.get_master_pid()
+ core.log.warn("send USR1 signal to master process [", pid, "] for reopening log file")
+ local ok, err = signal.kill(pid, signal.signum("USR1"))
+ if not ok then
+ core.log.error("failed to send USR1 signal for reopening log file: ", err)
+ end
+
+ if enable_compression then
+ -- Waiting for nginx reopen files
+ -- to avoid losing logs during compression
+ ngx_sleep(wait_time)
Review Comment:
ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #7869: fix: reload once when log rotate
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #7869:
URL: https://github.com/apache/apisix/pull/7869#discussion_r964441104
##########
apisix/plugins/log-rotate.lua:
##########
@@ -37,7 +37,9 @@ local str_format = string.format
local str_reverse = string.reverse
local tab_insert = table.insert
local tab_sort = table.sort
-
+local new_tab = require "table.new"
+local ngx_sleep = require("apisix.core.utils").sleep
+local isempty = require "table.isempty"
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #7869: fix: reload once when log rotate
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #7869:
URL: https://github.com/apache/apisix/pull/7869#discussion_r965551753
##########
apisix/plugins/log-rotate.lua:
##########
@@ -218,25 +221,42 @@ local function file_size(file)
end
-local function rotate_file(files, now_time, max_kept)
+local function rotate_file(files, now_time, max_kept, wait_time)
+ if isempty(files) then
+ return
+ end
+
+ local new_files = new_tab(2, 0)
+ -- rename the log files
for _, file in ipairs(files) do
local now_date = os_date("%Y-%m-%d_%H-%M-%S", now_time)
local new_file = rename_file(default_logs[file], now_date)
if not new_file then
return
end
- local pid = process.get_master_pid()
- core.log.warn("send USR1 signal to master process [", pid, "] for reopening log file")
- local ok, err = signal.kill(pid, signal.signum("USR1"))
- if not ok then
- core.log.error("failed to send USR1 signal for reopening log file: ", err)
- end
+ tab_insert(new_files, new_file)
+ end
+
+ -- send signal to reopen log files
+ local pid = process.get_master_pid()
+ core.log.warn("send USR1 signal to master process [", pid, "] for reopening log file")
+ local ok, err = signal.kill(pid, signal.signum("USR1"))
+ if not ok then
+ core.log.error("failed to send USR1 signal for reopening log file: ", err)
+ end
+
+ if enable_compression then
+ -- Waiting for nginx reopen files
+ -- to avoid losing logs during compression
+ ngx_sleep(wait_time)
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tokers commented on pull request #7869: fix: reload once when log rotate
Posted by GitBox <gi...@apache.org>.
tokers commented on PR #7869:
URL: https://github.com/apache/apisix/pull/7869#issuecomment-1238036792
Is it feasible to add some test cases?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #7869: fix: reload once when log rotate
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #7869:
URL: https://github.com/apache/apisix/pull/7869#discussion_r964429386
##########
apisix/plugins/log-rotate.lua:
##########
@@ -218,25 +221,42 @@ local function file_size(file)
end
-local function rotate_file(files, now_time, max_kept)
+local function rotate_file(files, now_time, max_kept, wait_time)
+ if isempty(files) then
+ return
+ end
+
+ local new_files = new_tab(2, 0)
+ -- rename the log files
for _, file in ipairs(files) do
local now_date = os_date("%Y-%m-%d_%H-%M-%S", now_time)
local new_file = rename_file(default_logs[file], now_date)
if not new_file then
return
end
- local pid = process.get_master_pid()
- core.log.warn("send USR1 signal to master process [", pid, "] for reopening log file")
- local ok, err = signal.kill(pid, signal.signum("USR1"))
- if not ok then
- core.log.error("failed to send USR1 signal for reopening log file: ", err)
- end
+ tab_insert(new_files, new_file)
+ end
+
+ -- send signal to reopen log files
+ local pid = process.get_master_pid()
+ core.log.warn("send USR1 signal to master process [", pid, "] for reopening log file")
+ local ok, err = signal.kill(pid, signal.signum("USR1"))
+ if not ok then
+ core.log.error("failed to send USR1 signal for reopening log file: ", err)
+ end
+
+ if enable_compression then
+ -- Waiting for nginx reopen files
+ -- to avoid losing logs during compression
+ ngx_sleep(wait_time)
Review Comment:
`wait_time` is just 0.5s by default. rotation time is much lager than 0.5s, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a diff in pull request #7869: fix: reload once when log rotate
Posted by GitBox <gi...@apache.org>.
spacewander commented on code in PR #7869:
URL: https://github.com/apache/apisix/pull/7869#discussion_r965519961
##########
apisix/plugins/log-rotate.lua:
##########
@@ -218,25 +221,42 @@ local function file_size(file)
end
-local function rotate_file(files, now_time, max_kept)
+local function rotate_file(files, now_time, max_kept, wait_time)
+ if isempty(files) then
+ return
+ end
+
+ local new_files = new_tab(2, 0)
+ -- rename the log files
for _, file in ipairs(files) do
local now_date = os_date("%Y-%m-%d_%H-%M-%S", now_time)
local new_file = rename_file(default_logs[file], now_date)
if not new_file then
return
end
- local pid = process.get_master_pid()
- core.log.warn("send USR1 signal to master process [", pid, "] for reopening log file")
- local ok, err = signal.kill(pid, signal.signum("USR1"))
- if not ok then
- core.log.error("failed to send USR1 signal for reopening log file: ", err)
- end
+ tab_insert(new_files, new_file)
+ end
+
+ -- send signal to reopen log files
+ local pid = process.get_master_pid()
+ core.log.warn("send USR1 signal to master process [", pid, "] for reopening log file")
+ local ok, err = signal.kill(pid, signal.signum("USR1"))
+ if not ok then
+ core.log.error("failed to send USR1 signal for reopening log file: ", err)
+ end
+
+ if enable_compression then
+ -- Waiting for nginx reopen files
+ -- to avoid losing logs during compression
+ ngx_sleep(wait_time)
Review Comment:
@monkeyDluffy6017
Currently, there is no limitation for the user-specific wait_time. BTW, we need to doc it if this argument is exposed to the user.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on pull request #7869: fix: reload once when log rotate
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on PR #7869:
URL: https://github.com/apache/apisix/pull/7869#issuecomment-1238241179
> Is it feasible to add some test cases?
We have log-rotate.t, log-rotate2.t, log-rotate3.t 3 test files to test its functionality.
But I have no idea how to add test case about losing log file or sending several reopen signal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander merged pull request #7869: fix: reload once when log rotate
Posted by GitBox <gi...@apache.org>.
spacewander merged PR #7869:
URL: https://github.com/apache/apisix/pull/7869
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org