httpd should be able to reopen log files without a restart.

[crocket <cr...@gmail.com>]

When some softwares(including stunnel) receive SIGUSR1, they reopen a
log file without re-reading configuration files or restarting.
Since httpd uses SIGUSR1 for graceful restart, it could use SIGUSR2
purely for reopening log files.

Why do people need this signal?

A graceful restart doesn't kill active children processes, and those
processes still write on an old log file.
logrotate can be configured to compress old log files, in which case
httpd would try to write onto a compressed log file.
Since httpd can't log to a file, it would show abnormal behaviors or
even abort downloads from httpd.

So SIGUSR2 can come in handy.

Re: httpd should be able to reopen log files without a restart.

[Eric Covener <co...@gmail.com>]

On Mon, Oct 10, 2011 at 9:16 PM, crocket <cr...@gmail.com> wrote:
> When some softwares(including stunnel) receive SIGUSR1, they reopen a
> log file without re-reading configuration files or restarting.
> Since httpd uses SIGUSR1 for graceful restart, it could use SIGUSR2
> purely for reopening log files.
>
> Why do people need this signal?
>
> A graceful restart doesn't kill active children processes, and those
> processes still write on an old log file.
> logrotate can be configured to compress old log files, in which case
> httpd would try to write onto a compressed log file.
> Since httpd can't log to a file, it would show abnormal behaviors or
> even abort downloads from httpd.

Does logrotate use the same inode for the compressed file? Either way,
why would httpd fail to write to the file and behave abnormally?

Re: httpd should be able to reopen log files without a restart.

["Brian J. France" <br...@brianfrance.com>]

For some reason the docs for trunk haven't updated.

Here are the details for what is in apr (which is a little different than my patch for work):

apr_file_open() has two new flags:

  APR_FOPEN_ROTATING - Do file file rotation checking
  APR_FOPEN_MANUAL_ROTATE  - Enable Manual rotation

Two new functions where added (no docs yet):

  APR_DECLARE(apr_status_t) apr_file_rotating_check(apr_file_t *thefile);
  APR_DECLARE(apr_status_t) apr_file_rotating_manual_check(apr_file_t *thefile, apr_time_t time);

Code was added in apr_file_write() and apr_file_writev() to check to see if the file needs rotated (the log file in this case) before writing data to it.  If rotating is not enabled or set to manual it doesn't do anything, but if it is rotating then it checks to see if the now - lastcheck > than the timeout (60 seconds default) and if it has then it stats the file and compares inode and device to make sure they are still the same.  If it finds they are different, then it re-opens the file, saves the new inode and device.

If manual check is enabled, then you have to call apr_file_rotating_check() or apr_file_rotating_manual_check() if you want a  different timeout.

So if we would added a option for adding APR_FOPEN_ROTATING to the ErrorLog, CustomLog, etc directives then we could get httpd to allow rotating logs (or may be adding a global setting to enable it by default).

Brian





On Oct 11, 2011, at 10:10 AM, crocket wrote:

> How exactly does it work? Will it be documented?
> 
> On Tue, Oct 11, 2011 at 10:28 AM, Brian J. France <br...@brianfrance.com> wrote:
>> apr trunk has APR_FOPEN_ROTATING support (stat checking and re-open), which is based on a patch we use at work.  This allows us to move the log file, wait 90 seconds and then compress the moved log file, all without touch the server as it will stat the log ever 60 and re-open it if needed before writing a log line.
>> 
>> Of course it is in trunk, so may take a bit to get it into a release.
>> 
>> Brian
>> 
>> 
>> On Oct 10, 2011, at 9:16 PM, crocket wrote:
>> 
>>> When some softwares(including stunnel) receive SIGUSR1, they reopen a
>>> log file without re-reading configuration files or restarting.
>>> Since httpd uses SIGUSR1 for graceful restart, it could use SIGUSR2
>>> purely for reopening log files.
>>> 
>>> Why do people need this signal?
>>> 
>>> A graceful restart doesn't kill active children processes, and those
>>> processes still write on an old log file.
>>> logrotate can be configured to compress old log files, in which case
>>> httpd would try to write onto a compressed log file.
>>> Since httpd can't log to a file, it would show abnormal behaviors or
>>> even abort downloads from httpd.
>>> 
>>> So SIGUSR2 can come in handy.
>> 
>> 
> 

Re: httpd should be able to reopen log files without a restart.

[crocket <cr...@gmail.com>]

How exactly does it work? Will it be documented?

On Tue, Oct 11, 2011 at 10:28 AM, Brian J. France <br...@brianfrance.com> wrote:
> apr trunk has APR_FOPEN_ROTATING support (stat checking and re-open), which is based on a patch we use at work.  This allows us to move the log file, wait 90 seconds and then compress the moved log file, all without touch the server as it will stat the log ever 60 and re-open it if needed before writing a log line.
>
> Of course it is in trunk, so may take a bit to get it into a release.
>
> Brian
>
>
> On Oct 10, 2011, at 9:16 PM, crocket wrote:
>
>> When some softwares(including stunnel) receive SIGUSR1, they reopen a
>> log file without re-reading configuration files or restarting.
>> Since httpd uses SIGUSR1 for graceful restart, it could use SIGUSR2
>> purely for reopening log files.
>>
>> Why do people need this signal?
>>
>> A graceful restart doesn't kill active children processes, and those
>> processes still write on an old log file.
>> logrotate can be configured to compress old log files, in which case
>> httpd would try to write onto a compressed log file.
>> Since httpd can't log to a file, it would show abnormal behaviors or
>> even abort downloads from httpd.
>>
>> So SIGUSR2 can come in handy.
>
>

Re: httpd should be able to reopen log files without a restart.

["Brian J. France" <br...@brianfrance.com>]

apr trunk has APR_FOPEN_ROTATING support (stat checking and re-open), which is based on a patch we use at work.  This allows us to move the log file, wait 90 seconds and then compress the moved log file, all without touch the server as it will stat the log ever 60 and re-open it if needed before writing a log line.

Of course it is in trunk, so may take a bit to get it into a release.

Brian


On Oct 10, 2011, at 9:16 PM, crocket wrote:

> When some softwares(including stunnel) receive SIGUSR1, they reopen a
> log file without re-reading configuration files or restarting.
> Since httpd uses SIGUSR1 for graceful restart, it could use SIGUSR2
> purely for reopening log files.
> 
> Why do people need this signal?
> 
> A graceful restart doesn't kill active children processes, and those
> processes still write on an old log file.
> logrotate can be configured to compress old log files, in which case
> httpd would try to write onto a compressed log file.
> Since httpd can't log to a file, it would show abnormal behaviors or
> even abort downloads from httpd.
> 
> So SIGUSR2 can come in handy.

Re: httpd should be able to reopen log files without a restart.

[Graham Leggett <mi...@sharp.fm>]

On 11 Oct 2011, at 3:16 AM, crocket wrote:

> When some softwares(including stunnel) receive SIGUSR1, they reopen a
> log file without re-reading configuration files or restarting.
> Since httpd uses SIGUSR1 for graceful restart, it could use SIGUSR2
> purely for reopening log files.
>
> Why do people need this signal?
>
> A graceful restart doesn't kill active children processes, and those
> processes still write on an old log file.
> logrotate can be configured to compress old log files, in which case
> httpd would try to write onto a compressed log file.
> Since httpd can't log to a file, it would show abnormal behaviors or
> even abort downloads from httpd.

A signal is unlikely to be enough.

Logfiles are opened using the permissions of the parent process, by  
the time a child runs the child no longer has access to the file, and  
does not have permission to close and reopen it.

If the parent received the signal, it would have no option but to  
replace the children (which is what gets done now), but because we  
want to do this gracefully, we allow the current set of children to  
run to completion. The alternative is to abort the request, and  
generally that's bad.

In our environments we work around this problem by using the fuser  
command to tell us whether anybody (ie httpd) has a logfile open  
before we attempt to compress it. If no, we go ahead. If yes, we wait  
till the top of the next hour, and try and compress it then.

Regards,
Graham
--