You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by KASPAR Sandro <sa...@hsr.ch> on 2017/05/10 13:59:28 UTC

[users@httpd] cgi script error output logging

Hi,

According to this documentation http://httpd.apache.org/docs/2.4/logs.html everything a cgi script sends to stderr is written to the apache  error log file. 

Unfortunately I can not control those scripts running on my server and often garbage is sent to stderr and then written to my error log. Because there isn't even a timestamp or any other useful information on those lines, I can't find out, which vhost created  the error. In addition those log lines often lack a "new line" at the end, and it the next log message gets appended to the current line instead of being written to a new line. Because of this, the next line which would have been in the correct format is also lost, because it can't be parsed automatically anymore.

Is there any way to change this behaviour? What I would like to achieve is to have the cgi errors in a seperate logfile or even better change the log format of these lines. For example prefix the log line with the vhost and a timestamp.
Because I don't have control over the cgi scripts, I would need to configure this in apaches main or vhost configuration. Unfortunately I could not find anything about cgi script output logging in the apache documentation (except that everything sent to  stderr is written to httpd-error.log) or mailing list archives. 
 
Any ideas appreciated. Thanks in advance!

Sandro Kaspar
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Re: [users@httpd] cgi script error output logging

Posted by KASPAR Sandro <sa...@hsr.ch>.
Hi Luca,


Yes I checked the error log format. Unfortunately, this only applies to log lines generated by apache and not to the lines created by cgi scripts. The lines created by a cgi script are just redirected to the httpd-error.log as they are. For example if you print "test" to stderr in a cgi script, this gets written just like that to the error log. No vhost information, no timestamp, no user, just what you printed.

My error log format is:


ErrorLogFormat "%v [%{u}t] [%-m:%l] [pid %P:tid %T] %7F: %E: [client\ %a] %M% ,\ referer\ %{Referer}i"


But as I said, the ErrorLogFormat is not applied to log lines generated by cgi scripts.

The apache version I am using is 2.4.25.


Sandro

________________________________
From: Luca Toscano <to...@gmail.com>
Sent: Thursday, May 11, 2017 2:53:35 PM
To: users@httpd.apache.org
Subject: Re: [users@httpd] cgi script error output logging

Hi Sandro,

have you checked https://httpd.apache.org/docs/2.4/mod/core.html#errorlogformat ? What is the current format that you are using? Also, what version of httpd?

Luca

2017-05-11 10:07 GMT+02:00 KASPAR Sandro <sa...@hsr.ch>>:

Hi suomi,


Thank you for your answer. Unfortunately I am not using php-fpm but fcgid. As far as I know there is no such possibility in fcgid.

Any other ideas?


Sandro


________________________________
From: fedora <fe...@ayni.com>>
Sent: Thursday, May 11, 2017 6:54 AM
To: users@httpd.apache.org<ma...@httpd.apache.org>
Subject: Re: [users@httpd] cgi script error output logging

Hi Sandro

are you using php-fpm as a cgi frontend? If yes: the stdout and stderr
are both redirectet to the php-fpm log (/var/log/php-fpm/*) if you have
in /etc/php-fpm/www.conf:

catch_workers_output = yes

I don't think this will solve all your problems, but it is a good
starting point.

suomi

On 05/10/2017 03:59 PM, KASPAR Sandro wrote:
>
> Hi,
>
> According to this documentation http://httpd.apache.org/docs/2.4/logs.html everything a cgi script sends to stderr is written to the apache  error log file.
>
> Unfortunately I can not control those scripts running on my server and often garbage is sent to stderr and then written to my error log. Because there isn't even a timestamp or any other useful information on those lines, I can't find out, which vhost created  the error. In addition those log lines often lack a "new line" at the end, and it the next log message gets appended to the current line instead of being written to a new line. Because of this, the next line which would have been in the correct format is also lost, because it can't be parsed automatically anymore.
>
> Is there any way to change this behaviour? What I would like to achieve is to have the cgi errors in a seperate logfile or even better change the log format of these lines. For example prefix the log line with the vhost and a timestamp.
> Because I don't have control over the cgi scripts, I would need to configure this in apaches main or vhost configuration. Unfortunately I could not find anything about cgi script output logging in the apache documentation (except that everything sent to  stderr is written to httpd-error.log) or mailing list archives.
>
> Any ideas appreciated. Thanks in advance!
>
> Sandro Kaspar
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org<ma...@httpd.apache.org>
> For additional commands, e-mail: users-help@httpd.apache.org<ma...@httpd.apache.org>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org<ma...@httpd.apache.org>
For additional commands, e-mail: users-help@httpd.apache.org<ma...@httpd.apache.org>



Re: [users@httpd] cgi script error output logging

Posted by Luca Toscano <to...@gmail.com>.
Hi Sandro,

have you checked
https://httpd.apache.org/docs/2.4/mod/core.html#errorlogformat ? What is
the current format that you are using? Also, what version of httpd?

Luca

2017-05-11 10:07 GMT+02:00 KASPAR Sandro <sa...@hsr.ch>:

> Hi suomi,
>
>
> Thank you for your answer. Unfortunately I am not using php-fpm but fcgid.
> As far as I know there is no such possibility in fcgid.
>
> Any other ideas?
>
>
> Sandro
>
>
> ------------------------------
> *From:* fedora <fe...@ayni.com>
> *Sent:* Thursday, May 11, 2017 6:54 AM
> *To:* users@httpd.apache.org
> *Subject:* Re: [users@httpd] cgi script error output logging
>
> Hi Sandro
>
> are you using php-fpm as a cgi frontend? If yes: the stdout and stderr
> are both redirectet to the php-fpm log (/var/log/php-fpm/*) if you have
> in /etc/php-fpm/www.conf:
>
> catch_workers_output = yes
>
> I don't think this will solve all your problems, but it is a good
> starting point.
>
> suomi
>
> On 05/10/2017 03:59 PM, KASPAR Sandro wrote:
> >
> > Hi,
> >
> > According to this documentation http://httpd.apache.org/docs/
> 2.4/logs.html everything a cgi script sends to stderr is written to the
> apache  error log file.
> >
> > Unfortunately I can not control those scripts running on my server and
> often garbage is sent to stderr and then written to my error log. Because
> there isn't even a timestamp or any other useful information on those
> lines, I can't find out, which vhost created  the error. In addition those
> log lines often lack a "new line" at the end, and it the next log message
> gets appended to the current line instead of being written to a new line.
> Because of this, the next line which would have been in the correct format
> is also lost, because it can't be parsed automatically anymore.
> >
> > Is there any way to change this behaviour? What I would like to achieve
> is to have the cgi errors in a seperate logfile or even better change the
> log format of these lines. For example prefix the log line with the vhost
> and a timestamp.
> > Because I don't have control over the cgi scripts, I would need to
> configure this in apaches main or vhost configuration. Unfortunately I
> could not find anything about cgi script output logging in the apache
> documentation (except that everything sent to  stderr is written to
> httpd-error.log) or mailing list archives.
> >
> > Any ideas appreciated. Thanks in advance!
> >
> > Sandro Kaspar
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> > For additional commands, e-mail: users-help@httpd.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>

Re: [users@httpd] cgi script error output logging

Posted by KASPAR Sandro <sa...@hsr.ch>.
To make sure it is not an OS specific problem I tested the same on ubuntu 16.04 with apache2.4.18 and mod_fcgid 2.3.9. Same problem there.


If you have any other ideas I could try I would appreciate it.


Sandro

________________________________
From: KASPAR Sandro <sa...@hsr.ch>
Sent: Wednesday, May 17, 2017 3:08:18 PM
To: users@httpd.apache.org
Subject: Re: [users@httpd] cgi script error output logging



Hi Petr,


Thank you for your help. I tried your tests on [1]. I tested everything wiht apache 2.4.25 and mod_fcgid 2.3.9 and apache 2.4.25 and mod_cgi. In both cases I get the same behaviour. Everything sent to stderr, in this case "I am an ERROR", is written to the apache error log without any further information. No timestamp etc. So my log looks like this:


...

I am an ERROR
I am an ERROR
I am an ERROR
[Wed May 17 14:52:22.653661 2017] [mpm_worker:notice] [pid 1274:tid 34401707008] AH00295: caught SIGTERM, shutting down
[Wed May 17 14:52:22.839149 2017] [mpm_worker:notice] [pid 1430:tid 34401707008] AH00292: Apache/2.4.25 (FreeBSD) mod_fcgid/2.3.9 configured -- resuming normal operations
[Wed May 17 14:52:22.839503 2017] [core:notice] [pid 1430:tid 34401707008] AH00094: Command line: '/usr/local/sbin/httpd -D NOHTTPACCEPT'
I am an ERROR

I tried this with apache default configuration, except for enabling mod_cgi or mod_fcgid. Did you have to configure anything special to get more information on the log lines?

Sandro


[1] https://github.com/pgajdos/apache-rex/blob/master/mod_cgi-stderr/run.sh


________________________________
From: Petr Gajdos <pg...@suse.cz>
Sent: Friday, May 12, 2017 10:37:17 AM
To: users@httpd.apache.org
Subject: Re: [users@httpd] cgi script error output logging

On Fri, May 12, 2017 at 09:16:29AM +0200, Petr Gajdos wrote:
> Hi,
>
> On Thu, May 11, 2017 at 08:07:28AM +0000, KASPAR Sandro wrote:
> > Hi suomi,
> >
> >
> > Thank you for your answer. Unfortunately I am not using php-fpm but fcgid. As
>
> JFYI, simple example [0] had shown that for me it does work with
> mod_cgi (tested with 2.4.25).

So I tried [1] also with httpd 2.4.25 and apache2-mod_fcgid 2.3.9 and
it seem to work for me,  if I understand correctly. The spec file
%check is a bit hard to read (work in progress), but I can provide you
%with minimal httpd.conf, if you wish.

Petr

[1] (search for APACHE MODULE TEST)
    https://build.opensuse.org/package/live_build_log/Apache:Modules/apache2-mod_fcgid/openSUSE_Tumbleweed/x86_64
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Re: [users@httpd] cgi script error output logging

Posted by KASPAR Sandro <sa...@hsr.ch>.
Hi Petr,


Thank you for your help. I tried your tests on [1]. I tested everything wiht apache 2.4.25 and mod_fcgid 2.3.9 and apache 2.4.25 and mod_cgi. In both cases I get the same behaviour. Everything sent to stderr, in this case "I am an ERROR", is written to the apache error log without any further information. No timestamp etc. So my log looks like this:


...

I am an ERROR
I am an ERROR
I am an ERROR
[Wed May 17 14:52:22.653661 2017] [mpm_worker:notice] [pid 1274:tid 34401707008] AH00295: caught SIGTERM, shutting down
[Wed May 17 14:52:22.839149 2017] [mpm_worker:notice] [pid 1430:tid 34401707008] AH00292: Apache/2.4.25 (FreeBSD) mod_fcgid/2.3.9 configured -- resuming normal operations
[Wed May 17 14:52:22.839503 2017] [core:notice] [pid 1430:tid 34401707008] AH00094: Command line: '/usr/local/sbin/httpd -D NOHTTPACCEPT'
I am an ERROR

I tried this with apache default configuration, except for enabling mod_cgi or mod_fcgid. Did you have to configure anything special to get more information on the log lines?

Sandro


[1] https://github.com/pgajdos/apache-rex/blob/master/mod_cgi-stderr/run.sh


________________________________
From: Petr Gajdos <pg...@suse.cz>
Sent: Friday, May 12, 2017 10:37:17 AM
To: users@httpd.apache.org
Subject: Re: [users@httpd] cgi script error output logging

On Fri, May 12, 2017 at 09:16:29AM +0200, Petr Gajdos wrote:
> Hi,
>
> On Thu, May 11, 2017 at 08:07:28AM +0000, KASPAR Sandro wrote:
> > Hi suomi,
> >
> >
> > Thank you for your answer. Unfortunately I am not using php-fpm but fcgid. As
>
> JFYI, simple example [0] had shown that for me it does work with
> mod_cgi (tested with 2.4.25).

So I tried [1] also with httpd 2.4.25 and apache2-mod_fcgid 2.3.9 and
it seem to work for me,  if I understand correctly. The spec file
%check is a bit hard to read (work in progress), but I can provide you
%with minimal httpd.conf, if you wish.

Petr

[1] (search for APACHE MODULE TEST)
    https://build.opensuse.org/package/live_build_log/Apache:Modules/apache2-mod_fcgid/openSUSE_Tumbleweed/x86_64
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Re: [users@httpd] cgi script error output logging

Posted by Petr Gajdos <pg...@suse.cz>.
On Fri, May 12, 2017 at 09:16:29AM +0200, Petr Gajdos wrote:
> Hi,
> 
> On Thu, May 11, 2017 at 08:07:28AM +0000, KASPAR Sandro wrote:
> > Hi suomi,
> > 
> > 
> > Thank you for your answer. Unfortunately I am not using php-fpm but fcgid. As
>  
> JFYI, simple example [0] had shown that for me it does work with
> mod_cgi (tested with 2.4.25).

So I tried [1] also with httpd 2.4.25 and apache2-mod_fcgid 2.3.9 and
it seem to work for me,  if I understand correctly. The spec file
%check is a bit hard to read (work in progress), but I can provide you
%with minimal httpd.conf, if you wish.

Petr

[1] (search for APACHE MODULE TEST)
    https://build.opensuse.org/package/live_build_log/Apache:Modules/apache2-mod_fcgid/openSUSE_Tumbleweed/x86_64
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Re: [users@httpd] cgi script error output logging

Posted by Petr Gajdos <pg...@suse.cz>.
Hi,

On Thu, May 11, 2017 at 08:07:28AM +0000, KASPAR Sandro wrote:
> Hi suomi,
> 
> 
> Thank you for your answer. Unfortunately I am not using php-fpm but fcgid. As
 
JFYI, simple example [0] had shown that for me it does work with
mod_cgi (tested with 2.4.25).

Petr

[0] https://github.com/pgajdos/apache-rex/tree/master/mod_cgi-stderr

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Re: [users@httpd] cgi script error output logging

Posted by KASPAR Sandro <sa...@hsr.ch>.
Hi suomi,


Thank you for your answer. Unfortunately I am not using php-fpm but fcgid. As far as I know there is no such possibility in fcgid.

Any other ideas?


Sandro


________________________________
From: fedora <fe...@ayni.com>
Sent: Thursday, May 11, 2017 6:54 AM
To: users@httpd.apache.org
Subject: Re: [users@httpd] cgi script error output logging

Hi Sandro

are you using php-fpm as a cgi frontend? If yes: the stdout and stderr
are both redirectet to the php-fpm log (/var/log/php-fpm/*) if you have
in /etc/php-fpm/www.conf:

catch_workers_output = yes

I don't think this will solve all your problems, but it is a good
starting point.

suomi

On 05/10/2017 03:59 PM, KASPAR Sandro wrote:
>
> Hi,
>
> According to this documentation http://httpd.apache.org/docs/2.4/logs.html everything a cgi script sends to stderr is written to the apache  error log file.
>
> Unfortunately I can not control those scripts running on my server and often garbage is sent to stderr and then written to my error log. Because there isn't even a timestamp or any other useful information on those lines, I can't find out, which vhost created  the error. In addition those log lines often lack a "new line" at the end, and it the next log message gets appended to the current line instead of being written to a new line. Because of this, the next line which would have been in the correct format is also lost, because it can't be parsed automatically anymore.
>
> Is there any way to change this behaviour? What I would like to achieve is to have the cgi errors in a seperate logfile or even better change the log format of these lines. For example prefix the log line with the vhost and a timestamp.
> Because I don't have control over the cgi scripts, I would need to configure this in apaches main or vhost configuration. Unfortunately I could not find anything about cgi script output logging in the apache documentation (except that everything sent to  stderr is written to httpd-error.log) or mailing list archives.
>
> Any ideas appreciated. Thanks in advance!
>
> Sandro Kaspar
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Re: [users@httpd] cgi script error output logging

Posted by fedora <fe...@ayni.com>.
Hi Sandro

are you using php-fpm as a cgi frontend? If yes: the stdout and stderr 
are both redirectet to the php-fpm log (/var/log/php-fpm/*) if you have
in /etc/php-fpm/www.conf:

catch_workers_output = yes

I don't think this will solve all your problems, but it is a good 
starting point.

suomi

On 05/10/2017 03:59 PM, KASPAR Sandro wrote:
> 
> Hi,
> 
> According to this documentation http://httpd.apache.org/docs/2.4/logs.html everything a cgi script sends to stderr is written to the apache  error log file.
> 
> Unfortunately I can not control those scripts running on my server and often garbage is sent to stderr and then written to my error log. Because there isn't even a timestamp or any other useful information on those lines, I can't find out, which vhost created  the error. In addition those log lines often lack a "new line" at the end, and it the next log message gets appended to the current line instead of being written to a new line. Because of this, the next line which would have been in the correct format is also lost, because it can't be parsed automatically anymore.
> 
> Is there any way to change this behaviour? What I would like to achieve is to have the cgi errors in a seperate logfile or even better change the log format of these lines. For example prefix the log line with the vhost and a timestamp.
> Because I don't have control over the cgi scripts, I would need to configure this in apaches main or vhost configuration. Unfortunately I could not find anything about cgi script output logging in the apache documentation (except that everything sent to  stderr is written to httpd-error.log) or mailing list archives.
>   
> Any ideas appreciated. Thanks in advance!
> 
> Sandro Kaspar
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org