You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@perl.apache.org by Geoffrey Young <ge...@modperlcookbook.org> on 2004/01/07 18:13:48 UTC

unescaping the error_log

hi all

this escaped error_log is making it impossible to debug anything within the
mod_perl test suite.  I'd like to do something like the attached patch to
make our lives (as well as the lives of our users) a bit easier.  it may not
be ideal, but it seems to make things better for me for the moment, and it
doesn't involve pipes so it should be platform safe.

other approaches or ways to make it better are welcome.

--Geoff

Re: unescaping the error_log

Posted by Stas Bekman <st...@stason.org>.
Geoffrey Young wrote:
> 
> Stas Bekman wrote:
> 
>>any follow ups on this one, before we try to take it back to httpd-dev?
> 
> 
> I already posted a patch to httpd-dev that made it possible to disable the
> escaping at compile time - a step in the right direction.  give it a review
> then commit :)

Thanks Geoff,

let's see what's the httpd-dev verdict on this.

__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org


Re: unescaping the error_log

Posted by Geoffrey Young <ge...@modperlcookbook.org>.

Stas Bekman wrote:
> any follow ups on this one, before we try to take it back to httpd-dev?

I already posted a patch to httpd-dev that made it possible to disable the
escaping at compile time - a step in the right direction.  give it a review
then commit :)

--Geoff


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org


Re: unescaping the error_log

Posted by Stas Bekman <st...@stason.org>.
any follow ups on this one, before we try to take it back to httpd-dev?

__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org


Re: unescaping the error_log

Posted by Stas Bekman <st...@stason.org>.
Geoffrey Young wrote:
>>I still want to see it solved on the Apache side. 
> 
> 
> I don't think you'll see this solved the way you want, so we're going to
> need to live with it one way or another.

so we need one centralized workaround (in the mod_perl core) and not in every 
place it's used as a separate random varying implementation.

>>Any workarounds suck,
>  
> but inevitable.

Not necessarily. I think suggesting to httpd-dev to make this escaping a 
compile time option (defaulting to the current behavior is a good idea).

>>because you unescape just a few things at the moment. Later you will
>>want to extend this list. 
> 
> 
> perhaps.  notice I didn't try an unescape \b or some of the others because I
> don't expect them to show up in the test suite and becuase really it's the
> newline thing that really makes this hell for us.

That's exactly the problem. You don't expect it, because you are trying to 
solve the problem for what you see. Others may get other output and they will 
need to unescape other chars as well.

>>And this
>>practice is very questionable in terms of circumventing the security
>>this change has been made for.
> 
> 
> I don't think so.  the security issue was for rogue people trying to access
> the server remotely via URL query.  for software on the box to do this, it
> hardly needs to resort to such antics - simply adding shell code to TEST.PL
> is sufficient.

Not only that. There were several reasons for this change. One of them was 
logisticals, where a malicious user create a URL that gets logged as two 
requests, second request carefully crafted by the user. It's not a security 
issue per se, IMHO. The main security issue as I understood it was the issue 
with some terminal programs which can be triggered into doing bad things while 
reading the log file.

>>I'm very unhappy about this change in Apache, but besides me everybody
>>keeps quiet and doesn't complain/looking for solutions in the core of
>>the problem, I won't be surprised that it'll stay that way.
> 
> 
> httpd core has chosen their path, giving a valid reason for doing so.
> complaining won't get anything else accomplished, other than removing karma.

complaining didn't help may be because I was the only voice. Wait till normal 
users pick this new server up (when it gets released) and then you will hear 
the screams of pain en masse. Also my complaining could be more helpful if I 
were to propose some sort of solution, rather just saying 'no, no, no'.

> the only thing that bothers me is that they don't have a compile-time option
> to turn it off.

right, that sounds good to me.

>>If Apache doesn't remove this change, I'm thinking that we will provide
>>an alternative implementation in mod_perl and have a compile time option
>>which will choose Apache's implementation vs. ours (ours will be just a
>>copy of Apache's core implementation before this change). How does this
>>sound? Of course users should be aware of the potential risks if they
>>choose ours.
> 
> 
> yucko.

what's your proposal?

__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org


Re: unescaping the error_log

Posted by Geoffrey Young <ge...@modperlcookbook.org>.
> I still want to see it solved on the Apache side. 

I don't think you'll see this solved the way you want, so we're going to
need to live with it one way or another.

> Any workarounds suck,

but inevitable.

> because you unescape just a few things at the moment. Later you will
> want to extend this list. 

perhaps.  notice I didn't try an unescape \b or some of the others because I
don't expect them to show up in the test suite and becuase really it's the
newline thing that really makes this hell for us.


> And it doesn't help non A-T users. 

twas never the intent.

> And this
> practice is very questionable in terms of circumventing the security
> this change has been made for.

I don't think so.  the security issue was for rogue people trying to access
the server remotely via URL query.  for software on the box to do this, it
hardly needs to resort to such antics - simply adding shell code to TEST.PL
is sufficient.

> 
> The particular problem with your patch is the slurp mode. Quite often I
> have enormous, up to 100MB error_log files. You don't want to slurp and
> rewrite them in memory. Even w/o the slurp issue it's also going to be
> quite slow.

yes, good point.

> 
> I'm very unhappy about this change in Apache, but besides me everybody
> keeps quiet and doesn't complain/looking for solutions in the core of
> the problem, I won't be surprised that it'll stay that way.

httpd core has chosen their path, giving a valid reason for doing so.
complaining won't get anything else accomplished, other than removing karma.

the only thing that bothers me is that they don't have a compile-time option
to turn it off.

> 
> If Apache doesn't remove this change, I'm thinking that we will provide
> an alternative implementation in mod_perl and have a compile time option
> which will choose Apache's implementation vs. ours (ours will be just a
> copy of Apache's core implementation before this change). How does this
> sound? Of course users should be aware of the potential risks if they
> choose ours.

yucko.

--Geoff


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org


Re: unescaping the error_log

Posted by Stas Bekman <st...@stason.org>.
Geoffrey Young wrote:
>>Well, being a bit slanted on the security side, I agree with mandatory
>>escaping of the error_logs, as annoying it might be for development
>>reasons.
> 
> 
> well, we've come up with a compromize in 2.1 - compile with
> -DAP_UNSAFE_ERROR_LOG_UNESCAPED and the error_log will remain in its
> unescaped format.
> 
> it's been integrated into 2.1 already, and I have patches for 2.0 and 1.3
> ready.  2.0 backport has 3 votes, but the last one is mine, so I'm not sure
> if I can just go an backport it to 2.0 at this point.

Yes, you can. Your a committer so it counts. I did that several times.

> as for 1.3, I'm not sure how that works.

Ping jim.


__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org


Re: unescaping the error_log

Posted by Geoffrey Young <ge...@modperlcookbook.org>.
> Well, being a bit slanted on the security side, I agree with mandatory
> escaping of the error_logs, as annoying it might be for development
> reasons.

well, we've come up with a compromize in 2.1 - compile with
-DAP_UNSAFE_ERROR_LOG_UNESCAPED and the error_log will remain in its
unescaped format.

it's been integrated into 2.1 already, and I have patches for 2.0 and 1.3
ready.  2.0 backport has 3 votes, but the last one is mine, so I'm not sure
if I can just go an backport it to 2.0 at this point.

as for 1.3, I'm not sure how that works.

--Geoff


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org


Re: unescaping the error_log

Posted by Stas Bekman <st...@stason.org>.
Philippe M. Chiasson wrote:
> On Wed, 2004-01-07 at 09:56, Stas Bekman wrote:
> 
>>Geoffrey Young wrote:
>>[...]
> 
> 
> Sorry for kicking in late on this thread, but lemme share my 0.02$
> 
> 
>>I'm very unhappy about this change in Apache, but besides me everybody keeps 
>>quiet and doesn't complain/looking for solutions in the core of the problem, I 
>>won't be surprised that it'll stay that way.
> 
> 
> Well, being a bit slanted on the security side, I agree with mandatory
> escaping of the error_logs, as annoying it might be for development
> reasons.
> 
> 
>>If Apache doesn't remove this change, I'm thinking that we will provide an 
>>alternative implementation in mod_perl and have a compile time option which 
>>will choose Apache's implementation vs. ours (ours will be just a copy of 
>>Apache's core implementation before this change). How does this sound? Of 
>>course users should be aware of the potential risks if they choose ours.
> 
> 
> I don't like much the idea of having mod_perl do something like that, at
> least not with *BIG RED LETTERING* warning joe-user.

It has been already solved, by introducing a compile time define in httpd 
which will prevent that escaping. Geoff, was it backported to 2.0 already? I 
suppose you have added your +1, right?

> But, why not work around this issue completely by introducing :
> ModPerlErrorLog logs/modperl_log
> 
> Or something similar? Then there is no problem anymore and we can do
> whatever we feel like with logs, color codes, etc.

That's a bad idea, IMHO. You will have to monitor two log files instead of one.

And even if you did that, what do you achive? You will have to drop the C API 
for logging and reimplement from scratch.

__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org


Re: unescaping the error_log

Posted by "Philippe M. Chiasson" <go...@cpan.org>.
On Wed, 2004-01-07 at 09:56, Stas Bekman wrote:
> Geoffrey Young wrote:
> [...]

Sorry for kicking in late on this thread, but lemme share my 0.02$

> I'm very unhappy about this change in Apache, but besides me everybody keeps 
> quiet and doesn't complain/looking for solutions in the core of the problem, I 
> won't be surprised that it'll stay that way.

Well, being a bit slanted on the security side, I agree with mandatory
escaping of the error_logs, as annoying it might be for development
reasons.

> If Apache doesn't remove this change, I'm thinking that we will provide an 
> alternative implementation in mod_perl and have a compile time option which 
> will choose Apache's implementation vs. ours (ours will be just a copy of 
> Apache's core implementation before this change). How does this sound? Of 
> course users should be aware of the potential risks if they choose ours.

I don't like much the idea of having mod_perl do something like that, at
least not with *BIG RED LETTERING* warning joe-user.

But, why not work around this issue completely by introducing :
ModPerlErrorLog logs/modperl_log

Or something similar? Then there is no problem anymore and we can do
whatever we feel like with logs, color codes, etc.

> __________________________________________________________________
> Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
> http://stason.org/     mod_perl Guide ---> http://perl.apache.org
> mailto:stas@stason.org http://use.perl.org http://apacheweek.com
> http://modperlbook.org http://apache.org   http://ticketmaster.com
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
> For additional commands, e-mail: dev-help@perl.apache.org
-- 
--------------------------------------------------------------------------------
Philippe M. Chiasson /gozer\@(cpan|ectoplasm)\.org/ 88C3A5A5 (122FF51B/C634E37B)
http://gozer.ectoplasm.org/    F9BF E0C2 480E 7680 1AE5 3631 CB32 A107 88C3 A5A5
Q: It is impossible to make anything foolproof because fools are so ingenious.
perl -e'$$=\${gozer};{$_=unpack(P7,pack(L,$$));/^JAm_pH\n$/&&print||$$++&&redo}'

Re: unescaping the error_log

Posted by Stas Bekman <st...@stason.org>.
Geoffrey Young wrote:
> hi all
> 
> this escaped error_log is making it impossible to debug anything within the
> mod_perl test suite.  I'd like to do something like the attached patch to
> make our lives (as well as the lives of our users) a bit easier.  it may not
> be ideal, but it seems to make things better for me for the moment, and it
> doesn't involve pipes so it should be platform safe.
> 
> other approaches or ways to make it better are welcome.

I still want to see it solved on the Apache side. Any workarounds suck, 
because you unescape just a few things at the moment. Later you will want to 
extend this list. And it doesn't help non A-T users. And this practice is very 
questionable in terms of circumventing the security this change has been made for.

The particular problem with your patch is the slurp mode. Quite often I have 
enormous, up to 100MB error_log files. You don't want to slurp and rewrite 
them in memory. Even w/o the slurp issue it's also going to be quite slow.

I'm very unhappy about this change in Apache, but besides me everybody keeps 
quiet and doesn't complain/looking for solutions in the core of the problem, I 
won't be surprised that it'll stay that way.

If Apache doesn't remove this change, I'm thinking that we will provide an 
alternative implementation in mod_perl and have a compile time option which 
will choose Apache's implementation vs. ours (ours will be just a copy of 
Apache's core implementation before this change). How does this sound? Of 
course users should be aware of the potential risks if they choose ours.

__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org


Re: unescaping the error_log

Posted by Geoffrey Young <ge...@modperlcookbook.org>.

Rafael Garcia-Suarez wrote:
> Geoffrey Young wrote:
> 
>>+    (my $version) = $config->server->{version} =~ m:^Apache/(\d\.\d+\.\d+):;
>>+
>>+    if ($version le 2.0.48) {
>>+        debug("skipping error_log fixup for httpd $version");
>>+        return;
>>+    } 
> 
> 
> This won't work. Remember that 2.0.48 is equivalent to chr(2).chr(0).chr(48).

ah, yes, you're right - it's not so much an issue now unless somebody is
using 2.0.5.  but it will be more of an issue when 2.1.10 compares as less
than 2.1.2.

too bad I copied the logic from elsewhere within Apache-Test :)

> 
> You could use the terse and hackish
> 	if (eval($version) le 2.0.48) ...
> or use a CPAN module :)
> or perform a clean separation of $version into major/minor/patchelevel
> and do 3 comparisons.

although it's a moot issue for the proposed patch, it will definitely need
to be fixed elsewhere in the suite.  thanks for bringing it up - I'll get on it.

--Geoff


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org


Re: unescaping the error_log

Posted by Rafael Garcia-Suarez <ra...@hexaflux.com>.
Geoffrey Young wrote:
>
> +    (my $version) = $config->server->{version} =~ m:^Apache/(\d\.\d+\.\d+):;
> +
> +    if ($version le 2.0.48) {
> +        debug("skipping error_log fixup for httpd $version");
> +        return;
> +    } 

This won't work. Remember that 2.0.48 is equivalent to chr(2).chr(0).chr(48).

You could use the terse and hackish
	if (eval($version) le 2.0.48) ...
or use a CPAN module :)
or perform a clean separation of $version into major/minor/patchelevel
and do 3 comparisons.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org


Re: unescaping the error_log

Posted by Geoffrey Young <ge...@modperlcookbook.org>.
> +    $log =~ s/\\n|\\t/\n\t/g;

duh - that's obviously not the proper regex :)

so, comments other than that...

--Geoff


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org