You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@perl.apache.org by "Pavel V." <pa...@ngs.ru> on 2014/08/31 10:43:37 UTC
ModPerl::RegistryCooker default_handler returned status
Hi all.
Can anybody explain historical reasons of ModPerl::RegistryCooker default_handler() implementation?
We can see the following piece of code there:
# handlers shouldn't set $r->status but return it, so we reset the
# status after running it
my $old_status = $self->{REQ}->status;
my $rc = $self->run;
my $new_status = $self->{REQ}->status($old_status);
return ($rc == Apache2::Const::OK && $old_status != $new_status)
? $new_status
: $rc;
The goal of ModPerl modules is "Run unaltered CGI scripts under mod_perl".
If scripts are unaltered, how they can change $r->status? I think, answer should be - 'in no way'.
Why we need to reset status then? I see no reasons. But this solution makes us impossible to set
status from 'altered' scripts correctly.
Did anyone can explain, why this code appears and for which scenarios it come?
Apache's mod_cgi.c module returns OK regardless of the r->status, which was set from script output.
Why ModPerl handler should behave differently?
One good example about problems, what are introduced by this solution, is 404 status.
Let`s look into small script:
#!/usr/bin/perl -w
use strict;
use CGI;
my $q = CGI->new;
#### Variant 1
print $q->header(-charset => "windows-1251", -type => "text/html", -status=>'404 Not Found');
print "SMALL RESPONSE";
#### Variant 2
print $q->header(-charset => "windows-1251", -type => "text/html", -status=>'404 Not Found');
#print 'BIG RESPONSE:' . '*' x 8192;
Under CGI it prints only "SMALL/BIG RESPONSE" string with 404 status code into browser.
Under mod_perl, browser get status 200 instead of 404 and script response is appended by default
Apache error-handler content (like 'Status: OK \n /path/to/script.pl was not found on this server').
If we print 'big response', then browser get status 404 and apache error-handler appended content
changes to 'Not Found \n The requested URL /perl/test.pl was not found on this server.'.
The reasons for such behaviour is what scripts are using CGI.pm which handles MOD_PERL.
So, they are practically 'altered' and can set $r-status inside of them. After script executes, we
have r->status = 404. But ModPerl::RegistryCooker set r->status to 200 and return 404 as handler
status - so apache ErrorDocument directive begin to work (under mod_cgi handler status is OK, so
no error processing occurs). Also, due to r->status changed, we get 200 into browser.
Most interesting, what we get 404 status logged into access log anyway.
Ok, the next example - completely unaltered script:
#!/usr/bin/perl -w
use strict;
print "Status: 404 Not Found\n";
print "Content-Type: text/html; charset=windows-1251\n\n";
#Small response
print 'NOTFOUND:' . '*' x 81;
#Big response
#print 'NOTFOUND:' . '*' x 8192;
When response is small, then 8k buffer is not filled, and r->status does not changed internally
before cgi headers parsed on buffer flush. So, we get r->status == 200, and handler status is 200
too. After headers are parsed, r->status become 404, but mod_perl handler returns status 200 to
apache
--
Regards,,
Pavel mailto:pavel2000@ngs.ru
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: ModPerl::RegistryCooker default_handler returned status
Posted by Perrin Harkins <pe...@elem.com>.
HI Pavel,
Your messages include a lot of good details, but they are a bit long. I'd
suggest trying to keep things as short as you can. Remember that everyone
working on open source is a volunteer, spending their own personal time to
read and answer your messages.
On Sun, Aug 31, 2014 at 4:15 PM, Pavel V. <pa...@ngs.ru> wrote:
> I completely misunderstand, why we need to return HTTP_* codes from
ModPerl handlers.
> Why Apache::OK or Apache::SERVER_ERROR are not enough?
Returning HTTP status code constants from a handler is legal and expected.
See "Return Values" here:
http://httpd.apache.org/docs/2.4/developer/modguide.html
> Can anybody explain, why this was needed with mp1?
Sure. CGI scripts don't have return values, since they are standalone
scripts and not subroutines. They set status by sending the headers out.
A proper handler can do this with the return code, but if you do that,
your script won't be correct under mod_cgi anymore. Many people needed
their scripts to run under both, so the workaround was to set $r->status()
if you detect a mod_perl environment, and Registry would pick that up and
use it as the return code.
It might have been better to do something different that would have left
status untouched, like set an $ENV variable, but I suspect people were
already trying to set $r->status() directly and Doug just wanted to make
that work. Registry has always been a balance between porting CGI scripts
and allowing people to do more mod_perl-specific things.
> Why we are unable to left $r->status, the new value set by script,
untouched?
I believe this is because we want the server to get the response code from
the handler and set the status itself.
> After reading mailing list I understand what nothing will change in
nearest .... century.
> Sorry to bother you.
There's no need to be rude. It's not a simple situation with a simple
answer. This is why the recent versions of Registry were designed to be
easy to override. It's impossible to please everyone with one set of
behavior. Most people who are trying to do things involving status codes
end up switching over to writing handlers, which is a better approach if
you're not concerned about your code working under mod_cgi.
- Perrin
Re: ModPerl::RegistryCooker default_handler returned status
Posted by "Pavel V." <pa...@ngs.ru>.
Hi, Perrin.
> Hi Pavel,
> You might get more interesting answers on the "users" mailing list, but there's lots of
> discussion about this in the archives.
Thanks for interesting reading. I will read for a last 4 hours.
> For example:
> http://www.gossamer-threads.com/lists/modperl/modperl/45559?search_string=registry%20status;#45559
This thread is about 'how to do things incorrectly'.
The handler status and response status are different things, and only broken implementation in
RegistryCooker mix them together.
I completely misunderstand, why we need to return HTTP_* codes from ModPerl handlers.
Why Apache::OK or Apache::SERVER_ERROR are not enough?
> http://www.gossamer-threads.com/lists/modperl/modperl/88690?search_string=registry%20status;#88690
This thread is exactly about this: '
> $r->status is used to communicate the registry handler return code,
> not really set r->status over in C-land (as it would be if you called it from a normal
> handler).'
Yes, I agreed with topicstarter - $r->status() is evil under ModPerl modules in current implementation.
Current implementation uses $r->status to set handler status, and r->status is can not be edited
using it. And this is ugly.
But WHY? What needs have led to this? I found no reasons for this in apache-1.3/apache-2.2
implementations. But it is written there:
> We have a bunch of old scripts that set $r->status explicitly. For
> handlers, that's deprecated and unnecessary, but I understand that was
> the usual way to do it in registry scripts in the old days of mp1.
Can anybody explain, why this was needed with mp1?
Why we are unable to left $r->status, the new value set by script, untouched?
---------
http://www.gossamer-threads.com/lists/modperl/modperl/54151?search_string=registry%20status;#54151
is more interesting - it is also about problem with non-200 response code.
Perrin, what personally you think about - 'why we need to reset status after running handler'?
Can you show discussion, explaining why this was neeeded? I know what this 'feature' appeared at
mod_perl-1.21_03 - March 15, 2000.
As I understand, it was your answer in
http://www.gossamer-threads.com/lists/modperl/modperl/99700?search_string=registry%20status;#99700 :
> You also need to return the correct code from the handler. This is
> not simple to do from Registry. Have a quick look at the list
> archives for things related to Registry and 304/404 handling.
Why this should not be simple? Why this become a problem?
Lot of people had problems because of 'we need to reset status after running handler' decision.
WHY ?? WHY ?? WHY ??
http://www.gossamer-threads.com/lists/modperl/modperl/54154?search_string=registry%20status;#54154
is ever more interesting, but nothing changed for last decade.
> I think the gist of it is that some people make their Registry scripts set $r->status because
> Registry doesn't scan the script output for a status header like mod_cgi does.
mod_cgi does such scan to set r->status by calling ap_scan_script_header_err_(buff|brigade|core) and
does not scan nothing to produce OK in cgi_handler() 'return OK;' last line.
mod_perl does same scan by calling modperl_wbucket_pass() -> modperl_cgi_header_parse() -> ap_scan_script_header_err_strs().
What do you think about how it runs the code like this:
print "Status: 404 Not Found\n";
print "Content-Type: text/html; charset=windows-1251\n\n";
print 'NOTFOUND:' . '*' x 81;
This code works under mod_cgi and mod_perl both. $r->status was not set, right?
This code works, but it works differently, despite of ModPerl modules goal to "Run unaltered CGI scripts under mod_perl".
> However, that means Apache doesn't know what the real status is.
> In order to tell Apache what the status is, Registry checks to see if you set $r->status and
> uses that as the return code, which may affect what status Apache decides to send.
You mix handler status and response status again. These are different concepts.
Why not leave r->status set by script execution? Moreover, r->status can be changed by
mod_perl.c/apache internals (when buffer flushed due to 8k overflow).
> I may be reading your use case incorrectly, but you seem to be trying to tell the server you have
> a 200 OK but really return a 404.
No. As many other people I want send 404 (non-200/3xx) status into browser, and got problems with this simple
task.
HTTP_OK and OK are different constants. I want to tell apache what mod_perl handler finished
it`s request processing, by returning OK code (0), not HTTP_OK (200). mod_cgi does this in this way and I
want understand why mod_perl does not. Why not leave r->status untouched and return
Apache2::Const::OK in ModPerl::RegistryCooker default_handler() ?
http://www.gossamer-threads.com/lists/modperl/modperl/100810?search_string=registry%20status;#100810
- proposes this solution and was completely ignored!
Also, I described a problem I see: my browser get HTTP 200 status from two lines of
code script:
print $q->header(-charset => "windows-1251", -type => "text/html", -status=>'404 Not Found');
print "Sorry, page you requested does not exists.";
I think this abnormal. Did you?
This is because RegistryCooker 'reset the status after running handler' back to 200 (from $old_status).
r->status == 200 for 404 response - great!
Also, script output was appended by standard apache error page.
This was discussed many times too.
This was because when Registry return 404 (or other than 200) as handler status, then response is
processed by apache as error.
When status is 200 $old_status == $new_status and $rc is returned in RegistryCooker.pm and things
works correctly. Note, there will be $rc == Apache2::Const::OK, not HTTP_OK!
Look into httpd-2.2.27/modules/http/http_request.c:258, ap_process_request():
access_status = ap_invoke_handler(r);
if (access_status == DONE) {
access_status = OK;
}
if (access_status == OK) {
ap_finalize_request_protocol(r); <---- Should be here in all non-error cases
}
else {
r->status = HTTP_OK;
ap_die(access_status, r); <----- But ALL NON-200 handler status, such as 3xx, 4xx, will be processed here
}
And ap_die() also adds 'custom_response' which is is not needed and can break things - I described
this detailed.
> I may be reading your use case incorrectly, but you seem to be trying to tell the server you have
> a 200 OK but really return a 404. That's discussed a little bit here:
> http://perl.apache.org/docs/2.0/api/Apache2/RequestRec.html#C_status_.
That description is wrong too, because it is about implementation which mixes request status and
handler status together. Typical apache module uses three constants for handler status:
#define DECLINED -1 /**< Module declines to handle */
#define DONE -2 /**< Module has served the response completely.
* - it's safe to die() with no more output
*/
#define OK 0 /**< Module has handled this stage. */
And this behavior is not a trick as written there, but the normal way - mod_cgi,mod_cgid examples
are good enough, I think.
> That is somewhat sophisticated for a Registry script, but you can always just make your own subclass and override
> that method in RegistryCooker to do what you want. It's intended to be customizable in that way.
Sure, of course I know about this solution. It fully works for me.
I described in detail some cases when different problems occurs due RegistryCooker implementation.
Will we treat ModPerl Registry modules behaviour/implementation as a bug? Rhetorical question.
After reading mailing list I understand what nothing will change in nearest .... century.
Sorry to bother you.
--
Regards,
Pavel mailto:pavel2000@ngs.ru
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org
Re: ModPerl::RegistryCooker default_handler returned status
Posted by Perrin Harkins <pe...@elem.com>.
Hi Pavel,
You might get more interesting answers on the "users" mailing list, but
there's lots of discussion about this in the archives. For example:
http://www.gossamer-threads.com/lists/modperl/modperl/45559?search_string=registry%20status;#45559
http://www.gossamer-threads.com/lists/modperl/modperl/88690?search_string=registry%20status;#88690
I think the gist of it is that some people make their Registry scripts set
$r->status because Registry doesn't scan the script output for a status
header like mod_cgi does. However, that means Apache doesn't know what the
real status is. In order to tell Apache what the status is, Registry
checks to see if you set $r->status and uses that as the return code, which
may affect what status Apache decides to send.
I may be reading your use case incorrectly, but you seem to be trying to
tell the server you have a 200 OK but really return a 404. That's
discussed a little bit here:
http://perl.apache.org/docs/2.0/api/Apache2/RequestRec.html#C_status_.
That is somewhat sophisticated for a Registry script, but you can always
just make your own subclass and override that method in RegistryCooker to
do what you want. It's intended to be customizable in that way.
- Perrin
On Sun, Aug 31, 2014 at 7:50 AM, Pavel V. <pa...@ngs.ru> wrote:
>
> Hi all.
>
> (sorry for unfinished letter I sent before)
>
> Can anybody explain historical reasons of ModPerl::RegistryCooker
> default_handler() implementation?
> We can see the following piece of code there:
>
> # handlers shouldn't set $r->status but return it, so we reset the
> # status after running it
> my $old_status = $self->{REQ}->status;
> my $rc = $self->run;
> my $new_status = $self->{REQ}->status($old_status);
> return ($rc == Apache2::Const::OK && $old_status != $new_status)
> ? $new_status
> : $rc;
>
> The goal of ModPerl modules is "Run unaltered CGI scripts under
> mod_perl".
> If scripts are unaltered, how they can change $r->status? I think,
> answer should be: 'in no way'.
> Why we need to reset status then? I see no reasons. But this solution
> makes us impossible to set
> r->status from 'altered' scripts correctly and produces strange effects
> by headers parser on
> responses > 8k size.
>
> Did anyone can explain, why this code appear and for which scenarios it
> come?
>
> Apache's mod_cgi.c module returns OK regardless of the r->status value,
> which was set from script output.
> Why ModPerl handler should behave differently?
>
> One good example about problems, introduced by this solution, is 404
> status.
> Let`s look into small script:
>
> #!/usr/bin/perl -w
> use strict;
> use CGI;
>
> my $q = CGI->new;
> #### Variant 1
> print $q->header(-charset => "windows-1251", -type => "text/html",
> -status=>'404 Not Found');
> print "SMALL RESPONSE";
> #### Variant 2
> #print $q->header(-charset => "windows-1251", -type => "text/html",
> -status=>'404 Not Found');
> #print 'BIG RESPONSE:' . '*' x 8192;
>
> Under CGI it prints only "SMALL/BIG RESPONSE" strings with 404 status
> code into browser.
> Under mod_perl, browser get status 200 instead of 404 and script
> response content is appended by
> default Apache error-handler content (like 'Status: OK \n /path/to/
> script.pl was not found on this
> server') for 'small response'.
> If we print 'big response', then browser get status 404 and apache
> error-handler appended content
> changes to 'Not Found \n The requested URL /perl/test.pl was not found
> on this server.'.
>
> The reasons for such behavior is what script uses CGI.pm which detects
> mod_perl.
> So, these scripts are practically 'altered' and can set $r-status inside
> of them. After script
> executes, we have r->status == 404. But ModPerl::RegistryCooker set
> r->status to 200 back and return
> 404 as handler status - so apache ErrorDocument directive begin to work.
> (under mod_cgi handler status is OK, so no error processing occurs).
> Also, due to r->status changed, we can get 200 response code in browser
> if response was not send
> yet.
> Most interesting thing, what we get 404 status logged into Apache access
> log anyway, so it is harder to
> detect this problem.
>
> Ok, maybe this is completely CGI.pm problem? Let's look into next
> example - fully unaltered script:
>
> #!/usr/bin/perl -w
> use strict;
> print "Status: 404 Not Found\n";
> print "Content-Type: text/html; charset=windows-1251\n\n";
> #Small response
> print 'NOTFOUND:' . '*' x 81;
> #Big response
> #print 'NOTFOUND:' . '*' x 8192;
>
> When response is small, then 8k buffer is not filled, and r->status does
> not changed internally
> before cgi headers parsed on buffer flush. So, we get r->status == 200,
> and handler status is 200
> too. After headers are parsed, r->status become 404, but mod_perl
> handler returns status 200 to
> apache.
>
> When response is big, then 8k buffer is filled and flushed before
> handler is done.
> So, we have r->status == 404, and handler status is 404 too. As result,
> we have ErrorDocument
> directive again.
>
> Browser get 404 status in all these cases, I don't understand
> completely, how this is processed.
>
> If we enable mod_deflate apache module, all things become ever more
> interesting.
> When small response - all shows as before.
> When big response - only apache error page displayed, no content from
> script output displayed.
>
> With mod_cgi we see output from script all the time, with or without
> mod_deflate.
>
> So, ModPerl::* modules does not fully reach their goal of "Run unaltered
> CGI scripts under
> mod_perl".
>
> As follows from my analysis of mod_perl, mod_cgi and other httpd core,
> we need no checks for new
> r->status value after script run at all. Due to mod_perl handler status
> not in (OK,DECLINED,DONE)
> for all r->statuses != 200, ap_die() is always called instead of
> ap_finalize_request_protocol() at
> ap_process_request(). Then all responses with non-200 statuses are
> processed through
> ErrorDocument, if exists for that code, and then
> ap_send_error_response() is called for them. This
> is wrong way, I think. 'Altered' scripts have no practical use for
> $r->status($newValue) in
> current code state.
>
> So, I propose to patch ModPerl::* modules and they would run
> unaltered/altered scripts more
> correctly:
>
> - # handlers shouldn't set $r->status but return it, so we reset the
> - # status after running it
> - my $old_status = $self->{REQ}->status;
> - my $rc = $self->run;
> - my $new_status = $self->{REQ}->status($old_status);
> - return ($rc == Apache2::Const::OK && $old_status != $new_status)
> - ? $new_status
> - : $rc;
> + return $self->run;
>
> If altered script needs to set new handler status by doing 'return
> $newHandleStatus', sub run() should
> be altered to get value, returned from eval {}; But I think, all who
> needs this wrote
> their module handlers already ;-)
>
> Thanks for your attention.
> I understand what it is too hard to make such global changes and there
> is not so much people
> interested in this. So, please look to my letter as a question about
> help in historical research ;-)
>
> Let me remind you my question:
> Did anyone can explain, why this code appears and for which scenarios
> it come?
>
> If anybody is interested, I propose to discuss this.
>
> Thanks.
>
> --
> Regards,
> Pavel mailto:pavel2000@ngs.ru
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
> For additional commands, e-mail: dev-help@perl.apache.org
>
>
ModPerl::RegistryCooker default_handler returned status
Posted by "Pavel V." <pa...@ngs.ru>.
Hi all.
(sorry for unfinished letter I sent before)
Can anybody explain historical reasons of ModPerl::RegistryCooker default_handler() implementation?
We can see the following piece of code there:
# handlers shouldn't set $r->status but return it, so we reset the
# status after running it
my $old_status = $self->{REQ}->status;
my $rc = $self->run;
my $new_status = $self->{REQ}->status($old_status);
return ($rc == Apache2::Const::OK && $old_status != $new_status)
? $new_status
: $rc;
The goal of ModPerl modules is "Run unaltered CGI scripts under mod_perl".
If scripts are unaltered, how they can change $r->status? I think, answer should be: 'in no way'.
Why we need to reset status then? I see no reasons. But this solution makes us impossible to set
r->status from 'altered' scripts correctly and produces strange effects by headers parser on
responses > 8k size.
Did anyone can explain, why this code appear and for which scenarios it come?
Apache's mod_cgi.c module returns OK regardless of the r->status value, which was set from script output.
Why ModPerl handler should behave differently?
One good example about problems, introduced by this solution, is 404 status.
Let`s look into small script:
#!/usr/bin/perl -w
use strict;
use CGI;
my $q = CGI->new;
#### Variant 1
print $q->header(-charset => "windows-1251", -type => "text/html", -status=>'404 Not Found');
print "SMALL RESPONSE";
#### Variant 2
#print $q->header(-charset => "windows-1251", -type => "text/html", -status=>'404 Not Found');
#print 'BIG RESPONSE:' . '*' x 8192;
Under CGI it prints only "SMALL/BIG RESPONSE" strings with 404 status code into browser.
Under mod_perl, browser get status 200 instead of 404 and script response content is appended by
default Apache error-handler content (like 'Status: OK \n /path/to/script.pl was not found on this
server') for 'small response'.
If we print 'big response', then browser get status 404 and apache error-handler appended content
changes to 'Not Found \n The requested URL /perl/test.pl was not found on this server.'.
The reasons for such behavior is what script uses CGI.pm which detects mod_perl.
So, these scripts are practically 'altered' and can set $r-status inside of them. After script
executes, we have r->status == 404. But ModPerl::RegistryCooker set r->status to 200 back and return
404 as handler status - so apache ErrorDocument directive begin to work.
(under mod_cgi handler status is OK, so no error processing occurs).
Also, due to r->status changed, we can get 200 response code in browser if response was not send
yet.
Most interesting thing, what we get 404 status logged into Apache access log anyway, so it is harder to
detect this problem.
Ok, maybe this is completely CGI.pm problem? Let's look into next example - fully unaltered script:
#!/usr/bin/perl -w
use strict;
print "Status: 404 Not Found\n";
print "Content-Type: text/html; charset=windows-1251\n\n";
#Small response
print 'NOTFOUND:' . '*' x 81;
#Big response
#print 'NOTFOUND:' . '*' x 8192;
When response is small, then 8k buffer is not filled, and r->status does not changed internally
before cgi headers parsed on buffer flush. So, we get r->status == 200, and handler status is 200
too. After headers are parsed, r->status become 404, but mod_perl handler returns status 200 to
apache.
When response is big, then 8k buffer is filled and flushed before handler is done.
So, we have r->status == 404, and handler status is 404 too. As result, we have ErrorDocument
directive again.
Browser get 404 status in all these cases, I don't understand completely, how this is processed.
If we enable mod_deflate apache module, all things become ever more interesting.
When small response - all shows as before.
When big response - only apache error page displayed, no content from script output displayed.
With mod_cgi we see output from script all the time, with or without mod_deflate.
So, ModPerl::* modules does not fully reach their goal of "Run unaltered CGI scripts under
mod_perl".
As follows from my analysis of mod_perl, mod_cgi and other httpd core, we need no checks for new
r->status value after script run at all. Due to mod_perl handler status not in (OK,DECLINED,DONE)
for all r->statuses != 200, ap_die() is always called instead of ap_finalize_request_protocol() at
ap_process_request(). Then all responses with non-200 statuses are processed through
ErrorDocument, if exists for that code, and then ap_send_error_response() is called for them. This
is wrong way, I think. 'Altered' scripts have no practical use for $r->status($newValue) in
current code state.
So, I propose to patch ModPerl::* modules and they would run unaltered/altered scripts more
correctly:
- # handlers shouldn't set $r->status but return it, so we reset the
- # status after running it
- my $old_status = $self->{REQ}->status;
- my $rc = $self->run;
- my $new_status = $self->{REQ}->status($old_status);
- return ($rc == Apache2::Const::OK && $old_status != $new_status)
- ? $new_status
- : $rc;
+ return $self->run;
If altered script needs to set new handler status by doing 'return $newHandleStatus', sub run() should
be altered to get value, returned from eval {}; But I think, all who needs this wrote
their module handlers already ;-)
Thanks for your attention.
I understand what it is too hard to make such global changes and there is not so much people
interested in this. So, please look to my letter as a question about help in historical research ;-)
Let me remind you my question:
Did anyone can explain, why this code appears and for which scenarios it come?
If anybody is interested, I propose to discuss this.
Thanks.
--
Regards,
Pavel mailto:pavel2000@ngs.ru
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org