You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Iosif Fettich <if...@netsoft.ro> on 2009/03/23 09:10:34 UTC
who's putting that tag in the output...?
Dear all,
as I'm still making no progress in finding what's up, I thought I'll try
to ask for some help.
I wrote a little PerlResponse header, like
---------
package Apache::Default_to_OLD;
use strict;
use warnings;
use Apache2::Const -compile => qw(OK DECLINED);
use Apache2::RequestRec ();
use Apache2::RequestIO ();
use Apache2::RequestUtil ();
use Apache2::SubRequest;
use APR::URI;
use Apache2::URI;
use HTTP::Request;
use LWP::UserAgent;
sub handler {
my $r = shift;
if ($r->is_initial_req) {
my $unparsed_uri = $r->unparsed_uri();
my $subr = $r->lookup_uri( $unparsed_uri );
my $rc = $subr->run;
if ($rc eq Apache2::Const::OK) {
# the new site is offering what is wanted
return $rc;
}
else {
# transform the request into a proxy request
my $url = $r->construct_url;
my $args = $r->args();
my $full_url = $url;
$full_url .= "?$args" if defined $args;
my $parsed = APR::URI->parse($r->pool, $full_url);
$parsed->hostname( 'oldsite.mydomain.com' );
my $new_url = $parsed->unparse;
# Create a new request.
my $request = HTTP::Request->new( GET => $new_url );
# Pass the request to the user agent and get a response back.
my $response = LWP::UserAgent->new->request($request);
my $content = $response->content;
$r->rflush();
print $content;
return Apache2::Const::OK if $response->is_success;
return $response->code;
}
}
return Apache2::Const::DECLINED;
}
1;
----
The problem is that what I want to be the handler's proxied response is
actual embedded instead in an construct like
<html>
<head/>
<body>
<pre>
...
</pre>
</body>
</html>
which I seem not to be able to get rid of. What am I doing wrong..?
What I want to achieve is that I'd want to allow for gradually replacement
of the content for an existing site. Customers and search engines should
_not_ be redirected to new URLs. The new content, if available, seems to
be served OK, but the old content, which should be served when there is
not yet any new content available, is shown as source, inside those <pre>
tags.
Are there any obvious/better ways to get the functionality I hope to get ?
Thanks in advance,
Iosif Fettich
Re: who's putting that tag in the output...?
Posted by Iosif Fettich <if...@netsoft.ro>.
Hi André,
> I have not looked at your code in detail, but in general : there is
> nothing in Apache or mod_perl that will automatically and magically wrap
> any response in any html tag sequence.
I think/suppose so - that's why I'm getting nervuous about not being able
to see where this comes from.
> So the only reasonable explanation, is that it is your back-end server which
> generates these tags (or your browser ?).
back-end server, as far as I can tell.
> Why don't you put some tracing code in your handler, to dump what it really
> receives from the back-end ?
> Like :
>
> $r->log_error("going to get : $new_url")
> ...
> my $content = $response->content;
> $r->log_error("got : " . $content);
> ...
>
> and look in your Apache error log.
evrything looks OK in the log, I get the content I expect.
It's just that when I print() it, I see the whole thing as HTML source
code, as it is embedded already inside some <html><head/><body><pre>...and
the corresponding ending tags.
Thank you,
Iosif Fettich
Re: who's putting that tag in the output...?
Posted by André Warnier <aw...@ice-sa.com>.
Iosif Fettich wrote:
[...]
Hi.
I have not looked at your code in detail, but in general : there is
nothing in Apache or mod_perl that will automatically and magically wrap
any response in any html tag sequence.
So the only reasonable explanation, is that it is your back-end server
which generates these tags (or your browser ?).
Why don't you put some tracing code in your handler, to dump what it
really receives from the back-end ?
Like :
$r->log_error("going to get : $new_url")
...
my $content = $response->content;
$r->log_error("got : " . $content);
...
and look in your Apache error log.
You could also use lwp-request (comes with perl), to retrieve directly
what you think your handler is retrieving.
lwp-request, as the name indicates, uses the LWP module, just like your
handler does.
Re: who's putting that tag in the output...?
Posted by Iosif Fettich <if...@netsoft.ro>.
Hi Torsten,
> An ErrorDocument is an internal redirect. These REDIRECT_... environment
> variables are copied from the previous ($r->prev) request's
> $r->subprocess_env just by copying everything and prepending REDIRECT_
> to each key. So if the original request has an environment variable
> named REQUEST_URI the error document should have a REDIRECT_REQUEST_URI,
> see rename_original_env() in httpd-x.y/modules/http/http_request.c.
>
> Since REQUEST_URI is the standard CGI environment variable (see
> ap_add_cgi_vars() httpd-x.y/server/util_script.c) I'd take
> REDIRECT_REQUEST_URI.
As it turned out, I was (entirely) wrong when I thought it is
working. It was wishfull thinking - but not a real solution - neighter one
of the REQUEST_URI, REDIRECT_URL and/or REDIRECT_QUERY_STRING environment
variables seemed to be good enough for a mod_rewrite solution, or at least
I was unable to build one. (I just made some errors in testing and
repeatedly out- and out-out-commenting various httpd.conf setting, but it
wasn't _really_ working whein I thought it would).
Summing up what I have so far ( which might be incomplete or even wrong):
looking for a cheap/good/working solution for a way to solve what
http://httpd.apache.org/docs/2.2/rewrite/rewrite_guide_advanced.html
describes under the title "Redirect Failing URLs to Another Web Server",
but with the (it seems important) difference that I want to hide the new
server from the eyes of the customers and as such _proxy_ the failing
requests instead of redirecting, the given receipt
RewriteEngine on
RewriteCond %{REQUEST_URI} !-U
RewriteRule ^(.+) http://webserverB.dom/$1
shows up to NOT work when I attempted to make it
RewriteEngine on
RewriteCond %{REQUEST_URI} !-U
RewriteRule ^(.+) http://webserverB.dom/$1 [P]
Neither was I able to use the Error_Document trick you sugegsted and use
Rewrite on/with it.
I've given up my first attempt - the earlier in the thread shown
PerlResponse handler - as I was unable to output the Content-Type header
as 'text/html'; I haven't however tried the solution suggested with adding
an extra filter for the end phase to substitute the 'text/plain' that I
was seeing and which actually generated the initial question for this
thread and it's subject.
I finally went the 'standard way' [?] and added
ErrorDocument 404 /cgi-bin/404_to_oldserver.pl
in httpd.conf, making /cgi-bin/404_to_oldserver.pl to be
----------------------------
#!/usr/bin/perl
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
my $url = $ENV{'REQUEST_URI'};
$url = "http://OLD.SERVER.COM/$url" ;
my $response = $ua->get( $url );
my $body = $response->content;
my $h = $response->{'_headers'};
$h->push_header( 'Status' => $response->code );
my $header = $h->as_string;
print $header;
print "\n";
print $body;
1;
-----------------------------------------------
This way might have it's own special problems too, but at least it seems
to work OK so far and give me a start.
I'm still [a bit] convinced that a mod_perl solution might or should be
available and be both better and more effective, but I wasn't able to get
it working - even after spending much more effort than I thought initially
that it will take - and gave up for now.
Many thanks to all those that offered advice or help.
Iosif Fettich
PS. Firebug once again proved to be an invaluable resource in helping
understand what's up and find a solution.
Re: who's putting that tag in the output...?
Posted by Torsten Foertsch <to...@gmx.net>.
On Mon 23 Mar 2009, Iosif Fettich wrote:
> Using the (obvious...!) REDIRECT_URL as you suggested works! :)
An ErrorDocument is an internal redirect. These REDIRECT_... environment
variables are copied from the previous ($r->prev) request's
$r->subprocess_env just by copying everything and prepending REDIRECT_
to each key. So if the original request has an environment variable
named REQUEST_URI the error document should have a
REDIRECT_REQUEST_URI, see rename_original_env() in
httpd-x.y/modules/http/http_request.c.
Since REQUEST_URI is the standard CGI environment variable (see
ap_add_cgi_vars() httpd-x.y/server/util_script.c) I'd take
REDIRECT_REQUEST_URI.
Torsten
--
Need professional mod_perl support?
Just hire me: torsten.foertsch@gmx.net
Re: who's putting that tag in the output...?
Posted by Iosif Fettich <if...@netsoft.ro>.
Hi Torsten,
just following up my previous message:
> [...]
> REDIRECT_URL=/404.xxx
> GATEWAY_INTERFACE=CGI/1.1
> [...]
>
> (Nothing like REDIRECT_URI).
Using the (obvious...!) REDIRECT_URL as you suggested works! :)
I think there's just a new member in the 'Torsten, you're the greatest!'
club ;)
Many many thanks, once again.
Iosif Fettich
Re: who's putting that tag in the output...?
Posted by Iosif Fettich <if...@netsoft.ro>.
Hi Torsten,
>> ? It seems to work fine for the subrequest (status=200) ...?
>
> This is exactly the problem. The 404 is normally generated in the
> response phase from the default response handler. The subreq lookup
> won't check that. It does not run the subreq but only checks if after
> fixup there is still no error.
Hmm... Would that mean that the example receipt in the rewrite guide
simply doesn't work as given/explained ? For no one ...?!
> So the simplest solution for you would be an ErrorDocument I think.
>
> Something like this:
>
> ErrorDocument 404 /-/404 <- put an url here you are not using otherwise
> RewriteRule ^/-/404 http://...%{REDIRECT_URI} [P]
>
> Write a simple printenv and activate it as ErrorDocument first to get
> the right environment variable names (not sure if it is REDIRECT_URI).
> If you use mod_include that would be
>
> <!--#printenv -->
>
> in an .shtml file.
I must do something wrong again. Outcommenting RewriteEngine and setting
ErrorDocument 404 /404.shtml
with 404.shtml contaiining the above <!--#printenv --> shows
[...]
REDIRECT_URL=/404.xxx
GATEWAY_INTERFACE=CGI/1.1
SERVER_PROTOCOL=HTTP/1.1
REQUEST_METHOD=GET
QUERY_STRING=
REQUEST_URI=/404.xxx
SCRIPT_NAME=/404.shtml
DATE_LOCAL=Monday, 23-Mar-2009 19:49:31 EET
DATE_GMT=Monday, 23-Mar-2009 17:49:31 GMT
LAST_MODIFIED=Monday, 23-Mar-2009 19:30:26 EET
DOCUMENT_URI=/404.shtml
DOCUMENT_NAME=404.shtml
(Nothing like REDIRECT_URI).
So I believe it is actually REQUEST_URI I'm after ?
Now, re-enabling the RewriteEngine and setting
ErrorDocument 404 /-/404
RewriteLog /tmp/REWRITE.log
RewriteLogLevel 99
RewriteEngine on
RewriteRule ^/-/404 http://OLD_SITE.COM%{REQUEST_URI} [P]
brings me the _error_ page of the old site, and the rewrite log for
my http://ACTUAL_SITE.COM/index.php looks like
[rid#2b5dc09e5c68/initial] (2) init rewrite engine with requested uri /index.php
[rid#2b5dc09e5c68/initial] (3) applying pattern '^/-/404' to uri '/index.php'
[rid#2b5dc09e5c68/initial] (1) pass through /index.php
[rid#2b5dc09e4d48/initial/redir#1] (2) init rewrite engine with requested uri /-/404
[rid#2b5dc09e4d48/initial/redir#1] (3) applying pattern '^/-/404' to uri '/-/404'
[rid#2b5dc09e4d48/initial/redir#1] (2) rewrite '/-/404' -> 'http://OLD_SITE.COM/-/404'
[rid#2b5dc09e4d48/initial/redir#1] (2) forcing proxy-throughput with http://OLD_SITE.COM/-/404
[rid#2b5dc09e4d48/initial/redir#1] (1) go-ahead with proxy request proxy:http://OLD_SITE.COM/-/404 [OK]
Still not there yet, not sure how I could get what you mean by
REDIRECT_URI.
> This solution should work for GET/HEAD. POST requests are still a
> problem. If you use them we'll make it work without ErrorDocument.
I'm not sure if POSTs are used, that would be the next step then.
> BTW, why don't you use the file lookup (-f). That is much easier. Is
> your uri->file mapping so complicated?
I'd say yes, it's some CMS in the back and almost all of the content is
dynamic, most requests look like
http://MYDOAMIN.COM/index.php?qs_sect_id=3456 etc.
Many thanks,
Iosif Fettich
Re: who's putting that tag in the output...?
Posted by Torsten Foertsch <to...@gmx.net>.
On Mon 23 Mar 2009, Iosif Fettich wrote:
> Or is the [P]roxy flag not working as it should or as I expect it to
> ? It seems to work fine for the subrequest (status=200) ...?
This is exactly the problem. The 404 is normally generated in the
response phase from the default response handler. The subreq lookup
won't check that. It does not run the subreq but only checks if after
fixup there is still no error.
So the simplest solution for you would be an ErrorDocument I think.
Something like this:
ErrorDocument 404 /-/404 <- put an url here you are not using otherwise
RewriteRule ^/-/404 http://...%{REDIRECT_URI} [P]
Write a simple printenv and activate it as ErrorDocument first to get
the right environment variable names (not sure if it is REDIRECT_URI).
If you use mod_include that would be
<!--#printenv -->
in an .shtml file.
This solution should work for GET/HEAD. POST requests are still a
problem. If you use them we'll make it work without ErrorDocument. BTW,
why don't you use the file lookup (-f). That is much easier. Is your
uri->file mapping so complicated?
Torsten
--
Need professional mod_perl support?
Just hire me: torsten.foertsch@gmx.net
Re: who's putting that tag in the output...?
Posted by Iosif Fettich <if...@netsoft.ro>.
Hi Torsten,
On Mon, 23 Mar 2009, Torsten Foertsch wrote:
> On Mon 23 Mar 2009, Iosif Fettich wrote:
>> So it seems to be very, very easy. Still, when using the above
>> receipt like
>> RewriteEngine on
>> RewriteCond %{REQUEST_URI} !-U
>> RewriteRule ^\/(.+) http://OLDDOMAIN.COM/$1 [QSA,P]
>
> The engine tries to resolve the request uri with a subrequest. Hence it
> goes through this rule twice. Try to add "NS" (no subrequest) to the
> flags [QSA,P,NS].
I have; the log now shows
[rid#2ba8b0168af8/initial] (3) applying pattern '^\/(.+)' to uri '/index.php'
[rid#2ba8b04b9e18/subreq] (2) init rewrite engine with requested uri /index.php
[rid#2ba8b04b9e18/subreq] (1) pass through /index.php
[rid#2ba8b0168af8/initial] (5) RewriteCond URI (-U) check: path=/index.php -> status=200
[rid#2ba8b0168af8/initial] (4) RewriteCond: input='/index.php' pattern='!-U' => not-matched
[rid#2ba8b0168af8/initial] (1) pass through /index.php
which looks a buit different, but still leaves mw with the "The requested
URL /index.php was not found on this server." error.
I'm not really understanding what the log is saying.
Or is the [P]roxy flag not working as it should or as I expect it to ?
It seems to work fine for the subrequest (status=200) ...?
So far, I just cannot understand what's going on here or where to look and
what to try.
Many thanks,
Iosif Fettich
Re: who's putting that tag in the output...?
Posted by Torsten Foertsch <to...@gmx.net>.
On Mon 23 Mar 2009, Iosif Fettich wrote:
> So it seems to be very, very easy. Still, when using the above
> receipt like
> RewriteEngine on
> RewriteCond %{REQUEST_URI} !-U
> RewriteRule ^\/(.+) http://OLDDOMAIN.COM/$1 [QSA,P]
The engine tries to resolve the request uri with a subrequest. Hence it
goes through this rule twice. Try to add "NS" (no subrequest) to the
flags [QSA,P,NS].
Torsten
--
Need professional mod_perl support?
Just hire me: torsten.foertsch@gmx.net
Re: who's putting that tag in the output...?
Posted by André Warnier <aw...@ice-sa.com>.
Perrin Harkins wrote:
> On Mon, Mar 23, 2009 at 7:14 AM, Iosif Fettich <if...@netsoft.ro> wrote:
>> I've ommited printing headers explicitely :(
>
> HTTP won't work without headers.
>
>> Have to see when and how I should do this; simply inserting a
>>
>> $r->content_type( 'text/html' );
>>
>> before my
>>
>> $r->print( $content );
>>
>> seems to be a NOOP..
>
> Nope, it's not a NOOP. Maybe you're setting it too late.
>
You may also want to have a look at this previous thread :
http://marc.info/?l=apache-modperl&m=123072345912551&w=2
In the second or third message, there is a paragraph by Rainer Jung
which explains why setting the Content-Type may sometimes appear as not
working. There is also later on a solution.
This does not mean that what Perrin says above is wrong.
Apparently, with the Content-Type header, timing is really of the essence.
Re: who's putting that tag in the output...?
Posted by Iosif Fettich <if...@netsoft.ro>.
Hi Perrin,
>> I'm just not aware yet that I could check the outcome of a subrequest and
>> put some proxied response in place if the subrequest is unsuccessful. Isn't
>> mod-rewrite just a _request_ rewrite ?
>
> It can do just about anything:
> http://httpd.apache.org/docs/1.3/misc/rewriteguide.html
Looks like it should be simple. And still I can't get it do what I want.
There is actually an almost _exact_ FAQ-like answer to my problem in the
doc, http://httpd.apache.org/docs/2.2/rewrite/rewrite_guide_advanced.html:
---
Redirect Failing URLs to Another Web Server
Description:
A typical FAQ about URL rewriting is how to redirect failing requests
on webserver A to webserver B. Usually this is done via ErrorDocument CGI
scripts in Perl, but there is also a mod_rewrite solution. But note that
this performs more poorly than using an ErrorDocument CGI script!
Solution:
The first solution has the best performance but less flexibility, and
is less safe:
RewriteEngine on
RewriteCond /your/docroot/%{REQUEST_FILENAME} !-f
RewriteRule ^(.+) http://webserverB.dom/$1
The problem here is that this will only work for pages inside the
DocumentRoot. While you can add more Conditions (for instance to also
handle homedirs, etc.) there is a better variant:
RewriteEngine on
RewriteCond %{REQUEST_URI} !-U
RewriteRule ^(.+) http://webserverB.dom/$1
This uses the URL look-ahead feature of mod_rewrite. The result is
that this will work for all types of URLs and is safe. But it does have a
performance impact on the web server, because for every request there is
one more internal subrequest. So, if your web server runs on a powerful
CPU, use this one. If it is a slow machine, use the first approach or
better an ErrorDocument CGI script.
---
So it seems to be very, very easy. Still, when using the above receipt
like
RewriteEngine on
RewriteCond %{REQUEST_URI} !-U
RewriteRule ^\/(.+) http://OLDDOMAIN.COM/$1 [QSA,P]
instead of getting the proxied content, I get
---
Not Found
The requested URL /index.php was not found on this server.
---
for a GET request like http://mydomain.com/index.php and the rewrite log
(with RewriteLogLevel 9) looks like
[rid#2ad8dc3dfb98/initial] (2) init rewrite engine with requested uri /index.php
[rid#2ad8dc3dfb98/initial] (3) applying pattern '^\/(.+)' to uri '/index.php'
[rid#2ad8dc3e5bc8/subreq] (2) init rewrite engine with requested uri /index.php
[rid#2ad8dc3e5bc8/subreq] (3) applying pattern '^\/(.+)' to uri '/index.php'
[rid#2ad8dc3e5bc8/subreq] (4) RewriteCond: input='/index.php' pattern='!-U' => matched
[rid#2ad8dc3e5bc8/subreq] (2) rewrite '/index.php' -> 'http://OLDDOMAIN.COM/index.php'
[rid#2ad8dc3e5bc8/subreq] (2) forcing proxy-throughput with http://OLDDOMAIN.COM/index.php
[rid#2ad8dc3e5bc8/subreq] (1) go-ahead with proxy request proxy:http://OLDDOMAIN.COM/index.php [OK]
[rid#2ad8dc3dfb98/initial] (5) RewriteCond URI (-U) check: path=/index.php -> status=200
[rid#2ad8dc3dfb98/initial] (4) RewriteCond: input='/index.php' pattern='!-U' => not-matched
[rid#2ad8dc3dfb98/initial] (1) pass through /index.php
If I do the proxying unconditionally, like
RewriteEngine on
# RewriteCond %{REQUEST_URI} !-U
RewriteRule ^\/(.+) http://OLDDOMAIN.COM/$1 [QSA,P]
it works OK.
Any clou/idea of what might be wrong here ?
Many thanks,
Iosif Fettich
Re: who's putting that tag in the output...?
Posted by Perrin Harkins <pe...@elem.com>.
On Mon, Mar 23, 2009 at 7:14 AM, Iosif Fettich <if...@netsoft.ro> wrote:
> I've ommited printing headers explicitely :(
HTTP won't work without headers.
> Have to see when and how I should do this; simply inserting a
>
> $r->content_type( 'text/html' );
>
> before my
>
> $r->print( $content );
>
> seems to be a NOOP..
Nope, it's not a NOOP. Maybe you're setting it too late.
>> Everything you've shown so far could be done more efficiently by a couple
>> of lines of mod_rewrite.
>
> I'll re-read mod_rewrite then ;)
>
> I'm just not aware yet that I could check the outcome of a subrequest and
> put some proxied response in place if the subrequest is unsuccessful. Isn't
> mod-rewrite just a _request_ rewrite ?
It can do just about anything:
http://httpd.apache.org/docs/1.3/misc/rewriteguide.html
- Perrin
Re: who's putting that tag in the output...?
Posted by Iosif Fettich <if...@netsoft.ro>.
Hi Perrin,
> I don't see you printing any content type or other headers. Those
> aren't in $response->content.
I've ommited printing headers explicitely :(
Have to see when and how I should do this; simply inserting a
$r->content_type( 'text/html' );
before my
$r->print( $content );
seems to be a NOOP..
> Everything you've shown so far could be done more efficiently by a
> couple of lines of mod_rewrite.
I'll re-read mod_rewrite then ;)
I'm just not aware yet that I could check the outcome of a subrequest and
put some proxied response in place if the subrequest is unsuccessful.
Isn't mod-rewrite just a _request_ rewrite ?
Thanks,
Iosif Fettich
Re: who's putting that tag in the output...?
Posted by Perrin Harkins <pe...@elem.com>.
On Mon, Mar 23, 2009 at 4:10 AM, Iosif Fettich <if...@netsoft.ro> wrote:
> The problem is that what I want to be the handler's proxied response is
> actual embedded instead in an construct like
>
> <html>
> <head/>
> <body>
> <pre>
> ...
>
> </pre>
> </body>
> </html>
>
> which I seem not to be able to get rid of. What am I doing wrong..?
I don't see you printing any content type or other headers. Those
aren't in $response->content.
> Are there any obvious/better ways to get the functionality I hope to get ?
Everything you've shown so far could be done more efficiently by a
couple of lines of mod_rewrite.
- Perrin