You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Adam Woodworth <mi...@gmail.com> on 2007/11/29 17:05:27 UTC
[users@httpd] Getting mod_perl to run after mod_cache with Apache 2.2.6
Hi Folks,
I originally posted this message on the mod_perl mailing list, but I
did not receive any responses, so I'm posting here in the hopes that
someone can help me out.
My current LAMP stack is using Apache 2.0.54 and mod_perl 2.0.0-RC4,
and I'm doing something very much like what is mentioned in a previous
mod_perl mailing list post from 2 years ago, the thread of which can
be seen here:
http://www.gossamer-threads.com/lists/modperl/modperl/79672
In summary (taking from the above posting), here's a simple flow of
what we have happening:
1 - mod_cache [..got valid content in cache? If so, go to 4; if not, go to 2]
2 - mod_proxy [fetch content from origin web]
3 - mod_cache [content cacheable? If so, cache it locally]
4 - *MY FILTER*
5 - deflate
Using the same modifications mentioned in the above posting, we were
able to get mod_cache to run *before* mod_perl by changing mod_cache.c
so that the CACHE_SAVE and CACHE_OUT filters hook in at
AP_FTYPE_CONTENT_SET-2, and changing mod_perl.c so that mod_perl hooks
in at AP_FTYPE_CONTENT_SET-1. This solution is mentioned at the
bottom of the above posting.
But now I am upgrading our LAMP stack to Apache 2.2.6 and mod_perl
2.0.3, and I'm having trouble getting the above flow to work.
First off, mod_cache.c has changed so that instead of just CACHE_SAVE
and CACHE_OUT, there are also CACHE_SAVE_SUBREQ and CACHE_OUT_SUBREQ.
I'm not sure what the subrequests are really for, but here's what I've
been doing to try to get my desired flow to work:
The original Apache 2.2.6 filter order for mod_cache is:
CACHE_SAVE = AP_FTYPE_CONTENT_SET+1
CACHE_SAVE_SUBREQ = AP_FTYPE_CONTENT_SET-1
CACHE_OUT = AP_FTYPE_CONTENT_SET+1
CACHE_OUT_SUBREQ = AP_FTYPE_CONTENT_SET-1
The original mod_perl 2.0.3 filter order is:
MODPERL_REQUEST_OUTPUT = AP_FTYPE_RESOURCE
I've modified these to be:
Modified Apache 2.2.6 filter order for mod_cache:
CACHE_SAVE = AP_FTYPE_CONTENT_SET-2
CACHE_SAVE_SUBREQ = AP_FTYPE_CONTENT_SET-3
CACHE_OUT = AP_FTYPE_CONTENT_SET-2
CACHE_OUT_SUBREQ = AP_FTYPE_CONTENT_SET-3
Modified mod_perl 2.0.3 filter order:
MODPERL_REQUEST_OUTPUT = AP_FTYPE_CONTENT_SET-1
These modifications make it (theoretically from what I understand
about the filtering order numbers and my experience with the older
Apache and mod_perl) so that mod_perl runs after mod_cache, in the
same way that I was able to do this for Apache 2.0.54 and mod_perl
2.0.0-RC4. However it is not working as I expected -- mod_cache
appears to not be returning the body of the content before my mod_perl
filter is run. So the user hits the site, mod_cache sees the page is
cached, mod_perl is run but doesn't see any content, only the headers,
then somewhere down the line mod_cache must be serving up the cached
content after mod_perl.
So, I'm stumped. Can anyone point me in the right direction?
Thanks!
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Getting mod_perl to run after mod_cache with Apache 2.2.6
Posted by Adam Woodworth <mi...@gmail.com>.
After some more digging, I've figured out (mostly) what is going on.
It turns out that our mod_perl filter actually does get the body
content from mod_cache -- originally I thought that our filter wasn't
seeing the body at all. But, the Content-Type header is missing.
In cache_storage.c, around line 120 (in the 2.2.6 source code) there
is this code:
apr_table_unset(h->resp_hdrs, "Content-Type");
/*
* Also unset possible Content-Type headers in r->headers_out and
* r->err_headers_out as they may be different to what we have received
* from the cache.
* Actually they are not needed as r->content_type set by
* ap_set_content_type above will be used in the store_headers functions
* of the storage providers as a fallback and the HTTP_HEADER filter
* does overwrite the Content-Type header with r->content_type anyway.
*/
apr_table_unset(r->headers_out, "Content-Type");
apr_table_unset(r->err_headers_out, "Content-Type");
While I'm not clear why this is happening, mod_cache is clearing out
the Content-Type header from the response. So, our mod_perl filter
doesn't see the content-type filter and I think that content-type gets
set again by Apache somewhere after all the filters run. Since
Content-Type is missing during mod_perl's running, and our mod_perl
filter triggers off of Content-Type being matching certain types
(normally, text/html), our filter doesn't do anything to the body
because the Content-Type is missing.
I was able to "fix" this by commenting out the
apr_table_unset(h->resp_hdrs, ... line above.
Does anyone know the affect that this could have on the rest of
mod_cache/etc (any ill effects, cases where this would break
something, etc?), and why exactly the Content-Type is removed in the
first place my mod_cache? I see that the comments above try to
explain it, but it doesn't quite make sense to me.
Cheers,
Adam
On Nov 29, 2007 2:27 PM, Adam Woodworth <mi...@gmail.com> wrote:
> Hi Nick,
>
> Thanks for the response!
>
> > > 1 - mod_cache [..got valid content in cache? If so, go to 4; if not,
> > > go to 2] 2 - mod_proxy [fetch content from origin web]
> > > 3 - mod_cache [content cacheable? If so, cache it locally]
> > > 4 - *MY FILTER*
> > > 5 - deflate
> >
> > That makes sense if and only if you want to repeat your-filter and
> > DEFLATE on every request rather than cache the ready-processed contents.
>
> Yup, due to the nature of our product, our mod_perl filter doesn't do
> the same thing for each request to the same page.
>
> > directive. The problem with that is that mod_cache does its own thing.
>
> Could you ellaborate on what you mean by mod_cache doing its own thing?
>
> The problem I seem to be running into is that when my mod_perl filter
> runs, mod_cache has served up the headers of the file, but the content
> of the file is empty, so mod_perl has nothing to process. Then
> somewhere down the line mod_cache must serve up the rest of the file.
> Do you know how this might be happening, or if it's just the way
> mod_cache operates? Is there perhaps some interaction with mod_proxy
> where mod_cache only spits out the cached data at some particular
> point in the filter chain...
>
> I'm going to dig into mod_cache deeper now...
>
> Thanks!
> Adam
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Getting mod_perl to run after mod_cache with Apache 2.2.6
Posted by Adam Woodworth <mi...@gmail.com>.
Hi Nick,
Thanks for the response!
> > 1 - mod_cache [..got valid content in cache? If so, go to 4; if not,
> > go to 2] 2 - mod_proxy [fetch content from origin web]
> > 3 - mod_cache [content cacheable? If so, cache it locally]
> > 4 - *MY FILTER*
> > 5 - deflate
>
> That makes sense if and only if you want to repeat your-filter and
> DEFLATE on every request rather than cache the ready-processed contents.
Yup, due to the nature of our product, our mod_perl filter doesn't do
the same thing for each request to the same page.
> directive. The problem with that is that mod_cache does its own thing.
Could you ellaborate on what you mean by mod_cache doing its own thing?
The problem I seem to be running into is that when my mod_perl filter
runs, mod_cache has served up the headers of the file, but the content
of the file is empty, so mod_perl has nothing to process. Then
somewhere down the line mod_cache must serve up the rest of the file.
Do you know how this might be happening, or if it's just the way
mod_cache operates? Is there perhaps some interaction with mod_proxy
where mod_cache only spits out the cached data at some particular
point in the filter chain...
I'm going to dig into mod_cache deeper now...
Thanks!
Adam
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Getting mod_perl to run after mod_cache with
Apache 2.2.6
Posted by Nick Kew <ni...@webthing.com>.
On Thu, 29 Nov 2007 11:05:27 -0500
"Adam Woodworth" <mi...@gmail.com> wrote:
> In summary (taking from the above posting), here's a simple flow of
> what we have happening:
>
> 1 - mod_cache [..got valid content in cache? If so, go to 4; if not,
> go to 2] 2 - mod_proxy [fetch content from origin web]
> 3 - mod_cache [content cacheable? If so, cache it locally]
> 4 - *MY FILTER*
> 5 - deflate
That makes sense if and only if you want to repeat your-filter and
DEFLATE on every request rather than cache the ready-processed contents.
In principle, mod_filter enables you to control the ordering of output
filters, so you could just configure the above with a FilterChain
directive. The problem with that is that mod_cache does its own thing.
I suspect the easiest solution may be to use mod_cache's filters
but without using mod_cache's handler. I'm not sure how much work
that'll involve: presumably mod_cache can largely be cloned, except
for the insert-filters bit. Doing it in Perl with mod_perl should
be an option.
--
Nick Kew
Application Development with Apache - the Apache Modules Book
http://www.apachetutor.org/
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org