You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Solutio at Gmail <so...@gmail.com> on 2009/02/01 08:11:15 UTC
[users@httpd] CAPTURING SERVER RESPONSE
Hi Everybody,
I wonder if there is a way to capture the entire Apache response, including its body, on the server side, and store it in a file, so we could check what the user actually gets by viewing the contents in a browser? We keep requests for audit purposes, but the data changes quickly, and the same request can result in a different response over time. I looked for ideas on Google and CPAN, but haven't been able to find any definite answer so far.
Thank you in advance.
Re: [users@httpd] Capturing server response
Posted by anson ho <ho...@gmail.com>.
for apache 2.0.x, I think apache filter is the way to go.
http://httpd.apache.org/docs/2.0/filter.html
On Sat, Feb 7, 2009 at 1:43 PM, Solutio at Gmail <so...@gmail.com> wrote:
> Thanks, I'll take a closer look at the module. But my first impression is
> that it works with Apache 1.3.x, and we have already switched to Apache
> 2.0.x.
>
> BTW, where the actual code of handlers and filters is supposed to be
> deployed? I'm afraid the documentation omits this, perhaps, assuming it's
> trivial.
>
> --------------------------------------------------
> From: "anson ho" <ho...@gmail.com>
> Sent: Friday, February 06, 2009 9:53 PM
> To: <us...@httpd.apache.org>
> Subject: Re: [users@httpd] Capturing server response
>
>> mod_trace_output do exactly what you want, except that headers are not
>> saved. But you can do it by myself with the source included.
>>
>> http://trace-output.sourceforge.net/
>>
>>
>> On Mon, Feb 2, 2009 at 2:56 AM, Solutio at Gmail <so...@gmail.com>
>> wrote:
>>>
>>> Thank you for the advice and introduction to the forum rules. I
>>> "lowercased"
>>> the subject.
>>>
>>> As for the options available, we would like to avoid sending the same
>>> request to the web server (Apache) if possible. Sure, LWP could emulate a
>>> web client, but our goal is to fork the very same HTTP response that
>>> Apache
>>> sends to the browser, into a file, rather than repeating the request. One
>>> of
>>> the reasons being that as I mentioned, the data in the response can
>>> already
>>> be different.
>>>
>>> Thanks again.
>>>
>>> --------------------------------------------------
>>> From: "André Warnier" <aw...@ice-sa.com>
>>> Sent: Sunday, February 01, 2009 3:46 AM
>>> To: <us...@httpd.apache.org>
>>> Subject: Re: [users@httpd] CAPTURING SERVER RESPONSE
>>>
>>>> Solutio at Gmail wrote:
>>>>>
>>>>> Hi Everybody,
>>>>>
>>>>> I wonder if there is a way to capture the entire Apache response,
>>>>> including its body, on the server side, and store it in a file, so we
>>>>> could
>>>>> check what the user actually gets by viewing the contents in a browser?
>>>>> We
>>>>> keep requests for audit purposes, but the data changes quickly, and the
>>>>> same
>>>>> request can result in a different response over time. I looked for
>>>>> ideas on
>>>>> Google and CPAN, but haven't been able to find any definite answer so
>>>>> far.
>>>>>
>>>> Hi.
>>>> First, just a small item relating to "etiquette" (politeness etc..) in
>>>> forums like this one : writing something in CAPITALS is generally
>>>> considered
>>>> as equivalent to shouting. So it's not a very good idea to use that in
>>>> the
>>>> subject of your posting.
>>>>
>>>> About your question :
>>>> There are may ways to capture the response of a HTTP server. The easiest
>>>> ones consist of programs that can act as a "client" (like a browser),
>>>> and
>>>> save the answer from the server to a disk file.
>>>> If you have perl installed wherever, I suggest to have a look at the
>>>> "lwp-request" command, which is very flexible and allows you to tailor
>>>> what
>>>> you send to the server as well as what you want to see (or save) at the
>>>> client level.
>>>> In the simplest form :
>>>> lwp-request -m GET "http://www.myserver.com/myurl"
>>>> or
>>>> lwp-request -m GET "http://www.myserver.com/myurl" > saved_file.html
>>>>
>>>> I suggest the above because it will run on whatever platform, as long as
>>>> perl is inatalled on it.
>>>>
>>>> There are a bunch of other programs that can do similar things,
>>>> including
>>>> saving a whole website if you wish. Look at "curl", "wget" etc.. Their
>>>> availability may depend on the platform.
>>>>
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> The official User-To-User support forum of the Apache HTTP Server
>>>> Project.
>>>> See <URL:http://httpd.apache.org/userslist.html> for more info.
>>>> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>>>> " from the digest: users-digest-unsubscribe@httpd.apache.org
>>>> For additional commands, e-mail: users-help@httpd.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> The official User-To-User support forum of the Apache HTTP Server
>>> Project.
>>> See <URL:http://httpd.apache.org/userslist.html> for more info.
>>> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>>> " from the digest: users-digest-unsubscribe@httpd.apache.org
>>> For additional commands, e-mail: users-help@httpd.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> The official User-To-User support forum of the Apache HTTP Server Project.
>> See <URL:http://httpd.apache.org/userslist.html> for more info.
>> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>> " from the digest: users-digest-unsubscribe@httpd.apache.org
>> For additional commands, e-mail: users-help@httpd.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Capturing server response
Posted by Solutio at Gmail <so...@gmail.com>.
Thanks, I'll take a closer look at the module. But my first impression is
that it works with Apache 1.3.x, and we have already switched to Apache
2.0.x.
BTW, where the actual code of handlers and filters is supposed to be
deployed? I'm afraid the documentation omits this, perhaps, assuming it's
trivial.
--------------------------------------------------
From: "anson ho" <ho...@gmail.com>
Sent: Friday, February 06, 2009 9:53 PM
To: <us...@httpd.apache.org>
Subject: Re: [users@httpd] Capturing server response
> mod_trace_output do exactly what you want, except that headers are not
> saved. But you can do it by myself with the source included.
>
> http://trace-output.sourceforge.net/
>
>
> On Mon, Feb 2, 2009 at 2:56 AM, Solutio at Gmail <so...@gmail.com>
> wrote:
>> Thank you for the advice and introduction to the forum rules. I
>> "lowercased"
>> the subject.
>>
>> As for the options available, we would like to avoid sending the same
>> request to the web server (Apache) if possible. Sure, LWP could emulate a
>> web client, but our goal is to fork the very same HTTP response that
>> Apache
>> sends to the browser, into a file, rather than repeating the request. One
>> of
>> the reasons being that as I mentioned, the data in the response can
>> already
>> be different.
>>
>> Thanks again.
>>
>> --------------------------------------------------
>> From: "André Warnier" <aw...@ice-sa.com>
>> Sent: Sunday, February 01, 2009 3:46 AM
>> To: <us...@httpd.apache.org>
>> Subject: Re: [users@httpd] CAPTURING SERVER RESPONSE
>>
>>> Solutio at Gmail wrote:
>>>>
>>>> Hi Everybody,
>>>>
>>>> I wonder if there is a way to capture the entire Apache response,
>>>> including its body, on the server side, and store it in a file, so we
>>>> could
>>>> check what the user actually gets by viewing the contents in a browser?
>>>> We
>>>> keep requests for audit purposes, but the data changes quickly, and the
>>>> same
>>>> request can result in a different response over time. I looked for
>>>> ideas on
>>>> Google and CPAN, but haven't been able to find any definite answer so
>>>> far.
>>>>
>>> Hi.
>>> First, just a small item relating to "etiquette" (politeness etc..) in
>>> forums like this one : writing something in CAPITALS is generally
>>> considered
>>> as equivalent to shouting. So it's not a very good idea to use that in
>>> the
>>> subject of your posting.
>>>
>>> About your question :
>>> There are may ways to capture the response of a HTTP server. The easiest
>>> ones consist of programs that can act as a "client" (like a browser),
>>> and
>>> save the answer from the server to a disk file.
>>> If you have perl installed wherever, I suggest to have a look at the
>>> "lwp-request" command, which is very flexible and allows you to tailor
>>> what
>>> you send to the server as well as what you want to see (or save) at the
>>> client level.
>>> In the simplest form :
>>> lwp-request -m GET "http://www.myserver.com/myurl"
>>> or
>>> lwp-request -m GET "http://www.myserver.com/myurl" > saved_file.html
>>>
>>> I suggest the above because it will run on whatever platform, as long as
>>> perl is inatalled on it.
>>>
>>> There are a bunch of other programs that can do similar things,
>>> including
>>> saving a whole website if you wish. Look at "curl", "wget" etc.. Their
>>> availability may depend on the platform.
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> The official User-To-User support forum of the Apache HTTP Server
>>> Project.
>>> See <URL:http://httpd.apache.org/userslist.html> for more info.
>>> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>>> " from the digest: users-digest-unsubscribe@httpd.apache.org
>>> For additional commands, e-mail: users-help@httpd.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> The official User-To-User support forum of the Apache HTTP Server
>> Project.
>> See <URL:http://httpd.apache.org/userslist.html> for more info.
>> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>> " from the digest: users-digest-unsubscribe@httpd.apache.org
>> For additional commands, e-mail: users-help@httpd.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Capturing server response
Posted by anson ho <ho...@gmail.com>.
mod_trace_output do exactly what you want, except that headers are not
saved. But you can do it by myself with the source included.
http://trace-output.sourceforge.net/
On Mon, Feb 2, 2009 at 2:56 AM, Solutio at Gmail <so...@gmail.com> wrote:
> Thank you for the advice and introduction to the forum rules. I "lowercased"
> the subject.
>
> As for the options available, we would like to avoid sending the same
> request to the web server (Apache) if possible. Sure, LWP could emulate a
> web client, but our goal is to fork the very same HTTP response that Apache
> sends to the browser, into a file, rather than repeating the request. One of
> the reasons being that as I mentioned, the data in the response can already
> be different.
>
> Thanks again.
>
> --------------------------------------------------
> From: "André Warnier" <aw...@ice-sa.com>
> Sent: Sunday, February 01, 2009 3:46 AM
> To: <us...@httpd.apache.org>
> Subject: Re: [users@httpd] CAPTURING SERVER RESPONSE
>
>> Solutio at Gmail wrote:
>>>
>>> Hi Everybody,
>>>
>>> I wonder if there is a way to capture the entire Apache response,
>>> including its body, on the server side, and store it in a file, so we could
>>> check what the user actually gets by viewing the contents in a browser? We
>>> keep requests for audit purposes, but the data changes quickly, and the same
>>> request can result in a different response over time. I looked for ideas on
>>> Google and CPAN, but haven't been able to find any definite answer so far.
>>>
>> Hi.
>> First, just a small item relating to "etiquette" (politeness etc..) in
>> forums like this one : writing something in CAPITALS is generally considered
>> as equivalent to shouting. So it's not a very good idea to use that in the
>> subject of your posting.
>>
>> About your question :
>> There are may ways to capture the response of a HTTP server. The easiest
>> ones consist of programs that can act as a "client" (like a browser), and
>> save the answer from the server to a disk file.
>> If you have perl installed wherever, I suggest to have a look at the
>> "lwp-request" command, which is very flexible and allows you to tailor what
>> you send to the server as well as what you want to see (or save) at the
>> client level.
>> In the simplest form :
>> lwp-request -m GET "http://www.myserver.com/myurl"
>> or
>> lwp-request -m GET "http://www.myserver.com/myurl" > saved_file.html
>>
>> I suggest the above because it will run on whatever platform, as long as
>> perl is inatalled on it.
>>
>> There are a bunch of other programs that can do similar things, including
>> saving a whole website if you wish. Look at "curl", "wget" etc.. Their
>> availability may depend on the platform.
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> The official User-To-User support forum of the Apache HTTP Server Project.
>> See <URL:http://httpd.apache.org/userslist.html> for more info.
>> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>> " from the digest: users-digest-unsubscribe@httpd.apache.org
>> For additional commands, e-mail: users-help@httpd.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Capturing server response
Posted by Solutio at Gmail <so...@gmail.com>.
Thanks again for your help. It would have taken me countless hours to get to
this point on my own...
--------------------------------------------------
From: "André Warnier" <aw...@ice-sa.com>
Sent: Sunday, February 01, 2009 2:41 PM
To: <us...@httpd.apache.org>
Subject: Re: [users@httpd] Capturing server response
> Solutio at Gmail wrote:
>> Gotcha. I haven't had to do that before, but I am sure I will figure that
>> out. We will try this solution as well as setting up a proxy, to see
>> which will have the least impact on the overall performance. Thanks
>> heaps.
>>
> If you need more specific help with mod_perl, I suggest to try the
> mod_perl list. You will find subscription information at
> "http://perl.apache.org".
>
> By the way, if you go here :
> http://perl.apache.org/docs/2.0/user/handlers/filters.html#Setting_the_Content_Length_Header_in_Request_Output_Filters
> you should find an example that is just about what you need.
> As shown, it modifies the Content-Length header, but you would just need
> to modify this section in the example :
> if ($seen_eos) {
> my $len = length $data;
> $filter->r->headers_out->set('Content-Length', $len);
> $filter->print($data) if $data;
> }..
>
> to do something like
> if ($seen_eos) {
> my $r_headers = $filter->r->headers_out;
> (log the headers) + (log $data)
> $filter->print($data) if $data; # pass the data further
> }..
>
> This is not the most efficient way, and ok only as long as your response
> data is not too large (because it buffers it all) but if this is your
> first try at mod_perl, it would be an easy starting point.
>
> In various places there, it is mentioned that a Request output filter (as
> opposed to a Connection output filter, which is harder to write) has no
> access to the HTTP response headers. That is true in the sense that the
> response headers do not really pass through the filter as data (only the
> response body does). But it still /can/ get the response headers (to log
> them e.g.) via $filter->r->headers_out().
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Capturing server response
Posted by André Warnier <aw...@ice-sa.com>.
Solutio at Gmail wrote:
> Gotcha. I haven't had to do that before, but I am sure I will figure
> that out. We will try this solution as well as setting up a proxy, to
> see which will have the least impact on the overall performance. Thanks
> heaps.
>
If you need more specific help with mod_perl, I suggest to try the
mod_perl list. You will find subscription information at
"http://perl.apache.org".
By the way, if you go here :
http://perl.apache.org/docs/2.0/user/handlers/filters.html#Setting_the_Content_Length_Header_in_Request_Output_Filters
you should find an example that is just about what you need.
As shown, it modifies the Content-Length header, but you would just need
to modify this section in the example :
if ($seen_eos) {
my $len = length $data;
$filter->r->headers_out->set('Content-Length', $len);
$filter->print($data) if $data;
}..
to do something like
if ($seen_eos) {
my $r_headers = $filter->r->headers_out;
(log the headers) + (log $data)
$filter->print($data) if $data; # pass the data further
}..
This is not the most efficient way, and ok only as long as your response
data is not too large (because it buffers it all) but if this is your
first try at mod_perl, it would be an easy starting point.
In various places there, it is mentioned that a Request output filter
(as opposed to a Connection output filter, which is harder to write) has
no access to the HTTP response headers. That is true in the sense that
the response headers do not really pass through the filter as data (only
the response body does). But it still /can/ get the response headers
(to log them e.g.) via $filter->r->headers_out().
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Capturing server response
Posted by André Warnier <aw...@ice-sa.com>.
Eric Covener wrote:
> On Sun, Feb 1, 2009 at 3:08 PM, Solutio at Gmail <so...@gmail.com> wrote:
>> Gotcha. I haven't had to do that before, but I am sure I will figure that
>> out. We will try this solution as well as setting up a proxy, to see which
>> will have the least impact on the overall performance. Thanks heaps.
>
> Maybe mod_disk_cache is worth looking at
>
Ooh, sneaky. Because it will save the served pages to disk ?
But it would not save the response headers, would it ?
And anyway, if the content changes at each request ...
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Capturing server response
Posted by Eric Covener <co...@gmail.com>.
On Sun, Feb 1, 2009 at 3:08 PM, Solutio at Gmail <so...@gmail.com> wrote:
> Gotcha. I haven't had to do that before, but I am sure I will figure that
> out. We will try this solution as well as setting up a proxy, to see which
> will have the least impact on the overall performance. Thanks heaps.
Maybe mod_disk_cache is worth looking at
--
Eric Covener
covener@gmail.com
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Capturing server response
Posted by Solutio at Gmail <so...@gmail.com>.
Gotcha. I haven't had to do that before, but I am sure I will figure that
out. We will try this solution as well as setting up a proxy, to see which
will have the least impact on the overall performance. Thanks heaps.
--------------------------------------------------
From: "André Warnier" <aw...@ice-sa.com>
Sent: Sunday, February 01, 2009 1:12 PM
To: <us...@httpd.apache.org>
Subject: Re: [users@httpd] Capturing server response
> Solutio at Gmail wrote:
>> Thank you for the advice and introduction to the forum rules. I
>> "lowercased" the subject.
>>
>> As for the options available, we would like to avoid sending the same
>> request to the web server (Apache) if possible. Sure, LWP could emulate a
>> web client, but our goal is to fork the very same HTTP response that
>> Apache sends to the browser, into a file, rather than repeating the
>> request. One of the reasons being that as I mentioned, the data in the
>> response can already be different.
>>
> Allright.
> First, you could have a look at
> http://httpd.apache.org/docs/2.2/mod/mod_dumpio.html
> but I don't think this is really what you want.
>
> If you want to be more selective, and choose to which file you are dumping
> these responses, I think you're going to have to use (and probably create)
> your own output filter.
>
> For me, the easiest way would be to do that using mod_perl.
> Have a look here :
> http://perl.apache.org/docs/2.0/api/Apache2/Filter.html
> and here
> http://perl.apache.org/docs/2.0/user/handlers/filters.html#Adding_OutputFilters_Dynamically
>
> Is that in your league ?
>
> Basically, the idea would be that you have a first module that is inserted
> somewhere early in the request processing, and based on some request
> characteristic, decides if yes or no this request's output should be
> logged. You probably do not want to do this permanently, not for all
> requests, because (a) it would have a major impact on server performance
> and (b) it would generate huge output volumes.
> (That module could just be mod_setenvif, to set a variable if you want
> logging).
>
> Second, you would have an output filter that actually does the logging
> work, if it has been turned on by the module above.
> mod_perl allows you to dynamically insert an output filter just for the
> current request, or not, depending.
> On the other hand, if your filter is configured to be there statically,
> then you would want to set some value whereby it can just return DECLINE
> to Apache, so that it will be bypassed for the current request.
>
> Still with me ?
>
>
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Capturing server response
Posted by André Warnier <aw...@ice-sa.com>.
Solutio at Gmail wrote:
> Thank you for the advice and introduction to the forum rules. I
> "lowercased" the subject.
>
> As for the options available, we would like to avoid sending the same
> request to the web server (Apache) if possible. Sure, LWP could emulate
> a web client, but our goal is to fork the very same HTTP response that
> Apache sends to the browser, into a file, rather than repeating the
> request. One of the reasons being that as I mentioned, the data in the
> response can already be different.
>
Allright.
First, you could have a look at
http://httpd.apache.org/docs/2.2/mod/mod_dumpio.html
but I don't think this is really what you want.
If you want to be more selective, and choose to which file you are
dumping these responses, I think you're going to have to use (and
probably create) your own output filter.
For me, the easiest way would be to do that using mod_perl.
Have a look here :
http://perl.apache.org/docs/2.0/api/Apache2/Filter.html
and here
http://perl.apache.org/docs/2.0/user/handlers/filters.html#Adding_OutputFilters_Dynamically
Is that in your league ?
Basically, the idea would be that you have a first module that is
inserted somewhere early in the request processing, and based on some
request characteristic, decides if yes or no this request's output
should be logged. You probably do not want to do this permanently, not
for all requests, because (a) it would have a major impact on server
performance and (b) it would generate huge output volumes.
(That module could just be mod_setenvif, to set a variable if you want
logging).
Second, you would have an output filter that actually does the logging
work, if it has been turned on by the module above.
mod_perl allows you to dynamically insert an output filter just for the
current request, or not, depending.
On the other hand, if your filter is configured to be there statically,
then you would want to set some value whereby it can just return DECLINE
to Apache, so that it will be bypassed for the current request.
Still with me ?
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Capturing server response
Posted by Solutio at Gmail <so...@gmail.com>.
Thank you for the advice and introduction to the forum rules. I "lowercased"
the subject.
As for the options available, we would like to avoid sending the same
request to the web server (Apache) if possible. Sure, LWP could emulate a
web client, but our goal is to fork the very same HTTP response that Apache
sends to the browser, into a file, rather than repeating the request. One of
the reasons being that as I mentioned, the data in the response can already
be different.
Thanks again.
--------------------------------------------------
From: "André Warnier" <aw...@ice-sa.com>
Sent: Sunday, February 01, 2009 3:46 AM
To: <us...@httpd.apache.org>
Subject: Re: [users@httpd] CAPTURING SERVER RESPONSE
> Solutio at Gmail wrote:
>> Hi Everybody,
>>
>> I wonder if there is a way to capture the entire Apache response,
>> including its body, on the server side, and store it in a file, so we
>> could check what the user actually gets by viewing the contents in a
>> browser? We keep requests for audit purposes, but the data changes
>> quickly, and the same request can result in a different response over
>> time. I looked for ideas on Google and CPAN, but haven't been able to
>> find any definite answer so far.
>>
> Hi.
> First, just a small item relating to "etiquette" (politeness etc..) in
> forums like this one : writing something in CAPITALS is generally
> considered as equivalent to shouting. So it's not a very good idea to use
> that in the subject of your posting.
>
> About your question :
> There are may ways to capture the response of a HTTP server. The easiest
> ones consist of programs that can act as a "client" (like a browser), and
> save the answer from the server to a disk file.
> If you have perl installed wherever, I suggest to have a look at the
> "lwp-request" command, which is very flexible and allows you to tailor
> what you send to the server as well as what you want to see (or save) at
> the client level.
> In the simplest form :
> lwp-request -m GET "http://www.myserver.com/myurl"
> or
> lwp-request -m GET "http://www.myserver.com/myurl" > saved_file.html
>
> I suggest the above because it will run on whatever platform, as long as
> perl is inatalled on it.
>
> There are a bunch of other programs that can do similar things, including
> saving a whole website if you wish. Look at "curl", "wget" etc.. Their
> availability may depend on the platform.
>
>
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] CAPTURING SERVER RESPONSE
Posted by André Warnier <aw...@ice-sa.com>.
Solutio at Gmail wrote:
> Hi Everybody,
>
> I wonder if there is a way to capture the entire Apache response, including its body, on the server side, and store it in a file, so we could check what the user actually gets by viewing the contents in a browser? We keep requests for audit purposes, but the data changes quickly, and the same request can result in a different response over time. I looked for ideas on Google and CPAN, but haven't been able to find any definite answer so far.
>
Hi.
First, just a small item relating to "etiquette" (politeness etc..) in
forums like this one : writing something in CAPITALS is generally
considered as equivalent to shouting. So it's not a very good idea to
use that in the subject of your posting.
About your question :
There are may ways to capture the response of a HTTP server. The easiest
ones consist of programs that can act as a "client" (like a browser),
and save the answer from the server to a disk file.
If you have perl installed wherever, I suggest to have a look at the
"lwp-request" command, which is very flexible and allows you to tailor
what you send to the server as well as what you want to see (or save) at
the client level.
In the simplest form :
lwp-request -m GET "http://www.myserver.com/myurl"
or
lwp-request -m GET "http://www.myserver.com/myurl" > saved_file.html
I suggest the above because it will run on whatever platform, as long as
perl is inatalled on it.
There are a bunch of other programs that can do similar things,
including saving a whole website if you wish. Look at "curl", "wget"
etc.. Their availability may depend on the platform.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org