You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by "Paul J. Reder" <re...@remulak.net> on 2002/10/01 22:07:09 UTC
Speaking of pipes from cgis causing trouble...
I've been working on the caching code and ran across a core dump...
A particular file contains an SSI call to a cgi. The cgi causes a pipe
bucket to pass down the chain. cache_in_filter tries to save the bucket
away and core dumps.
Since a pipe bucket can be of any length, and could take any amount of
time to complete, I would assume that the cache code should decide not
to cache any response with a pipe bucket, correct?
--
Paul J. Reder
-----------------------------------------------------------
"The strength of the Constitution lies entirely in the determination of each
citizen to defend it. Only if every single citizen feels duty bound to do
his share in this defense are the constitutional rights secure."
-- Albert Einstein
Re: Speaking of pipes from cgis causing trouble...
Posted by Cliff Woolley <jw...@virginia.edu>.
On Tue, 1 Oct 2002, Cliff Woolley wrote:
> Indeterminate length is indicated by bkt->length == -1.
Oops, I meant to delete this sentence along with the rest of that thought.
In this context, checking ->length for -1 is not useful because there
might be known-length buckets you can't copy. So ignore this sentence and
just go with the checking apr_bucket_copy() for APR_ENOTIMPL thing. :)
Re: Speaking of pipes from cgis causing trouble...
Posted by Cliff Woolley <jw...@virginia.edu>.
On Tue, 1 Oct 2002, Paul J. Reder wrote:
> Are there any other bucket types to worry about (i.e. that can't just be
> copied)?
Currently no, but your code shouldn't have that kind of knowledge. There
might be third-party buckets of this sort. Indeterminate length is
indicated by bkt->length == -1. Just do an rv=apr_bucket_copy(bkt,©);
and if rv comes back as APR_ENOTIMPL, you know you have to read from the
bucket and try again.
--Cliff
Re: Speaking of pipes from cgis causing trouble...
Posted by "Paul J. Reder" <re...@remulak.net>.
William A. Rowe, Jr. wrote:
> At 03:27 PM 10/1/2002, Paul J. Reder wrote:
>
>
>>Ryan Bloom wrote:
>>
>>
>>>On Tue, 1 Oct 2002, Paul J. Reder wrote:
>>>
>>>
>>>>I've been working on the caching code and ran across a core dump...
>>>>
>>>>A particular file contains an SSI call to a cgi. The cgi causes a pipe
>>>>bucket to pass down the chain. cache_in_filter tries to save the bucket
>>>>away and core dumps.
>>>>
>>>>Since a pipe bucket can be of any length, and could take any amount of
>>>>time to complete, I would assume that the cache code should decide not
>>>>to cache any response with a pipe bucket, correct?
>>>>
>>>Not necessarily. The cache code should stream the data to the cache, and
>>>allow the data to also stream to the core_output_filter.
>>>
>>Until it reaches the specified max cache size?
>>
>>So instead of trying to just copy and insert the pipe bucket, it should read
>>
>>from the pipe and add buckets to the cache (and forward copies on)? Seems
>
>>like it would defeat some of the sendfile optimizations and such... Is it worth
>>the code and lost optimizations to read/store/pass the piped data?
>>
>
> Not if you read off the headers, determine cacheability, and then send the
> remaining pipe bucket down and ignore it when we can't cache the result
> anyways. If the resource can be cached, the penalty for reading and then
> storing + sending is nothing when you weigh serving out of the cache,
> next time around.
Well, considering that we wouldn't even be in the core dumping code if it hadn't
passed the cacheability checks, I think *this* part is safe. My concern was more
along the lines of, how much are we worried that reading and storing pipe buckets
will result in wasted cycles when the cache entry has to be tossed after exceeding
the specified maximum size. Is the potential cache gain worth the price of reading,
storing, and passing the pipe contents given the potential for having to toss it
for growing too big?
If the probability of tossing is low, or largely under user configuration control,
then I'll adjust the cache code to be able to read and store pipe and socket bucket
contents. Are there any other bucket types to worry about (i.e. that can't just be
copied)?
--
Paul J. Reder
-----------------------------------------------------------
"The strength of the Constitution lies entirely in the determination of each
citizen to defend it. Only if every single citizen feels duty bound to do
his share in this defense are the constitutional rights secure."
-- Albert Einstein
Re: Speaking of pipes from cgis causing trouble...
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
At 03:27 PM 10/1/2002, Paul J. Reder wrote:
>Ryan Bloom wrote:
>
>>On Tue, 1 Oct 2002, Paul J. Reder wrote:
>>
>>>I've been working on the caching code and ran across a core dump...
>>>
>>>A particular file contains an SSI call to a cgi. The cgi causes a pipe
>>>bucket to pass down the chain. cache_in_filter tries to save the bucket
>>>away and core dumps.
>>>
>>>Since a pipe bucket can be of any length, and could take any amount of
>>>time to complete, I would assume that the cache code should decide not
>>>to cache any response with a pipe bucket, correct?
>>Not necessarily. The cache code should stream the data to the cache, and
>>allow the data to also stream to the core_output_filter.
>
>Until it reaches the specified max cache size?
>
>So instead of trying to just copy and insert the pipe bucket, it should read
>from the pipe and add buckets to the cache (and forward copies on)? Seems
>like it would defeat some of the sendfile optimizations and such... Is it worth
>the code and lost optimizations to read/store/pass the piped data?
Not if you read off the headers, determine cacheability, and then send the
remaining pipe bucket down and ignore it when we can't cache the result
anyways. If the resource can be cached, the penalty for reading and then
storing + sending is nothing when you weigh serving out of the cache,
next time around.
Bill
Re: Speaking of pipes from cgis causing trouble...
Posted by "Paul J. Reder" <re...@remulak.net>.
Ryan Bloom wrote:
> On Tue, 1 Oct 2002, Paul J. Reder wrote:
>
>
>>I've been working on the caching code and ran across a core dump...
>>
>>A particular file contains an SSI call to a cgi. The cgi causes a pipe
>>bucket to pass down the chain. cache_in_filter tries to save the bucket
>>away and core dumps.
>>
>>Since a pipe bucket can be of any length, and could take any amount of
>>time to complete, I would assume that the cache code should decide not
>>to cache any response with a pipe bucket, correct?
>>
>
> Not necessarily. The cache code should stream the data to the cache, and
> allow the data to also stream to the core_output_filter.
Until it reaches the specified max cache size?
So instead of trying to just copy and insert the pipe bucket, it should read
from the pipe and add buckets to the cache (and forward copies on)? Seems
like it would defeat some of the sendfile optimizations and such... Is it worth
the code and lost optimizations to read/store/pass the piped data?
--
Paul J. Reder
-----------------------------------------------------------
"The strength of the Constitution lies entirely in the determination of each
citizen to defend it. Only if every single citizen feels duty bound to do
his share in this defense are the constitutional rights secure."
-- Albert Einstein
Re: Speaking of pipes from cgis causing trouble...
Posted by Ryan Bloom <rb...@ntrnet.net>.
On Tue, 1 Oct 2002, Paul J. Reder wrote:
> I've been working on the caching code and ran across a core dump...
>
> A particular file contains an SSI call to a cgi. The cgi causes a pipe
> bucket to pass down the chain. cache_in_filter tries to save the bucket
> away and core dumps.
>
> Since a pipe bucket can be of any length, and could take any amount of
> time to complete, I would assume that the cache code should decide not
> to cache any response with a pipe bucket, correct?
Not necessarily. The cache code should stream the data to the cache, and
allow the data to also stream to the core_output_filter.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
550 Jean St
Oakland CA 94610
-------------------------------------------------------------------------------