You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modules-dev@httpd.apache.org by Vlastimil Pavicek <vl...@centrum.cz> on 2006/10/14 11:23:17 UTC
Traversing the bucket brigade from last to first in the output filter
Hello,
at first sorry for a quite longer mail(and for a bad english).
I am a student working on a project including an apache output filter. I encountered a strange behavior of a bucket brigade
when it is traversed from the last bucket (APR_BRIGADE_LAST(..)) to the first one (APR_BRIGADE_FIRST(..)) using
APR_BUCKET_PREV(..).
For testing purposes I splitted incoming bucket brigade (size 18567) using:
b=APR_BRIGADE_FIRST(bb);
if(apr_bucket_split(b,100)!=APR_SUCCESS) my_error(1);
b=APR_BUCKET_NEXT(b);
if(apr_bucket_split(b,100)!=APR_SUCCESS) my_error(2);
b=APR_BUCKET_NEXT(b);
if(apr_bucket_split(b,100)!=APR_SUCCESS) my_error(3);
Then I try to traverse the brigade from the last bucket to the first one:
b=APR_BRIGADE_LAST(bb);
while(b!=APR_BRIGADE_SENTINEL(bb)) {
..dump the contents of bucket into file to see it's contents
b=APR_BUCKET_PREV(b);
}
I got a set of files with the contents of individual buckets.
Then I traverse the brigade from the first bucket to the last one:
b=APR_BRIGADE_FIRST(bb);
while(b!=APR_BRIGADE_SENTINEL(bb)) {
..dump the contents of bucket into file to see it's contents
b=APR_BUCKET_NEXT(b);
}
I got an another set files with the contents of individual buckets.
The problem is, that the bucket contents set I get when traversing from the last to the first is not complete.
This way I get five files sized 0, 8000, 100, 100 and 100 -> that is 8300 total instead of 16888
While traversing from the first to the last I got 7 files sized 100, 100, 100, 8000, 8000 and 2267
-> that is correctly 18567.
I wonder, why there is a difference. The contents of corresponding files are right. After a brief look
into the apache sources (apr_bucket_shared_split, apr_bucket_simple_split, APR_BUCKET_INSERT_AFTER,
APR_RING_INSERT_AFTER, APR_RING_SPLICE_AFTER) everything seems OK to me (connecting buckets).
Please does anyone have an idea why this is not working? Is the way I split the buckets correct?
I am not sure if it is a bug in apache or in my code. I use apache 2.0.55 from debian testing.
I look forward to any reply, greetings
Vlastimil Pavicek, CTU Prague
Re: Traversing the bucket brigade from last to first in the output filter
Posted by Vlastimil Pavicek <vl...@centrum.cz>.
Thank to all for their effort, It seems like a bug in apache so I am going to commit it to bugzilla.
Vlastimil Pavicek, CTU Prague
Re: Traversing the bucket brigade from last to first in the output filter
Posted by Vlastimil Pavicek <vl...@centrum.cz>.
Thank you for your reply, I will try to answer your questions.
> > I got following for the first bucket in this brigade just when I got it:
> > APR_BUCKET_IS_FILE -> 1
>
> OK, a static file.
>
> > There are two buckets sized 18567 and 0.
>
> FILE + EOS.
>
> > > > b=APR_BRIGADE_LAST(bb);
> > >
> > > If we hypothesize that your input was a single bucket, then it's
> > > 3*100 + LAST. And what's wrong is the size of LAST at this point.
> > > Is LAST by any chance a Heap bucket?
>
> So how exactly did you read that last bucket, and what exactly
> did the read function return? Perhaps you could put your demo
> source code up where we can look at it.
Source code is at
http://www.volny.cz/jeword/a/mod_txt.c
Some of you might not see this code for the first time :)
The problematic traversal starts at line 222.
> > LAST (size 0):
> > APR_BUCKET_IS_METADATA -> 1
> > APR_BUCKET_IS_EOS -> 1
> >
> > LAST->PREV (size 8000)
> > APR_BUCKET_IS_HEAP -> 1
>
> I'd guess that's *after* reading it? What does it look like before?
Before reading:
LAST (size 0):
APR_BUCKET_IS_METADATA -> 1
APR_BUCKET_IS_EOS -> 1
LAST->PREV (size 8000):
APR_BUCKET_IS_FILE -> 1
LAST->PREV->PREV (size 100):
APR_BUCKET_IS_FILE -> 1
LAST->PREV->PREV->PREV (size 100):
APR_BUCKET_IS_FILE -> 1
LAST->PREV->PREV->PREV->PREV (size 100):
APR_BUCKET_IS_FILE -> 1
The brigade stays truncated.
Thank to all for their effort, I look forward to the explanation
Vlastimil Pavicek, CTU Prague
Re: Traversing the bucket brigade from last to first in the output filter
Posted by Nick Kew <ni...@webthing.com>.
On Saturday 14 October 2006 21:20, Vlastimil Pavicek wrote:
> I got following for the first bucket in this brigade just when I got it:
> APR_BUCKET_IS_FILE -> 1
OK, a static file.
> There are two buckets sized 18567 and 0.
FILE + EOS.
> > > b=APR_BRIGADE_LAST(bb);
> >
> > If we hypothesize that your input was a single bucket, then it's
> > 3*100 + LAST. And what's wrong is the size of LAST at this point.
> > Is LAST by any chance a Heap bucket?
So how exactly did you read that last bucket, and what exactly
did the read function return? Perhaps you could put your demo
source code up where we can look at it.
> LAST (size 0):
> APR_BUCKET_IS_METADATA -> 1
> APR_BUCKET_IS_EOS -> 1
>
> LAST->PREV (size 8000)
> APR_BUCKET_IS_HEAP -> 1
I'd guess that's *after* reading it? What does it look like before?
--
Nick Kew
Application Development with Apache - the Apache Modules Book
http://www.prenhallprofessional.com/title/0132409674
Re: Traversing the bucket brigade from last to first in the output
filter
Posted by Joachim Zobel <jz...@heute-morgen.de>.
Hi.
I do not fully understand your problem, but you probably encountered
morphing. If you call apr_bucket_read on a file bucket, it turns into
two buckets. A heap bucket, that holds the buffer you read and a file
bucket that has a different start offset.
You may find this helpful
http://www.cs.virginia.edu/~jcw5q/talks/apache/bucketbrigades.ac2002.pdf
Sincerely,
Joachim
Re: Traversing the bucket brigade from last to first in the output filter
Posted by Vlastimil Pavicek <vl...@centrum.cz>.
Thank you for a reply I will try to answer your questions
> > at first sorry for a quite longer mail(and for a bad english).
> >
> > I am a student working on a project including an apache output filter. I
> > encountered a strange behavior of a bucket brigade when it is traversed
> > from the last bucket (APR_BRIGADE_LAST(..)) to the first one
> > (APR_BRIGADE_FIRST(..)) using APR_BUCKET_PREV(..).
> >
> > For testing purposes I splitted incoming bucket brigade (size 18567) using:
>
> Where does that come from, and what breakdown of individual buckets in there?
> Is it by any chance a source (eg pipe or socket) rather than a store?
That is the bucket brigade I got from upper level. I assume, that this is the file
I am testing the filter on.
I got following for the first bucket in this brigade just when I got it:
APR_BUCKET_IS_METADATA -> 0
APR_BUCKET_IS_FLUSH -> 0
APR_BUCKET_IS_EOS -> 0
APR_BUCKET_IS_FILE -> 1
APR_BUCKET_IS_PIPE -> 0
APR_BUCKET_IS_SOCKET -> 0
APR_BUCKET_IS_HEAP -> 0
APR_BUCKET_IS_TRANSIENT -> 0
APR_BUCKET_IS_IMMORTAL -> 0
APR_BUCKET_IS_MMAP -> 0
APR_BUCKET_IS_POOL -> 0
There are two buckets sized 18567 and 0.
> > b=APR_BRIGADE_FIRST(bb);
> > if(apr_bucket_split(b,100)!=APR_SUCCESS) my_error(1);
> >
> > b=APR_BUCKET_NEXT(b);
> > if(apr_bucket_split(b,100)!=APR_SUCCESS) my_error(2);
> >
> > b=APR_BUCKET_NEXT(b);
> > if(apr_bucket_split(b,100)!=APR_SUCCESS) my_error(3);
> >
> > Then I try to traverse the brigade from the last bucket to the first one:
> >
> > b=APR_BRIGADE_LAST(bb);
>
> If we hypothesize that your input was a single bucket, then it's
> 3*100 + LAST. And what's wrong is the size of LAST at this point.
> Is LAST by any chance a Heap bucket?
LAST (size 0):
APR_BUCKET_IS_METADATA -> 1
APR_BUCKET_IS_EOS -> 1
LAST->PREV (size 8000)
APR_BUCKET_IS_HEAP -> 1
LAST->PREV->PREV (size 100)
APR_BUCKET_IS_HEAP -> 1
LAST->PREV->PREV->PREV (size 100)
APR_BUCKET_IS_HEAP -> 1
LAST->PREV->PREV->PREV->PREV (size 100)
APR_BUCKET_IS_MMAP -> 1
all other tested flags are 0
> > while(b!=APR_BRIGADE_SENTINEL(bb)) {
> >
> > ..dump the contents of bucket into file to see it's contents
> >
> > b=APR_BUCKET_PREV(b);
> > }
> >
> > I got a set of files with the contents of individual buckets.
> >
> > Then I traverse the brigade from the first bucket to the last one:
> >
> > b=APR_BRIGADE_FIRST(bb);
> > while(b!=APR_BRIGADE_SENTINEL(bb)) {
> >
> > ..dump the contents of bucket into file to see it's contents
> >
> > b=APR_BUCKET_NEXT(b);
> > }
> >
> > I got an another set files with the contents of individual buckets.
>
> And what happens if you traverse it backwards after going forwards?
Then everything is fixed.
In fact everything is OK even if I traverse it forward before doing the splitting.
Does this mean anything?
Thank you a lot for your reply.
Greetings,
Vlastimil Pavicek, CTU Prague
Re: Traversing the bucket brigade from last to first in the output filter
Posted by Nick Kew <ni...@webthing.com>.
On Saturday 14 October 2006 10:23, Vlastimil Pavicek wrote:
> Hello,
>
> at first sorry for a quite longer mail(and for a bad english).
>
> I am a student working on a project including an apache output filter. I
> encountered a strange behavior of a bucket brigade when it is traversed
> from the last bucket (APR_BRIGADE_LAST(..)) to the first one
> (APR_BRIGADE_FIRST(..)) using APR_BUCKET_PREV(..).
>
> For testing purposes I splitted incoming bucket brigade (size 18567) using:
Where does that come from, and what breakdown of individual buckets in there?
Is it by any chance a source (eg pipe or socket) rather than a store?
> b=APR_BRIGADE_FIRST(bb);
> if(apr_bucket_split(b,100)!=APR_SUCCESS) my_error(1);
>
> b=APR_BUCKET_NEXT(b);
> if(apr_bucket_split(b,100)!=APR_SUCCESS) my_error(2);
>
> b=APR_BUCKET_NEXT(b);
> if(apr_bucket_split(b,100)!=APR_SUCCESS) my_error(3);
>
> Then I try to traverse the brigade from the last bucket to the first one:
>
> b=APR_BRIGADE_LAST(bb);
If we hypothesize that your input was a single bucket, then it's
3*100 + LAST. And what's wrong is the size of LAST at this point.
Is LAST by any chance a Heap bucket?
> while(b!=APR_BRIGADE_SENTINEL(bb)) {
>
> ..dump the contents of bucket into file to see it's contents
>
> b=APR_BUCKET_PREV(b);
> }
>
> I got a set of files with the contents of individual buckets.
>
> Then I traverse the brigade from the first bucket to the last one:
>
> b=APR_BRIGADE_FIRST(bb);
> while(b!=APR_BRIGADE_SENTINEL(bb)) {
>
> ..dump the contents of bucket into file to see it's contents
>
> b=APR_BUCKET_NEXT(b);
> }
>
> I got an another set files with the contents of individual buckets.
And what happens if you traverse it backwards after going forwards?
--
Nick Kew
Application Development with Apache - the Apache Modules Book
http://www.prenhallprofessional.com/title/0132409674