You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Jacob Lewallen <jl...@cs.ucr.edu> on 2003/02/21 18:57:31 UTC
[PATCH] Problem with the queues
Some more problems with apr_queue. . .
Erwin De Wolff wrote:
>My application uses queues and I notice that I had some difficulties when
>reaching the end and start of the queue. I'm using the latest (21 febr 2003)
>snapshot of apr and apr-util. Platforms on which it occurs, Windows and
>Linux. To make the problem more accessible I use most of the same code as
>the testqueue.c provided in the distribution. Only to simplify I use one
>producer and one consumer only and a queue of size 1. The producer pushes
>values 0,1,2,3,... and the consumer pops them one by one. The following
>output is obtained:
>
> 0: value 0 succesfully pushed.
> 1: value 1 succesfully pushed.
> 0: value 1 succesfully popped.
> 2: value 2 succesfully pushed.
> 1: value 2 succesfully popped.
> 3: value 3 succesfully pushed.
> 2: value 3 succesfully popped.
> 3: value 3 succesfully popped.
>
>You can notice a few things:
>1. value 0 is never popped
>2. there are two values pushed on a queue of size 1!
>3. value 3 is popped twice on a queue of size 1.
>
>I don't think I made a mistake in the code, but just for those who might
>doubt that, I attached the used code.
>
>
[code removed]
Here's what's going on here. With a queue size of 1, there's only one
position in the queue for placing data. In apr_queue_pop, we return the
address of the popped item "in the queue's own array" (hence us having
to dereference twice to get our integer pointer above). So here's how
things breakdown:
P: we push 0, its placed in data[0]
P: we try to push 1, its blocked, because we're full
C: we pop, and are returned the address of data[0] (in our case, a
pointer to where the pointer to 0 is zero)
P: unblock, push 1 into the queue (overwriting data[0] with the new
pointer)
C: we dereference the pointer to data[0] that was returned, which has
since been replaced with a pointer to 1.
And this goes on. I think that this is responsible for all of the
behavior we see above. I'm almost positive that I've seen someone
complain about how the apr_queue code returns the address of the item,
rather than the value itself. This is what the attached patch does, if
anybody has any reason for keeping the existing behavior, I'd love to
hear them. This, of course, breaks any code that uses apr_queue_pop, so
it's a big deal.
jacob lewallen
jlewalle@cs.ucr.edu
Re: [PATCH] Problem with the queues
Posted by Jacob Lewallen <jl...@cs.ucr.edu>.
Ian Holsman wrote:
>Thank you Paul & Jacob.
>the patch has been comitted.
>
Thanks, good stuff.
>ps.
>I'm interested in what your both using APR, and the queue code for.
>
>
>
>
I'm developing a network monitoring suite that we use here in our CS
department. The queue code is just handy since its multithreaded and
there's message passing type stuff going on, nothing too fancy here.
jacob lewallen
jlewalle@cs.ucr.edu
Re: [PATCH] Problem with the queues
Posted by Ian Holsman <li...@holsman.net>.
Thank you Paul & Jacob.
the patch has been comitted.
in the future, if your patch has been 'ignored' just
post it again in a week.
people on the list sometimes assume that someone else will do it,
and it falls through the cracks.
--Ian
ps.
I'm interested in what your both using APR, and the queue code for.
On Fri, 21 Feb 2003 09:57:31 -0800, Jacob Lewallen wrote:
> Some more problems with apr_queue. . .
>
> Erwin De Wolff wrote:
>
>>My application uses queues and I notice that I had some difficulties when
>>reaching the end and start of the queue. I'm using the latest (21 febr
>>2003) snapshot of apr and apr-util. Platforms on which it occurs, Windows
>>and Linux. To make the problem more accessible I use most of the same
>>code as the testqueue.c provided in the distribution. Only to simplify I
>>use one producer and one consumer only and a queue of size 1. The
>>producer pushes values 0,1,2,3,... and the consumer pops them one by one.
>>The following output is obtained:
>>
>> 0: value 0 succesfully pushed.
>> 1: value 1 succesfully pushed.
>> 0: value 1 succesfully popped.
>> 2: value 2 succesfully pushed.
>> 1: value 2 succesfully popped.
>> 3: value 3 succesfully pushed.
>> 2: value 3 succesfully popped.
>> 3: value 3 succesfully popped.
>>
>>You can notice a few things:
>>1. value 0 is never popped
>>2. there are two values pushed on a queue of size 1! 3. value 3 is popped
>>twice on a queue of size 1.
>>
>>I don't think I made a mistake in the code, but just for those who might
>>doubt that, I attached the used code.
>>
>>
> [code removed]
>
> Here's what's going on here. With a queue size of 1, there's only one
> position in the queue for placing data. In apr_queue_pop, we return the
> address of the popped item "in the queue's own array" (hence us having to
> dereference twice to get our integer pointer above). So here's how things
> breakdown:
>
> P: we push 0, its placed in data[0]
> P: we try to push 1, its blocked, because we're full C: we pop, and are
> returned the address of data[0] (in our case, a
> pointer to where the pointer to 0 is zero)
> P: unblock, push 1 into the queue (overwriting data[0] with the new
> pointer)
> C: we dereference the pointer to data[0] that was returned, which has
> since been replaced with a pointer to 1.
>
> And this goes on. I think that this is responsible for all of the behavior
> we see above. I'm almost positive that I've seen someone complain about
> how the apr_queue code returns the address of the item, rather than the
> value itself. This is what the attached patch does, if anybody has any
> reason for keeping the existing behavior, I'd love to hear them. This, of
> course, breaks any code that uses apr_queue_pop, so it's a big deal.
>
> jacob lewallen
> jlewalle@cs.ucr.edu
Re: [PATCH] Problem with the queues - commited
Posted by Ian Holsman <li...@holsman.net>.
Thanks guys.
this code has been committed.
On Fri, 21 Feb 2003 09:57:31 -0800, Jacob Lewallen wrote:
> Some more problems with apr_queue. . .
>
> Erwin De Wolff wrote:
>
>>My application uses queues and I notice that I had some difficulties when
>>reaching the end and start of the queue. I'm using the latest (21 febr
>>2003) snapshot of apr and apr-util. Platforms on which it occurs, Windows
>>and Linux. To make the problem more accessible I use most of the same
>>code as the testqueue.c provided in the distribution. Only to simplify I
>>use one producer and one consumer only and a queue of size 1. The
>>producer pushes values 0,1,2,3,... and the consumer pops them one by one.
>>The following output is obtained:
>>
>> 0: value 0 succesfully pushed.
>> 1: value 1 succesfully pushed.
>> 0: value 1 succesfully popped.
>> 2: value 2 succesfully pushed.
>> 1: value 2 succesfully popped.
>> 3: value 3 succesfully pushed.
>> 2: value 3 succesfully popped.
>> 3: value 3 succesfully popped.
>>
>>You can notice a few things:
>>1. value 0 is never popped
>>2. there are two values pushed on a queue of size 1! 3. value 3 is popped
>>twice on a queue of size 1.
>>
>>I don't think I made a mistake in the code, but just for those who might
>>doubt that, I attached the used code.
>>
>>
> [code removed]
>
> Here's what's going on here. With a queue size of 1, there's only one
> position in the queue for placing data. In apr_queue_pop, we return the
> address of the popped item "in the queue's own array" (hence us having to
> dereference twice to get our integer pointer above). So here's how things
> breakdown:
>
> P: we push 0, its placed in data[0]
> P: we try to push 1, its blocked, because we're full C: we pop, and are
> returned the address of data[0] (in our case, a
> pointer to where the pointer to 0 is zero)
> P: unblock, push 1 into the queue (overwriting data[0] with the new
> pointer)
> C: we dereference the pointer to data[0] that was returned, which has
> since been replaced with a pointer to 1.
>
> And this goes on. I think that this is responsible for all of the behavior
> we see above. I'm almost positive that I've seen someone complain about
> how the apr_queue code returns the address of the item, rather than the
> value itself. This is what the attached patch does, if anybody has any
> reason for keeping the existing behavior, I'd love to hear them. This, of
> course, breaks any code that uses apr_queue_pop, so it's a big deal.
>
> jacob lewallen
> jlewalle@cs.ucr.edu
Re: [PATCH] Problem with the queues
Posted by Paul Marquis <pm...@pobox.com>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Yeah, that was me. :-P
FYI, the patch I submitted to the list is identical to the one you
suggested.
On Friday 21 February 2003 12:57 pm, Jacob Lewallen wrote:
> Here's what's going on here. With a queue size of 1, there's only
> one position in the queue for placing data. In apr_queue_pop, we
> return the address of the popped item "in the queue's own array"
> (hence us having to dereference twice to get our integer pointer
> above). So here's how things breakdown:
>
> P: we push 0, its placed in data[0]
> P: we try to push 1, its blocked, because we're full
> C: we pop, and are returned the address of data[0] (in our case, a
> pointer to where the pointer to 0 is zero)
> P: unblock, push 1 into the queue (overwriting data[0] with the
> new pointer)
> C: we dereference the pointer to data[0] that was returned, which
> has since been replaced with a pointer to 1.
>
> And this goes on. I think that this is responsible for all of the
> behavior we see above. I'm almost positive that I've seen someone
> complain about how the apr_queue code returns the address of the
> item, rather than the value itself. This is what the attached patch
> does, if anybody has any reason for keeping the existing behavior,
> I'd love to hear them. This, of course, breaks any code that uses
> apr_queue_pop, so it's a big deal.
>
> jacob lewallen
> jlewalle@cs.ucr.edu
- --
Paul Marquis
pmarquis@pobox.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (FreeBSD)
iD8DBQE+VoLfIuLAUo3z6B0RAkW5AKDEOHRZoEhEXp+GbVk23Rdz7eSZFgCbBL2n
CvwuuB3MLphXhO1MTKFgJpc=
=BNwk
-----END PGP SIGNATURE-----
Re: [PATCH] Problem with the queues
Posted by Jacob Lewallen <jl...@cs.ucr.edu>.
Ian Holsman wrote:
>
> Thanks for the patch.
> I've been humbled twice now ;-)
Naw, all the activitiy is just a testament to how incredibly handy
apr_queue_t really is. :)
jacob lewalle
jlewalle@cs.ucr.edu
Re: [PATCH] Problem with the queues
Posted by Ian Holsman <ia...@apache.org>.
Thanks for the patch.
I've been humbled twice now ;-)
I'll try to get to this tonight.
--Ian
Jacob Lewallen wrote:
> Jacob Lewallen wrote:
>
>> Some more problems with apr_queue. . .
>>
>> Erwin De Wolff wrote:
>>
>>> My application uses queues and I notice that I had some difficulties
>>> when
>>> reaching the end and start of the queue. I'm using the latest (21
>>> febr 2003)
>>> snapshot of apr and apr-util. Platforms on which it occurs, Windows and
>>> Linux. To make the problem more accessible I use most of the same
>>> code as
>>> the testqueue.c provided in the distribution. Only to simplify I use one
>>> producer and one consumer only and a queue of size 1. The producer
>>> pushes
>>> values 0,1,2,3,... and the consumer pops them one by one. The following
>>> output is obtained:
>>>
>>> 0: value 0 succesfully pushed.
>>> 1: value 1 succesfully pushed.
>>> 0: value 1 succesfully popped.
>>> 2: value 2 succesfully pushed.
>>> 1: value 2 succesfully popped.
>>> 3: value 3 succesfully pushed.
>>> 2: value 3 succesfully popped.
>>> 3: value 3 succesfully popped.
>>>
>>> You can notice a few things:
>>> 1. value 0 is never popped
>>> 2. there are two values pushed on a queue of size 1!
>>> 3. value 3 is popped twice on a queue of size 1.
>>>
>>> I don't think I made a mistake in the code, but just for those who might
>>> doubt that, I attached the used code.
>>>
>>>
>> [code removed]
>>
>> Here's what's going on here. With a queue size of 1, there's only one
>> position in the queue for placing data. In apr_queue_pop, we return
>> the address of the popped item "in the queue's own array" (hence us
>> having to dereference twice to get our integer pointer above). So
>> here's how things breakdown:
>>
>> P: we push 0, its placed in data[0]
>> P: we try to push 1, its blocked, because we're full
>> C: we pop, and are returned the address of data[0] (in our case, a
>> pointer to where the pointer to 0 is zero)
>> P: unblock, push 1 into the queue (overwriting data[0] with the new
>> pointer)
>> C: we dereference the pointer to data[0] that was returned, which has
>> since been replaced with a pointer to 1.
>>
>> And this goes on. I think that this is responsible for all of the
>> behavior we see above. I'm almost positive that I've seen someone
>> complain about how the apr_queue code returns the address of the item,
>> rather than the value itself. This is what the attached patch does, if
>> anybody has any reason for keeping the existing behavior, I'd love to
>> hear them. This, of course, breaks any code that uses apr_queue_pop,
>> so it's a big deal.
>>
>> jacob lewallen
>> jlewalle@cs.ucr.edu
>>
>
>
> ------------------------------------------------------------------------
>
> ? apr_queue_overwrite_fix
> Index: apr_queue.c
> ===================================================================
> RCS file: /home/cvspublic/apr-util/misc/apr_queue.c,v
> retrieving revision 1.11
> diff -u -u -r1.11 apr_queue.c
> --- apr_queue.c 14 Feb 2003 04:39:07 -0000 1.11
> +++ apr_queue.c 21 Feb 2003 17:50:34 -0000
> @@ -333,7 +333,7 @@
> }
> }
>
> - *data = &queue->data[queue->out];
> + *data = queue->data[queue->out];
> queue->nelts--;
>
> queue->out = (queue->out + 1) % queue->bounds;
> @@ -375,7 +375,7 @@
> return APR_EAGAIN;
> }
>
> - *data = &queue->data[queue->out];
> + *data = queue->data[queue->out];
> queue->nelts--;
>
> queue->out = (queue->out + 1) % queue->bounds;
Re: [PATCH] Problem with the queues
Posted by Jacob Lewallen <jl...@cs.ucr.edu>.
Jacob Lewallen wrote:
> Some more problems with apr_queue. . .
>
> Erwin De Wolff wrote:
>
>> My application uses queues and I notice that I had some difficulties
>> when
>> reaching the end and start of the queue. I'm using the latest (21
>> febr 2003)
>> snapshot of apr and apr-util. Platforms on which it occurs, Windows and
>> Linux. To make the problem more accessible I use most of the same
>> code as
>> the testqueue.c provided in the distribution. Only to simplify I use one
>> producer and one consumer only and a queue of size 1. The producer
>> pushes
>> values 0,1,2,3,... and the consumer pops them one by one. The following
>> output is obtained:
>>
>> 0: value 0 succesfully pushed.
>> 1: value 1 succesfully pushed.
>> 0: value 1 succesfully popped.
>> 2: value 2 succesfully pushed.
>> 1: value 2 succesfully popped.
>> 3: value 3 succesfully pushed.
>> 2: value 3 succesfully popped.
>> 3: value 3 succesfully popped.
>>
>> You can notice a few things:
>> 1. value 0 is never popped
>> 2. there are two values pushed on a queue of size 1!
>> 3. value 3 is popped twice on a queue of size 1.
>>
>> I don't think I made a mistake in the code, but just for those who might
>> doubt that, I attached the used code.
>>
>>
> [code removed]
>
> Here's what's going on here. With a queue size of 1, there's only one
> position in the queue for placing data. In apr_queue_pop, we return
> the address of the popped item "in the queue's own array" (hence us
> having to dereference twice to get our integer pointer above). So
> here's how things breakdown:
>
> P: we push 0, its placed in data[0]
> P: we try to push 1, its blocked, because we're full
> C: we pop, and are returned the address of data[0] (in our case, a
> pointer to where the pointer to 0 is zero)
> P: unblock, push 1 into the queue (overwriting data[0] with the new
> pointer)
> C: we dereference the pointer to data[0] that was returned, which has
> since been replaced with a pointer to 1.
>
> And this goes on. I think that this is responsible for all of the
> behavior we see above. I'm almost positive that I've seen someone
> complain about how the apr_queue code returns the address of the item,
> rather than the value itself. This is what the attached patch does, if
> anybody has any reason for keeping the existing behavior, I'd love to
> hear them. This, of course, breaks any code that uses apr_queue_pop,
> so it's a big deal.
>
> jacob lewallen
> jlewalle@cs.ucr.edu
>