You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Yann <yl...@bee-ware.net> on 2008/06/06 11:02:37 UTC
Opaque structures in general (was Re: Opaque apr_pool_t structure)
I'm sorry to insist on this point but, in general, maybe you should consider having a function that provides the
sizeof() each opaque structure/type of the APR (maybe not all of them, but at least some).
That would let the user "play" with the opaque types without breaking the interest (portability, evolution ...) of this
ones.
As far as I know, the APR is a C programming API, so let us play with the pointers if we find it usefull.
I could propose the patch(s) if you agree and/or are interested by that.
If not, maybe you could let me know why ?
Thanks for your answers,
Yann.
>> Sander Striker wrote:
>>> Hello,
>>>
>>> I'm making a piece of software that recycles apr_sockets in
>>> sockets-pools.
>>
>> Have you looked at apr_reslist in apr-util?
>>
>
> It looks very interesting, thank you for the hint, however it seems that
> I have to do the same trick to prevent a leak with my_socket (a stucture
> containing the apr_socket and other things associated with it).
>
> As the pool used to create the reslist is passed to the contructor for
> its own allocations (my_sockets here), I suppose I have to create a
> subpool in the constructor where my_socket will be allocated, and keep a
> reference to the subpool in my_socket to be able to release all that in
> the destructor.
>
> Without this, that is using the reslist pool directly, memory will leak
> as my_socket allocated fields or even the apr_socket internals (addr and
> like) will stay in the pool after destruction.
>
> Did I miss something ?
>
>>> Each socket is allocated and created on its own pool (not a subpool, or
>>> exactly a subpool of the internal global pool only).
>>> I've read on svn programming pages that it is not recommanded to do
>>> this,
>>> but I find it very usefull, that's the power of pools, IMHO.
>>
>> The problem is the memory footprint that this brings with it. Each pool
>> will pre-allocate 8k of memory.
>>
>
> Maybe a subpool doesn't pre-allocate memory, or the problem remain with
> reslists (and the method above).
>
>>> Anyway, to make the things clean, I need a way to attach/detach pools
>>> to/from others, whether the socket belongs to the socket-pool's pool
>>> or to
>>> the user's one.
>>>
>>> As the apr_pool_join() function is a noop (and I don't knwow if
>>> that's its
>>> purpose anyway), I used the apr_pool_cleanup_register()ing /
>>> apr_pool_cleanup_kill()ing mechanism to attach / detach pools between
>>> them.
>>>
>>> I create a small new API with my pools based on the APR ones, where I
>>> can
>>> attach/detach pools, as I said, but also malloc() and realloc() memories
>>> attached to a pool (with the same registering mechanism).
>>>
>>> My pool struct is like :
>>>
>>> struct my_pool_t {
>>> apr_pool_t pool;
>>> apr_pool_t *owner;
>>> };
>>>
>>> And the functions :
>>>
>>> apr_status_t my_pool_create(my_pool_t **pool, void *owner);
>>> apr_status_t my_pool_attach(my_pool_t *pool, void *owner);
>>> apr_status_t my_pool_detach(my_pool_t *pool);
>>>
>>> apr_status_t my_pmalloc(void *pool, apr_size_t size);
>>> apr_status_t my_pzalloc(void *pool, apr_size_t size);
>>> apr_status_t my_prealloc(void *pool, void *ptr, apr_size_t size);
>>> void my_pfree(void *pool, void *ptr);
>>>
>>> As you can see with the first element of the struct which is an APR
>>> pool and
>>> 'void*' types used in functions, I expect the functions to be
>>> "compatible"
>>> with the APR ones, that is I can use my_pool_t and apr_pool_t pools
>>> indistinctly with them (where the type is void*) ...
>>
>> That's not the case, see below.
>>
>
> Hum, I see, the pool stands in an allocator node, see below.
>
>>> My problem is that I can't initialize my_pool because I haven't got the
>>> sizeof(apr_pool_t), an opaque structure ...
>>>
>>> I understand the advantages of opaque types, but is there a way an
>>> apr_pool_sizeof() function be added (and exported) in the APR, simply
>>> like :
>>>
>>> APR_DECLARE(apr_size_t) apr_pool_sizeof(void)
>>> {
>>> /* maybe the aligned SIZEOF_POOL_T should be used */
>>> return sizeof(apr_pool_t);
>>> }
>>>
>>> With it, I could at least do :
>>> apr_pool_create(&p, NULL);
>>> memcpy(my_pool->pool, p, apr_pool_sizeof())
>>
>> This would be a bad idea, as you'd be copying pointers and the like,
>> which
>> would be off as soon as copied. Even if that were not the case, the
>> apr_pool_t
>> struct instance lives in an allocated block of memory from which the pool
>> will hand out memory. This block in turn is managed by an allocator.
>>
>
> But the allocator node shouldn't be destroyed (or reused) while the pool
> isn't, so I don't see why a copy of the pool couldn't do the job as well.
>
> The copy of the pointers and the like still reference the allocator
> node, and it should work.
>
> However all this becomes cheat and I'd better look at apr_reslist.
>
>>> Or should I consider using my_p*() functions strictly with my_pools ?
>>>
>>> Thanks for your advices, answers.
>>
>> Cheers,
>>
>> Sander
>>
>
> Cheers, and thanks for your answers,
> Yann.
>
>
Re: Opaque structures in general (was Re: Opaque apr_pool_t structure)
Posted by Branko Čibej <br...@xbc.nu>.
Yann wrote:
> Branko Čibej wrote:
>> Yann wrote:
>>> Sorry I didn't explain myself very well.
>>>
>>> I don't wan't to play with pointers, of course, a single sizeof
>>> won't allow me to do much more then a memcpy().
>>>
>>> For example ( innocently :p ) :
>>>
>>> struct mystuct {
>>> apr_pool_t p;
>>> ...
>>> } myvar;
>>> apr_pool_t p;
>>>
>>> apr_pool_create(&p, NULL);
>>> memcpy(&myvar.p, p, apr_pool_sizeof());
>>
>> How is that better than just putting an apr_pool_t* in your struct?
>> So it's inheritance by reference instead of by containment,
>> semantically it's equivalent (given that this is C).
>>
>> -- Brane
>>
>
> Think at this struct instead :
>
> struct mypool_t {
> apr_pool_t p;
> apr_pool_t *owner;
> ...
> };
>
> It is strictly compatible with apr_pool_t one, as I can use mypool_t*
> or apr_pool_t* with the APR or derivative functions.
Maybe you should just live with the fact that this is C, not C++, so any
kind of fake polymorphism will be just that -- fake. Even if apr_pool_t
wasn't opaqe, you'd still have to add a cast in order to make the
regular apr_pool functions digest your new struct, and frankly, an
explicit indirection is better than a cast any day -- because it
preserves type information.
-- Brane
Re: Opaque structures in general (was Re: Opaque apr_pool_t structure)
Posted by Yann <yl...@bee-ware.net>.
Branko Čibej wrote:
> Yann wrote:
>> Sorry I didn't explain myself very well.
>>
>> I don't wan't to play with pointers, of course, a single sizeof won't
>> allow me to do much more then a memcpy().
>>
>> For example ( innocently :p ) :
>>
>> struct mystuct {
>> apr_pool_t p;
>> ...
>> } myvar;
>> apr_pool_t p;
>>
>> apr_pool_create(&p, NULL);
>> memcpy(&myvar.p, p, apr_pool_sizeof());
>
> How is that better than just putting an apr_pool_t* in your struct? So
> it's inheritance by reference instead of by containment, semantically
> it's equivalent (given that this is C).
>
> -- Brane
>
Think at this struct instead :
struct mypool_t {
apr_pool_t p;
apr_pool_t *owner;
...
};
It is strictly compatible with apr_pool_t one, as I can use mypool_t* or apr_pool_t* with the APR or derivative functions.
But I can add my own functions which, for example, attach/detach pools to/from each others with the
apr_pool_cleanup_register()ing mechanism (or make the my_prealloc() function).
So that I can create objects that can be affected to different pools, depending on whom they belong to for a particular
treatment (handled by a thread that can die with its pool anytime for example again).
The actual apr_pool implementation does not allow to join subpools to another pool after creation (the apr_pool_join is
a noop as far as I know).
Yann.
Re: Opaque structures in general (was Re: Opaque apr_pool_t structure)
Posted by Branko Čibej <br...@xbc.nu>.
Yann wrote:
> Sorry I didn't explain myself very well.
>
> I don't wan't to play with pointers, of course, a single sizeof won't
> allow me to do much more then a memcpy().
>
> For example ( innocently :p ) :
>
> struct mystuct {
> apr_pool_t p;
> ...
> } myvar;
> apr_pool_t p;
>
> apr_pool_create(&p, NULL);
> memcpy(&myvar.p, p, apr_pool_sizeof());
How is that better than just putting an apr_pool_t* in your struct? So
it's inheritance by reference instead of by containment, semantically
it's equivalent (given that this is C).
-- Brane
Re: Opaque structures in general (was Re: Opaque apr_pool_t structure)
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Yann wrote:
> Sorry I didn't explain myself very well.
>
> I don't wan't to play with pointers, of course, a single sizeof won't
> allow me to do much more then a memcpy().
>
> For example ( innocently :p ) :
>
> struct mystuct {
> apr_pool_t p;
> ...
> } myvar;
> apr_pool_t p;
>
> apr_pool_create(&p, NULL);
> memcpy(&myvar.p, p, apr_pool_sizeof());
>
> That's all I can do, I can't access any apr_pool field ...
Keep in mind that the origin p may have internal reference pointers or
other objects may have reference pointers into the original p, so that
if you begin manipulating p at a different location you may break both
p itself and other objects referencing p.
That said, I can see where you might be going with this... I'd think
that some serialization wrapper for some of our object types into a
transferable, storable/cachable and later queryable form might be useful.
Re: Opaque structures in general (was Re: Opaque apr_pool_t structure)
Posted by Ryan Bloom <rb...@gmail.com>.
I agree with Sander, this should be relatively easy to implement using
parent relationships. And, I didn't intend sarcasm. I really meant
that if you have a feature that APR is missing, it is better to
implement it inside APR, rather than implement a wrapper around the
existing concept.
Ryan
On Fri, Jun 6, 2008 at 12:16 PM, Sander Striker <s....@striker.nl> wrote:
> On Fri, Jun 6, 2008 at 5:10 PM, Yann <yl...@bee-ware.net> wrote:
>> Christopher Key wrote:
>
> [...]
>> Ryan Bloom wrote:
>>> If this is what you really want/need, then I would suggest focusing on
>>> a patch that implements this functionality.
>>>
>>> Ryan
>>>
>>
>> OK, I can try to do that although I feel some sarcasm in your suggestion,
>> don't you think it is possible ?
>
> I'm not sure if I really understand what you need. It sounds like you'd want to
> have something like
>
> /**
> * Detach @a p from it's parent, making it an orphan. Destruction or cleanup of
> * the original parent pool will not destruct @a p.
> *
> * @warning Use this function wisely, you're changing original lifetimes.
> */
> apr_status_t apr_pool_orphan(apr_pool_t *p);
>
> /**
> * Attach @a p to @a parent, making it a child pool. Destruction or cleanup of
> * @a parent will also destroy @a p.
> *
> * @warning Use this function wisely, you're changing original lifetimes.
> */
> apr_status_t apr_pool_adopt(apr_pool_t *p, apr_pool_t *parent);
>
> If that's what you want it's not particularly hard to do.
>
>> Moreover, my solution looks quite simple and I'm afraid it has already been
>> discussed in this list, but I give a try ...
>>
>> If I had to do a patch, I would try to use the
>> apr_pool_cleanup_register()ing / _kill()ing mechanism to bind the pools each
>> others.
>
> I would just manipulate the child/parent relationships.
>
>> Including the subpools that would just be pools registered in the cleanups
>> list of their parent/owner.
>>
>> Do you see another constraint, for a pool to be lately attached to another
>> parent/owner, than the two pools have to use the same allocator and the
>> to-attach pool *not* to be an ancestor of the attached owner/parent pool
>> (that is, not having its own cleanup registered in the owner) ?
>>
>> I'm surely missing something, and I surely need some advices,
>> regards,
>> Yann.
>
> Cook up a patch and we'll give it a look :)
>
> Cheers,
>
> Sander
>
--
Ryan Bloom
rbb@apache.org
rbb@rkbloom.net
rbloom@gmail.com
Re: Opaque structures in general (was Re: Opaque apr_pool_t structure)
Posted by Sander Striker <s....@striker.nl>.
On Fri, Jun 6, 2008 at 5:10 PM, Yann <yl...@bee-ware.net> wrote:
> Christopher Key wrote:
[...]
> Ryan Bloom wrote:
>> If this is what you really want/need, then I would suggest focusing on
>> a patch that implements this functionality.
>>
>> Ryan
>>
>
> OK, I can try to do that although I feel some sarcasm in your suggestion,
> don't you think it is possible ?
I'm not sure if I really understand what you need. It sounds like you'd want to
have something like
/**
* Detach @a p from it's parent, making it an orphan. Destruction or cleanup of
* the original parent pool will not destruct @a p.
*
* @warning Use this function wisely, you're changing original lifetimes.
*/
apr_status_t apr_pool_orphan(apr_pool_t *p);
/**
* Attach @a p to @a parent, making it a child pool. Destruction or cleanup of
* @a parent will also destroy @a p.
*
* @warning Use this function wisely, you're changing original lifetimes.
*/
apr_status_t apr_pool_adopt(apr_pool_t *p, apr_pool_t *parent);
If that's what you want it's not particularly hard to do.
> Moreover, my solution looks quite simple and I'm afraid it has already been
> discussed in this list, but I give a try ...
>
> If I had to do a patch, I would try to use the
> apr_pool_cleanup_register()ing / _kill()ing mechanism to bind the pools each
> others.
I would just manipulate the child/parent relationships.
> Including the subpools that would just be pools registered in the cleanups
> list of their parent/owner.
>
> Do you see another constraint, for a pool to be lately attached to another
> parent/owner, than the two pools have to use the same allocator and the
> to-attach pool *not* to be an ancestor of the attached owner/parent pool
> (that is, not having its own cleanup registered in the owner) ?
>
> I'm surely missing something, and I surely need some advices,
> regards,
> Yann.
Cook up a patch and we'll give it a look :)
Cheers,
Sander
Re: Opaque structures in general (was Re: Opaque apr_pool_t structure)
Posted by Yann <yl...@bee-ware.net>.
Christopher Key wrote:
> Yann Lavictoire wrote:
>
> <Big snip>
>
> Ok, given your most recent post, I can see what you're doing now. I
> also think a mail client somewhere along the way had borken you're
> original code slightly, did you mean,
>
> struct {
> ...
> } myvar;
> apr_pool_t *p;
>
>
> This still seems like a bad idea, albeit somewhat less less than before,
> and with examples of situations where things break being a little less
> obvious. Nevertheless, it does remove some of the freedom the APR had
> with the way it implemented things behind the scenes, and means that
> there's more to be mindful of whilst coding platform specific
> implementations of things.
>
>
> A couple of examples:
> APR may reference members of apr_pool_t, both directly and via contained
> pointers:
>
> struct s {
> int a;
> int b;
> int *c;
> }
>
> int main(...) {
> struct s *s1;
> struct s s2;
>
> make(&s1);
> memcpy(&s2, s1, sizeof...);
> do(&s2);
> }
>
> void make(struct s **ptr) {
> *ptr = malloc(sizeof(**ptr));
> ptr.a = 0;
> ptr.b = 0;
>
> if (...) { ptr.c = &(ptr.a); } else {ptr.c = &(ptr.b); }
> }
>
> void do(struct s *ptr) {
> ptr->a++;
> ptr->b++;
> *(ptr->cur) *=2;
> }
>
>
> APR may also want to keep track of all pools created and their usage,
> maybe for a future memory usage API or similar. Obviously, APR will
> only know about the apr_pool_t that it returned a pointer to, and
> nothing about the copied version, so can't keep track of how its being
> used.
>
> I would say that alignment could still be an issue, although is probably
> unlikely to. A platform may require that an apr_pool_t end up on a
> specific boundary for some reason (quite independent of its size). APR
> can force this when it allocates storage is required, but when the
> storage is allocated for myvar, there's no such guarantee.
>
> Admittedly, these are rather contrived examples, but they are still
> illustrating extra requirements for APR implementations that aren't
> there with purely opaque types and pointers. This increase in
> complexity of the how APR can implement things cannot be good, and just
> seems likely to lead to problems to me.
>
>
> Kind regards,
>
> Chris
>
Good point, thanks for your example.
Since apr_pool_t only contains "direct" pointers, it doesn't apply to it but I admit the sizeof() and memcpy() doesn't
work on opaque structs.
My changes have to be done in the apr_pools implementation I'm afraid ...
Ryan Bloom wrote:
> On Fri, Jun 6, 2008 at 9:18 AM, Yann <yl...@bee-ware.net> wrote:
>> Erik Huelsmann wrote:
>>> On 6/6/08, Ryan Bloom <rb...@gmail.com> wrote:
>>>> I don't think that there is any reason to not have a sizeof()
>>>> function, other than any code that does "play" with the pointers will
>>>> be non-portable code. The reason that I originally went with opaque
>>>> data structures (I did it before giving the code to the ASF), was that
>>>> most of the structures are defined totally differently on each
>>>> platform. By making the structures opaque, it became much harder for
>>>> a developer to write code with APR that worked on some APR platforms,
>>>> but not others. If you play with the pointers, your code is very
>>>> likely to work only on the platforms that you code on.
>>>>
>>>> But, I would like to hear from some of the active developers about this
>>>> as well.
>>> Well, as soon as you provide its size, it's not completely opaque
>>> anymore, now is it :-)
>>>
>> Yes of course, but as soon you provide an accessor it isn't neither ...
>>
>>> I think the entire issue is centered around the fact that Yann doesn't
>>> really want to play by the pool-rules...
>>>
>> I would love to play with the APR pools if they could become subpools of
>> others after their creation.
>>
>> That is, a pool being the whole object (destroyed with its pool), and
>> objects living or dying with their changing owners.
>>
>> Why couldn't that be compatible with the pool rules ?
>
Ryan Bloom wrote:
> If this is what you really want/need, then I would suggest focusing on
> a patch that implements this functionality.
>
> Ryan
>
OK, I can try to do that although I feel some sarcasm in your suggestion, don't you think it is possible ?
Moreover, my solution looks quite simple and I'm afraid it has already been discussed in this list, but I give a try ...
If I had to do a patch, I would try to use the apr_pool_cleanup_register()ing / _kill()ing mechanism to bind the pools
each others.
Including the subpools that would just be pools registered in the cleanups list of their parent/owner.
Do you see another constraint, for a pool to be lately attached to another parent/owner, than the two pools have to use
the same allocator and the to-attach pool *not* to be an ancestor of the attached owner/parent pool (that is, not having
its own cleanup registered in the owner) ?
I'm surely missing something, and I surely need some advices,
regards,
Yann.
Re: Opaque structures in general (was Re: Opaque apr_pool_t structure)
Posted by Yann Lavictoire <yl...@bee-ware.net>.
Christopher Key wrote:
> Yann wrote:
>> Sorry I didn't explain myself very well.
>>
>> I don't wan't to play with pointers, of course, a single sizeof won't
>> allow me to do much more then a memcpy().
>>
>> For example ( innocently :p ) :
>>
>> struct mystuct {
>> apr_pool_t p;
>> ...
>> } myvar;
>> apr_pool_t p;
>>
>> apr_pool_create(&p, NULL);
>> memcpy(&myvar.p, p, apr_pool_sizeof());
>>
>> That's all I can do, I can't access any apr_pool field ...
>
>
> That sounds like a bad idea to me. At a conceptual level, just
> duplicating a number of bytes is not the same as creating a copy.
> Indeed, the concept of creating a copy is something that can only be
> defined specifically for each object.
>
Unless you only want to copy the pointers, which are still valid pointers.
Indeed with the memcpy() you don't create a real copy of the object, but the you can use this pseudo-copy as if it were
the original object (until this latter is destroyed of course).
Fields other then pointers in this pseudo-copy are nothing else than unused/wasted memory space.
> Specific examples why this is so:
> The apr_pool_t struct may contain pointers to other data that cannot be
> shared between pools. Again, this data would also need to be copied,
> but wouldn't in the above example.
As I said above, the goal can't be to create a real copy, and a pseudo-copy use as the original object is still working
here.
> The apr_pool_t struct may contain pointers to items within itself.
> These would no longer point to the correct location after the copy.
> There may be alignement issues with where an apr_pool_t can be stored on
> specific platforms. There's no way to guarantee that the copy your
> create will end up with the correct alignment.
> Plenty of others.
>
Same applies here, the pointers of the pseudo-copy still point to the original pool (the one in the allocator), and its
internal items.
I don't see any platform/alignement issue since apr_pool_sizeof() *is* platform specific too.
> Regards,
>
> Chris
>
Regards,
Yann.
Re: Opaque structures in general (was Re: Opaque apr_pool_t structure)
Posted by Yann <yl...@bee-ware.net>.
Sorry I didn't explain myself very well.
I don't wan't to play with pointers, of course, a single sizeof won't allow me to do much more then a memcpy().
For example ( innocently :p ) :
struct mystuct {
apr_pool_t p;
...
} myvar;
apr_pool_t p;
apr_pool_create(&p, NULL);
memcpy(&myvar.p, p, apr_pool_sizeof());
That's all I can do, I can't access any apr_pool field ...
Yann.
Ryan Bloom wrote:
> I don't think that there is any reason to not have a sizeof()
> function, other than any code that does "play" with the pointers will
> be non-portable code. The reason that I originally went with opaque
> data structures (I did it before giving the code to the ASF), was that
> most of the structures are defined totally differently on each
> platform. By making the structures opaque, it became much harder for
> a developer to write code with APR that worked on some APR platforms,
> but not others. If you play with the pointers, your code is very
> likely to work only on the platforms that you code on.
>
> But, I would like to hear from some of the active developers about this as well.
>
> Ryan
>
> On Fri, Jun 6, 2008 at 5:02 AM, Yann <yl...@bee-ware.net> wrote:
>> I'm sorry to insist on this point but, in general, maybe you should consider
>> having a function that provides the sizeof() each opaque structure/type of
>> the APR (maybe not all of them, but at least some).
>>
>> That would let the user "play" with the opaque types without breaking the
>> interest (portability, evolution ...) of this ones.
>>
>> As far as I know, the APR is a C programming API, so let us play with the
>> pointers if we find it usefull.
>>
>> I could propose the patch(s) if you agree and/or are interested by that.
>> If not, maybe you could let me know why ?
>>
>> Thanks for your answers,
>> Yann.
>>
>>>> Sander Striker wrote:
>>>>> Hello,
>>>>>
>>>>> I'm making a piece of software that recycles apr_sockets in
>>>>> sockets-pools.
>>>> Have you looked at apr_reslist in apr-util?
>>>>
>>> It looks very interesting, thank you for the hint, however it seems that I
>>> have to do the same trick to prevent a leak with my_socket (a stucture
>>> containing the apr_socket and other things associated with it).
>>>
>>> As the pool used to create the reslist is passed to the contructor for its
>>> own allocations (my_sockets here), I suppose I have to create a subpool in
>>> the constructor where my_socket will be allocated, and keep a reference to
>>> the subpool in my_socket to be able to release all that in the destructor.
>>>
>>> Without this, that is using the reslist pool directly, memory will leak as
>>> my_socket allocated fields or even the apr_socket internals (addr and like)
>>> will stay in the pool after destruction.
>>>
>>> Did I miss something ?
>>>
>>>>> Each socket is allocated and created on its own pool (not a subpool, or
>>>>> exactly a subpool of the internal global pool only).
>>>>> I've read on svn programming pages that it is not recommanded to do
>>>>> this,
>>>>> but I find it very usefull, that's the power of pools, IMHO.
>>>> The problem is the memory footprint that this brings with it. Each pool
>>>> will pre-allocate 8k of memory.
>>>>
>>> Maybe a subpool doesn't pre-allocate memory, or the problem remain with
>>> reslists (and the method above).
>>>
>>>>> Anyway, to make the things clean, I need a way to attach/detach pools
>>>>> to/from others, whether the socket belongs to the socket-pool's pool or
>>>>> to
>>>>> the user's one.
>>>>>
>>>>> As the apr_pool_join() function is a noop (and I don't knwow if that's
>>>>> its
>>>>> purpose anyway), I used the apr_pool_cleanup_register()ing /
>>>>> apr_pool_cleanup_kill()ing mechanism to attach / detach pools between
>>>>> them.
>>>>>
>>>>> I create a small new API with my pools based on the APR ones, where I
>>>>> can
>>>>> attach/detach pools, as I said, but also malloc() and realloc() memories
>>>>> attached to a pool (with the same registering mechanism).
>>>>>
>>>>> My pool struct is like :
>>>>>
>>>>> struct my_pool_t {
>>>>> apr_pool_t pool;
>>>>> apr_pool_t *owner;
>>>>> };
>>>>>
>>>>> And the functions :
>>>>>
>>>>> apr_status_t my_pool_create(my_pool_t **pool, void *owner);
>>>>> apr_status_t my_pool_attach(my_pool_t *pool, void *owner);
>>>>> apr_status_t my_pool_detach(my_pool_t *pool);
>>>>>
>>>>> apr_status_t my_pmalloc(void *pool, apr_size_t size);
>>>>> apr_status_t my_pzalloc(void *pool, apr_size_t size);
>>>>> apr_status_t my_prealloc(void *pool, void *ptr, apr_size_t size);
>>>>> void my_pfree(void *pool, void *ptr);
>>>>>
>>>>> As you can see with the first element of the struct which is an APR pool
>>>>> and
>>>>> 'void*' types used in functions, I expect the functions to be
>>>>> "compatible"
>>>>> with the APR ones, that is I can use my_pool_t and apr_pool_t pools
>>>>> indistinctly with them (where the type is void*) ...
>>>> That's not the case, see below.
>>>>
>>> Hum, I see, the pool stands in an allocator node, see below.
>>>
>>>>> My problem is that I can't initialize my_pool because I haven't got the
>>>>> sizeof(apr_pool_t), an opaque structure ...
>>>>>
>>>>> I understand the advantages of opaque types, but is there a way an
>>>>> apr_pool_sizeof() function be added (and exported) in the APR, simply
>>>>> like :
>>>>>
>>>>> APR_DECLARE(apr_size_t) apr_pool_sizeof(void)
>>>>> {
>>>>> /* maybe the aligned SIZEOF_POOL_T should be used */
>>>>> return sizeof(apr_pool_t);
>>>>> }
>>>>>
>>>>> With it, I could at least do :
>>>>> apr_pool_create(&p, NULL);
>>>>> memcpy(my_pool->pool, p, apr_pool_sizeof())
>>>> This would be a bad idea, as you'd be copying pointers and the like,
>>>> which
>>>> would be off as soon as copied. Even if that were not the case, the
>>>> apr_pool_t
>>>> struct instance lives in an allocated block of memory from which the pool
>>>> will hand out memory. This block in turn is managed by an allocator.
>>>>
>>> But the allocator node shouldn't be destroyed (or reused) while the pool
>>> isn't, so I don't see why a copy of the pool couldn't do the job as well.
>>>
>>> The copy of the pointers and the like still reference the allocator node,
>>> and it should work.
>>>
>>> However all this becomes cheat and I'd better look at apr_reslist.
>>>
>>>>> Or should I consider using my_p*() functions strictly with my_pools ?
>>>>>
>>>>> Thanks for your advices, answers.
>>>> Cheers,
>>>>
>>>> Sander
>>>>
>>> Cheers, and thanks for your answers,
>>> Yann.
>>>
>>>
>>
>
>
>
Re: Opaque structures in general (was Re: Opaque apr_pool_t structure)
Posted by Ryan Bloom <rb...@gmail.com>.
On Fri, Jun 6, 2008 at 9:18 AM, Yann <yl...@bee-ware.net> wrote:
> Erik Huelsmann wrote:
>>
>> On 6/6/08, Ryan Bloom <rb...@gmail.com> wrote:
>>>
>>> I don't think that there is any reason to not have a sizeof()
>>> function, other than any code that does "play" with the pointers will
>>> be non-portable code. The reason that I originally went with opaque
>>> data structures (I did it before giving the code to the ASF), was that
>>> most of the structures are defined totally differently on each
>>> platform. By making the structures opaque, it became much harder for
>>> a developer to write code with APR that worked on some APR platforms,
>>> but not others. If you play with the pointers, your code is very
>>> likely to work only on the platforms that you code on.
>>>
>>> But, I would like to hear from some of the active developers about this
>>> as well.
>>
>> Well, as soon as you provide its size, it's not completely opaque
>> anymore, now is it :-)
>>
>
> Yes of course, but as soon you provide an accessor it isn't neither ...
>
>> I think the entire issue is centered around the fact that Yann doesn't
>> really want to play by the pool-rules...
>>
>
> I would love to play with the APR pools if they could become subpools of
> others after their creation.
>
> That is, a pool being the whole object (destroyed with its pool), and
> objects living or dying with their changing owners.
>
> Why couldn't that be compatible with the pool rules ?
If this is what you really want/need, then I would suggest focusing on
a patch that implements this functionality.
Ryan
--
Ryan Bloom
rbb@apache.org
rbb@rkbloom.net
rbloom@gmail.com
Re: Opaque structures in general (was Re: Opaque apr_pool_t structure)
Posted by Yann <yl...@bee-ware.net>.
Erik Huelsmann wrote:
> On 6/6/08, Ryan Bloom <rb...@gmail.com> wrote:
>> I don't think that there is any reason to not have a sizeof()
>> function, other than any code that does "play" with the pointers will
>> be non-portable code. The reason that I originally went with opaque
>> data structures (I did it before giving the code to the ASF), was that
>> most of the structures are defined totally differently on each
>> platform. By making the structures opaque, it became much harder for
>> a developer to write code with APR that worked on some APR platforms,
>> but not others. If you play with the pointers, your code is very
>> likely to work only on the platforms that you code on.
>>
>> But, I would like to hear from some of the active developers about this as well.
>
> Well, as soon as you provide its size, it's not completely opaque
> anymore, now is it :-)
>
Yes of course, but as soon you provide an accessor it isn't neither ...
> I think the entire issue is centered around the fact that Yann doesn't
> really want to play by the pool-rules...
>
I would love to play with the APR pools if they could become subpools of others after their creation.
That is, a pool being the whole object (destroyed with its pool), and objects living or dying with their changing owners.
Why couldn't that be compatible with the pool rules ?
> I've been subscribed to this list a few years now. I've heard people
> regretting that APR uses pools for all of its portability
> functionality. I've seen the Subversion devs juggle with pool life
> time problems, but I've never heard anybody wanting to copy pools.
> Having done quite my share of pool life time juggling myself, I really
> don't understand the desire to copy pools around: you have the
> pointer, you don't know what else has a pointer...
>
> Bye,
>
> Erik.
>
I explained myself for what I intended by a copy, this is not really a copy, just the possibility to use the apr_pool_t
as a first field of a structure to make this one compatible with the apr_pool_t struct.
I can do that without the sizeof(apr_pool_t), or a function that returns it ...
Regards,
Yann.
Re: Opaque structures in general (was Re: Opaque apr_pool_t structure)
Posted by Erik Huelsmann <eh...@gmail.com>.
On 6/6/08, Ryan Bloom <rb...@gmail.com> wrote:
> I don't think that there is any reason to not have a sizeof()
> function, other than any code that does "play" with the pointers will
> be non-portable code. The reason that I originally went with opaque
> data structures (I did it before giving the code to the ASF), was that
> most of the structures are defined totally differently on each
> platform. By making the structures opaque, it became much harder for
> a developer to write code with APR that worked on some APR platforms,
> but not others. If you play with the pointers, your code is very
> likely to work only on the platforms that you code on.
>
> But, I would like to hear from some of the active developers about this as well.
Well, as soon as you provide its size, it's not completely opaque
anymore, now is it :-)
I think the entire issue is centered around the fact that Yann doesn't
really want to play by the pool-rules...
I've been subscribed to this list a few years now. I've heard people
regretting that APR uses pools for all of its portability
functionality. I've seen the Subversion devs juggle with pool life
time problems, but I've never heard anybody wanting to copy pools.
Having done quite my share of pool life time juggling myself, I really
don't understand the desire to copy pools around: you have the
pointer, you don't know what else has a pointer...
Bye,
Erik.
Re: Opaque structures in general (was Re: Opaque apr_pool_t structure)
Posted by Ryan Bloom <rb...@gmail.com>.
I don't think that there is any reason to not have a sizeof()
function, other than any code that does "play" with the pointers will
be non-portable code. The reason that I originally went with opaque
data structures (I did it before giving the code to the ASF), was that
most of the structures are defined totally differently on each
platform. By making the structures opaque, it became much harder for
a developer to write code with APR that worked on some APR platforms,
but not others. If you play with the pointers, your code is very
likely to work only on the platforms that you code on.
But, I would like to hear from some of the active developers about this as well.
Ryan
On Fri, Jun 6, 2008 at 5:02 AM, Yann <yl...@bee-ware.net> wrote:
> I'm sorry to insist on this point but, in general, maybe you should consider
> having a function that provides the sizeof() each opaque structure/type of
> the APR (maybe not all of them, but at least some).
>
> That would let the user "play" with the opaque types without breaking the
> interest (portability, evolution ...) of this ones.
>
> As far as I know, the APR is a C programming API, so let us play with the
> pointers if we find it usefull.
>
> I could propose the patch(s) if you agree and/or are interested by that.
> If not, maybe you could let me know why ?
>
> Thanks for your answers,
> Yann.
>
>>> Sander Striker wrote:
>>>>
>>>> Hello,
>>>>
>>>> I'm making a piece of software that recycles apr_sockets in
>>>> sockets-pools.
>>>
>>> Have you looked at apr_reslist in apr-util?
>>>
>>
>> It looks very interesting, thank you for the hint, however it seems that I
>> have to do the same trick to prevent a leak with my_socket (a stucture
>> containing the apr_socket and other things associated with it).
>>
>> As the pool used to create the reslist is passed to the contructor for its
>> own allocations (my_sockets here), I suppose I have to create a subpool in
>> the constructor where my_socket will be allocated, and keep a reference to
>> the subpool in my_socket to be able to release all that in the destructor.
>>
>> Without this, that is using the reslist pool directly, memory will leak as
>> my_socket allocated fields or even the apr_socket internals (addr and like)
>> will stay in the pool after destruction.
>>
>> Did I miss something ?
>>
>>>> Each socket is allocated and created on its own pool (not a subpool, or
>>>> exactly a subpool of the internal global pool only).
>>>> I've read on svn programming pages that it is not recommanded to do
>>>> this,
>>>> but I find it very usefull, that's the power of pools, IMHO.
>>>
>>> The problem is the memory footprint that this brings with it. Each pool
>>> will pre-allocate 8k of memory.
>>>
>>
>> Maybe a subpool doesn't pre-allocate memory, or the problem remain with
>> reslists (and the method above).
>>
>>>> Anyway, to make the things clean, I need a way to attach/detach pools
>>>> to/from others, whether the socket belongs to the socket-pool's pool or
>>>> to
>>>> the user's one.
>>>>
>>>> As the apr_pool_join() function is a noop (and I don't knwow if that's
>>>> its
>>>> purpose anyway), I used the apr_pool_cleanup_register()ing /
>>>> apr_pool_cleanup_kill()ing mechanism to attach / detach pools between
>>>> them.
>>>>
>>>> I create a small new API with my pools based on the APR ones, where I
>>>> can
>>>> attach/detach pools, as I said, but also malloc() and realloc() memories
>>>> attached to a pool (with the same registering mechanism).
>>>>
>>>> My pool struct is like :
>>>>
>>>> struct my_pool_t {
>>>> apr_pool_t pool;
>>>> apr_pool_t *owner;
>>>> };
>>>>
>>>> And the functions :
>>>>
>>>> apr_status_t my_pool_create(my_pool_t **pool, void *owner);
>>>> apr_status_t my_pool_attach(my_pool_t *pool, void *owner);
>>>> apr_status_t my_pool_detach(my_pool_t *pool);
>>>>
>>>> apr_status_t my_pmalloc(void *pool, apr_size_t size);
>>>> apr_status_t my_pzalloc(void *pool, apr_size_t size);
>>>> apr_status_t my_prealloc(void *pool, void *ptr, apr_size_t size);
>>>> void my_pfree(void *pool, void *ptr);
>>>>
>>>> As you can see with the first element of the struct which is an APR pool
>>>> and
>>>> 'void*' types used in functions, I expect the functions to be
>>>> "compatible"
>>>> with the APR ones, that is I can use my_pool_t and apr_pool_t pools
>>>> indistinctly with them (where the type is void*) ...
>>>
>>> That's not the case, see below.
>>>
>>
>> Hum, I see, the pool stands in an allocator node, see below.
>>
>>>> My problem is that I can't initialize my_pool because I haven't got the
>>>> sizeof(apr_pool_t), an opaque structure ...
>>>>
>>>> I understand the advantages of opaque types, but is there a way an
>>>> apr_pool_sizeof() function be added (and exported) in the APR, simply
>>>> like :
>>>>
>>>> APR_DECLARE(apr_size_t) apr_pool_sizeof(void)
>>>> {
>>>> /* maybe the aligned SIZEOF_POOL_T should be used */
>>>> return sizeof(apr_pool_t);
>>>> }
>>>>
>>>> With it, I could at least do :
>>>> apr_pool_create(&p, NULL);
>>>> memcpy(my_pool->pool, p, apr_pool_sizeof())
>>>
>>> This would be a bad idea, as you'd be copying pointers and the like,
>>> which
>>> would be off as soon as copied. Even if that were not the case, the
>>> apr_pool_t
>>> struct instance lives in an allocated block of memory from which the pool
>>> will hand out memory. This block in turn is managed by an allocator.
>>>
>>
>> But the allocator node shouldn't be destroyed (or reused) while the pool
>> isn't, so I don't see why a copy of the pool couldn't do the job as well.
>>
>> The copy of the pointers and the like still reference the allocator node,
>> and it should work.
>>
>> However all this becomes cheat and I'd better look at apr_reslist.
>>
>>>> Or should I consider using my_p*() functions strictly with my_pools ?
>>>>
>>>> Thanks for your advices, answers.
>>>
>>> Cheers,
>>>
>>> Sander
>>>
>>
>> Cheers, and thanks for your answers,
>> Yann.
>>
>>
>
>
--
Ryan Bloom
rbb@apache.org
rbb@rkbloom.net
rbloom@gmail.com