You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Fraser Adams <fr...@blueyonder.co.uk> on 2014/03/28 09:20:09 UTC
question on Proton memory management
I mentioned the other day about the lack of API documentation for things
like message and codec/data
It'd be really useful to have documentation for at least the non-obvious
calls.
One thing I'm currently struggling with is understanding the
ownership/responsibility for the memory of items that consume or return
pn_bytes_t
What I *think* is the case is that for say pn_data_get_binary if I were
applying that to say the message body then it refers to memory that is
*owned* by the message, so I could access the bytes pointed to by
bytes.start whilst the message was in scope and I wouldn't need to do
any explicit free of bytes.start, however if I wanted to retain that
data I'd have to copy the data into my own buffer before I did say
pn_messenger_get again into the same message instance.
I guess in the case of data retrieval I'm asking if I don't generally
need to explicitly free the underlying data from a pn_bytes_t because
it's owned by the underlying message (or pn_data if I'm doing lower
level things).
So what about the reverse case? If I'm going to do pn_data_put_binary I
clearly need to create a pn_bytes_t and the start pointer for that would
likely be a block of memory that I've malloc'd - so at which point is
the ownership of that data "transferred" so that I can free my client
side buffer? I'm *guessing* that pn_data_put_binary copies the data from
the byte array pointed to by the pn_bytes_t start somewhere into the
underlying pn_data_t but that's only an assumption on my part because
that behaviour seems to make logical sense to me, it's far from clear
that this is what is actually going own.
Clearly understanding ownership of dynamically allocated memory is
pretty important for application efficiency (I'd like to avoid
unnecessary copies) and correctness (I definitely want to avoid leaks)
and this sort of thing gets even more important to understand if one
ends up passing or retrieving more complex data structures such as maps
or lists that might contain binary elements.
MTIA
Frase
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: question on Proton memory management
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
Thanks v. much for the quick response Rafael.
Cheers,
Frase
On 28/03/14 14:38, Rafael Schloming wrote:
> On Fri, Mar 28, 2014 at 4:20 AM, Fraser Adams <fraser.adams@blueyonder.co.uk
>> wrote:
>> I mentioned the other day about the lack of API documentation for things
>> like message and codec/data
>>
> Yeah, I missed those in my documentation pass a few weeks ago.
>
>
>> It'd be really useful to have documentation for at least the non-obvious
>> calls.
>>
>> One thing I'm currently struggling with is understanding the
>> ownership/responsibility for the memory of items that consume or return
>> pn_bytes_t
>>
>> What I *think* is the case is that for say pn_data_get_binary if I were
>> applying that to say the message body then it refers to memory that is
>> *owned* by the message, so I could access the bytes pointed to by
>> bytes.start whilst the message was in scope and I wouldn't need to do any
>> explicit free of bytes.start, however if I wanted to retain that data I'd
>> have to copy the data into my own buffer before I did say pn_messenger_get
>> again into the same message instance.
>>
>> I guess in the case of data retrieval I'm asking if I don't generally need
>> to explicitly free the underlying data from a pn_bytes_t because it's owned
>> by the underlying message (or pn_data if I'm doing lower level things).
>>
> Your assumption is correct. When pn_bytes_t is returned it is giving you a
> pointer to memory owned by the object you are accessing.
>
>
>>
>> So what about the reverse case? If I'm going to do pn_data_put_binary I
>> clearly need to create a pn_bytes_t and the start pointer for that would
>> likely be a block of memory that I've malloc'd - so at which point is the
>> ownership of that data "transferred" so that I can free my client side
>> buffer? I'm *guessing* that pn_data_put_binary copies the data from the
>> byte array pointed to by the pn_bytes_t start somewhere into the underlying
>> pn_data_t but that's only an assumption on my part because that behaviour
>> seems to make logical sense to me, it's far from clear that this is what is
>> actually going own.
>>
> Your assumption is again correct.
>
>
>> Clearly understanding ownership of dynamically allocated memory is pretty
>> important for application efficiency (I'd like to avoid unnecessary copies)
>> and correctness (I definitely want to avoid leaks) and this sort of thing
>> gets even more important to understand if one ends up passing or retrieving
>> more complex data structures such as maps or lists that might contain
>> binary elements.
>>
> The missing docs should be up soon.
>
> --Rafael
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: question on Proton memory management
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Fri, Mar 28, 2014 at 4:20 AM, Fraser Adams <fraser.adams@blueyonder.co.uk
> wrote:
> I mentioned the other day about the lack of API documentation for things
> like message and codec/data
>
Yeah, I missed those in my documentation pass a few weeks ago.
>
> It'd be really useful to have documentation for at least the non-obvious
> calls.
>
> One thing I'm currently struggling with is understanding the
> ownership/responsibility for the memory of items that consume or return
> pn_bytes_t
>
> What I *think* is the case is that for say pn_data_get_binary if I were
> applying that to say the message body then it refers to memory that is
> *owned* by the message, so I could access the bytes pointed to by
> bytes.start whilst the message was in scope and I wouldn't need to do any
> explicit free of bytes.start, however if I wanted to retain that data I'd
> have to copy the data into my own buffer before I did say pn_messenger_get
> again into the same message instance.
>
> I guess in the case of data retrieval I'm asking if I don't generally need
> to explicitly free the underlying data from a pn_bytes_t because it's owned
> by the underlying message (or pn_data if I'm doing lower level things).
>
Your assumption is correct. When pn_bytes_t is returned it is giving you a
pointer to memory owned by the object you are accessing.
>
>
> So what about the reverse case? If I'm going to do pn_data_put_binary I
> clearly need to create a pn_bytes_t and the start pointer for that would
> likely be a block of memory that I've malloc'd - so at which point is the
> ownership of that data "transferred" so that I can free my client side
> buffer? I'm *guessing* that pn_data_put_binary copies the data from the
> byte array pointed to by the pn_bytes_t start somewhere into the underlying
> pn_data_t but that's only an assumption on my part because that behaviour
> seems to make logical sense to me, it's far from clear that this is what is
> actually going own.
>
Your assumption is again correct.
>
> Clearly understanding ownership of dynamically allocated memory is pretty
> important for application efficiency (I'd like to avoid unnecessary copies)
> and correctness (I definitely want to avoid leaks) and this sort of thing
> gets even more important to understand if one ends up passing or retrieving
> more complex data structures such as maps or lists that might contain
> binary elements.
>
The missing docs should be up soon.
--Rafael