You are viewing a plain text version of this content. The canonical link for it is here.
Posted to proton@qpid.apache.org by "Darryl L. Pierce" <dp...@redhat.com> on 2015/05/07 15:41:59 UTC
Introducing the Ruby Reactive APIs
I've been working on this codebase since the beginning of the year. The
two branches [1, 2] in my git repo represent the low-level engine APIs
and the higher-level reactive APIs, respectively.
I'm still working through the set of example apps for the reactive APIs,
but at this point I feel this is close enough that I want to start
getting feedback from people.
== Memory Concerns
Of particular important is memory management: the Proton libraries use
reference counting to manage object lifespans, while Ruby uses mark and
sweep operations for garbage collection. So ensuring that pure Ruby
objects aren't reaped when they've only known to the Proton libraries,
in the case of event handlers specifically, has been a challenge and one
that's sure to have some cases that need fixing.
The first model explored was to attachment the Ruby wrapper objects to
the Swig-generated wrappers for the underlying C structs in Proton.
Which worked at first, but turned out to be not useful. The reason being
that the Swig bindings were themselves being reaped when they went out
of scope; i.e., Swig doesn't maintain them by providing a mark operation
until disposal of the underlying C structs. So this path, while
initially promising, was discarded.
The current model uses a hash table that is attached to the Qpid::Proton
module. When objects are stored for use by the C libraries, they are
tucked away in this hash table with a unique key generated based on
memory addresses. A copy of that key, as a char*, is given to Proton to
use later when the object is being retrieved.
To help with this, two additional callback APIs were added to the Proton
libraries: pn_record_set_callback and pn_record_has_callback. These two
functions work to help register a method to be called whenever a record
is deleted to enable memory management. This way the above-mentioned key
can be properly deleted, and the value stored in the hash table
discarded.
The reference counting aspect of the Proton libraries is a concern as
well. The code currently increments and decrements references in the
same places as the Python code, but there are likely more places where
such reference accounting need to be added.
[1] http://github.com/mcpierce/Proton/tree/PROTON-799-Ruby-engine-apis
[2] http://github.com/mcpierce/Proton/tree/PROTON-781-reactive-ruby-apis
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: Introducing the Ruby Reactive APIs
Posted by "Darryl L. Pierce" <dp...@redhat.com>.
On Tue, May 12, 2015 at 04:10:54PM -0400, Darryl L. Pierce wrote:
> On Tue, May 12, 2015 at 11:12:40AM -0400, Rafael Schloming wrote:
> > I made a bunch of line comments.
>
> Things appear to be working in the scaled back example except for one
> thing: when retrieving the record with pn_record_get the Swig wrapper
> returned is of type SWIG:TYPE_p_void rather than the expected wrapper
> for pn_rbkey_t. I think that Swig is confused by the type being returned
> by pn_record_get and can't distinguish what the appropriate wrapper
> should be.
>
> When the return value is then used for fetching the key, it fails with:
>
> TypeError: Expected argument 0 of type pn_rbkey_t *, but got SWIG::TYPE_p_void #<SWIG::TYPE_p_void:0x00000000...
Fixed it with a cast function of pni_void2rbkey(). My tests are now
flying in Ruby, so I'm far more confident that this approach is cleaner
and more efficient so far.
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: Introducing the Ruby Reactive APIs
Posted by "Darryl L. Pierce" <dp...@redhat.com>.
On Tue, May 12, 2015 at 11:12:40AM -0400, Rafael Schloming wrote:
> I made a bunch of line comments.
Excellent, thanks for that. I'll update the code and reply with my
results.
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: Introducing the Ruby Reactive APIs
Posted by "Darryl L. Pierce" <dp...@redhat.com>.
On Tue, May 12, 2015 at 11:12:40AM -0400, Rafael Schloming wrote:
> I made a bunch of line comments.
Things appear to be working in the scaled back example except for one
thing: when retrieving the record with pn_record_get the Swig wrapper
returned is of type SWIG:TYPE_p_void rather than the expected wrapper
for pn_rbkey_t. I think that Swig is confused by the type being returned
by pn_record_get and can't distinguish what the appropriate wrapper
should be.
When the return value is then used for fetching the key, it fails with:
TypeError: Expected argument 0 of type pn_rbkey_t *, but got SWIG::TYPE_p_void #<SWIG::TYPE_p_void:0x00000000...
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: Introducing the Ruby Reactive APIs
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Tue, May 12, 2015 at 10:34 AM, Darryl L. Pierce <dp...@redhat.com>
wrote:
> On Tue, May 12, 2015 at 09:44:41AM -0400, Rafael Schloming wrote:
> > On Tue, May 12, 2015 at 8:34 AM, Darryl L. Pierce <dp...@redhat.com>
> > wrote:
> >
> > > On Tue, May 12, 2015 at 05:45:20AM -0400, Rafael Schloming wrote:
> > > > Can you post an isolated reproducer with just your definition of
> > > pn_rbkey_t
> > > > and a code version of the 5 steps that lead to the seg fault?
> > >
> > > On my PROTON-781-reactive-ruby-apis branch is an example named
> > > "$REPO/examples/ruby/registry_test.rb" which does it. It's very pared
> > > down, only
> > > creating 3 Transport instances and consistently produces the segfault.
> > >
> >
> > That branch has about a 15 thousand line delta from master. That's a lot
> of
> > lines of code to hide a subtle memory bug. The pn_rbkey_t definition and
> > enough ruby code to use it in a proof of concept should only require a
> few
> > hundred line delta from master. I suggest producing just this delta for
> two
> > reasons. 1) Just the act of producing it will help narrow down where the
> > bug is, and 2) it gives me a much smaller delta to look at so I can be
> more
> > useful to you.
> >
> > (I did run the registry_test.rb that you pointed to through a debugger,
> > however its not obvious what the issue is upon inspection of the trace,
> and
> > while the ruby.i delta is pretty self contained, the details of how you
> are
> > using it from ruby are somewhere buried in the 15K diff.)
>
> I've pulled the pertinent pieces into a new branch:
>
> http://github.com/mcpierce/Proton/tree/rbkey-isolation
>
> The changes are all here:
>
>
> http://github.com/mcpierce/Proton/commit/2dc770992a7c02fe2054a4a77325a199e39b9c94
>
> and it blows up exactly as I've been experiencing.
>
I made a bunch of line comments.
--Rafael
Re: Introducing the Ruby Reactive APIs
Posted by "Darryl L. Pierce" <dp...@redhat.com>.
On Tue, May 12, 2015 at 09:44:41AM -0400, Rafael Schloming wrote:
> On Tue, May 12, 2015 at 8:34 AM, Darryl L. Pierce <dp...@redhat.com>
> wrote:
>
> > On Tue, May 12, 2015 at 05:45:20AM -0400, Rafael Schloming wrote:
> > > Can you post an isolated reproducer with just your definition of
> > pn_rbkey_t
> > > and a code version of the 5 steps that lead to the seg fault?
> >
> > On my PROTON-781-reactive-ruby-apis branch is an example named
> > "$REPO/examples/ruby/registry_test.rb" which does it. It's very pared
> > down, only
> > creating 3 Transport instances and consistently produces the segfault.
> >
>
> That branch has about a 15 thousand line delta from master. That's a lot of
> lines of code to hide a subtle memory bug. The pn_rbkey_t definition and
> enough ruby code to use it in a proof of concept should only require a few
> hundred line delta from master. I suggest producing just this delta for two
> reasons. 1) Just the act of producing it will help narrow down where the
> bug is, and 2) it gives me a much smaller delta to look at so I can be more
> useful to you.
>
> (I did run the registry_test.rb that you pointed to through a debugger,
> however its not obvious what the issue is upon inspection of the trace, and
> while the ruby.i delta is pretty self contained, the details of how you are
> using it from ruby are somewhere buried in the 15K diff.)
I've pulled the pertinent pieces into a new branch:
http://github.com/mcpierce/Proton/tree/rbkey-isolation
The changes are all here:
http://github.com/mcpierce/Proton/commit/2dc770992a7c02fe2054a4a77325a199e39b9c94
and it blows up exactly as I've been experiencing.
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: Introducing the Ruby Reactive APIs
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Tue, May 12, 2015 at 8:34 AM, Darryl L. Pierce <dp...@redhat.com>
wrote:
> On Tue, May 12, 2015 at 05:45:20AM -0400, Rafael Schloming wrote:
> > Can you post an isolated reproducer with just your definition of
> pn_rbkey_t
> > and a code version of the 5 steps that lead to the seg fault?
>
> On my PROTON-781-reactive-ruby-apis branch is an example named
> "$REPO/examples/ruby/registry_test.rb" which does it. It's very pared
> down, only
> creating 3 Transport instances and consistently produces the segfault.
>
That branch has about a 15 thousand line delta from master. That's a lot of
lines of code to hide a subtle memory bug. The pn_rbkey_t definition and
enough ruby code to use it in a proof of concept should only require a few
hundred line delta from master. I suggest producing just this delta for two
reasons. 1) Just the act of producing it will help narrow down where the
bug is, and 2) it gives me a much smaller delta to look at so I can be more
useful to you.
(I did run the registry_test.rb that you pointed to through a debugger,
however its not obvious what the issue is upon inspection of the trace, and
while the ruby.i delta is pretty self contained, the details of how you are
using it from ruby are somewhere buried in the 15K diff.)
--Rafael
Re: Introducing the Ruby Reactive APIs
Posted by "Darryl L. Pierce" <dp...@redhat.com>.
On Tue, May 12, 2015 at 05:45:20AM -0400, Rafael Schloming wrote:
> Can you post an isolated reproducer with just your definition of pn_rbkey_t
> and a code version of the 5 steps that lead to the seg fault?
On my PROTON-781-reactive-ruby-apis branch is an example named
"$REPO/examples/ruby/registry_test.rb" which does it. It's very pared down, only
creating 3 Transport instances and consistently produces the segfault.
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: Introducing the Ruby Reactive APIs
Posted by Rafael Schloming <rh...@alum.mit.edu>.
Can you post an isolated reproducer with just your definition of pn_rbkey_t
and a code version of the 5 steps that lead to the seg fault?
--Rafael
On Mon, May 11, 2015 at 5:21 PM, Darryl L. Pierce <dp...@redhat.com>
wrote:
> On Thu, May 07, 2015 at 01:02:13PM -0400, Rafael Schloming wrote:
> > The way the python code does this is by checking whenever a C object is
> > returned to python code. If the record contains an attachment indicating
> > that the C object has previously been wrapped, it uses this to
> > construct/retrieve an appropriate wrapper object. If it doesn't have the
> > appropriate attachment then it uses the record API to define/set the
> > attachment to the appropriate value. I presume you could do something
> > similar with ruby.
>
> After we chatted the other day, I've tried the following approach, using
> the pn_transport_t type as my test bed since it has relatively fewer
> dependencies. However, the plumbing in Proton for objects isn't quite
> clear to me and my code's not quite working the way we had discussed and
> I'm not sure why.
>
> The goal is to have a type that will live for as long as one of the
> impls in Proton lives; i.e., when we create something like
> pn_transport_t, the attachment created for this would hold some type
> that will get finalized when the pn_transport_t type is finalized. And
> that type would be the hook to clean up the single instance of a Ruby
> class that wraps the underlying C type.
>
> I've created a new type in the ruby.i descriptor for Swig and named it
> pn_rbkey_t, with three files: void* registry (a pointer to an object
> held in Ruby), char* method (the name of a method to invoke on that
> object), and char* key_value (the argument to be passed to that method).
>
> The code defines pn_rbkey_initialize and pn_rbkey_finalize methods, as
> well as getter and setter methods for the three fields. But I've put
> debugging into the code and never see the pn_rbkey_finalize method being
> invoked.
>
> My registry_test app does the following:
>
> 1. create an instance of pn_transport_t: impl = Cproton.pn_transport
> 2. create a Ruby Transport object: transport = Transport.wrap(impl)
> a. puts a weak reference to the Transport into the hashtable
> b. creates a pn_rbkey_t object and sets it as the sole record for the
> pn_transport_t object
> c. calls Cproton.pn_incref on the pn_rbkey_t instance
> 3. remove the reference: transport = nil
> 4. call garbage collection: ObjectSpace.garbage_collect
> 5. get the object back: transport = Transport.wrap(impl)
> a. calls pn_transport_attachment and retrieves the record created in 2
> b. should then be able to get the key_value from the pn_rbkey_t type
> c. should then get the object out of the hashtable to return
>
> It's at step 5 that the example app segfaults. The segfault happens
> when, from Ruby, there's a call to print the attachment retrieve in 5a.
> Swig isn't failing since it's returning a value that seems to have been
> cached. But when Swig tries to retrieve data from the pn_rbkey_t struct
> underneath of it, *THAT* seems to have been reaped by Proton and Swig
> then segfaults, thinking there was an object still under the covers.
>
> Any ideas or suggestions of where to look for what's going on?
>
> --
> Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
> Delivering value year after year.
> Red Hat ranks #1 in value among software vendors.
> http://www.redhat.com/promo/vendor/
>
>
Re: Introducing the Ruby Reactive APIs
Posted by "Darryl L. Pierce" <dp...@redhat.com>.
On Thu, May 07, 2015 at 01:02:13PM -0400, Rafael Schloming wrote:
> The way the python code does this is by checking whenever a C object is
> returned to python code. If the record contains an attachment indicating
> that the C object has previously been wrapped, it uses this to
> construct/retrieve an appropriate wrapper object. If it doesn't have the
> appropriate attachment then it uses the record API to define/set the
> attachment to the appropriate value. I presume you could do something
> similar with ruby.
After we chatted the other day, I've tried the following approach, using
the pn_transport_t type as my test bed since it has relatively fewer
dependencies. However, the plumbing in Proton for objects isn't quite
clear to me and my code's not quite working the way we had discussed and
I'm not sure why.
The goal is to have a type that will live for as long as one of the
impls in Proton lives; i.e., when we create something like
pn_transport_t, the attachment created for this would hold some type
that will get finalized when the pn_transport_t type is finalized. And
that type would be the hook to clean up the single instance of a Ruby
class that wraps the underlying C type.
I've created a new type in the ruby.i descriptor for Swig and named it
pn_rbkey_t, with three files: void* registry (a pointer to an object
held in Ruby), char* method (the name of a method to invoke on that
object), and char* key_value (the argument to be passed to that method).
The code defines pn_rbkey_initialize and pn_rbkey_finalize methods, as
well as getter and setter methods for the three fields. But I've put
debugging into the code and never see the pn_rbkey_finalize method being
invoked.
My registry_test app does the following:
1. create an instance of pn_transport_t: impl = Cproton.pn_transport
2. create a Ruby Transport object: transport = Transport.wrap(impl)
a. puts a weak reference to the Transport into the hashtable
b. creates a pn_rbkey_t object and sets it as the sole record for the
pn_transport_t object
c. calls Cproton.pn_incref on the pn_rbkey_t instance
3. remove the reference: transport = nil
4. call garbage collection: ObjectSpace.garbage_collect
5. get the object back: transport = Transport.wrap(impl)
a. calls pn_transport_attachment and retrieves the record created in 2
b. should then be able to get the key_value from the pn_rbkey_t type
c. should then get the object out of the hashtable to return
It's at step 5 that the example app segfaults. The segfault happens
when, from Ruby, there's a call to print the attachment retrieve in 5a.
Swig isn't failing since it's returning a value that seems to have been
cached. But when Swig tries to retrieve data from the pn_rbkey_t struct
underneath of it, *THAT* seems to have been reaped by Proton and Swig
then segfaults, thinking there was an object still under the covers.
Any ideas or suggestions of where to look for what's going on?
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: Introducing the Ruby Reactive APIs
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Thu, May 7, 2015 at 11:49 AM, Darryl L. Pierce <dp...@redhat.com>
wrote:
> On Thu, May 07, 2015 at 11:32:33AM -0400, Rafael Schloming wrote:
> > On Thu, May 7, 2015 at 10:40 AM, Darryl L. Pierce <dp...@redhat.com>
> > wrote:
> >
> > > On Thu, May 07, 2015 at 09:57:49AM -0400, Rafael Schloming wrote:
> > > > On Thu, May 7, 2015 at 9:41 AM, Darryl L. Pierce <dpierce@redhat.com
> >
> > > wrote:
> > > <snip>
> > > > > To help with this, two additional callback APIs were added to the
> > > Proton
> > > > > libraries: pn_record_set_callback and pn_record_has_callback.
> These two
> > > > > functions work to help register a method to be called whenever a
> record
> > > > > is deleted to enable memory management. This way the
> above-mentioned
> > > key
> > > > > can be properly deleted, and the value stored in the hash table
> > > > > discarded.
> > > >
> > > > I would need to see the code in detail, but I suspect you don't need
> to
> > > add
> > > > a pn_record_set_callback/get_callback to achieve roughly the
> > > functionality.
> > > > I *think* you could simply define a pn_class_t that is a reference
> > > counted
> > > > holder of your key. You could then put your callback logic in the
> > > finalizer
> > > > for that class, and when proton's reference counting triggers the
> > > > finalizer, it will run the callback logic at the appropriate time.
> > >
> > > (edit)
> > >
> > > As I was writing up a description of the code I realized I have already
> > > done what you suggest above WRT the pni_rbhandler_t type. I could use
> > > the same logic to create a pni_rbrecord_t type and manage its lifecycle
> > > the same way the handler's lifecycles are managed, yeah?
> >
> > Yes, I believe so.
>
> Since records are created when a struct if initially created, I'm not
> sure how to go about attaching the key to its lifecycle since the
> dynamic language isn't explicitly creating the record.
>
The way the python code does this is by checking whenever a C object is
returned to python code. If the record contains an attachment indicating
that the C object has previously been wrapped, it uses this to
construct/retrieve an appropriate wrapper object. If it doesn't have the
appropriate attachment then it uses the record API to define/set the
attachment to the appropriate value. I presume you could do something
similar with ruby.
--Rafael
Re: Introducing the Ruby Reactive APIs
Posted by "Darryl L. Pierce" <dp...@redhat.com>.
On Thu, May 07, 2015 at 11:32:33AM -0400, Rafael Schloming wrote:
> On Thu, May 7, 2015 at 10:40 AM, Darryl L. Pierce <dp...@redhat.com>
> wrote:
>
> > On Thu, May 07, 2015 at 09:57:49AM -0400, Rafael Schloming wrote:
> > > On Thu, May 7, 2015 at 9:41 AM, Darryl L. Pierce <dp...@redhat.com>
> > wrote:
> > <snip>
> > > > To help with this, two additional callback APIs were added to the
> > Proton
> > > > libraries: pn_record_set_callback and pn_record_has_callback. These two
> > > > functions work to help register a method to be called whenever a record
> > > > is deleted to enable memory management. This way the above-mentioned
> > key
> > > > can be properly deleted, and the value stored in the hash table
> > > > discarded.
> > >
> > > I would need to see the code in detail, but I suspect you don't need to
> > add
> > > a pn_record_set_callback/get_callback to achieve roughly the
> > functionality.
> > > I *think* you could simply define a pn_class_t that is a reference
> > counted
> > > holder of your key. You could then put your callback logic in the
> > finalizer
> > > for that class, and when proton's reference counting triggers the
> > > finalizer, it will run the callback logic at the appropriate time.
> >
> > (edit)
> >
> > As I was writing up a description of the code I realized I have already
> > done what you suggest above WRT the pni_rbhandler_t type. I could use
> > the same logic to create a pni_rbrecord_t type and manage its lifecycle
> > the same way the handler's lifecycles are managed, yeah?
>
> Yes, I believe so.
Since records are created when a struct if initially created, I'm not
sure how to go about attaching the key to its lifecycle since the
dynamic language isn't explicitly creating the record.
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: Introducing the Ruby Reactive APIs
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Thu, May 7, 2015 at 10:40 AM, Darryl L. Pierce <dp...@redhat.com>
wrote:
> On Thu, May 07, 2015 at 09:57:49AM -0400, Rafael Schloming wrote:
> > On Thu, May 7, 2015 at 9:41 AM, Darryl L. Pierce <dp...@redhat.com>
> wrote:
> <snip>
> > > To help with this, two additional callback APIs were added to the
> Proton
> > > libraries: pn_record_set_callback and pn_record_has_callback. These two
> > > functions work to help register a method to be called whenever a record
> > > is deleted to enable memory management. This way the above-mentioned
> key
> > > can be properly deleted, and the value stored in the hash table
> > > discarded.
> >
> > I would need to see the code in detail, but I suspect you don't need to
> add
> > a pn_record_set_callback/get_callback to achieve roughly the
> functionality.
> > I *think* you could simply define a pn_class_t that is a reference
> counted
> > holder of your key. You could then put your callback logic in the
> finalizer
> > for that class, and when proton's reference counting triggers the
> > finalizer, it will run the callback logic at the appropriate time.
>
> (edit)
>
> As I was writing up a description of the code I realized I have already
> done what you suggest above WRT the pni_rbhandler_t type. I could use
> the same logic to create a pni_rbrecord_t type and manage its lifecycle
> the same way the handler's lifecycles are managed, yeah?
>
Yes, I believe so.
--Rafael
Re: Introducing the Ruby Reactive APIs
Posted by "Darryl L. Pierce" <dp...@redhat.com>.
On Thu, May 07, 2015 at 09:57:49AM -0400, Rafael Schloming wrote:
> On Thu, May 7, 2015 at 9:41 AM, Darryl L. Pierce <dp...@redhat.com> wrote:
<snip>
> > To help with this, two additional callback APIs were added to the Proton
> > libraries: pn_record_set_callback and pn_record_has_callback. These two
> > functions work to help register a method to be called whenever a record
> > is deleted to enable memory management. This way the above-mentioned key
> > can be properly deleted, and the value stored in the hash table
> > discarded.
>
> I would need to see the code in detail, but I suspect you don't need to add
> a pn_record_set_callback/get_callback to achieve roughly the functionality.
> I *think* you could simply define a pn_class_t that is a reference counted
> holder of your key. You could then put your callback logic in the finalizer
> for that class, and when proton's reference counting triggers the
> finalizer, it will run the callback logic at the appropriate time.
(edit)
As I was writing up a description of the code I realized I have already
done what you suggest above WRT the pni_rbhandler_t type. I could use
the same logic to create a pni_rbrecord_t type and manage its lifecycle
the same way the handler's lifecycles are managed, yeah?
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: Introducing the Ruby Reactive APIs
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Thu, May 7, 2015 at 9:41 AM, Darryl L. Pierce <dp...@redhat.com> wrote:
> I've been working on this codebase since the beginning of the year. The
> two branches [1, 2] in my git repo represent the low-level engine APIs
> and the higher-level reactive APIs, respectively.
>
> I'm still working through the set of example apps for the reactive APIs,
> but at this point I feel this is close enough that I want to start
> getting feedback from people.
>
> == Memory Concerns
>
> Of particular important is memory management: the Proton libraries use
> reference counting to manage object lifespans, while Ruby uses mark and
> sweep operations for garbage collection. So ensuring that pure Ruby
> objects aren't reaped when they've only known to the Proton libraries,
> in the case of event handlers specifically, has been a challenge and one
> that's sure to have some cases that need fixing.
>
> The first model explored was to attachment the Ruby wrapper objects to
> the Swig-generated wrappers for the underlying C structs in Proton.
> Which worked at first, but turned out to be not useful. The reason being
> that the Swig bindings were themselves being reaped when they went out
> of scope; i.e., Swig doesn't maintain them by providing a mark operation
> until disposal of the underlying C structs. So this path, while
> initially promising, was discarded.
>
> The current model uses a hash table that is attached to the Qpid::Proton
> module. When objects are stored for use by the C libraries, they are
> tucked away in this hash table with a unique key generated based on
> memory addresses. A copy of that key, as a char*, is given to Proton to
> use later when the object is being retrieved.
>
> To help with this, two additional callback APIs were added to the Proton
> libraries: pn_record_set_callback and pn_record_has_callback. These two
> functions work to help register a method to be called whenever a record
> is deleted to enable memory management. This way the above-mentioned key
> can be properly deleted, and the value stored in the hash table
> discarded.
>
I would need to see the code in detail, but I suspect you don't need to add
a pn_record_set_callback/get_callback to achieve roughly the functionality.
I *think* you could simply define a pn_class_t that is a reference counted
holder of your key. You could then put your callback logic in the finalizer
for that class, and when proton's reference counting triggers the
finalizer, it will run the callback logic at the appropriate time.
> The reference counting aspect of the Proton libraries is a concern as
> well. The code currently increments and decrements references in the
> same places as the Python code, but there are likely more places where
> such reference accounting need to be added.
>
> [1] http://github.com/mcpierce/Proton/tree/PROTON-799-Ruby-engine-apis
> [2] http://github.com/mcpierce/Proton/tree/PROTON-781-reactive-ruby-apis
> --
> Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
> Delivering value year after year.
> Red Hat ranks #1 in value among software vendors.
> http://www.redhat.com/promo/vendor/
>
>