Ruby, Proton Engine and Records

["Darryl L. Pierce" <dp...@redhat.com>]

So over the last week I've been working on a way to avoid leaking memory
or causing segfaults when the underlying C library is giving a reference
to a Ruby object. And after much coding and gnashing of teeth I foudn
the easiest way to do this with manually wrapping the pn_record_t
struct. That would allow for adding a mark function to mark any Ruby
objects so that they could not be reaped during garbage collection.

And this would work fine but for one problem: the instances of pn_record
are created by transport, etc. as part of *their* initializations. So
there is no way for us to manually wrap the pn_record_t type for Ruby
use in Swig.

So looking at things, I have an alternate method that's not brittle and
which gives us the feature we're looking for. From the Swig code, we can
wrap the call to pn_record_set, at which point we have access to the
Ruby wrapper for the pn_record_t type. We can then add/modify instance
variables on that Ruby wrapper. Specifically, we can add and then update
one named pn_record_attrs (or any other arbitrary name) and use it to
track any objects that're added or replaced for the record. 

And the bonus is that, when the pn_record_t is deleted, it's Ruby
wrapper is deleted as well and _all_ instance variables (including our
Ruby objects) will be reaped along with it.

I'm going to do some more testing on this, but my initial findings
support this memory management scheme as viable. I have a test
application that creates 500k+ values, storing them in a single record
instance.

Iterating back over the records and recovering them after running garbage
collect returns variables as expected.

Commenting out the portion that manages the wrapper instance variable
and the same tests fail with a segfault while accessing the records.

The work can all be checked out here [1], with the changes to the ruby.i
file as the most recent commit in the branch. I'd love to get some
feedback on this work in general as well.

[1] https://github.com/mcpierce/Proton/tree/PROTON-799-Ruby-engine-apis

-- 
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/

Re: Ruby, Proton Engine and Records

["Darryl L. Pierce" <dp...@redhat.com>]

On Fri, Feb 06, 2015 at 08:37:07AM -0500, Rafael Schloming wrote:
> Is this the approach you referenced in your other email? (Same request here
> for a pointer to the highlighted key bits of the approach.)

No, this was a third approach to the problem, since we can't directly
affect the object created by Swig via Data_Wrap_Struct. You can see the
specific changes to ruby.i here [1].

http://github.com/mcpierce/Proton/commit/1e66f39535e71dc376d69abde7b5741929d33a3d

-- 
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/

Re: Ruby, Proton Engine and Records

[Rafael Schloming <rh...@alum.mit.edu>]

Is this the approach you referenced in your other email? (Same request here
for a pointer to the highlighted key bits of the approach.)

On Fri, Jan 30, 2015 at 4:11 PM, Darryl L. Pierce <dp...@redhat.com>
wrote:

> So over the last week I've been working on a way to avoid leaking memory
> or causing segfaults when the underlying C library is giving a reference
> to a Ruby object. And after much coding and gnashing of teeth I foudn
> the easiest way to do this with manually wrapping the pn_record_t
> struct. That would allow for adding a mark function to mark any Ruby
> objects so that they could not be reaped during garbage collection.
>
> And this would work fine but for one problem: the instances of pn_record
> are created by transport, etc. as part of *their* initializations. So
> there is no way for us to manually wrap the pn_record_t type for Ruby
> use in Swig.
>
> So looking at things, I have an alternate method that's not brittle and
> which gives us the feature we're looking for. From the Swig code, we can
> wrap the call to pn_record_set, at which point we have access to the
> Ruby wrapper for the pn_record_t type. We can then add/modify instance
> variables on that Ruby wrapper. Specifically, we can add and then update
> one named pn_record_attrs (or any other arbitrary name) and use it to
> track any objects that're added or replaced for the record.
>
> And the bonus is that, when the pn_record_t is deleted, it's Ruby
> wrapper is deleted as well and _all_ instance variables (including our
> Ruby objects) will be reaped along with it.
>
> I'm going to do some more testing on this, but my initial findings
> support this memory management scheme as viable. I have a test
> application that creates 500k+ values, storing them in a single record
> instance.
>
> Iterating back over the records and recovering them after running garbage
> collect returns variables as expected.
>
> Commenting out the portion that manages the wrapper instance variable
> and the same tests fail with a segfault while accessing the records.
>
> The work can all be checked out here [1], with the changes to the ruby.i
> file as the most recent commit in the branch. I'd love to get some
> feedback on this work in general as well.
>
> [1] https://github.com/mcpierce/Proton/tree/PROTON-799-Ruby-engine-apis
>
> --
> Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
> Delivering value year after year.
> Red Hat ranks #1 in value among software vendors.
> http://www.redhat.com/promo/vendor/
>
>