You are viewing a plain text version of this content. The canonical link for it is here.
Posted to proton@qpid.apache.org by "Darryl L. Pierce" <dp...@redhat.com> on 2015/03/06 22:41:34 UTC
Proton and Ruby object references...
Revisiting this topic given newly discovered issues:
Back in January I had a solution that worked quite well with passing in
Ruby objects to the C library. Ruby objects could be attached to Swig
wrappers around structs so that their lifecycle's were kept; i.e., when
C object X was garbaged collected, its wrapper would go away, and
associated objects Y & Z were reaped since they no longer had a parent.
However, it's now come up that the Swig wrappers themselves don't seem
to live a terribly long time in some cases. Also, asking Swig for the
same struct more than once results in a completely different Swig
wrapper coming back; i.e., there's no one-to-one mapping between the
Swig wrapper and the underlying struct. And, the big problem, the Ruby
object passed to the C structs are still vulnerable to being reaped as a
result of this.
To prove this, I created two test APIs to store and retrieve a single
pn_transport_t instance [1]. I create a transport, store it and then
fetch it again and get two different objects that are are not equal:
t = Cproton.pn_transport
=> t now holds a reference to an instance of pn_transport_t
Cproton.pn_transport_set_mine t
=> that pn_transport_t struct is now pointed to in the C libraries
u = Cproton.pn_transport_get_mine
=> u now holds a reference to the same instance of pn_transport_t
t == u
=> false -- comparison fails
Additionally, the pn_record_get API seems to have an issue with
retrieving stored objects that may or may not be related to this issue.
I've written a reproducer [2] that simply stores and retrieves attachments
for a transport, creates a hash, forces garbage collection and then
shows object statistics for the MyHash and wrapper for the pn_record_t
type in Ruby space.
Specifically, if lines 101 and 103-4 are commented out, the reproducer
runs fine and creates a whole lot of new transports before going back
for the original and getting its attachments.
If lines 102 thru 104 are commented out the reproducer runs fine and
retrieves the same attachments over and over without a problem.
BUT, if 101-2 are commented out and the reproducer alternates between
getting the stored one and a new one, it blows up on the first call to
retrieve the stored reference after getting a single new transport. This
doesn't seem to make sense since I can create 1k new transports and then
get my old one back, but creating a single new one and then getting the
old one back it blows up. And where it blows up is on displaying the
retrieved attrs object.
I know that this issue is related to garbage collection: I can add the
attrs variable to a global array and the error never happens in the last
scenario. But this doesn't seem to be a good solution since it would
require a lot of extra code to track and remove objects when a C struct
is deleted.
So, in the end, I need a better way to store and retrieve Ruby objects
in the library than is currently being done. It works well with the
engine APIs since you're not creating and then re-creating objects, so
there's not a lot of storing and fetching going on. But with the
reactive APIs, it wants to store object state details and recreate them
via the various class wrap methods. And this is failing for Ruby since
Ruby's mark and sweep is deleting objects that C is holding.
[1] pn_transport_set_mine, pn_transport_get_mine
[2] http://github.com/mcpierce/Proton/blob/PROTON-799-Ruby-engine-apis/examples/ruby/wrapper_test.rb
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: Proton and Ruby object references...
Posted by "Darryl L. Pierce" <dp...@redhat.com>.
On Fri, Mar 06, 2015 at 04:41:34PM -0500, Darryl L. Pierce wrote:
> Revisiting this topic given newly discovered issues:
<snip>
Thinking about this over the weekend, it feels like it would be a huge
nightmare to try and put a shim in place to keep a Ruby object held by a
C struct from being reaped. Especially given that Swig is recreating
Ruby wrappers and losing state information on them.
But I had an idea taken from a previous project that feels like it would
solve this issue.
Instead of having a Proton C struct hold a reference to a Ruby object
and dealing with the GC issues, why not instead have that C struct hold
a unique string?
Within Ruby we can maintain a private (not visible to the application)
hashtable that uses that string value as a key to point to a Ruby object
that is the true value of interest.
To assist this, we would add a function pointer to the record APIs
(pn_record_remove) that gets called when a record is deleted. It passes
in the void* value for each field in the record, if the function is
assigned, and that function can work as a hook to remove the object from
the global hash.
Why do this instead of having C hold a Ruby object and stuffing the Ruby
object into some sort of array? I think it would be a simpler, cleaner
operation to have C not know if's holding onto Ruby. So, for example, if
something happened to the hash or the object held in the hash, we
wouldn't get a segfault to access the value from C: instead, we would
have a key with no matching value. Something that would be easier to
debug than what could appear to be a random segfault.
Thoughts?
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/