You are viewing a plain text version of this content. The canonical link for it is here.
Posted to proton@qpid.apache.org by "Darryl L. Pierce" <dp...@redhat.com> on 2012/09/10 15:26:54 UTC
Ruby stable API design
I've been working the past few weeks on providing a stable API layer in
Ruby on top of the swig bindings. I've made a few decisions in the
design and want to now float them out here to get feedback.
You can take a look at the WIP in my Github repo [1].
NAMESPACES
==========
The APIs all live within the Qpid::Proton namespace. Each major group of
APIs exist within namespaces below them. For example, the Driver APIs
live within Qpid::Proton::Driver, Qpid::Proton::Driver::Connector and
Qpid::Proton::Driver::Listener.
CLASSES VS MODULE METHODS AND CONSTANTS
=======================================
All of the APIs are module-level methods in the design. There's no need
to instantiate an object to work with them. You only need to pass in the
object types returned by the underlying Proton C implementation.
Enumerated types are represented by classes that provide a simple,
consistent interface for working with symbols that have roughly the same
names as the enum. For example, Qpid::Proton::Trace represents pn_trace_t,
and values can be accessed using Qpid::Proton::Trace::RAW.
UNIT AND FEATURE TESTING
========================
For now I've focused on writing Rspec tests to cover as many conditions
as I can setup, with the goal of keeping the coverage at 100% or within
2 points. Some of the conditions aren't easy to setup, so those I'm
documentingn with a note to fix them in future.
For feature tests, I plan to write those using Cucumber. But I'd like to
start a second conversation there since we can use Cucumber to test
features across all currently supported languages (Ruby, Python, Perl,
PHP, Java). Since Cucumber supports these [2] we can write one set of
feature tests and ensure all implementations and bindings work as
expected.
Questions? Thoughts?
[1] https://github.com/mcpierce/Proton/tree/Ruby-language-APIs
[2] https://github.com/cucumber/cucumber/wiki/
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: Ruby stable API design
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Mon, Sep 10, 2012 at 2:04 PM, Darryl L. Pierce <dp...@redhat.com>wrote:
> On Mon, Sep 10, 2012 at 01:36:35PM -0400, Rafael Schloming wrote:
> > So my general thought is that we should focus on the messenger stuff
> first
> > for high level languages like Ruby. The main reason being that the lower
> > level interfaces might want to get tweaked a bit more for performance
> > reasons, whereas the messenger interface itself is higher level and
> > therefore less likely to change. So given that the point of this is that
> > it's a stable API, that is really the most stable place to start. I also
> > think in a lot of ways the other stuff is distracting for a language like
> > python or ruby and not as interesting to those communities.
> >
> > I took a look at the messenger example and I'm a little surprised not to
> > see Messenger and Message exposed as objects. Granted the bindings are
> > procedural because the C API is procedural, but they're pretty much
> begging
> > to be exposed as objects. As they are it looks like you still need to do
> > manual memory management, I would think by mapping the procedures into
> > methods on Messenger and Message objects and by tying pn_messenger_free
> and
> > pn_message_free into the respective destructors, we could make things a
> > whole lot safer, e.g. avoid dangling pointers and the like that a user
> > could use to segfault the interpreter.
>
> Okay, I get what you're saying there. I'll switch gears at this point
> and tackle a Messenger/Message framework for Ruby.
>
Cool, looking forward to it.
> Regarding cucumber, this is the first I've seen of or heard of it. My
> initial reaction could probably be described as mildly skeptical by
someone
> prone to extreme understatement, but I'll try to keep an open mind and
read
> up on it sometime soon. ;-)
I would definitely like to hear your feedback after looking at the
> Cucumber website. It's a great framework that other teams I know use
> regularly. We also have such tests in the old Qpid Ruby bindings as
> well.
>
Thanks, I'll check those out. It will be good to see it in a familiar
context.
--Rafael
Re: Ruby stable API design
Posted by "Darryl L. Pierce" <dp...@redhat.com>.
On Mon, Sep 10, 2012 at 01:36:35PM -0400, Rafael Schloming wrote:
> So my general thought is that we should focus on the messenger stuff first
> for high level languages like Ruby. The main reason being that the lower
> level interfaces might want to get tweaked a bit more for performance
> reasons, whereas the messenger interface itself is higher level and
> therefore less likely to change. So given that the point of this is that
> it's a stable API, that is really the most stable place to start. I also
> think in a lot of ways the other stuff is distracting for a language like
> python or ruby and not as interesting to those communities.
>
> I took a look at the messenger example and I'm a little surprised not to
> see Messenger and Message exposed as objects. Granted the bindings are
> procedural because the C API is procedural, but they're pretty much begging
> to be exposed as objects. As they are it looks like you still need to do
> manual memory management, I would think by mapping the procedures into
> methods on Messenger and Message objects and by tying pn_messenger_free and
> pn_message_free into the respective destructors, we could make things a
> whole lot safer, e.g. avoid dangling pointers and the like that a user
> could use to segfault the interpreter.
Okay, I get what you're saying there. I'll switch gears at this point
and tackle a Messenger/Message framework for Ruby.
> Regarding cucumber, this is the first I've seen of or heard of it. My
> initial reaction could probably be described as mildly skeptical by someone
> prone to extreme understatement, but I'll try to keep an open mind and read
> up on it sometime soon. ;-)
I would definitely like to hear your feedback after looking at the
Cucumber website. It's a great framework that other teams I know use
regularly. We also have such tests in the old Qpid Ruby bindings as
well.
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: Ruby stable API design
Posted by "Darryl L. Pierce" <dp...@redhat.com>.
On Thu, Sep 13, 2012 at 04:46:00PM -0400, Rajith Attapattu wrote:
> On Mon, Sep 10, 2012 at 1:36 PM, Rafael Schloming <rh...@alum.mit.edu> wrote:
> > On Mon, Sep 10, 2012 at 9:26 AM, Darryl L. Pierce <dp...@redhat.com>wrote:
> > methods on Messenger and Message objects and by tying pn_messenger_free and
> > pn_message_free into the respective destructors, we could make things a
> > whole lot safer, e.g. avoid dangling pointers and the like that a user
> > could use to segfault the interpreter.
>
> Darryl, you could use %typemap(freearg) to ensure the respective free
> function is called to cleanup.
> How it gets called is also automatically handled by swig based on the
> host language.
I'll take a look at that. I did use some simple Ruby to make sure that a
destructor method is invoked on garbage collection and also shutdown,
but will look at something more automated like this. Thanks. :)
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: Ruby stable API design
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Thu, Sep 13, 2012 at 5:27 PM, Rajith Attapattu <ra...@gmail.com>wrote:
> On Thu, Sep 13, 2012 at 5:05 PM, Rafael Schloming <rh...@alum.mit.edu>
> wrote:
> > On Thu, Sep 13, 2012 at 4:46 PM, Rajith Attapattu <rajith77@gmail.com
> >wrote:
> >
> >> On Mon, Sep 10, 2012 at 1:36 PM, Rafael Schloming <rh...@alum.mit.edu>
> >> wrote:
> >> > On Mon, Sep 10, 2012 at 9:26 AM, Darryl L. Pierce <dpierce@redhat.com
> >> >wrote:
> >> > methods on Messenger and Message objects and by tying
> pn_messenger_free
> >> and
> >> > pn_message_free into the respective destructors, we could make things
> a
> >> > whole lot safer, e.g. avoid dangling pointers and the like that a user
> >> > could use to segfault the interpreter.
> >>
> >> Darryl, you could use %typemap(freearg) to ensure the respective free
> >> function is called to cleanup.
> >> How it gets called is also automatically handled by swig based on the
> >> host language.
> >>
> >
> > Maybe I'm missing something, but I don't think %typemap(freearg) helps in
> > this case. From what I understand it's used to clean up temporary memory
> > allocated for the purpose of argument processing. The situation I'm
> talking
> > about is when you allocate an object in C and return it to the
> interpreter,
> > e.g.:
>
> Sorry I misunderstood. I thought it was for methods.
>
> It appears Ruby does have support for %newobject
> http://www.swig.org/Doc1.3/Ruby.html#Ruby_nn59
> But it maybe simpler to use the strategy Rafi has used for the python
> binding.
>
I'm not trying to discourage use of %newobject and %delobject, I just
didn't bother messing with them myself as they seem to be targeted at
making the naked swig API more palatable, and IMHO the swig APIs shouldn't
ever really be used directly, rather the simplest possible idomatic APIs
like the one I posted earlier should be what we support as a public
interface. They might well be worth investigating though if they can save
us a bit of work inside the idiomatic APIs (aka the buttercream frosting
layer).
--Rafael
Re: Ruby stable API design
Posted by "Darryl L. Pierce" <dp...@redhat.com>.
On Thu, Sep 13, 2012 at 05:27:14PM -0400, Rajith Attapattu wrote:
> >> Darryl, you could use %typemap(freearg) to ensure the respective free
> >> function is called to cleanup.
> >> How it gets called is also automatically handled by swig based on the
> >> host language.
> >>
> >
> > Maybe I'm missing something, but I don't think %typemap(freearg) helps in
> > this case. From what I understand it's used to clean up temporary memory
> > allocated for the purpose of argument processing. The situation I'm talking
> > about is when you allocate an object in C and return it to the interpreter,
> > e.g.:
>
> Sorry I misunderstood. I thought it was for methods.
>
> It appears Ruby does have support for %newobject
> http://www.swig.org/Doc1.3/Ruby.html#Ruby_nn59
> But it maybe simpler to use the strategy Rafi has used for the python binding.
The way I'm going at it currently is to have each constructor setup a
finalizer method that gets invoked when the object is garbage collected.
For example, in Messenger we have:
def initialize(name = nil)
@impl = Cproton.pn_messenger(name)
ObjectSpace.define_finalizer(self, self.class.finalize!(@impl))
end
def self.finalize!(impl) # :nodoc:
proc {
Cproton.pn_messenger_stop(impl)
Cproton.pn_messenger_free(impl)
}
end
What this does it register a class-level callback for each instance that
will call the finalize! method, passing in the underlying pn_messenger_t
instance. That will get stopped (if it's running) and then its resources
released.
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: Ruby stable API design
Posted by Rajith Attapattu <ra...@gmail.com>.
On Thu, Sep 13, 2012 at 5:05 PM, Rafael Schloming <rh...@alum.mit.edu> wrote:
> On Thu, Sep 13, 2012 at 4:46 PM, Rajith Attapattu <ra...@gmail.com>wrote:
>
>> On Mon, Sep 10, 2012 at 1:36 PM, Rafael Schloming <rh...@alum.mit.edu>
>> wrote:
>> > On Mon, Sep 10, 2012 at 9:26 AM, Darryl L. Pierce <dpierce@redhat.com
>> >wrote:
>> > methods on Messenger and Message objects and by tying pn_messenger_free
>> and
>> > pn_message_free into the respective destructors, we could make things a
>> > whole lot safer, e.g. avoid dangling pointers and the like that a user
>> > could use to segfault the interpreter.
>>
>> Darryl, you could use %typemap(freearg) to ensure the respective free
>> function is called to cleanup.
>> How it gets called is also automatically handled by swig based on the
>> host language.
>>
>
> Maybe I'm missing something, but I don't think %typemap(freearg) helps in
> this case. From what I understand it's used to clean up temporary memory
> allocated for the purpose of argument processing. The situation I'm talking
> about is when you allocate an object in C and return it to the interpreter,
> e.g.:
Sorry I misunderstood. I thought it was for methods.
It appears Ruby does have support for %newobject
http://www.swig.org/Doc1.3/Ruby.html#Ruby_nn59
But it maybe simpler to use the strategy Rafi has used for the python binding.
> msg = pn_message()
>
> What swig returns to the interpreter here is a wrapper around a pointer. If
> that variable goes out of scope and the interpreter garbage collects it,
> then only the wrapper is freed, and the memory pointed to by the internal
> pointer is leaked.
>
> You can fix this by doing pn_message_free(msg) before it goes out of scope,
> however for a short time you end up with the wrapping object having an
> internal pointer that is invalid since it now points to freed memory. If
> you the try to do things you can cause the interpreter to seg fault just
> like the C runtime would seg fault if you were to try to use a pointer
> after freeing it.
>
> You can use %newobject and %delobject to address this, however I'm given to
> understand that support for that varies across target languages. It's
> fairly easy to work around with a destructor in the wrapper layer as I've
> done in the idiomatic python API layer.
>
> --Rafael
Re: Ruby stable API design
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Thu, Sep 13, 2012 at 4:46 PM, Rajith Attapattu <ra...@gmail.com>wrote:
> On Mon, Sep 10, 2012 at 1:36 PM, Rafael Schloming <rh...@alum.mit.edu>
> wrote:
> > On Mon, Sep 10, 2012 at 9:26 AM, Darryl L. Pierce <dpierce@redhat.com
> >wrote:
> > methods on Messenger and Message objects and by tying pn_messenger_free
> and
> > pn_message_free into the respective destructors, we could make things a
> > whole lot safer, e.g. avoid dangling pointers and the like that a user
> > could use to segfault the interpreter.
>
> Darryl, you could use %typemap(freearg) to ensure the respective free
> function is called to cleanup.
> How it gets called is also automatically handled by swig based on the
> host language.
>
Maybe I'm missing something, but I don't think %typemap(freearg) helps in
this case. From what I understand it's used to clean up temporary memory
allocated for the purpose of argument processing. The situation I'm talking
about is when you allocate an object in C and return it to the interpreter,
e.g.:
msg = pn_message()
What swig returns to the interpreter here is a wrapper around a pointer. If
that variable goes out of scope and the interpreter garbage collects it,
then only the wrapper is freed, and the memory pointed to by the internal
pointer is leaked.
You can fix this by doing pn_message_free(msg) before it goes out of scope,
however for a short time you end up with the wrapping object having an
internal pointer that is invalid since it now points to freed memory. If
you the try to do things you can cause the interpreter to seg fault just
like the C runtime would seg fault if you were to try to use a pointer
after freeing it.
You can use %newobject and %delobject to address this, however I'm given to
understand that support for that varies across target languages. It's
fairly easy to work around with a destructor in the wrapper layer as I've
done in the idiomatic python API layer.
--Rafael
Re: Ruby stable API design
Posted by Rajith Attapattu <ra...@gmail.com>.
On Mon, Sep 10, 2012 at 1:36 PM, Rafael Schloming <rh...@alum.mit.edu> wrote:
> On Mon, Sep 10, 2012 at 9:26 AM, Darryl L. Pierce <dp...@redhat.com>wrote:
> methods on Messenger and Message objects and by tying pn_messenger_free and
> pn_message_free into the respective destructors, we could make things a
> whole lot safer, e.g. avoid dangling pointers and the like that a user
> could use to segfault the interpreter.
Darryl, you could use %typemap(freearg) to ensure the respective free
function is called to cleanup.
How it gets called is also automatically handled by swig based on the
host language.
Regards,
Rajith
Re: Ruby stable API design
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Mon, Sep 10, 2012 at 9:26 AM, Darryl L. Pierce <dp...@redhat.com>wrote:
> I've been working the past few weeks on providing a stable API layer in
> Ruby on top of the swig bindings. I've made a few decisions in the
> design and want to now float them out here to get feedback.
>
> You can take a look at the WIP in my Github repo [1].
>
> NAMESPACES
> ==========
> The APIs all live within the Qpid::Proton namespace. Each major group of
> APIs exist within namespaces below them. For example, the Driver APIs
> live within Qpid::Proton::Driver, Qpid::Proton::Driver::Connector and
> Qpid::Proton::Driver::Listener.
>
> CLASSES VS MODULE METHODS AND CONSTANTS
> =======================================
> All of the APIs are module-level methods in the design. There's no need
> to instantiate an object to work with them. You only need to pass in the
> object types returned by the underlying Proton C implementation.
>
> Enumerated types are represented by classes that provide a simple,
> consistent interface for working with symbols that have roughly the same
> names as the enum. For example, Qpid::Proton::Trace represents pn_trace_t,
> and values can be accessed using Qpid::Proton::Trace::RAW.
>
> UNIT AND FEATURE TESTING
> ========================
> For now I've focused on writing Rspec tests to cover as many conditions
> as I can setup, with the goal of keeping the coverage at 100% or within
> 2 points. Some of the conditions aren't easy to setup, so those I'm
> documentingn with a note to fix them in future.
>
> For feature tests, I plan to write those using Cucumber. But I'd like to
> start a second conversation there since we can use Cucumber to test
> features across all currently supported languages (Ruby, Python, Perl,
> PHP, Java). Since Cucumber supports these [2] we can write one set of
> feature tests and ensure all implementations and bindings work as
> expected.
>
> Questions? Thoughts?
>
So my general thought is that we should focus on the messenger stuff first
for high level languages like Ruby. The main reason being that the lower
level interfaces might want to get tweaked a bit more for performance
reasons, whereas the messenger interface itself is higher level and
therefore less likely to change. So given that the point of this is that
it's a stable API, that is really the most stable place to start. I also
think in a lot of ways the other stuff is distracting for a language like
python or ruby and not as interesting to those communities.
I took a look at the messenger example and I'm a little surprised not to
see Messenger and Message exposed as objects. Granted the bindings are
procedural because the C API is procedural, but they're pretty much begging
to be exposed as objects. As they are it looks like you still need to do
manual memory management, I would think by mapping the procedures into
methods on Messenger and Message objects and by tying pn_messenger_free and
pn_message_free into the respective destructors, we could make things a
whole lot safer, e.g. avoid dangling pointers and the like that a user
could use to segfault the interpreter.
Regarding cucumber, this is the first I've seen of or heard of it. My
initial reaction could probably be described as mildly skeptical by someone
prone to extreme understatement, but I'll try to keep an open mind and read
up on it sometime soon. ;-)
--Rafael