You are viewing a plain text version of this content. The canonical link for it is here.
Posted to proton@qpid.apache.org by Rafael Schloming <rh...@alum.mit.edu> on 2013/12/19 14:49:59 UTC
improving cross language maintainability
Hi Everyone,
I've been struggling with some of the cross language maintenance aspects of
proton for a while, and I believe we need to take some steps to improve the
situation. I'm one of a tiny number of people (two possibly) who regularly
commit changes to both the Java and C codebase and attempt to keep the two
in sync and at feature parity. Because of this, not everyone is necessarily
aware of the process, but to summarize the issue, currently there are far
too many moving parts and layers of indirection involved. This is not only
a significant drag on my personal productivity, but perhaps more
importantly it is a significant barrier to growing the number of proton
contributors in general as all those moving parts and layers of indirection
need to be understood before being able to make complete contributions.
To get an idea of what's going on here I think it helps to look at what's
involved in a simple change. Recently I noticed an edge case around status
updates in the messenger interface. The fix involved adding another value
to messenger's status enum and making a trivial logic change to make use of
that value under the appropriate circumstances. You can look at the full
commit here[1] if you like, but it breaks down in the following way:
Changes to C implementation (.h and .c file): 8 LOC
Changes to the python test shim for C: 3 LOC
Changes to the Java API: 2 LOC
Changes to the pure Java implementation: 6 LOC
Changes to the JNI binding: 4 LOC
Changes to the python test shim for Java: 3 LOC
Changes to the python test suite: 2 LOC
Now obviously from a personal productivity perspective it is at a minimum
annoying to have to touch so many different parts in order to make even a
trivial change, but setting that aside for the moment, what is really
sobering about this is that each one of the above parts involve a non
trivial learning curve on their own, and while it's true that only a few
lines of code are needed in each area, it is necessary to understand each
piece before being able to write the correct few lines of code. This
presents a pretty daunting hurdle for bringing new contributors to the
codebase up to the level needed to make even a trivial change like the one
above.
The JNI binding and the python test shim for Java both exist to serve
similar purposes, namely to facilitate running a common test suite against
both implementations in order to ensure common behaviour. The python test
shims allow the python test suite to run against both proton-c and proton-j
(via jython), and the JNI binding allows pure Java tests to be run against
proton-c as well as proton-j.
Currently by my count there are about 19 such tests in Java. By comparison
there are about 266 tests in the python test suite. Also, judging by
commits, the python test suite is growing/actively maintained, and the java
system tests (the subset of java tests that are run against both
implementations) are neither. On top of this the JNI binding itself has
suffered significantly from bit rot. As far as I know it is not made use of
outside of our own test infrastructure, and currently about 50% of the
tests run against it are skipped because it is only minimally maintained
when necessary to get the tests to pass.
Because of all this I'm proposing that we remove the JNI binding and roll
back the Java API/impl split that was (hastily) done to facilitate the test
infrastructure. This should significantly simplify the development process
relative to what it is now, and while there is still more learning curve
than desired with the python shims, I believe this will put us in a
position to improve the shims, remove duplication, and minimize the
overhead associated with them, ultimately allowing us to make the codebase
more transparently communicate its design and hopefully lessen the
learning curve for new contributions.
I would love to hear thoughts and/or alternative ideas on how to improve
things. I would like to start addressing this early in the 0.7 development
cycle.
[1] http://svn.apache.org/r1551950
Re: improving cross language maintainability
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Tue, Jan 7, 2014 at 3:24 PM, Rob Godfrey <ro...@gmail.com> wrote:
> On 20 December 2013 19:49, Rafael Schloming <rh...@alum.mit.edu> wrote:
>
> > On Fri, Dec 20, 2013 at 11:43 AM, Fraser Adams <
> > fraser.adams@blueyonder.co.uk> wrote:
> >
> > > I've been following this thread with interest and I guess that Rob's
> > > comment
> > >
> > >
> > > "
> > >
> > > However I think that would require us to be
> > > organised differently with a recognition that the Engine and Messenger
> > are
> > > conceptually separate (with Messenger depending on the Engine API, and
> > > having no more authority over defining/testing the engine API than any
> > > other client
> > > "
> > >
> > > Struck some resonance.
> > >
> > > I think perhaps the layering is OK if you're familiar with the code
> base
> > > and perhaps it's more about "packaging" than layering, but FWIW coming
> > into
> > > proton quite "cold" it seems like messenger and engine are essentially
> > > "combined". Now I know that's not the case now from the responses to my
> > > "how does it all hang together" question of a few weeks ago, but in
> terms
> > > of packaging that's still kind of the case.
> > >
> > > So what I mean by that is that qpid::messaging and proton messenger are
> > as
> > > far as has been discussed with me "peer" APIs - both high level that
> can
> > > achieve similar things albeit with different semantics, whereas engine
> > is a
> > > lower level API.
> > >
> > > Why is it in that case that the proton library is built that contains
> > both
> > > messenger and engine? OK, so it's convenient, but as far as I'm aware
> > > neither qpid::messaging nor Ted's dispatch router actually use
> messenger
> > at
> > > all, they both use engine? That would suggest to me that engine and
> > > messenger probably ought to be exposed as separate libraries? (and
> > > presumably there's a similar position in Java where the JMS
> > implementation
> > > is engine not messenger based - though I've not looked at the jar
> > > packaging).
> > >
> > > I guess (perhaps related) it just might be better if messenger was in a
> > > separate source tree, which I'd agree might be a faff, but would
> clarify
> > > the hierarchical rather than peer relationship between messenger and
> > engine.
> > >
> > > So I guess that my take is that Rafael is perfectly accurate when he
> says
> > > that "I believe they are currently layered precisely the way you
> > > describe" that certainly seems to be the case at the logical level, but
> > at
> > > the "physical" level it's an awful lot less clear of the layering -
> > > certainly from the perspective of a novice trying to navigate the code
> > base.
> > >
> >
> > There is certainly a pragmatic element in keeping messenger and engine
> > bundled the way they are. From a development perspective, having
> messenger
> > in the same code base makes testing significantly easier.
>
>
> It might make it easier for you (since you have a complete view of every
> piece of the code and design) - for myself I found that it made things
> *much* harder, as tests were running through multiple layers rather than
> being isolated to a single component. When a test "failed" it was entirely
> unclear which layer the failure resided in or even why the outcome should
> be as the test expeted. Personally I'd say the testing is a strong
> argument for separation (although it means more work upfront it makes it
> easier for everyone in the long run).
>
I'm not suggesting messenger tests are in any way a substitute for API
tests against the engine itself, rather that are other important categories
of testing, e.g. soak testing, that would require building something that
is pretty much equivalent to what messenger already does.
>
>
> > Writing a simple
> > command line program to fire off a few messages in order to reproduce
> some
> > scenario is quite trivial with messenger, but would involve lots of
> boiler
> > plate with just the engine directly. That's something I would like to
> work
> > on improving about the engine API, but it is very slow and difficult to
> > make changes given the current setup I described in my original post.
>
>
>
> > Given
> > that, realistically I think if we were to pull messenger into a separate
> > code base, there would be multiple sources of duplicate effort, not only
> > having to duplicate/produce another non trivial multi lingual build
> system,
> > but we would also need to build up a test harness that duplicates a good
> > chunk of what messenger already does.
>
>
> Every other project that wants to use the Proton engine will also be
> writing tests for their use of the API. And they'll have different use
> cases and different pattern that may not be something supported by
> Messenger.
>
> Sure, it's great that Messenger tests will implicitly test the Engine, but
> they are not a substitute for Engine tests.
>
As I said above, I don't believe we have been using them as a substitute
for engine tests, rather they are a tool for discovering more bugs. Just
having shallow test coverage of an API doesn't provide much confidence that
the API will work in real world scenarios, e.g. early engine tests verified
that sending/receiving 1, 2, or even 10 messages worked as expected. Soak
tests implemented with messenger encountered bugs involving much longer
runs or more complex configurations, e.g. near credit/session window edges
and such, or when interleaving messages across multiple links/sessions.
These bugs were then isolated and turned into specific engine test cases,
however the messenger level testing is still an invaluable tool for finding
them, and replacing that category of messenger based test in the current
test suite would require writing test specific client/server code that is
comparable to what messenger already does.
>
> > Granted a chunk of that is
> > boilerplate that should be somehow refactored into the engine API proper
> so
> > that it is easier to use in general, and if this were to happen such a
> > split would probably be more feasible, but that kind of refactoring is
> > really prohibitively expensive given all the different shims and bindings
> > that would be impacted.
> >
> >
> And this points to an issue, with the focus on Messenger the Engine API is
> not being enhanced... and it needs people and effort focussed on it.
> Messenger should be a client of the Engine API, the Engine shouldn't be a
> slave to Messenger. Other users (whether in Qpid or not) should have equal
> influence of the Engine API design, and equal ability to try to shape it.
> Right now I find the Engine unattaractive to try to use within the Qpid
> Java Broker... I know Gordon has also expressed thought on things that he
> would like to see changed based on his work intergrating in the C++
> broker. I feel like the Engine needs to set free to evolve on its own, but
> with an API that everyone (Messenger, Qpid Brokers, etc) can rely on.
>
I agree the engine API needs more focussed effort (that is in fact exactly
why I'm trying to remove excess development overhead associated with
working on it), however your implication that somehow focus on Messenger
is preventing focus on the engine doesn't make sense to me. This is not a
zero sum game, and even if it were that is all the more reason to remove as
much overhead as possible.
As for undue influence, I don't know of any areas where messenger has
unduly influenced or constrained the development of the engine API, but if
you believe that it has then please provide some specifics. I would be very
interested in hearing how the engine API could be improved for your use
cases. I have some of my own ideas about improving it, but I will post
about those separately.
>
>
> > I'd also point out that from a user perspective I think there is some
> > synergy in bundling a higher level tool together with the engine. It is
> > very rare that you are going to want to embed the engine in something and
> > not want some convenient high level way to send that thing a message.
> Case
> > in point, Ted's dispatch router proper doesn't actually use messenger as
> > you say, however he ships it with several command line scripts that do
> use
> > messenger. Splitting them up would result in both him and his users
> having
> > to deal with an extra dependency.
> >
> >
> Sure, but maybe if someone else had written a different API Ted would have
> wanted to use that. Having said that, I'm not necessarily against
> bundling... but I think then we need to be clearer in the loosening of the
> coupling and allow for the horizontal scaling of allowing different people
> to evolve the Engine and Messenger APIs.
>
>
> >
> > > I'd have to agree also with Rob about the horizontal scaling aspects of
> > > lack of comments and documentation, particularly with respect to
> engine.
> > > I'm currently trying to get a JavaScript_messenger_ working because
> that
> > > seems tractable given the examples and documentation around messenger
> > > (though as you know even there I've had a fair few complications) my
> > > personal preference/familiarity is however around the
> JMS/qpid::messaging
> > > API and I'd ultimately quite like to have a JavaScript "binding" for
> > that,
> > > but at this stage I wouldn't know where to begin and I'd probably end
> up
> > > "reverse engineering" the work Gordon has done with qpid::messaging
> > and/or
> > > Rob and co. are doing with the new JMS implementation.
> > >
> > > I'd be interested to hear from Gordon and Ted about how they went about
> > > building capability on top of the engine API.
> > >
> > > I hope I'm not coming across as over-critical - email is a bit rubbish
> at
> > > conveying emotion :-) I guess that I just want to add my 2p from the
> > > perspective of someone coming in new with no previous exposure to the
> > code
> > > base, which is hopefully a fairly objective position.
> > >
> >
> > No worries, I'm well aware of the issues you mention and I'm as eager to
> > see a solution as anyone else. I kicked off this thread asking for
> thoughts
> > and ideas, so I certainly can't blame anyone for speaking up.
> >
> >
> In terms of your desire to remove the JNI implementation... I agree that
> it's not something that in its current form would be production
> deployable. However as an "independent" developer trying to work on the
> Java implementation previously, I found having the JNI implementation
> invaluable in terms of being able to compare and contrast what the C impl
> was doing to the pure Java impl (I don't really see any value in the JNI
> Driver or JNI Messenger... but then I'm not really interested in those
> components anyway). Using the python tests against the Java was incredibly
> painful for myself and others who I work with.
>
I believe that it was a valuable experience for you to develop the JNI
binding as the process of writing it required you to become familiar with
both the C and Java APIs, however I don't think that it serves that same
purpose for anyone else. It's certainly not being used as a form of
documentation, and even if we wish to develop such documentation, the
jython shim would be a much better starting point as it is both more
complete and significantly more readable than the JNI binding.
> I guess really it comes down to whether our aim here is to make it as
> frictionless as possible for you to work on the codebase... Or whether we
> should make a concerted effort to make it easier for others to work on (and
> integrate with) the work we are doing. If not removing it makes it even a
> tenth as painful for you to work on as I and others found it... then
> perhaps we can *all* find some motivation to reshape the process and the
> mechanics. :-)
>
>
Why do you think the JNI binding makes it easier for others to work on the
code? Nobody has ever submitted a patch that has included the necessary
modifications to that layer, so I find it hard to believe that it is
actually helping anyone.
The original purpose of the JNI binding was to provide some way for tests
written in Java to be accessible to the C implementation, i.e. it was a
means for Java developers to never leave Java or worry about/interact with
other languages. I can see how this reduces development overhead for Java
only, but it creates additional overhead overall and depends on those of us
who develop across both languages to make up the difference. This makes
me/ken even more of a bottleneck if we want to keep the implementations in
sync.
That said, if you have any specific ideas about what to do I would love to
hear them. You seem to be suggesting that making my life harder will
motivate me to make everyone else's life easier, but my life is already
harder, and the solution I've come up with, aside from documentation that
is already in progress, is to remove a lot of the unnecessary complexity
surrounding the development process, starting with the JNI Binding.
--Rafael
Re: improving cross language maintainability
Posted by Rob Godfrey <ro...@gmail.com>.
On 20 December 2013 19:49, Rafael Schloming <rh...@alum.mit.edu> wrote:
> On Fri, Dec 20, 2013 at 11:43 AM, Fraser Adams <
> fraser.adams@blueyonder.co.uk> wrote:
>
> > I've been following this thread with interest and I guess that Rob's
> > comment
> >
> >
> > "
> >
> > However I think that would require us to be
> > organised differently with a recognition that the Engine and Messenger
> are
> > conceptually separate (with Messenger depending on the Engine API, and
> > having no more authority over defining/testing the engine API than any
> > other client
> > "
> >
> > Struck some resonance.
> >
> > I think perhaps the layering is OK if you're familiar with the code base
> > and perhaps it's more about "packaging" than layering, but FWIW coming
> into
> > proton quite "cold" it seems like messenger and engine are essentially
> > "combined". Now I know that's not the case now from the responses to my
> > "how does it all hang together" question of a few weeks ago, but in terms
> > of packaging that's still kind of the case.
> >
> > So what I mean by that is that qpid::messaging and proton messenger are
> as
> > far as has been discussed with me "peer" APIs - both high level that can
> > achieve similar things albeit with different semantics, whereas engine
> is a
> > lower level API.
> >
> > Why is it in that case that the proton library is built that contains
> both
> > messenger and engine? OK, so it's convenient, but as far as I'm aware
> > neither qpid::messaging nor Ted's dispatch router actually use messenger
> at
> > all, they both use engine? That would suggest to me that engine and
> > messenger probably ought to be exposed as separate libraries? (and
> > presumably there's a similar position in Java where the JMS
> implementation
> > is engine not messenger based - though I've not looked at the jar
> > packaging).
> >
> > I guess (perhaps related) it just might be better if messenger was in a
> > separate source tree, which I'd agree might be a faff, but would clarify
> > the hierarchical rather than peer relationship between messenger and
> engine.
> >
> > So I guess that my take is that Rafael is perfectly accurate when he says
> > that "I believe they are currently layered precisely the way you
> > describe" that certainly seems to be the case at the logical level, but
> at
> > the "physical" level it's an awful lot less clear of the layering -
> > certainly from the perspective of a novice trying to navigate the code
> base.
> >
>
> There is certainly a pragmatic element in keeping messenger and engine
> bundled the way they are. From a development perspective, having messenger
> in the same code base makes testing significantly easier.
It might make it easier for you (since you have a complete view of every
piece of the code and design) - for myself I found that it made things
*much* harder, as tests were running through multiple layers rather than
being isolated to a single component. When a test "failed" it was entirely
unclear which layer the failure resided in or even why the outcome should
be as the test expeted. Personally I'd say the testing is a strong
argument for separation (although it means more work upfront it makes it
easier for everyone in the long run).
> Writing a simple
> command line program to fire off a few messages in order to reproduce some
> scenario is quite trivial with messenger, but would involve lots of boiler
> plate with just the engine directly. That's something I would like to work
> on improving about the engine API, but it is very slow and difficult to
> make changes given the current setup I described in my original post.
> Given
> that, realistically I think if we were to pull messenger into a separate
> code base, there would be multiple sources of duplicate effort, not only
> having to duplicate/produce another non trivial multi lingual build system,
> but we would also need to build up a test harness that duplicates a good
> chunk of what messenger already does.
Every other project that wants to use the Proton engine will also be
writing tests for their use of the API. And they'll have different use
cases and different pattern that may not be something supported by
Messenger.
Sure, it's great that Messenger tests will implicitly test the Engine, but
they are not a substitute for Engine tests.
> Granted a chunk of that is
> boilerplate that should be somehow refactored into the engine API proper so
> that it is easier to use in general, and if this were to happen such a
> split would probably be more feasible, but that kind of refactoring is
> really prohibitively expensive given all the different shims and bindings
> that would be impacted.
>
>
And this points to an issue, with the focus on Messenger the Engine API is
not being enhanced... and it needs people and effort focussed on it.
Messenger should be a client of the Engine API, the Engine shouldn't be a
slave to Messenger. Other users (whether in Qpid or not) should have equal
influence of the Engine API design, and equal ability to try to shape it.
Right now I find the Engine unattaractive to try to use within the Qpid
Java Broker... I know Gordon has also expressed thought on things that he
would like to see changed based on his work intergrating in the C++
broker. I feel like the Engine needs to set free to evolve on its own, but
with an API that everyone (Messenger, Qpid Brokers, etc) can rely on.
> I'd also point out that from a user perspective I think there is some
> synergy in bundling a higher level tool together with the engine. It is
> very rare that you are going to want to embed the engine in something and
> not want some convenient high level way to send that thing a message. Case
> in point, Ted's dispatch router proper doesn't actually use messenger as
> you say, however he ships it with several command line scripts that do use
> messenger. Splitting them up would result in both him and his users having
> to deal with an extra dependency.
>
>
Sure, but maybe if someone else had written a different API Ted would have
wanted to use that. Having said that, I'm not necessarily against
bundling... but I think then we need to be clearer in the loosening of the
coupling and allow for the horizontal scaling of allowing different people
to evolve the Engine and Messenger APIs.
>
> > I'd have to agree also with Rob about the horizontal scaling aspects of
> > lack of comments and documentation, particularly with respect to engine.
> > I'm currently trying to get a JavaScript_messenger_ working because that
> > seems tractable given the examples and documentation around messenger
> > (though as you know even there I've had a fair few complications) my
> > personal preference/familiarity is however around the JMS/qpid::messaging
> > API and I'd ultimately quite like to have a JavaScript "binding" for
> that,
> > but at this stage I wouldn't know where to begin and I'd probably end up
> > "reverse engineering" the work Gordon has done with qpid::messaging
> and/or
> > Rob and co. are doing with the new JMS implementation.
> >
> > I'd be interested to hear from Gordon and Ted about how they went about
> > building capability on top of the engine API.
> >
> > I hope I'm not coming across as over-critical - email is a bit rubbish at
> > conveying emotion :-) I guess that I just want to add my 2p from the
> > perspective of someone coming in new with no previous exposure to the
> code
> > base, which is hopefully a fairly objective position.
> >
>
> No worries, I'm well aware of the issues you mention and I'm as eager to
> see a solution as anyone else. I kicked off this thread asking for thoughts
> and ideas, so I certainly can't blame anyone for speaking up.
>
>
In terms of your desire to remove the JNI implementation... I agree that
it's not something that in its current form would be production
deployable. However as an "independent" developer trying to work on the
Java implementation previously, I found having the JNI implementation
invaluable in terms of being able to compare and contrast what the C impl
was doing to the pure Java impl (I don't really see any value in the JNI
Driver or JNI Messenger... but then I'm not really interested in those
components anyway). Using the python tests against the Java was incredibly
painful for myself and others who I work with.
I guess really it comes down to whether our aim here is to make it as
frictionless as possible for you to work on the codebase... Or whether we
should make a concerted effort to make it easier for others to work on (and
integrate with) the work we are doing. If not removing it makes it even a
tenth as painful for you to work on as I and others found it... then
perhaps we can *all* find some motivation to reshape the process and the
mechanics. :-)
-- Rob
Re: improving cross language maintainability
Posted by Rafael Schloming <rh...@alum.mit.edu>.
So far there has been some good discussion on this thread. I can certainly
appreciate the frustration that has been expressed, specifically around
under defined APIs, and confusion regarding the various components of
proton. I believe there are steps we can take to improve both of those
situations. I've created additional components in JIRA for tracking changes
across bindings, and I'm currently working on some content describing
proton's overall architecture and how messenger and engine fit within it
and relate to each other.
That said, no amount of documentation or JIRA usage will help with the
issues I raised in my original post. In particular making even small code
changes requires touching too many different parts: (C code, Java impl,
Java API, JNI binding, C shim, Java shim, and python tests). As a first
step towards improving this I'd like to reiterate my proposal of removing
the JNI binding for the following reasons:
- It is experiencing significant bit rot.
- It is not currently used outside of testing.
- It only provides a minimal amount of additional coverage (some 19 or so
tests as compared to the 266 tests already in the shared python test suite).
- It is a significant maintenance burden as it covers the entire engine
API rather than just messenger.
- It is unlikely to ever be of production value, because a JNI layer
designed for testing an existing C library has fundamentally different
requirements than a JNI layer designed to improve the performance of an
existing Java library.
--Rafael
Re: improving cross language maintainability
Posted by Ted Ross <tr...@redhat.com>.
On 12/20/2013 01:49 PM, Rafael Schloming wrote:
> On Fri, Dec 20, 2013 at 11:43 AM, Fraser Adams <
> fraser.adams@blueyonder.co.uk> wrote:
>
>> I've been following this thread with interest and I guess that Rob's
>> comment
>>
>>
>> "
>>
>> However I think that would require us to be
>> organised differently with a recognition that the Engine and Messenger are
>> conceptually separate (with Messenger depending on the Engine API, and
>> having no more authority over defining/testing the engine API than any
>> other client
>> "
>>
>> Struck some resonance.
>>
>> I think perhaps the layering is OK if you're familiar with the code base
>> and perhaps it's more about "packaging" than layering, but FWIW coming into
>> proton quite "cold" it seems like messenger and engine are essentially
>> "combined". Now I know that's not the case now from the responses to my
>> "how does it all hang together" question of a few weeks ago, but in terms
>> of packaging that's still kind of the case.
>>
>> So what I mean by that is that qpid::messaging and proton messenger are as
>> far as has been discussed with me "peer" APIs - both high level that can
>> achieve similar things albeit with different semantics, whereas engine is a
>> lower level API.
>>
>> Why is it in that case that the proton library is built that contains both
>> messenger and engine? OK, so it's convenient, but as far as I'm aware
>> neither qpid::messaging nor Ted's dispatch router actually use messenger at
>> all, they both use engine? That would suggest to me that engine and
>> messenger probably ought to be exposed as separate libraries? (and
>> presumably there's a similar position in Java where the JMS implementation
>> is engine not messenger based - though I've not looked at the jar
>> packaging).
>>
>> I guess (perhaps related) it just might be better if messenger was in a
>> separate source tree, which I'd agree might be a faff, but would clarify
>> the hierarchical rather than peer relationship between messenger and engine.
>>
>> So I guess that my take is that Rafael is perfectly accurate when he says
>> that "I believe they are currently layered precisely the way you
>> describe" that certainly seems to be the case at the logical level, but at
>> the "physical" level it's an awful lot less clear of the layering -
>> certainly from the perspective of a novice trying to navigate the code base.
>>
> There is certainly a pragmatic element in keeping messenger and engine
> bundled the way they are. From a development perspective, having messenger
> in the same code base makes testing significantly easier. Writing a simple
> command line program to fire off a few messages in order to reproduce some
> scenario is quite trivial with messenger, but would involve lots of boiler
> plate with just the engine directly. That's something I would like to work
> on improving about the engine API, but it is very slow and difficult to
> make changes given the current setup I described in my original post. Given
> that, realistically I think if we were to pull messenger into a separate
> code base, there would be multiple sources of duplicate effort, not only
> having to duplicate/produce another non trivial multi lingual build system,
> but we would also need to build up a test harness that duplicates a good
> chunk of what messenger already does. Granted a chunk of that is
> boilerplate that should be somehow refactored into the engine API proper so
> that it is easier to use in general, and if this were to happen such a
> split would probably be more feasible, but that kind of refactoring is
> really prohibitively expensive given all the different shims and bindings
> that would be impacted.
>
> I'd also point out that from a user perspective I think there is some
> synergy in bundling a higher level tool together with the engine. It is
> very rare that you are going to want to embed the engine in something and
> not want some convenient high level way to send that thing a message. Case
> in point, Ted's dispatch router proper doesn't actually use messenger as
> you say, however he ships it with several command line scripts that do use
> messenger. Splitting them up would result in both him and his users having
> to deal with an extra dependency.
This is true. The system tests in the Dispatch test suite are also
built on Messenger. I think Messenger is sufficiently lightweight that
there's no issue with it being bundled with Engine. The separation I'm
more interested in is Messenger+Engine and Driver. I would like to see
it be easier to use alternate drivers for Proton-based software.
>
>> I'd have to agree also with Rob about the horizontal scaling aspects of
>> lack of comments and documentation, particularly with respect to engine.
>> I'm currently trying to get a JavaScript_messenger_ working because that
>> seems tractable given the examples and documentation around messenger
>> (though as you know even there I've had a fair few complications) my
>> personal preference/familiarity is however around the JMS/qpid::messaging
>> API and I'd ultimately quite like to have a JavaScript "binding" for that,
>> but at this stage I wouldn't know where to begin and I'd probably end up
>> "reverse engineering" the work Gordon has done with qpid::messaging and/or
>> Rob and co. are doing with the new JMS implementation.
>>
>> I'd be interested to hear from Gordon and Ted about how they went about
>> building capability on top of the engine API.
>>
>> I hope I'm not coming across as over-critical - email is a bit rubbish at
>> conveying emotion :-) I guess that I just want to add my 2p from the
>> perspective of someone coming in new with no previous exposure to the code
>> base, which is hopefully a fairly objective position.
>>
> No worries, I'm well aware of the issues you mention and I'm as eager to
> see a solution as anyone else. I kicked off this thread asking for thoughts
> and ideas, so I certainly can't blame anyone for speaking up.
>
> --Rafael
>
Re: improving cross language maintainability
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Fri, Dec 20, 2013 at 11:43 AM, Fraser Adams <
fraser.adams@blueyonder.co.uk> wrote:
> I've been following this thread with interest and I guess that Rob's
> comment
>
>
> "
>
> However I think that would require us to be
> organised differently with a recognition that the Engine and Messenger are
> conceptually separate (with Messenger depending on the Engine API, and
> having no more authority over defining/testing the engine API than any
> other client
> "
>
> Struck some resonance.
>
> I think perhaps the layering is OK if you're familiar with the code base
> and perhaps it's more about "packaging" than layering, but FWIW coming into
> proton quite "cold" it seems like messenger and engine are essentially
> "combined". Now I know that's not the case now from the responses to my
> "how does it all hang together" question of a few weeks ago, but in terms
> of packaging that's still kind of the case.
>
> So what I mean by that is that qpid::messaging and proton messenger are as
> far as has been discussed with me "peer" APIs - both high level that can
> achieve similar things albeit with different semantics, whereas engine is a
> lower level API.
>
> Why is it in that case that the proton library is built that contains both
> messenger and engine? OK, so it's convenient, but as far as I'm aware
> neither qpid::messaging nor Ted's dispatch router actually use messenger at
> all, they both use engine? That would suggest to me that engine and
> messenger probably ought to be exposed as separate libraries? (and
> presumably there's a similar position in Java where the JMS implementation
> is engine not messenger based - though I've not looked at the jar
> packaging).
>
> I guess (perhaps related) it just might be better if messenger was in a
> separate source tree, which I'd agree might be a faff, but would clarify
> the hierarchical rather than peer relationship between messenger and engine.
>
> So I guess that my take is that Rafael is perfectly accurate when he says
> that "I believe they are currently layered precisely the way you
> describe" that certainly seems to be the case at the logical level, but at
> the "physical" level it's an awful lot less clear of the layering -
> certainly from the perspective of a novice trying to navigate the code base.
>
There is certainly a pragmatic element in keeping messenger and engine
bundled the way they are. From a development perspective, having messenger
in the same code base makes testing significantly easier. Writing a simple
command line program to fire off a few messages in order to reproduce some
scenario is quite trivial with messenger, but would involve lots of boiler
plate with just the engine directly. That's something I would like to work
on improving about the engine API, but it is very slow and difficult to
make changes given the current setup I described in my original post. Given
that, realistically I think if we were to pull messenger into a separate
code base, there would be multiple sources of duplicate effort, not only
having to duplicate/produce another non trivial multi lingual build system,
but we would also need to build up a test harness that duplicates a good
chunk of what messenger already does. Granted a chunk of that is
boilerplate that should be somehow refactored into the engine API proper so
that it is easier to use in general, and if this were to happen such a
split would probably be more feasible, but that kind of refactoring is
really prohibitively expensive given all the different shims and bindings
that would be impacted.
I'd also point out that from a user perspective I think there is some
synergy in bundling a higher level tool together with the engine. It is
very rare that you are going to want to embed the engine in something and
not want some convenient high level way to send that thing a message. Case
in point, Ted's dispatch router proper doesn't actually use messenger as
you say, however he ships it with several command line scripts that do use
messenger. Splitting them up would result in both him and his users having
to deal with an extra dependency.
> I'd have to agree also with Rob about the horizontal scaling aspects of
> lack of comments and documentation, particularly with respect to engine.
> I'm currently trying to get a JavaScript_messenger_ working because that
> seems tractable given the examples and documentation around messenger
> (though as you know even there I've had a fair few complications) my
> personal preference/familiarity is however around the JMS/qpid::messaging
> API and I'd ultimately quite like to have a JavaScript "binding" for that,
> but at this stage I wouldn't know where to begin and I'd probably end up
> "reverse engineering" the work Gordon has done with qpid::messaging and/or
> Rob and co. are doing with the new JMS implementation.
>
> I'd be interested to hear from Gordon and Ted about how they went about
> building capability on top of the engine API.
>
> I hope I'm not coming across as over-critical - email is a bit rubbish at
> conveying emotion :-) I guess that I just want to add my 2p from the
> perspective of someone coming in new with no previous exposure to the code
> base, which is hopefully a fairly objective position.
>
No worries, I'm well aware of the issues you mention and I'm as eager to
see a solution as anyone else. I kicked off this thread asking for thoughts
and ideas, so I certainly can't blame anyone for speaking up.
--Rafael
Re: improving cross language maintainability
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
I've been following this thread with interest and I guess that Rob's comment
"
However I think that would require us to be
organised differently with a recognition that the Engine and Messenger are
conceptually separate (with Messenger depending on the Engine API, and
having no more authority over defining/testing the engine API than any
other client
"
Struck some resonance.
I think perhaps the layering is OK if you're familiar with the code base and perhaps it's more about "packaging" than layering, but FWIW coming into proton quite "cold" it seems like messenger and engine are essentially "combined". Now I know that's not the case now from the responses to my "how does it all hang together" question of a few weeks ago, but in terms of packaging that's still kind of the case.
So what I mean by that is that qpid::messaging and proton messenger are as far as has been discussed with me "peer" APIs - both high level that can achieve similar things albeit with different semantics, whereas engine is a lower level API.
Why is it in that case that the proton library is built that contains both messenger and engine? OK, so it's convenient, but as far as I'm aware neither qpid::messaging nor Ted's dispatch router actually use messenger at all, they both use engine? That would suggest to me that engine and messenger probably ought to be exposed as separate libraries? (and presumably there's a similar position in Java where the JMS implementation is engine not messenger based - though I've not looked at the jar packaging).
I guess (perhaps related) it just might be better if messenger was in a separate source tree, which I'd agree might be a faff, but would clarify the hierarchical rather than peer relationship between messenger and engine.
So I guess that my take is that Rafael is perfectly accurate when he says that "I believe they are currently layered precisely the way you
describe" that certainly seems to be the case at the logical level, but at the "physical" level it's an awful lot less clear of the layering - certainly from the perspective of a novice trying to navigate the code base.
I'd have to agree also with Rob about the horizontal scaling aspects of lack of comments and documentation, particularly with respect to engine. I'm currently trying to get a JavaScript_messenger_ working because that seems tractable given the examples and documentation around messenger (though as you know even there I've had a fair few complications) my personal preference/familiarity is however around the JMS/qpid::messaging API and I'd ultimately quite like to have a JavaScript "binding" for that, but at this stage I wouldn't know where to begin and I'd probably end up "reverse engineering" the work Gordon has done with qpid::messaging and/or Rob and co. are doing with the new JMS implementation.
I'd be interested to hear from Gordon and Ted about how they went about building capability on top of the engine API.
I hope I'm not coming across as over-critical - email is a bit rubbish at conveying emotion :-) I guess that I just want to add my 2p from the perspective of someone coming in new with no previous exposure to the code base, which is hopefully a fairly objective position.
Best regards,
Frase
On 20/12/13 15:57, Rafael Schloming wrote:
> On Fri, Dec 20, 2013 at 5:46 AM, Rob Godfrey <ro...@gmail.com>wrote:
>
>> Since I have pretty much given up on trying to work on proton, I'm not sure
>> that my opinion counts for much anymore... however I suppose I'm a little
>> curious how the JNI binding differs in "extra work" vs. PHP or Ruby or
>> such. Similarly why the API distinction is "hasty" when again it should
>> (modulo some language idiom changes) reflect the API that the C is
>> presenting to Ruby/PHP/Python/etc.
>>
> That's a really good question. One obvious factor is that the JNI bindings
> cover the full surface of the engine API, whereas the PHP and Ruby bindings
> only cover the messenger API, the latter being much smaller. The other
> factor is that we have a different workflow with how we maintain the
> scripting language bindings. From a design perspective each of those
> bindings are more tightly coupled to the C code since that is the only
> implementation they use, but from a workflow perspective each one is
> actually more independent than the Java binding. When we add a feature to
> the C messenger implementation we don't try to add it simultaneously across
> all the bindings. It generally goes into python because that is how we
> test, and it goes into perl and ruby pretty quickly after that because
> Darryl is good about keeping those up to date, but, e.g., in the case of
> php, we haven't tried to keep the binding at full feature parity. Finally,
> the JNI binding is a bit of a different animal from the others in that it
> is not just performing a simple wrapping of the C API and directly exposing
> that to Java. It is adapting between the C API underneath it and the Java
> API above it.
>
> As for the hastiness, perhaps that was a poor choice of words. The problem
> with the Java API/impl split is that the API was produced from an existing
> implementation after the fact in order to facilitate testing. As such it
> isn't well suited to be implemented by two independent implementations,
> particularly one that is JNI based.
>
>
>> To me the fact that APIs aren't defined but that instead the only way of
>> knowing what a particular call should do seems to be 1) look at the code in
>> C and, if it's not clear, then 2) ask Rafi; is why I no longer feel it
>> productive to try to contribute. As such, it seems that this change is
>> more about vertical scaling (making it easier for Rafi, and those sitting
>> very near him, to work more productively), than horizontal scaling
>> (allowing people who are not Rafi or with easy access to him to work on
>> it).
>>
> Really there are two separate issues here. You're talking about conceptual
> barriers to contribution, and I'm talking about mechanical barriers. Both
> are barriers to contribution, and both prevent horizontal scaling. Fixing
> some of the mechanical barriers to contribution does happen to also improve
> vertical scaling, but as I said in my original post, I think the more
> important factor is making it easier/possible for people to make complete
> contributions.
>
> FWIW, I agree we need lots of additional work around API definition and
> documentation. Having more time for that is also one of the reasons I would
> like to reduce some of the overhead involved in the current development
> process.
>
>
>> Overall I don't care very much about Messenger in Java (either pure Java or
>> JNI), but I do care about the Engine. When the Proton Engine was first
>> discussed it was always said that the C code should be able to be called
>> from within Java as an alternative to a pure Java impl. - has this changed?
>>
> I'm fine with that as a goal, however I can't say it's a priority for me.
> More importantly, as I've said before, I don't believe the use of JNI in
> production is at all compatible with the use of JNI for testing purposes.
> The current JNI binding exposes each part of the engine API in very fine
> detail. This what you want for testing, so you can exercise every part of
> the C API from Java. This is probably exactly the opposite of what you'd
> want for production use though. I expect you'd want to minimize the number
> of times you cross the C/Java boundary and just tie into the engine API at
> key points that are shifting a lot of bytes around.
>
>
>> Going back to the horizontal vs. vertical scaling... If the project were
>> structured differently I would be very happy to contribute to the engine
>> side of things in Java. However I think that would require us to be
>> organised differently with a recognition that the Engine and Messenger are
>> conceptually separate (with Messenger depending on the Engine API, and
>> having no more authority over defining/testing the engine API than any
>> other client - such as the C++ broker, ActiveMQ, or the upcoming JMS clent,
>> etc.). As such I would rather see us fix the definition of the API (if you
>> believe it to be "hasty") rather than simply try to remove any notion of
>> their being an API which is distinct from the implementation.
>>
> I'm not sure exactly what you're asking for here with respect to Messenger
> and Engine. I believe they are currently layered precisely the way you
> describe. There certainly aren't any code level dependencies from Engine to
> Messenger. Are there specific ways that you think Messenger has influenced
> the Engine that are problematic? Is there something concrete you would like
> to see happen here?
>
> As for the Java API, I'm not suggesting eliminating the notion of there
> being an API. What I'm suggesting is that what sits in the proton-api
> module is not actually an API that is well suited for multiple
> implementations, but is really a single implementation API. As such I would
> like to put it back into the proton package and focus on making it a good
> single implementation API. I don't believe that this step would detract at
> all from it being a well defined API, in fact I think it would make it
> clearer and more transparent as there are currently plenty of cases where
> the split obscures the design. I believe a distinct question is whether we
> need a multi-implementation API, and if so for what purpose, e.g.
> production use of a JNI based engine vs testing of the C based engine.
>
> --Rafael
>
Re: improving cross language maintainability
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Fri, Dec 20, 2013 at 5:46 AM, Rob Godfrey <ro...@gmail.com>wrote:
> Since I have pretty much given up on trying to work on proton, I'm not sure
> that my opinion counts for much anymore... however I suppose I'm a little
> curious how the JNI binding differs in "extra work" vs. PHP or Ruby or
> such. Similarly why the API distinction is "hasty" when again it should
> (modulo some language idiom changes) reflect the API that the C is
> presenting to Ruby/PHP/Python/etc.
>
That's a really good question. One obvious factor is that the JNI bindings
cover the full surface of the engine API, whereas the PHP and Ruby bindings
only cover the messenger API, the latter being much smaller. The other
factor is that we have a different workflow with how we maintain the
scripting language bindings. From a design perspective each of those
bindings are more tightly coupled to the C code since that is the only
implementation they use, but from a workflow perspective each one is
actually more independent than the Java binding. When we add a feature to
the C messenger implementation we don't try to add it simultaneously across
all the bindings. It generally goes into python because that is how we
test, and it goes into perl and ruby pretty quickly after that because
Darryl is good about keeping those up to date, but, e.g., in the case of
php, we haven't tried to keep the binding at full feature parity. Finally,
the JNI binding is a bit of a different animal from the others in that it
is not just performing a simple wrapping of the C API and directly exposing
that to Java. It is adapting between the C API underneath it and the Java
API above it.
As for the hastiness, perhaps that was a poor choice of words. The problem
with the Java API/impl split is that the API was produced from an existing
implementation after the fact in order to facilitate testing. As such it
isn't well suited to be implemented by two independent implementations,
particularly one that is JNI based.
>
> To me the fact that APIs aren't defined but that instead the only way of
> knowing what a particular call should do seems to be 1) look at the code in
> C and, if it's not clear, then 2) ask Rafi; is why I no longer feel it
> productive to try to contribute. As such, it seems that this change is
> more about vertical scaling (making it easier for Rafi, and those sitting
> very near him, to work more productively), than horizontal scaling
> (allowing people who are not Rafi or with easy access to him to work on
> it).
>
Really there are two separate issues here. You're talking about conceptual
barriers to contribution, and I'm talking about mechanical barriers. Both
are barriers to contribution, and both prevent horizontal scaling. Fixing
some of the mechanical barriers to contribution does happen to also improve
vertical scaling, but as I said in my original post, I think the more
important factor is making it easier/possible for people to make complete
contributions.
FWIW, I agree we need lots of additional work around API definition and
documentation. Having more time for that is also one of the reasons I would
like to reduce some of the overhead involved in the current development
process.
>
> Overall I don't care very much about Messenger in Java (either pure Java or
> JNI), but I do care about the Engine. When the Proton Engine was first
> discussed it was always said that the C code should be able to be called
> from within Java as an alternative to a pure Java impl. - has this changed?
>
I'm fine with that as a goal, however I can't say it's a priority for me.
More importantly, as I've said before, I don't believe the use of JNI in
production is at all compatible with the use of JNI for testing purposes.
The current JNI binding exposes each part of the engine API in very fine
detail. This what you want for testing, so you can exercise every part of
the C API from Java. This is probably exactly the opposite of what you'd
want for production use though. I expect you'd want to minimize the number
of times you cross the C/Java boundary and just tie into the engine API at
key points that are shifting a lot of bytes around.
>
> Going back to the horizontal vs. vertical scaling... If the project were
> structured differently I would be very happy to contribute to the engine
> side of things in Java. However I think that would require us to be
> organised differently with a recognition that the Engine and Messenger are
> conceptually separate (with Messenger depending on the Engine API, and
> having no more authority over defining/testing the engine API than any
> other client - such as the C++ broker, ActiveMQ, or the upcoming JMS clent,
> etc.). As such I would rather see us fix the definition of the API (if you
> believe it to be "hasty") rather than simply try to remove any notion of
> their being an API which is distinct from the implementation.
>
I'm not sure exactly what you're asking for here with respect to Messenger
and Engine. I believe they are currently layered precisely the way you
describe. There certainly aren't any code level dependencies from Engine to
Messenger. Are there specific ways that you think Messenger has influenced
the Engine that are problematic? Is there something concrete you would like
to see happen here?
As for the Java API, I'm not suggesting eliminating the notion of there
being an API. What I'm suggesting is that what sits in the proton-api
module is not actually an API that is well suited for multiple
implementations, but is really a single implementation API. As such I would
like to put it back into the proton package and focus on making it a good
single implementation API. I don't believe that this step would detract at
all from it being a well defined API, in fact I think it would make it
clearer and more transparent as there are currently plenty of cases where
the split obscures the design. I believe a distinct question is whether we
need a multi-implementation API, and if so for what purpose, e.g.
production use of a JNI based engine vs testing of the C based engine.
--Rafael
Re: improving cross language maintainability
Posted by Rob Godfrey <ro...@gmail.com>.
Since I have pretty much given up on trying to work on proton, I'm not sure
that my opinion counts for much anymore... however I suppose I'm a little
curious how the JNI binding differs in "extra work" vs. PHP or Ruby or
such. Similarly why the API distinction is "hasty" when again it should
(modulo some language idiom changes) reflect the API that the C is
presenting to Ruby/PHP/Python/etc.
To me the fact that APIs aren't defined but that instead the only way of
knowing what a particular call should do seems to be 1) look at the code in
C and, if it's not clear, then 2) ask Rafi; is why I no longer feel it
productive to try to contribute. As such, it seems that this change is
more about vertical scaling (making it easier for Rafi, and those sitting
very near him, to work more productively), than horizontal scaling
(allowing people who are not Rafi or with easy access to him to work on
it).
Overall I don't care very much about Messenger in Java (either pure Java or
JNI), but I do care about the Engine. When the Proton Engine was first
discussed it was always said that the C code should be able to be called
from within Java as an alternative to a pure Java impl. - has this changed?
Going back to the horizontal vs. vertical scaling... If the project were
structured differently I would be very happy to contribute to the engine
side of things in Java. However I think that would require us to be
organised differently with a recognition that the Engine and Messenger are
conceptually separate (with Messenger depending on the Engine API, and
having no more authority over defining/testing the engine API than any
other client - such as the C++ broker, ActiveMQ, or the upcoming JMS clent,
etc.). As such I would rather see us fix the definition of the API (if you
believe it to be "hasty") rather than simply try to remove any notion of
their being an API which is distinct from the implementation.
-- Rob
On 19 December 2013 14:49, Rafael Schloming <rh...@alum.mit.edu> wrote:
> Hi Everyone,
>
> I've been struggling with some of the cross language maintenance aspects of
> proton for a while, and I believe we need to take some steps to improve the
> situation. I'm one of a tiny number of people (two possibly) who regularly
> commit changes to both the Java and C codebase and attempt to keep the two
> in sync and at feature parity. Because of this, not everyone is necessarily
> aware of the process, but to summarize the issue, currently there are far
> too many moving parts and layers of indirection involved. This is not only
> a significant drag on my personal productivity, but perhaps more
> importantly it is a significant barrier to growing the number of proton
> contributors in general as all those moving parts and layers of indirection
> need to be understood before being able to make complete contributions.
>
> To get an idea of what's going on here I think it helps to look at what's
> involved in a simple change. Recently I noticed an edge case around status
> updates in the messenger interface. The fix involved adding another value
> to messenger's status enum and making a trivial logic change to make use of
> that value under the appropriate circumstances. You can look at the full
> commit here[1] if you like, but it breaks down in the following way:
>
> Changes to C implementation (.h and .c file): 8 LOC
> Changes to the python test shim for C: 3 LOC
> Changes to the Java API: 2 LOC
> Changes to the pure Java implementation: 6 LOC
> Changes to the JNI binding: 4 LOC
> Changes to the python test shim for Java: 3 LOC
> Changes to the python test suite: 2 LOC
>
> Now obviously from a personal productivity perspective it is at a minimum
> annoying to have to touch so many different parts in order to make even a
> trivial change, but setting that aside for the moment, what is really
> sobering about this is that each one of the above parts involve a non
> trivial learning curve on their own, and while it's true that only a few
> lines of code are needed in each area, it is necessary to understand each
> piece before being able to write the correct few lines of code. This
> presents a pretty daunting hurdle for bringing new contributors to the
> codebase up to the level needed to make even a trivial change like the one
> above.
>
> The JNI binding and the python test shim for Java both exist to serve
> similar purposes, namely to facilitate running a common test suite against
> both implementations in order to ensure common behaviour. The python test
> shims allow the python test suite to run against both proton-c and proton-j
> (via jython), and the JNI binding allows pure Java tests to be run against
> proton-c as well as proton-j.
>
> Currently by my count there are about 19 such tests in Java. By comparison
> there are about 266 tests in the python test suite. Also, judging by
> commits, the python test suite is growing/actively maintained, and the java
> system tests (the subset of java tests that are run against both
> implementations) are neither. On top of this the JNI binding itself has
> suffered significantly from bit rot. As far as I know it is not made use of
> outside of our own test infrastructure, and currently about 50% of the
> tests run against it are skipped because it is only minimally maintained
> when necessary to get the tests to pass.
>
> Because of all this I'm proposing that we remove the JNI binding and roll
> back the Java API/impl split that was (hastily) done to facilitate the test
> infrastructure. This should significantly simplify the development process
> relative to what it is now, and while there is still more learning curve
> than desired with the python shims, I believe this will put us in a
> position to improve the shims, remove duplication, and minimize the
> overhead associated with them, ultimately allowing us to make the codebase
> more transparently communicate its design and hopefully lessen the
> learning curve for new contributions.
>
> I would love to hear thoughts and/or alternative ideas on how to improve
> things. I would like to start addressing this early in the 0.7 development
> cycle.
>
> [1] http://svn.apache.org/r1551950
>
Re: improving cross language maintainability
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Thu, Dec 19, 2013 at 2:44 PM, Ken Giusti <kg...@redhat.com> wrote:
> Sorry for top-posting. I'm trying to understand the consequences of what
> you are proposing.
>
> First, as I understand it, there are two separate test suites in the
> proton tree: one written in Java - containing 19 tests as you point out -
> and a much larger one written in python. Each test suite exercises both
> the Java and C implementations. By testing both implementations using the
> same tests, we ensure consistency across the two implementations (some, as
> a lot of the python tests are skipped)
>
> What you're proposing would remove the ability for the Java test suite to
> exercise the C implementation, right?
>
Yes, it would remove the ability for Java tests to exercise C code by
calling it through JNI. We do have the interop test suite which checks that
there is common behaviour without using JNI, i.e. by comparing binary
output of codec and such across languages and I can imagine a number of
other ways we could test for common behaviour that would not involve Java
code calling into C code, e.g. comparing protocol traces, or running
interop scenarios over the wire.
>
> This means that only the python test suite would be used to ensure cross
> implementation consistency (Java v. C), right?
>
Yes
>
> What doesn't change is the two python wrapper implementations - one that
> wraps the Java API, the other wraps the C API - that are used by the python
> test suite to test both implementations. We'd still have to keep both of
> those sync'ed.
>
Correct, although I believe there are some ways that we can significantly
improve the commonality between the two python wrapper implementations
(shims), and I suspect this will also improve the consistency of the APIs
as well.
>
> If all my assumptions above are correct, then I can live with this. It
> still ensures (some) consistency checking between the two implementations.
> Drift between the two python wrappers could be caught by the tests
> themselves, so I'm not too concerned about that.
>
> I'd like to see that time saved maintaining two test beds invested in
> bringing both implementations to parity - IMHO we're skipping far too many
> tests due to feature disparity.
>
Agreed.
--Rafael
Re: improving cross language maintainability
Posted by Ken Giusti <kg...@redhat.com>.
Sorry for top-posting. I'm trying to understand the consequences of what you are proposing.
First, as I understand it, there are two separate test suites in the proton tree: one written in Java - containing 19 tests as you point out - and a much larger one written in python. Each test suite exercises both the Java and C implementations. By testing both implementations using the same tests, we ensure consistency across the two implementations (some, as a lot of the python tests are skipped)
What you're proposing would remove the ability for the Java test suite to exercise the C implementation, right?
This means that only the python test suite would be used to ensure cross implementation consistency (Java v. C), right?
What doesn't change is the two python wrapper implementations - one that wraps the Java API, the other wraps the C API - that are used by the python test suite to test both implementations. We'd still have to keep both of those sync'ed.
If all my assumptions above are correct, then I can live with this. It still ensures (some) consistency checking between the two implementations. Drift between the two python wrappers could be caught by the tests themselves, so I'm not too concerned about that.
I'd like to see that time saved maintaining two test beds invested in bringing both implementations to parity - IMHO we're skipping far too many tests due to feature disparity.
----- Original Message -----
> From: "Rafael Schloming" <rh...@alum.mit.edu>
> To: proton@qpid.apache.org
> Sent: Thursday, December 19, 2013 8:49:59 AM
> Subject: improving cross language maintainability
>
> Hi Everyone,
>
> I've been struggling with some of the cross language maintenance aspects of
> proton for a while, and I believe we need to take some steps to improve the
> situation. I'm one of a tiny number of people (two possibly) who regularly
> commit changes to both the Java and C codebase and attempt to keep the two
> in sync and at feature parity. Because of this, not everyone is necessarily
> aware of the process, but to summarize the issue, currently there are far
> too many moving parts and layers of indirection involved. This is not only
> a significant drag on my personal productivity, but perhaps more
> importantly it is a significant barrier to growing the number of proton
> contributors in general as all those moving parts and layers of indirection
> need to be understood before being able to make complete contributions.
>
> To get an idea of what's going on here I think it helps to look at what's
> involved in a simple change. Recently I noticed an edge case around status
> updates in the messenger interface. The fix involved adding another value
> to messenger's status enum and making a trivial logic change to make use of
> that value under the appropriate circumstances. You can look at the full
> commit here[1] if you like, but it breaks down in the following way:
>
> Changes to C implementation (.h and .c file): 8 LOC
> Changes to the python test shim for C: 3 LOC
> Changes to the Java API: 2 LOC
> Changes to the pure Java implementation: 6 LOC
> Changes to the JNI binding: 4 LOC
> Changes to the python test shim for Java: 3 LOC
> Changes to the python test suite: 2 LOC
>
> Now obviously from a personal productivity perspective it is at a minimum
> annoying to have to touch so many different parts in order to make even a
> trivial change, but setting that aside for the moment, what is really
> sobering about this is that each one of the above parts involve a non
> trivial learning curve on their own, and while it's true that only a few
> lines of code are needed in each area, it is necessary to understand each
> piece before being able to write the correct few lines of code. This
> presents a pretty daunting hurdle for bringing new contributors to the
> codebase up to the level needed to make even a trivial change like the one
> above.
>
> The JNI binding and the python test shim for Java both exist to serve
> similar purposes, namely to facilitate running a common test suite against
> both implementations in order to ensure common behaviour. The python test
> shims allow the python test suite to run against both proton-c and proton-j
> (via jython), and the JNI binding allows pure Java tests to be run against
> proton-c as well as proton-j.
>
> Currently by my count there are about 19 such tests in Java. By comparison
> there are about 266 tests in the python test suite. Also, judging by
> commits, the python test suite is growing/actively maintained, and the java
> system tests (the subset of java tests that are run against both
> implementations) are neither. On top of this the JNI binding itself has
> suffered significantly from bit rot. As far as I know it is not made use of
> outside of our own test infrastructure, and currently about 50% of the
> tests run against it are skipped because it is only minimally maintained
> when necessary to get the tests to pass.
>
> Because of all this I'm proposing that we remove the JNI binding and roll
> back the Java API/impl split that was (hastily) done to facilitate the test
> infrastructure. This should significantly simplify the development process
> relative to what it is now, and while there is still more learning curve
> than desired with the python shims, I believe this will put us in a
> position to improve the shims, remove duplication, and minimize the
> overhead associated with them, ultimately allowing us to make the codebase
> more transparently communicate its design and hopefully lessen the
> learning curve for new contributions.
>
> I would love to hear thoughts and/or alternative ideas on how to improve
> things. I would like to start addressing this early in the 0.7 development
> cycle.
>
> [1] http://svn.apache.org/r1551950
>
--
-K
Re: improving cross language maintainability
Posted by "Darryl L. Pierce" <dp...@redhat.com>.
On Thu, Dec 19, 2013 at 04:07:38PM -0500, Rafael Schloming wrote:
> On Thu, Dec 19, 2013 at 9:01 AM, Darryl L. Pierce <dp...@redhat.com>wrote:
> > On Thu, Dec 19, 2013 at 08:49:59AM -0500, Rafael Schloming wrote:
> > <snip>
> > > I would love to hear thoughts and/or alternative ideas on how to improve
> > > things. I would like to start addressing this early in the 0.7
> > development
> > > cycle.
> >
> > In a similar way, I'm trying to keep our Ruby and Perl bindings in
> > parity as best I can with what's going on in the C and Python code. Can
> > we use JIRA to create umbrella tasks for when new features are added,
> > with subtasks that are binding specific? Or if there's a change to the C
> > code that would require a change in the bindings, have the C code be the
> > top JIRA and the language bindings be subtasks to that? That way I
> > wouldn't need to look through commits to see what's changed in C to know
> > what should be added to the other languages.
> >
> > Also, could we add a component for each of the language bindings? It
> > doesn't feel right to add a JIRA for something in Ruby that's at the
> > Ruby level but have its component be "proton-c".
> >
>
> It certainly makes sense to make it as easy as possible to track what is
> going on, and I can see how that would help keep bindings up to date where
> there is interest and resources to do so. However we do this though, I
> don't want to just brainlessly duplicate each C jira across every binding
> (not that you're necessarily suggesting this).
>
> The problem I have with that approach is that there isn't equivalent
> interest/resources associated with each binding, so e.g. if we were to make
> every JIRA a full umbrella that depends on sub tasks for each binding we
> would continually accumulate php jiras that never end up getting closed off
> because we don't keep php as up to date as the other bindings, and this in
> turn would cause the umbrella JIRAs to never get closed off. Jira is really
> a task oriented tool, and I think JIRAs should really only be created when
> there is intention/interest to actually complete the task they represent,
> otherwise they usually end up being noise/clutter that will eventually be
> irrelevant and out of date. I'd suggest that perhaps a more document
> oriented description of those features for which we are trying to keep
> parity would possibly be helpful.
Definitely. Maybe a wiki page that lists the set of feeatures being done
in C can be used as a backlog for the other language bindingss. Not
necessarily a full blown, prioritized scrum backlog. But minimally a
central list of what's done in C/Python that can be used as a guide?
> All that said, I'm certainly sure we can improve our usage of JIRA, and
> I've gone ahead and added ruby-binding, python-binding, perl-binding, and
> php-binding components as you suggest.
Thank you! :D
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
Re: improving cross language maintainability
Posted by Rafael Schloming <rh...@alum.mit.edu>.
On Thu, Dec 19, 2013 at 9:01 AM, Darryl L. Pierce <dp...@redhat.com>wrote:
> On Thu, Dec 19, 2013 at 08:49:59AM -0500, Rafael Schloming wrote:
> <snip>
> > I would love to hear thoughts and/or alternative ideas on how to improve
> > things. I would like to start addressing this early in the 0.7
> development
> > cycle.
>
> In a similar way, I'm trying to keep our Ruby and Perl bindings in
> parity as best I can with what's going on in the C and Python code. Can
> we use JIRA to create umbrella tasks for when new features are added,
> with subtasks that are binding specific? Or if there's a change to the C
> code that would require a change in the bindings, have the C code be the
> top JIRA and the language bindings be subtasks to that? That way I
> wouldn't need to look through commits to see what's changed in C to know
> what should be added to the other languages.
>
> Also, could we add a component for each of the language bindings? It
> doesn't feel right to add a JIRA for something in Ruby that's at the
> Ruby level but have its component be "proton-c".
>
It certainly makes sense to make it as easy as possible to track what is
going on, and I can see how that would help keep bindings up to date where
there is interest and resources to do so. However we do this though, I
don't want to just brainlessly duplicate each C jira across every binding
(not that you're necessarily suggesting this).
The problem I have with that approach is that there isn't equivalent
interest/resources associated with each binding, so e.g. if we were to make
every JIRA a full umbrella that depends on sub tasks for each binding we
would continually accumulate php jiras that never end up getting closed off
because we don't keep php as up to date as the other bindings, and this in
turn would cause the umbrella JIRAs to never get closed off. Jira is really
a task oriented tool, and I think JIRAs should really only be created when
there is intention/interest to actually complete the task they represent,
otherwise they usually end up being noise/clutter that will eventually be
irrelevant and out of date. I'd suggest that perhaps a more document
oriented description of those features for which we are trying to keep
parity would possibly be helpful.
All that said, I'm certainly sure we can improve our usage of JIRA, and
I've gone ahead and added ruby-binding, python-binding, perl-binding, and
php-binding components as you suggest.
--Rafael
Re: improving cross language maintainability
Posted by "Darryl L. Pierce" <dp...@redhat.com>.
On Thu, Dec 19, 2013 at 08:49:59AM -0500, Rafael Schloming wrote:
<snip>
> I would love to hear thoughts and/or alternative ideas on how to improve
> things. I would like to start addressing this early in the 0.7 development
> cycle.
In a similar way, I'm trying to keep our Ruby and Perl bindings in
parity as best I can with what's going on in the C and Python code. Can
we use JIRA to create umbrella tasks for when new features are added,
with subtasks that are binding specific? Or if there's a change to the C
code that would require a change in the bindings, have the C code be the
top JIRA and the language bindings be subtasks to that? That way I
wouldn't need to look through commits to see what's changed in C to know
what should be added to the other languages.
Also, could we add a component for each of the language bindings? It
doesn't feel right to add a JIRA for something in Ruby that's at the
Ruby level but have its component be "proton-c".
--
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/