A case in favor of separate repos for language bindings

[Flavio Percoco <fl...@redhat.com>]

Greetings,

I'd like to take a chance, now that 0.10 is out, to make a point in
favor of using submodules rather than having all bindings in the same
repository.

I think qpid-proton has reached the point where it's mature enough to
have some of the bindings promoted to having their own repos. More
importantly, I believe these bindings are mature enough and ready to
build a community on their own.

Lets take the python bindings, for instance. I've been contributing to
the python bindings quite a bit in the last couple of months. The
bindings went from depending on cmake for tests to having their own,
still dependant on proton-test, test suite that is self-maintained.

Furthermore, the python bindings went from depending on cmake to be
built to being self-built using python's internal build system. It's
still possible - and in fact, it still happens - to build these
drivers running cmake, although it's, in my personal opinion, not
recommended.

As you can see from the above, the python bindings depend on proton-c
but they don't need to live in the same codebase. Therefore, I've
prepared a small POC to show case this repo structure:

- https://github.com/FlaPer87/test-python-qpid-proton

The above repo contains the python-qpid-proton code. As you can tell
from the commit history, not many changes were required to set it up.
The relevant bits of this repo, I believe, are the ones related to the
submodule. The qpid-proton submodule points qpid-proton's master
branch in the test-python-qpid-proton branch, whereas in the 0.10
branch, it tracks qpid-proton's 0.10.x. This is important for the
bindings to keep track of the right code/release/tests.

- https://github.com/FlaPer87/qpid-proton/tree/submodules

The above is a fork of the qpid-proton repo we use already but rather
than having the python-qpid-bindings in tree, it just tracks the
master branch. (I created a `submodules` branch because I didn't want
to mess with the master branch in my fork).

I don't think tracking the binding in the qpid-proton repo is
necessary but I see how it's useful to avoid introducing breaking
changes on drivers. To be more precise, I don't think the above is
necessary because if we are testing bindings in cmake to make sure we
don't break backwards compatibility, we could simply install the
latest stable version and test against that, rather than testing
against the latest binding's master, which could, but shouldn't, be
broken.

I wouldn't probably advocate for the above to applied to every binding
we have since some of them might not be ready for this but I
definitely want to advocate to make this happen for the python binding
(and other bindings).

This structure will make it easier to mainting bindings, to build a
community around these bindings - I contribute more to the python
binding than proton-c. It'll also help keeping a clean history for
each project and to issue binding releases, therefore creating
tags for bindings, as needed.

I get it's worked well so far and probably some of you won't be happy
with this proposal but, git is powerful enough to support the above
structure, which is ideal for these kind of projects. Correct me if
I'm wrong but I believe the current structure also comes from the old
times when qpid-proton used to be in subversion.

Hope the above makes sense and I'm sorry for such a long email, I
should've drunk less coffee.
Flavio

P.S: I'm more than happy to help setting this up and to bring it back
if the experiment fails.

-- 
@flaper87
Flavio Percoco

Re: A case in favor of separate repos for language bindings

[aconway <ac...@redhat.com>]

My 2c.

The "use python to drive C and Java tests" was a good idea at the time
(because we only had C, Java and Python and python is the most
productive language to write tests in) and has served us well, but we
have outgrown it. 

There are two things we need:

- good unit tests for each binding/component that can be built/run
easily by developers in that language without excessive dependencies.

- good interop testing between *all* the components, not just a
"special relationship" between C, Java and python.

There is already work underway on an AMQP-focused (rather than language
focused) qpid interop test suite to cover all Qpid components.

So I would suggest reviwing the existing python tests to identify tests
that are *really* unit tests, and tests that are really about AMQP. The
latter to be adopted into the Qpid interop test framework to benefit
all the languages/components we care about. I suspect a lot of them are
the latter.

I see no issue with using python as part of the driver for C tests
since C is such a painful language to work with, but that shouldn't
require the entire python binding.

That does mean that you need to check-out, build and run tests in 2
repos for any given task: the component repo and the tests repo. On the
up side you *don't* have to check out and build every binding to work
on the core: just the core and the tests (which depends on the
bindings, which can be installed) That actually helps in the sense that
if you make core changes that break released bindings you know about it
before you ship.

Cheers,
Alan.

On Wed, 2015-08-19 at 19:05 +0100, Robbie Gemmell wrote:
> On 19 August 2015 at 13:05, Flavio Percoco <fl...@redhat.com> wrote:
> > On 19/08/15 12:34 +0100, Robbie Gemmell wrote:
> > > 
> > > I can see certain benefits to such a separation, mainly for folks
> > > interested only in the bindings, but if I'm honest I'm not sure 
> > > those
> > > outweigh the additional complication it seems it may bring in 
> > > some of
> > > the other areas.
> > 
> > 
> > I think they actually do but probably because I don't see any
> > complications being added to proton-c itself but rather a
> > simplification of the current structure.
> 
> Ken has covered much of what I was thinking about.
> 
> >  It'll bring a more intuitive
> > structure to the project rather than having the bindings under
> > `proton-c`, which is very confusing :)
> > 
> 
> I can see having them elsewhere might be more intuitive, especially
> for folks who dont know or care that they are using proton-c under 
> the
> covers while using them.
> 
> I guess if they were kept in-tree they could be moved at to the top
> level to make them more obvious but then presumably the same might
> need done for the others, and that still wouldn't make a difference 
> to
> the various other points in your suggestion.
> 
> > > The python bindings are slightly more interesting than the others 
> > > due
> > > to being at the heart of the python based tests that exist for
> > > proton-c and proton-j, so splitting them out actually creates a 
> > > bit of
> > > a circular dependency with proton / its tests (especially when 
> > > having
> > > the submodules in both repos). You mentioned thinking we wouldn't 
> > > need
> > > to track the binding within the proton tree (but could do so to
> > > prevent a breaking change for existing things), and instead use 
> > > an
> > > installed version of the bindings. In practice I guess it would 
> > > be
> > > about the same in terms of updating proton-c and the bindings and 
> > > the
> > > related python-based tests at the same time (adding an install, 
> > > or
> > > possibly two, in the middle).
> > 
> > 
> > It won't be the same.
> 
> I just meant in terms of which files people would need to update it
> would be similar, but with some install steps added (depending on 
> what
> you change), not any effect on what is or isnt being tested.
> 
> > The reason is that we're currently ensuring that
> > proton-c doesn't break the bindings 'master' version. This is being
> > done on the assumption that bindings will be released along with
> > proton-c and there'll be a binding release per every proton-c 
> > minor.
> > 
> > The above is not an ideal scenarion in my opinion. Ideally, we'd 
> > have
> > a release of the bindings that will be able to track several minor
> > releases of proton-c, but this will probably come later once proton
> > -c
> > stabilizes a bit more. However, I believe the current structure 
> > allows
> > for breaking changes to land because it tests things against master
> > only rather than making sure older, still maintained, versions are 
> > not
> > being broken.
> > 
> > Going back to the proposal, I think the circular dependency is not
> > really necessary and we could just have the python bindings being
> > installed. The installed version can either be the current master
> > branch (pip knows how to install from git) or the latest stable
> > version. Furthermore, we could even have a list of versions that we
> > want to test against.
> > 
> > To expand the above issue a bit more, I think it's wrong to have 
> > the
> > python tests as part of the in-tree tests for proton-c. Ideally, 
> > this
> > should be the work of a CI system responsible for doing integration
> > tests. Not saying we should solve this issue now but the proposed
> > structure allows for it to happen.
> > 
> > > Releasing (in terms of source, here at
> > > Apache) seems like it would become more 'interesting' though, 
> > > with
> > > potential again for the bindings needing proton-c (+proton-j) 
> > > updates,
> > > but proton-c (+proton-j) needing the bindings updates for updated
> > > tests. I guess that might point to splitting the python tests out 
> > > on
> > > their own as well so it depended on both separately.
> > 
> > 
> > What python tests are you referring to? The ones under 
> > `tests/python`
> > ?
> > 
> > Those are the only ones I know of and those are still in the
> > `proton-c` repo.
> 
> Those are the ones. As Ken has largely covered, its important to note
> that those exist mostly (entirely?) to test proton-c and proton-j
> rather than the python bindings themselves (though given how they
> work, they kinda do that implicitly too). Their need of the python
> bindings (and in turn proton again) to work however would present
> challenges as Ken mentioned.
> 
> > The python bindings access those through the
> > submodule for now but yes, I'd agree that eventually, those tests
> > could even be on their own.
> > 
> > > Perhaps I'm overthinking this though; how would you see some of 
> > > those
> > > things working?
> > 
> > 
> > Totally honest? I think Java should have its own tests and let 
> > python
> > alone. Maintaining python2.5 - because that's what the tests 
> > support -
> > plus all the self-written tests libraries and whatnot is a burden 
> > that
> > I think we should not carry. Most of the things being maintained 
> > under
> > the `tests/python` 'package' exists already in the python community
> > and are widely used and tests. I know it seems easier to maintain 1 
> > test
> > suite for 2 bindings rather than 2 but it really isn't. Not in this
> > case, at the very least.
> > 
> 
> I have never really liked the python tests, they are a pain, but they
> are for the most part all that exists. Replacing them in each 
> language
> at this point doesnt seem like great fun either though :)
> 
> I'm not sure it affects things too much with regards to splitting
> things up, but the tests only use 2.5 still because noone has updated
> the jython used to 2.7 yet, that presumably could be done if its
> causing an issue.
> 
> > However, based on the current situation, I'd either have the test 
> > code
> > on its own or simply follow the proposed structure and have the
> > bindings following proton-c and consuming tests from there.
> > 
> > > I also can't say that I'd be a particular fan of using 
> > > submodules,
> > > most of the things I've read about or tried with them have been a
> > > turnoff...but I'm mostly skipping over that for now.
> > 
> > 
> > In my experience, if done right, they work and they're not hard to
> > update/maintain. Circular dependencies are to avoid, agreed. One 
> > case
> > where I've had good experience with submodules is the rustlang. I
> > guess giving this a try would tell better.
> > 
> > Thanks for your reply,
> > Flavio
> > 
> > 
> > > 
> > > Robbie
> > > 
> > > On 18 August 2015 at 08:17, Flavio Percoco <fl...@redhat.com> 
> > > wrote:
> > > > 
> > > > Greetings,
> > > > 
> > > > I'd like to take a chance, now that 0.10 is out, to make a 
> > > > point in
> > > > favor of using submodules rather than having all bindings in 
> > > > the same
> > > > repository.
> > > > 
> > > > I think qpid-proton has reached the point where it's mature 
> > > > enough to
> > > > have some of the bindings promoted to having their own repos. 
> > > > More
> > > > importantly, I believe these bindings are mature enough and 
> > > > ready to
> > > > build a community on their own.
> > > > 
> > > > Lets take the python bindings, for instance. I've been 
> > > > contributing to
> > > > the python bindings quite a bit in the last couple of months. 
> > > > The
> > > > bindings went from depending on cmake for tests to having their 
> > > > own,
> > > > still dependant on proton-test, test suite that is self
> > > > -maintained.
> > > > 
> > > > Furthermore, the python bindings went from depending on cmake 
> > > > to be
> > > > built to being self-built using python's internal build system. 
> > > > It's
> > > > still possible - and in fact, it still happens - to build these
> > > > drivers running cmake, although it's, in my personal opinion, 
> > > > not
> > > > recommended.
> > > > 
> > > > As you can see from the above, the python bindings depend on 
> > > > proton-c
> > > > but they don't need to live in the same codebase. Therefore, 
> > > > I've
> > > > prepared a small POC to show case this repo structure:
> > > > 
> > > > - https://github.com/FlaPer87/test-python-qpid-proton
> > > > 
> > > > The above repo contains the python-qpid-proton code. As you can 
> > > > tell
> > > > from the commit history, not many changes were required to set 
> > > > it up.
> > > > The relevant bits of this repo, I believe, are the ones related 
> > > > to the
> > > > submodule. The qpid-proton submodule points qpid-proton's 
> > > > master
> > > > branch in the test-python-qpid-proton branch, whereas in the 
> > > > 0.10
> > > > branch, it tracks qpid-proton's 0.10.x. This is important for 
> > > > the
> > > > bindings to keep track of the right code/release/tests.
> > > > 
> > > > - https://github.com/FlaPer87/qpid-proton/tree/submodules
> > > > 
> > > > The above is a fork of the qpid-proton repo we use already but 
> > > > rather
> > > > than having the python-qpid-bindings in tree, it just tracks 
> > > > the
> > > > master branch. (I created a `submodules` branch because I 
> > > > didn't want
> > > > to mess with the master branch in my fork).
> > > > 
> > > > I don't think tracking the binding in the qpid-proton repo is
> > > > necessary but I see how it's useful to avoid introducing 
> > > > breaking
> > > > changes on drivers. To be more precise, I don't think the above 
> > > > is
> > > > necessary because if we are testing bindings in cmake to make 
> > > > sure we
> > > > don't break backwards compatibility, we could simply install 
> > > > the
> > > > latest stable version and test against that, rather than 
> > > > testing
> > > > against the latest binding's master, which could, but 
> > > > shouldn't, be
> > > > broken.
> > > > 
> > > > I wouldn't probably advocate for the above to applied to every 
> > > > binding
> > > > we have since some of them might not be ready for this but I
> > > > definitely want to advocate to make this happen for the python 
> > > > binding
> > > > (and other bindings).
> > > > 
> > > > This structure will make it easier to mainting bindings, to 
> > > > build a
> > > > community around these bindings - I contribute more to the 
> > > > python
> > > > binding than proton-c. It'll also help keeping a clean history 
> > > > for
> > > > each project and to issue binding releases, therefore creating
> > > > tags for bindings, as needed.
> > > > 
> > > > I get it's worked well so far and probably some of you won't be 
> > > > happy
> > > > with this proposal but, git is powerful enough to support the 
> > > > above
> > > > structure, which is ideal for these kind of projects. Correct 
> > > > me if
> > > > I'm wrong but I believe the current structure also comes from 
> > > > the old
> > > > times when qpid-proton used to be in subversion.
> > > > 
> > > > Hope the above makes sense and I'm sorry for such a long email, 
> > > > I
> > > > should've drunk less coffee.
> > > > Flavio
> > > > 
> > > > P.S: I'm more than happy to help setting this up and to bring 
> > > > it back
> > > > if the experiment fails.
> > > > 
> > > > --
> > > > @flaper87
> > > > Flavio Percoco
> > 
> > 
> > --
> > @flaper87
> > Flavio Percoco

Re: A case in favor of separate repos for language bindings

[Robbie Gemmell <ro...@gmail.com>]

On 19 August 2015 at 13:05, Flavio Percoco <fl...@redhat.com> wrote:
> On 19/08/15 12:34 +0100, Robbie Gemmell wrote:
>>
>> I can see certain benefits to such a separation, mainly for folks
>> interested only in the bindings, but if I'm honest I'm not sure those
>> outweigh the additional complication it seems it may bring in some of
>> the other areas.
>
>
> I think they actually do but probably because I don't see any
> complications being added to proton-c itself but rather a
> simplification of the current structure.

Ken has covered much of what I was thinking about.

>  It'll bring a more intuitive
> structure to the project rather than having the bindings under
> `proton-c`, which is very confusing :)
>

I can see having them elsewhere might be more intuitive, especially
for folks who dont know or care that they are using proton-c under the
covers while using them.

I guess if they were kept in-tree they could be moved at to the top
level to make them more obvious but then presumably the same might
need done for the others, and that still wouldn't make a difference to
the various other points in your suggestion.

>> The python bindings are slightly more interesting than the others due
>> to being at the heart of the python based tests that exist for
>> proton-c and proton-j, so splitting them out actually creates a bit of
>> a circular dependency with proton / its tests (especially when having
>> the submodules in both repos). You mentioned thinking we wouldn't need
>> to track the binding within the proton tree (but could do so to
>> prevent a breaking change for existing things), and instead use an
>> installed version of the bindings. In practice I guess it would be
>> about the same in terms of updating proton-c and the bindings and the
>> related python-based tests at the same time (adding an install, or
>> possibly two, in the middle).
>
>
> It won't be the same.

I just meant in terms of which files people would need to update it
would be similar, but with some install steps added (depending on what
you change), not any effect on what is or isnt being tested.

>The reason is that we're currently ensuring that
> proton-c doesn't break the bindings 'master' version. This is being
> done on the assumption that bindings will be released along with
> proton-c and there'll be a binding release per every proton-c minor.
>
> The above is not an ideal scenarion in my opinion. Ideally, we'd have
> a release of the bindings that will be able to track several minor
> releases of proton-c, but this will probably come later once proton-c
> stabilizes a bit more. However, I believe the current structure allows
> for breaking changes to land because it tests things against master
> only rather than making sure older, still maintained, versions are not
> being broken.
>
> Going back to the proposal, I think the circular dependency is not
> really necessary and we could just have the python bindings being
> installed. The installed version can either be the current master
> branch (pip knows how to install from git) or the latest stable
> version. Furthermore, we could even have a list of versions that we
> want to test against.
>
> To expand the above issue a bit more, I think it's wrong to have the
> python tests as part of the in-tree tests for proton-c. Ideally, this
> should be the work of a CI system responsible for doing integration
> tests. Not saying we should solve this issue now but the proposed
> structure allows for it to happen.
>
>> Releasing (in terms of source, here at
>> Apache) seems like it would become more 'interesting' though, with
>> potential again for the bindings needing proton-c (+proton-j) updates,
>> but proton-c (+proton-j) needing the bindings updates for updated
>> tests. I guess that might point to splitting the python tests out on
>> their own as well so it depended on both separately.
>
>
> What python tests are you referring to? The ones under `tests/python`
> ?
>
> Those are the only ones I know of and those are still in the
> `proton-c` repo.

Those are the ones. As Ken has largely covered, its important to note
that those exist mostly (entirely?) to test proton-c and proton-j
rather than the python bindings themselves (though given how they
work, they kinda do that implicitly too). Their need of the python
bindings (and in turn proton again) to work however would present
challenges as Ken mentioned.

> The python bindings access those through the
> submodule for now but yes, I'd agree that eventually, those tests
> could even be on their own.
>
>> Perhaps I'm overthinking this though; how would you see some of those
>> things working?
>
>
> Totally honest? I think Java should have its own tests and let python
> alone. Maintaining python2.5 - because that's what the tests support -
> plus all the self-written tests libraries and whatnot is a burden that
> I think we should not carry. Most of the things being maintained under
> the `tests/python` 'package' exists already in the python community
> and are widely used and tests. I know it seems easier to maintain 1 test
> suite for 2 bindings rather than 2 but it really isn't. Not in this
> case, at the very least.
>

I have never really liked the python tests, they are a pain, but they
are for the most part all that exists. Replacing them in each language
at this point doesnt seem like great fun either though :)

I'm not sure it affects things too much with regards to splitting
things up, but the tests only use 2.5 still because noone has updated
the jython used to 2.7 yet, that presumably could be done if its
causing an issue.

> However, based on the current situation, I'd either have the test code
> on its own or simply follow the proposed structure and have the
> bindings following proton-c and consuming tests from there.
>
>> I also can't say that I'd be a particular fan of using submodules,
>> most of the things I've read about or tried with them have been a
>> turnoff...but I'm mostly skipping over that for now.
>
>
> In my experience, if done right, they work and they're not hard to
> update/maintain. Circular dependencies are to avoid, agreed. One case
> where I've had good experience with submodules is the rustlang. I
> guess giving this a try would tell better.
>
> Thanks for your reply,
> Flavio
>
>
>>
>> Robbie
>>
>> On 18 August 2015 at 08:17, Flavio Percoco <fl...@redhat.com> wrote:
>>>
>>> Greetings,
>>>
>>> I'd like to take a chance, now that 0.10 is out, to make a point in
>>> favor of using submodules rather than having all bindings in the same
>>> repository.
>>>
>>> I think qpid-proton has reached the point where it's mature enough to
>>> have some of the bindings promoted to having their own repos. More
>>> importantly, I believe these bindings are mature enough and ready to
>>> build a community on their own.
>>>
>>> Lets take the python bindings, for instance. I've been contributing to
>>> the python bindings quite a bit in the last couple of months. The
>>> bindings went from depending on cmake for tests to having their own,
>>> still dependant on proton-test, test suite that is self-maintained.
>>>
>>> Furthermore, the python bindings went from depending on cmake to be
>>> built to being self-built using python's internal build system. It's
>>> still possible - and in fact, it still happens - to build these
>>> drivers running cmake, although it's, in my personal opinion, not
>>> recommended.
>>>
>>> As you can see from the above, the python bindings depend on proton-c
>>> but they don't need to live in the same codebase. Therefore, I've
>>> prepared a small POC to show case this repo structure:
>>>
>>> - https://github.com/FlaPer87/test-python-qpid-proton
>>>
>>> The above repo contains the python-qpid-proton code. As you can tell
>>> from the commit history, not many changes were required to set it up.
>>> The relevant bits of this repo, I believe, are the ones related to the
>>> submodule. The qpid-proton submodule points qpid-proton's master
>>> branch in the test-python-qpid-proton branch, whereas in the 0.10
>>> branch, it tracks qpid-proton's 0.10.x. This is important for the
>>> bindings to keep track of the right code/release/tests.
>>>
>>> - https://github.com/FlaPer87/qpid-proton/tree/submodules
>>>
>>> The above is a fork of the qpid-proton repo we use already but rather
>>> than having the python-qpid-bindings in tree, it just tracks the
>>> master branch. (I created a `submodules` branch because I didn't want
>>> to mess with the master branch in my fork).
>>>
>>> I don't think tracking the binding in the qpid-proton repo is
>>> necessary but I see how it's useful to avoid introducing breaking
>>> changes on drivers. To be more precise, I don't think the above is
>>> necessary because if we are testing bindings in cmake to make sure we
>>> don't break backwards compatibility, we could simply install the
>>> latest stable version and test against that, rather than testing
>>> against the latest binding's master, which could, but shouldn't, be
>>> broken.
>>>
>>> I wouldn't probably advocate for the above to applied to every binding
>>> we have since some of them might not be ready for this but I
>>> definitely want to advocate to make this happen for the python binding
>>> (and other bindings).
>>>
>>> This structure will make it easier to mainting bindings, to build a
>>> community around these bindings - I contribute more to the python
>>> binding than proton-c. It'll also help keeping a clean history for
>>> each project and to issue binding releases, therefore creating
>>> tags for bindings, as needed.
>>>
>>> I get it's worked well so far and probably some of you won't be happy
>>> with this proposal but, git is powerful enough to support the above
>>> structure, which is ideal for these kind of projects. Correct me if
>>> I'm wrong but I believe the current structure also comes from the old
>>> times when qpid-proton used to be in subversion.
>>>
>>> Hope the above makes sense and I'm sorry for such a long email, I
>>> should've drunk less coffee.
>>> Flavio
>>>
>>> P.S: I'm more than happy to help setting this up and to bring it back
>>> if the experiment fails.
>>>
>>> --
>>> @flaper87
>>> Flavio Percoco
>
>
> --
> @flaper87
> Flavio Percoco

Re: A case in favor of separate repos for language bindings

[Ken Giusti <kg...@redhat.com>]

While I'm in favor of breaking out the bindings (actually, I'd like to see wider changes to the overall structure of proton, specifically proton-c.  But I won't thread-jack - I'll start a separate thread).

However - I think the python binding is the one binding where it may not be practical, sadly.

See inline:

----- Original Message -----
> From: "Flavio Percoco" <fl...@redhat.com>
> To: proton@qpid.apache.org
> Sent: Wednesday, August 19, 2015 8:05:10 AM
> Subject: Re: A case in favor of separate repos for language bindings
> 
> On 19/08/15 12:34 +0100, Robbie Gemmell wrote:
> >I can see certain benefits to such a separation, mainly for folks
> >interested only in the bindings, but if I'm honest I'm not sure those
> >outweigh the additional complication it seems it may bring in some of
> >the other areas.
> 
> I think they actually do but probably because I don't see any
> complications being added to proton-c itself but rather a
> simplification of the current structure. It'll bring a more intuitive
> structure to the project rather than having the bindings under
> `proton-c`, which is very confusing :)
> 
> >The python bindings are slightly more interesting than the others due
> >to being at the heart of the python based tests that exist for
> >proton-c and proton-j, so splitting them out actually creates a bit of
> >a circular dependency with proton / its tests (especially when having
> >the submodules in both repos). You mentioned thinking we wouldn't need
> >to track the binding within the proton tree (but could do so to
> >prevent a breaking change for existing things), and instead use an
> >installed version of the bindings. In practice I guess it would be
> >about the same in terms of updating proton-c and the bindings and the
> >related python-based tests at the same time (adding an install, or
> >possibly two, in the middle).
> 
> It won't be the same. The reason is that we're currently ensuring that
> proton-c doesn't break the bindings 'master' version. This is being
> done on the assumption that bindings will be released along with
> proton-c and there'll be a binding release per every proton-c minor.
> 
> The above is not an ideal scenarion in my opinion. Ideally, we'd have
> a release of the bindings that will be able to track several minor
> releases of proton-c, but this will probably come later once proton-c
> stabilizes a bit more. However, I believe the current structure allows
> for breaking changes to land because it tests things against master
> only rather than making sure older, still maintained, versions are not
> being broken.
> 
> Going back to the proposal, I think the circular dependency is not
> really necessary and we could just have the python bindings being
> installed. The installed version can either be the current master
> branch (pip knows how to install from git) or the latest stable
> version. Furthermore, we could even have a list of versions that we
> want to test against.
> 


I don't see how this would work in the case of proton-c (or proton-j) API changes.   Robbie raises that point, and I don't see how this would work in practice with the bindings being out-of-tree (and the unit tests being implemented in python).

Specifically:
Proton-c adds a new API.  That new API cannot be tested using the python unit tests until the python binding is updated.  So the dev would have to also update the python binding and juggle the two in order to complete testing.

One way to address this: stop developing unit tests in python.  Instead, going forward all new proton-c tests should be done in C and put in the existing proton-c/tests dir.   See my next point:


> To expand the above issue a bit more, I think it's wrong to have the
> python tests as part of the in-tree tests for proton-c. 


I agree with this point, though I suspect we won't do it due to inertia alone.

I think each binding and language implementation (Java/C) should have their own in tree unit tests written in the implementation language.  Each binding should have just enough tests to test the binding itself - not the underlying C code.  The C code should have its own unit tests, written in C.

Doing the above would simplify and, in the case of python, speed up the unit test runs.  Currently, the python unit tests may be repeated several times: once under cmake, and, if tox is installed, once for each version of python available on the system.  That can be fixed by configuring tox to just run a subset of the unit tests, but since the unit tests are written to exercise more than just the python interface we'd have to be careful selecting the proper subset for python.

But... well.. inertia.


> Ideally, this
> should be the work of a CI system responsible for doing integration
> tests. Not saying we should solve this issue now but the proposed
> structure allows for it to happen.
> 

Nope.  There has to be an in-tree set of unit tests that devs run (and pass) before any modifications are pushed.  Full stop.  

Good or bad, this project doesn't have the CI infrastructure that Openstack does - using gerrit and running each proposed patch under CI *before* the patch can be committed isn't our workflow.  Maybe that should change, but I'm not sure I'd want that workflow here in QPID.

That said, the project _should_ have a set of CI tests for interoperability across the various bindings and Java/C.  But simple unit tests should be available to the dev locally, IMHO.



> > Releasing (in terms of source, here at
> >Apache) seems like it would become more 'interesting' though, with
> >potential again for the bindings needing proton-c (+proton-j) updates,
> >but proton-c (+proton-j) needing the bindings updates for updated
> >tests. I guess that might point to splitting the python tests out on
> >their own as well so it depended on both separately.
> 
> What python tests are you referring to? The ones under `tests/python`
> ?
> 
> Those are the only ones I know of and those are still in the
> `proton-c` repo. The python bindings access those through the
> submodule for now but yes, I'd agree that eventually, those tests
> could even be on their own.
> 

The proton-c/tests unit tests are C based unit tests that apply only to proton.c.  There's no python there.  It's the top level tests/python that concern me.


> >Perhaps I'm overthinking this though; how would you see some of those
> >things working?
> 
> Totally honest? I think Java should have its own tests and let python
> alone. Maintaining python2.5 - because that's what the tests support -
> plus all the self-written tests libraries and whatnot is a burden that
> I think we should not carry. Most of the things being maintained under
> the `tests/python` 'package' exists already in the python community
> and are widely used and tests. I know it seems easier to maintain 1 test
> suite for 2 bindings rather than 2 but it really isn't. Not in this
> case, at the very least.
> 
> However, based on the current situation, I'd either have the test code
> on its own or simply follow the proposed structure and have the
> bindings following proton-c and consuming tests from there.
> 
> >I also can't say that I'd be a particular fan of using submodules,
> >most of the things I've read about or tried with them have been a
> >turnoff...but I'm mostly skipping over that for now.
> 
> In my experience, if done right, they work and they're not hard to
> update/maintain. Circular dependencies are to avoid, agreed. One case
> where I've had good experience with submodules is the rustlang. I
> guess giving this a try would tell better.
> 
> Thanks for your reply,
> Flavio
> 
> >
> >Robbie
> >
> >On 18 August 2015 at 08:17, Flavio Percoco <fl...@redhat.com> wrote:
> >> Greetings,
> >>
> >> I'd like to take a chance, now that 0.10 is out, to make a point in
> >> favor of using submodules rather than having all bindings in the same
> >> repository.
> >>
> >> I think qpid-proton has reached the point where it's mature enough to
> >> have some of the bindings promoted to having their own repos. More
> >> importantly, I believe these bindings are mature enough and ready to
> >> build a community on their own.
> >>
> >> Lets take the python bindings, for instance. I've been contributing to
> >> the python bindings quite a bit in the last couple of months. The
> >> bindings went from depending on cmake for tests to having their own,
> >> still dependant on proton-test, test suite that is self-maintained.
> >>
> >> Furthermore, the python bindings went from depending on cmake to be
> >> built to being self-built using python's internal build system. It's
> >> still possible - and in fact, it still happens - to build these
> >> drivers running cmake, although it's, in my personal opinion, not
> >> recommended.
> >>
> >> As you can see from the above, the python bindings depend on proton-c
> >> but they don't need to live in the same codebase. Therefore, I've
> >> prepared a small POC to show case this repo structure:
> >>
> >> - https://github.com/FlaPer87/test-python-qpid-proton
> >>
> >> The above repo contains the python-qpid-proton code. As you can tell
> >> from the commit history, not many changes were required to set it up.
> >> The relevant bits of this repo, I believe, are the ones related to the
> >> submodule. The qpid-proton submodule points qpid-proton's master
> >> branch in the test-python-qpid-proton branch, whereas in the 0.10
> >> branch, it tracks qpid-proton's 0.10.x. This is important for the
> >> bindings to keep track of the right code/release/tests.
> >>
> >> - https://github.com/FlaPer87/qpid-proton/tree/submodules
> >>
> >> The above is a fork of the qpid-proton repo we use already but rather
> >> than having the python-qpid-bindings in tree, it just tracks the
> >> master branch. (I created a `submodules` branch because I didn't want
> >> to mess with the master branch in my fork).
> >>
> >> I don't think tracking the binding in the qpid-proton repo is
> >> necessary but I see how it's useful to avoid introducing breaking
> >> changes on drivers. To be more precise, I don't think the above is
> >> necessary because if we are testing bindings in cmake to make sure we
> >> don't break backwards compatibility, we could simply install the
> >> latest stable version and test against that, rather than testing
> >> against the latest binding's master, which could, but shouldn't, be
> >> broken.
> >>
> >> I wouldn't probably advocate for the above to applied to every binding
> >> we have since some of them might not be ready for this but I
> >> definitely want to advocate to make this happen for the python binding
> >> (and other bindings).
> >>
> >> This structure will make it easier to mainting bindings, to build a
> >> community around these bindings - I contribute more to the python
> >> binding than proton-c. It'll also help keeping a clean history for
> >> each project and to issue binding releases, therefore creating
> >> tags for bindings, as needed.
> >>
> >> I get it's worked well so far and probably some of you won't be happy
> >> with this proposal but, git is powerful enough to support the above
> >> structure, which is ideal for these kind of projects. Correct me if
> >> I'm wrong but I believe the current structure also comes from the old
> >> times when qpid-proton used to be in subversion.
> >>
> >> Hope the above makes sense and I'm sorry for such a long email, I
> >> should've drunk less coffee.
> >> Flavio
> >>
> >> P.S: I'm more than happy to help setting this up and to bring it back
> >> if the experiment fails.
> >>
> >> --
> >> @flaper87
> >> Flavio Percoco
> 
> --
> @flaper87
> Flavio Percoco
> 

-- 
-K

Re: A case in favor of separate repos for language bindings

[Flavio Percoco <fl...@redhat.com>]

On 19/08/15 12:34 +0100, Robbie Gemmell wrote:
>I can see certain benefits to such a separation, mainly for folks
>interested only in the bindings, but if I'm honest I'm not sure those
>outweigh the additional complication it seems it may bring in some of
>the other areas.

I think they actually do but probably because I don't see any
complications being added to proton-c itself but rather a
simplification of the current structure. It'll bring a more intuitive
structure to the project rather than having the bindings under
`proton-c`, which is very confusing :)

>The python bindings are slightly more interesting than the others due
>to being at the heart of the python based tests that exist for
>proton-c and proton-j, so splitting them out actually creates a bit of
>a circular dependency with proton / its tests (especially when having
>the submodules in both repos). You mentioned thinking we wouldn't need
>to track the binding within the proton tree (but could do so to
>prevent a breaking change for existing things), and instead use an
>installed version of the bindings. In practice I guess it would be
>about the same in terms of updating proton-c and the bindings and the
>related python-based tests at the same time (adding an install, or
>possibly two, in the middle).

It won't be the same. The reason is that we're currently ensuring that
proton-c doesn't break the bindings 'master' version. This is being
done on the assumption that bindings will be released along with
proton-c and there'll be a binding release per every proton-c minor.

The above is not an ideal scenarion in my opinion. Ideally, we'd have
a release of the bindings that will be able to track several minor
releases of proton-c, but this will probably come later once proton-c
stabilizes a bit more. However, I believe the current structure allows
for breaking changes to land because it tests things against master
only rather than making sure older, still maintained, versions are not
being broken.

Going back to the proposal, I think the circular dependency is not
really necessary and we could just have the python bindings being
installed. The installed version can either be the current master
branch (pip knows how to install from git) or the latest stable
version. Furthermore, we could even have a list of versions that we
want to test against.

To expand the above issue a bit more, I think it's wrong to have the
python tests as part of the in-tree tests for proton-c. Ideally, this
should be the work of a CI system responsible for doing integration
tests. Not saying we should solve this issue now but the proposed
structure allows for it to happen.

> Releasing (in terms of source, here at
>Apache) seems like it would become more 'interesting' though, with
>potential again for the bindings needing proton-c (+proton-j) updates,
>but proton-c (+proton-j) needing the bindings updates for updated
>tests. I guess that might point to splitting the python tests out on
>their own as well so it depended on both separately.

What python tests are you referring to? The ones under `tests/python`
?

Those are the only ones I know of and those are still in the
`proton-c` repo. The python bindings access those through the
submodule for now but yes, I'd agree that eventually, those tests
could even be on their own.

>Perhaps I'm overthinking this though; how would you see some of those
>things working?

Totally honest? I think Java should have its own tests and let python
alone. Maintaining python2.5 - because that's what the tests support -
plus all the self-written tests libraries and whatnot is a burden that
I think we should not carry. Most of the things being maintained under
the `tests/python` 'package' exists already in the python community
and are widely used and tests. I know it seems easier to maintain 1 test
suite for 2 bindings rather than 2 but it really isn't. Not in this
case, at the very least.

However, based on the current situation, I'd either have the test code
on its own or simply follow the proposed structure and have the
bindings following proton-c and consuming tests from there.

>I also can't say that I'd be a particular fan of using submodules,
>most of the things I've read about or tried with them have been a
>turnoff...but I'm mostly skipping over that for now.

In my experience, if done right, they work and they're not hard to
update/maintain. Circular dependencies are to avoid, agreed. One case
where I've had good experience with submodules is the rustlang. I
guess giving this a try would tell better.

Thanks for your reply,
Flavio

>
>Robbie
>
>On 18 August 2015 at 08:17, Flavio Percoco <fl...@redhat.com> wrote:
>> Greetings,
>>
>> I'd like to take a chance, now that 0.10 is out, to make a point in
>> favor of using submodules rather than having all bindings in the same
>> repository.
>>
>> I think qpid-proton has reached the point where it's mature enough to
>> have some of the bindings promoted to having their own repos. More
>> importantly, I believe these bindings are mature enough and ready to
>> build a community on their own.
>>
>> Lets take the python bindings, for instance. I've been contributing to
>> the python bindings quite a bit in the last couple of months. The
>> bindings went from depending on cmake for tests to having their own,
>> still dependant on proton-test, test suite that is self-maintained.
>>
>> Furthermore, the python bindings went from depending on cmake to be
>> built to being self-built using python's internal build system. It's
>> still possible - and in fact, it still happens - to build these
>> drivers running cmake, although it's, in my personal opinion, not
>> recommended.
>>
>> As you can see from the above, the python bindings depend on proton-c
>> but they don't need to live in the same codebase. Therefore, I've
>> prepared a small POC to show case this repo structure:
>>
>> - https://github.com/FlaPer87/test-python-qpid-proton
>>
>> The above repo contains the python-qpid-proton code. As you can tell
>> from the commit history, not many changes were required to set it up.
>> The relevant bits of this repo, I believe, are the ones related to the
>> submodule. The qpid-proton submodule points qpid-proton's master
>> branch in the test-python-qpid-proton branch, whereas in the 0.10
>> branch, it tracks qpid-proton's 0.10.x. This is important for the
>> bindings to keep track of the right code/release/tests.
>>
>> - https://github.com/FlaPer87/qpid-proton/tree/submodules
>>
>> The above is a fork of the qpid-proton repo we use already but rather
>> than having the python-qpid-bindings in tree, it just tracks the
>> master branch. (I created a `submodules` branch because I didn't want
>> to mess with the master branch in my fork).
>>
>> I don't think tracking the binding in the qpid-proton repo is
>> necessary but I see how it's useful to avoid introducing breaking
>> changes on drivers. To be more precise, I don't think the above is
>> necessary because if we are testing bindings in cmake to make sure we
>> don't break backwards compatibility, we could simply install the
>> latest stable version and test against that, rather than testing
>> against the latest binding's master, which could, but shouldn't, be
>> broken.
>>
>> I wouldn't probably advocate for the above to applied to every binding
>> we have since some of them might not be ready for this but I
>> definitely want to advocate to make this happen for the python binding
>> (and other bindings).
>>
>> This structure will make it easier to mainting bindings, to build a
>> community around these bindings - I contribute more to the python
>> binding than proton-c. It'll also help keeping a clean history for
>> each project and to issue binding releases, therefore creating
>> tags for bindings, as needed.
>>
>> I get it's worked well so far and probably some of you won't be happy
>> with this proposal but, git is powerful enough to support the above
>> structure, which is ideal for these kind of projects. Correct me if
>> I'm wrong but I believe the current structure also comes from the old
>> times when qpid-proton used to be in subversion.
>>
>> Hope the above makes sense and I'm sorry for such a long email, I
>> should've drunk less coffee.
>> Flavio
>>
>> P.S: I'm more than happy to help setting this up and to bring it back
>> if the experiment fails.
>>
>> --
>> @flaper87
>> Flavio Percoco

-- 
@flaper87
Flavio Percoco

Re: A case in favor of separate repos for language bindings

[Robbie Gemmell <ro...@gmail.com>]

I can see certain benefits to such a separation, mainly for folks
interested only in the bindings, but if I'm honest I'm not sure those
outweigh the additional complication it seems it may bring in some of
the other areas.

The python bindings are slightly more interesting than the others due
to being at the heart of the python based tests that exist for
proton-c and proton-j, so splitting them out actually creates a bit of
a circular dependency with proton / its tests (especially when having
the submodules in both repos). You mentioned thinking we wouldn't need
to track the binding within the proton tree (but could do so to
prevent a breaking change for existing things), and instead use an
installed version of the bindings. In practice I guess it would be
about the same in terms of updating proton-c and the bindings and the
related python-based tests at the same time (adding an install, or
possibly two, in the middle). Releasing (in terms of source, here at
Apache) seems like it would become more 'interesting' though, with
potential again for the bindings needing proton-c (+proton-j) updates,
but proton-c (+proton-j) needing the bindings updates for updated
tests. I guess that might point to splitting the python tests out on
their own as well so it depended on both separately.

Perhaps I'm overthinking this though; how would you see some of those
things working?

I also can't say that I'd be a particular fan of using submodules,
most of the things I've read about or tried with them have been a
turnoff...but I'm mostly skipping over that for now.

Robbie

On 18 August 2015 at 08:17, Flavio Percoco <fl...@redhat.com> wrote:
> Greetings,
>
> I'd like to take a chance, now that 0.10 is out, to make a point in
> favor of using submodules rather than having all bindings in the same
> repository.
>
> I think qpid-proton has reached the point where it's mature enough to
> have some of the bindings promoted to having their own repos. More
> importantly, I believe these bindings are mature enough and ready to
> build a community on their own.
>
> Lets take the python bindings, for instance. I've been contributing to
> the python bindings quite a bit in the last couple of months. The
> bindings went from depending on cmake for tests to having their own,
> still dependant on proton-test, test suite that is self-maintained.
>
> Furthermore, the python bindings went from depending on cmake to be
> built to being self-built using python's internal build system. It's
> still possible - and in fact, it still happens - to build these
> drivers running cmake, although it's, in my personal opinion, not
> recommended.
>
> As you can see from the above, the python bindings depend on proton-c
> but they don't need to live in the same codebase. Therefore, I've
> prepared a small POC to show case this repo structure:
>
> - https://github.com/FlaPer87/test-python-qpid-proton
>
> The above repo contains the python-qpid-proton code. As you can tell
> from the commit history, not many changes were required to set it up.
> The relevant bits of this repo, I believe, are the ones related to the
> submodule. The qpid-proton submodule points qpid-proton's master
> branch in the test-python-qpid-proton branch, whereas in the 0.10
> branch, it tracks qpid-proton's 0.10.x. This is important for the
> bindings to keep track of the right code/release/tests.
>
> - https://github.com/FlaPer87/qpid-proton/tree/submodules
>
> The above is a fork of the qpid-proton repo we use already but rather
> than having the python-qpid-bindings in tree, it just tracks the
> master branch. (I created a `submodules` branch because I didn't want
> to mess with the master branch in my fork).
>
> I don't think tracking the binding in the qpid-proton repo is
> necessary but I see how it's useful to avoid introducing breaking
> changes on drivers. To be more precise, I don't think the above is
> necessary because if we are testing bindings in cmake to make sure we
> don't break backwards compatibility, we could simply install the
> latest stable version and test against that, rather than testing
> against the latest binding's master, which could, but shouldn't, be
> broken.
>
> I wouldn't probably advocate for the above to applied to every binding
> we have since some of them might not be ready for this but I
> definitely want to advocate to make this happen for the python binding
> (and other bindings).
>
> This structure will make it easier to mainting bindings, to build a
> community around these bindings - I contribute more to the python
> binding than proton-c. It'll also help keeping a clean history for
> each project and to issue binding releases, therefore creating
> tags for bindings, as needed.
>
> I get it's worked well so far and probably some of you won't be happy
> with this proposal but, git is powerful enough to support the above
> structure, which is ideal for these kind of projects. Correct me if
> I'm wrong but I believe the current structure also comes from the old
> times when qpid-proton used to be in subversion.
>
> Hope the above makes sense and I'm sorry for such a long email, I
> should've drunk less coffee.
> Flavio
>
> P.S: I'm more than happy to help setting this up and to bring it back
> if the experiment fails.
>
> --
> @flaper87
> Flavio Percoco