You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-dev@jackrabbit.apache.org by Michael Dürig <md...@apache.org> on 2015/12/07 18:24:08 UTC
Semantic version in Oak
Hi,
We started to annotate packages with their export versions (aka.
semantic versioning [1]) a while ago. After the 1.2 release we enforced
proper API management through the baseline functionality of the Maven
bundle plugin. This means that on current trunk the build fails should
an API change but the related package export version is not reflecting
this.
So far this approach worked relatively well. Going forward however I
presume things to get more complicated. Once Oak 1.4 is out we are going
to have to maintain a 1.4 branch while continuing development on trunk
(i.e. 1.5). With this we suddenly will have two potentially diverging
branches, which begs the question on how to do release from these
branches while at the same time maintaining consistent semantic versions
of the contained packages across the branches?
Michael
[1] http://www.osgi.org/wp-content/uploads/SemanticVersioning1.pdf
[2] https://issues.apache.org/jira/browse/OAK-2006
Re: Semantic version in Oak
Posted by Felix Meschberger <fm...@adobe.com>.
Hi
> Am 08.12.2015 um 09:49 schrieb Thomas Mueller <mu...@adobe.com>:
>
> Hi,
>
> I think the main difference between Oak and Sling is, AFAIK, that Sling is
> "forward only", and does not maintain branches, and does not backport
> things.
Just to be clear, this is not about Sling vs. Oak.
This is whether Oak can support semantic versioning of its exported API. If you find, that Oak cannot do it. Fine, then don’t do it.
As a consumer I will have to live with uncertainties. But that is still better than being misleaded.
>
> In Oak, we add new features in trunk (changing the API), and backport some
> of those features, and not necessarily all of them, and not necessarily in
> the same order as they were implemented:
>
> == Trunk ==
>
> add feature A => bump export version to 1.1
> ... later on ...
> add feature B => bump export version to 2.0
> ... later on ...
> add feature C => bump export version to 2.1
>
>
> == Branch ==
>
> backport feature C => bump export version to ?
to get C you also need A and B and hence would be at 2.1
* you cannot do 1.1 because this means feature A but neither B nor C
* you cannot just do 2.1 (or even 2.2) because you are omitting both
A and B and thus consumers of 2.1 (or 2.2) expecting A or B besides
C would break
* you cannot do 3.x because this would later break migration to
trunk which is 2.1
> ... later on ...
> backport feature A => bump export version to ?
Since you already have C you already have A (and B) and hence are at 2.1 already.
Please note, though: This is *not* about an implementation detail. This is about something visible like a new public (or protected) method or a class/interface.
Semantic versioning is about resonating about something and agreeing on what level of compatibility I as a consumer can expect from the provider/producer - consider it being the part of an API contract dealing with evolution. It comes with a price for the provider/producer. If the provider is not able to abide by this contract, I suggest not to enter it.
Regards
Felix
>
>
> Regards,
> Thomas
>
>
>
> On 08/12/15 09:41, "Michael Dürig" <md...@apache.org> wrote:
>
>>
>>>> Packages evolve independently, but they do in potentially
>>>> divergent branches. This is the kind of timeline that we usually
>>>> face:
>>>>
>>>> - Oak 1.4 has a package org.foo.bar 1.0 - Some changes happen on
>>>> the development branch 1.5 - Oak 1.5 now has a package org.foo.bar
>>>> 1.1 - A change X happen in the development branch 1.5 - Oak 1.5 now
>>>> has a package org.foo.bar 1.2 - The change X has to be backported
>>>> to the maintenance branch 1.4 - Oak 1.4 now should have a package
>>>> org.foo.bar 1.1
>>>>
>>>> Assuming that the versions were incremented following the semantic
>>>> versioning rules, we now have two packages - both called
>>>> org.foo.bar and both having version 1.1 - that live on two
>>>> different branches and contain different code.
>>>>
>>>> The only obvious solution that comes to my mind is to bump the
>>>> major version of every package right after the development branch
>>>> 1.5 is started, but I don't like this approach very much because it
>>>> would break compatibility with existing clients for no obvious
>>>> reason.
>>>
>>> This scenario is the exact problem you are facing while branching and
>>> evolving the branches in parallel to trunk.
>>>
>>> The only end-developer friendly solution is to byte the bullet and do
>>> it really properly and make sure you evolve exported packages (being
>>> your API) in a truly diligent matter: Consider a package name and its
>>> export version as the package¹s identity and always make sure this
>>> identity (label) refers to the identical exported API.
>>>
>>
>> I fail to see how this would work with branches. For Francesco's example
>> this would mean that we'd need to backport everything into the branch
>> effectively aligning it with trunk and thus obviating its purpose.
>>
>> Michael
>>
>>
>>
>
Re: Semantic version in Oak
Posted by Felix Meschberger <fm...@adobe.com>.
Hi
> Am 08.12.2015 um 12:38 schrieb Michael Dürig <md...@apache.org>:
>
>
>>
>> The development model of Oak is currently setup in a way which makes it
>> impossible to do semantic versioning of the API. The main reason is that
>> the branches are not just maintenance branches but also features are
>> backported to the branches. And the way this is done also means that the
>> API on such a branch could be "different" to the main branch and evolve
>> differently. Which in consequence means it's not compatible anymore
>> (things added to the branch might evolve differently on the main branch
>> - this might be more theory than practice, but it can happen).
>
> Currently this happens all the time and I fear there is little we can do about it for the time being.
>
>> Therefore, with the current way of doing development in Oak, semantic
>> versioning is not possible. Period.
>
> Right, this is also my impression.
>
>>
>> You have two options: provide meaning to the API versions (which means
>> changing the way development is done in general), or stick to the
>> current way and clearly state that there is no semantic versioning.
>
> My question would be, how can we get there incrementally? There is no way for us to do a big bang refactoring as there are too much dependencies already.
> One idea that crossed my mind was to remove semantic versioning for the stuff we only export for consumption by ourselves and which we optimally would want to hide from the rest of the world anyway. These are the parts that see most changes. The "real public APIs" are much more stable and we should be able do proper semantic versioning for those. Could such an approach work?
Well, in this case, I really suggest to omit version numbers completely since you cannot resonate about versions. As a customer I would not be happy, but that would be the best you as a provider can do.
API comes in two flavours: Proper end-developer APIs like the JCR API and the Jackrabbit API. I think this is under control, not the least by having them in separate projects. The other flavour of API is what you might call SPI. This is API that extension developers would leverage to build such things as data stores or indexers. This is AFAICT the problematic one. Maybe you can solve this problem incrementally by starting to create separate, and clean SPI packages which you properly version and then start to deprecate the old API.
Regards
Felix
>
> Michael
>
>>
>> Everything else would be a fake compromise which would be totally
>> confusing and misleading.
>>
>> Again, I'm not arguing for one or the other - just stating the options.
>>
>> Carsten
>>
Re: Semantic version in Oak
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi,
On Tue, Dec 8, 2015 at 12:38 PM, Michael Dürig <md...@apache.org> wrote:
> ...One idea that crossed my mind was to remove semantic versioning for the
> stuff we only export for consumption by ourselves...
>...The "real public APIs" are much more stable and we
> should be able do proper semantic versioning for those....
I suppose moving those "real public APIs" to a distinct module (or set
of modules) with its own release cycle, and using semantic versioning
for that would be a big help, if that's possible.
-Bertrand
Re: Semantic version in Oak
Posted by Carsten Ziegeler <cz...@apache.org>.
Michael Dürig wrote
>
>>
>> The development model of Oak is currently setup in a way which makes it
>> impossible to do semantic versioning of the API. The main reason is that
>> the branches are not just maintenance branches but also features are
>> backported to the branches. And the way this is done also means that the
>> API on such a branch could be "different" to the main branch and evolve
>> differently. Which in consequence means it's not compatible anymore
>> (things added to the branch might evolve differently on the main branch
>> - this might be more theory than practice, but it can happen).
>
> Currently this happens all the time and I fear there is little we can do
> about it for the time being.
>
>> Therefore, with the current way of doing development in Oak, semantic
>> versioning is not possible. Period.
>
> Right, this is also my impression.
>
>>
>> You have two options: provide meaning to the API versions (which means
>> changing the way development is done in general), or stick to the
>> current way and clearly state that there is no semantic versioning.
>
> My question would be, how can we get there incrementally? There is no
> way for us to do a big bang refactoring as there are too much
> dependencies already.
> One idea that crossed my mind was to remove semantic versioning for the
> stuff we only export for consumption by ourselves and which we optimally
> would want to hide from the rest of the world anyway. These are the
> parts that see most changes. The "real public APIs" are much more stable
> and we should be able do proper semantic versioning for those. Could
> such an approach work?
>
I think so. This distinction makes sense to me and it would give at
least the
typical users a stable, semantically versioned API.
Carsten
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: Semantic version in Oak
Posted by Michael Dürig <md...@apache.org>.
>
> The development model of Oak is currently setup in a way which makes it
> impossible to do semantic versioning of the API. The main reason is that
> the branches are not just maintenance branches but also features are
> backported to the branches. And the way this is done also means that the
> API on such a branch could be "different" to the main branch and evolve
> differently. Which in consequence means it's not compatible anymore
> (things added to the branch might evolve differently on the main branch
> - this might be more theory than practice, but it can happen).
Currently this happens all the time and I fear there is little we can do
about it for the time being.
> Therefore, with the current way of doing development in Oak, semantic
> versioning is not possible. Period.
Right, this is also my impression.
>
> You have two options: provide meaning to the API versions (which means
> changing the way development is done in general), or stick to the
> current way and clearly state that there is no semantic versioning.
My question would be, how can we get there incrementally? There is no
way for us to do a big bang refactoring as there are too much
dependencies already.
One idea that crossed my mind was to remove semantic versioning for the
stuff we only export for consumption by ourselves and which we optimally
would want to hide from the rest of the world anyway. These are the
parts that see most changes. The "real public APIs" are much more stable
and we should be able do proper semantic versioning for those. Could
such an approach work?
Michael
>
> Everything else would be a fake compromise which would be totally
> confusing and misleading.
>
> Again, I'm not arguing for one or the other - just stating the options.
>
> Carsten
>
Re: Semantic version in Oak
Posted by Carsten Ziegeler <cz...@apache.org>.
Disclaimer: I'm not judging the way things are done in Oak and I'm not
trying to imply that it's bad or wrong. The following is just an
observation.
The development model of Oak is currently setup in a way which makes it
impossible to do semantic versioning of the API. The main reason is that
the branches are not just maintenance branches but also features are
backported to the branches. And the way this is done also means that the
API on such a branch could be "different" to the main branch and evolve
differently. Which in consequence means it's not compatible anymore
(things added to the branch might evolve differently on the main branch
- this might be more theory than practice, but it can happen).
Therefore, with the current way of doing development in Oak, semantic
versioning is not possible. Period.
You have two options: provide meaning to the API versions (which means
changing the way development is done in general), or stick to the
current way and clearly state that there is no semantic versioning.
Everything else would be a fake compromise which would be totally
confusing and misleading.
Again, I'm not arguing for one or the other - just stating the options.
Carsten
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: Semantic version in Oak
Posted by Michael Dürig <md...@apache.org>.
On 8.12.15 9:49 , Thomas Mueller wrote:
> I think the main difference between Oak and Sling is, AFAIK, that Sling is
> "forward only", and does not maintain branches, and does not backport
> things.
Right. However, even in this world it is not clear to me how you would
deliver fixes for earlier versions: say a customer has package
org.foo.bar at version 1.0.0 deployed. While time goes by org.foo.bar
goes through several iterations and now is at say 3.0.0. Now the
customer runs into a bug which calls for an increase of the microversion
only. In the forward only world this would lead to 3.0.1. However that
version is not compatible to the customers deployment anymore. He would
rather need a 1.0.1. I don't see how this could be achieved without
branching.
Michael
Re: Semantic version in Oak
Posted by Thomas Mueller <mu...@adobe.com>.
Hi,
I think the main difference between Oak and Sling is, AFAIK, that Sling is
"forward only", and does not maintain branches, and does not backport
things.
In Oak, we add new features in trunk (changing the API), and backport some
of those features, and not necessarily all of them, and not necessarily in
the same order as they were implemented:
== Trunk ==
add feature A => bump export version to 1.1
... later on ...
add feature B => bump export version to 2.0
... later on ...
add feature C => bump export version to 2.1
== Branch ==
backport feature C => bump export version to ?
... later on ...
backport feature A => bump export version to ?
Regards,
Thomas
On 08/12/15 09:41, "Michael Dürig" <md...@apache.org> wrote:
>
>>> Packages evolve independently, but they do in potentially
>>> divergent branches. This is the kind of timeline that we usually
>>> face:
>>>
>>> - Oak 1.4 has a package org.foo.bar 1.0 - Some changes happen on
>>> the development branch 1.5 - Oak 1.5 now has a package org.foo.bar
>>> 1.1 - A change X happen in the development branch 1.5 - Oak 1.5 now
>>> has a package org.foo.bar 1.2 - The change X has to be backported
>>> to the maintenance branch 1.4 - Oak 1.4 now should have a package
>>> org.foo.bar 1.1
>>>
>>> Assuming that the versions were incremented following the semantic
>>> versioning rules, we now have two packages - both called
>>> org.foo.bar and both having version 1.1 - that live on two
>>> different branches and contain different code.
>>>
>>> The only obvious solution that comes to my mind is to bump the
>>> major version of every package right after the development branch
>>> 1.5 is started, but I don't like this approach very much because it
>>> would break compatibility with existing clients for no obvious
>>> reason.
>>
>> This scenario is the exact problem you are facing while branching and
>> evolving the branches in parallel to trunk.
>>
>> The only end-developer friendly solution is to byte the bullet and do
>> it really properly and make sure you evolve exported packages (being
>> your API) in a truly diligent matter: Consider a package name and its
>> export version as the package¹s identity and always make sure this
>> identity (label) refers to the identical exported API.
>>
>
>I fail to see how this would work with branches. For Francesco's example
>this would mean that we'd need to backport everything into the branch
>effectively aligning it with trunk and thus obviating its purpose.
>
>Michael
>
>
>
Re: Semantic version in Oak
Posted by Michael Dürig <md...@apache.org>.
>> Packages evolve independently, but they do in potentially
>> divergent branches. This is the kind of timeline that we usually
>> face:
>>
>> - Oak 1.4 has a package org.foo.bar 1.0 - Some changes happen on
>> the development branch 1.5 - Oak 1.5 now has a package org.foo.bar
>> 1.1 - A change X happen in the development branch 1.5 - Oak 1.5 now
>> has a package org.foo.bar 1.2 - The change X has to be backported
>> to the maintenance branch 1.4 - Oak 1.4 now should have a package
>> org.foo.bar 1.1
>>
>> Assuming that the versions were incremented following the semantic
>> versioning rules, we now have two packages - both called
>> org.foo.bar and both having version 1.1 - that live on two
>> different branches and contain different code.
>>
>> The only obvious solution that comes to my mind is to bump the
>> major version of every package right after the development branch
>> 1.5 is started, but I don't like this approach very much because it
>> would break compatibility with existing clients for no obvious
>> reason.
>
> This scenario is the exact problem you are facing while branching and
> evolving the branches in parallel to trunk.
>
> The only end-developer friendly solution is to byte the bullet and do
> it really properly and make sure you evolve exported packages (being
> your API) in a truly diligent matter: Consider a package name and its
> export version as the package’s identity and always make sure this
> identity (label) refers to the identical exported API.
>
I fail to see how this would work with branches. For Francesco's example
this would mean that we'd need to backport everything into the branch
effectively aligning it with trunk and thus obviating its purpose.
Michael
Re: Semantic version in Oak
Posted by Felix Meschberger <fm...@adobe.com>.
Hi
> Am 08.12.2015 um 00:29 schrieb Francesco Mari <ma...@gmail.com>:
>
> 2015-12-07 21:02 GMT+01:00 David Bosschaert <da...@gmail.com>:
>
> ...
> The more frequent case is a change in the development branch (1.5) that has
> to be backported to the maintenance branches (1.4, 1.2, 1.0). If the change
> breaks the API of the affected packages, it could be troublesome - see
> below.
>
>
>>
>> 1 and 2 above should be pretty simple and 3 should pretty much never
>> happen anyway...
>>
>> Two additional notes. While we're talking about 'Oak 1.4' I assume
>> that is the version of the bundle. I assume each package in that
>> bundle evolves independently which means that they could have versions
>> other than 1.4 (e.g. org.apache.foo = 1.1.2 and org.apache.bar = 1.3
>> etc).
>>
>
> Packages evolve independently, but they do in potentially divergent
> branches. This is the kind of timeline that we usually face:
>
> - Oak 1.4 has a package org.foo.bar 1.0
> - Some changes happen on the development branch 1.5
> - Oak 1.5 now has a package org.foo.bar 1.1
> - A change X happen in the development branch 1.5
> - Oak 1.5 now has a package org.foo.bar 1.2
> - The change X has to be backported to the maintenance branch 1.4
> - Oak 1.4 now should have a package org.foo.bar 1.1
>
> Assuming that the versions were incremented following the semantic
> versioning rules, we now have two packages - both called org.foo.bar and
> both having version 1.1 - that live on two different branches and contain
> different code.
>
> The only obvious solution that comes to my mind is to bump the major
> version of every package right after the development branch 1.5 is started,
> but I don't like this approach very much because it would break
> compatibility with existing clients for no obvious reason.
This scenario is the exact problem you are facing while branching and evolving the branches in parallel to trunk.
The only end-developer friendly solution is to byte the bullet and do it really properly and make sure you evolve exported packages (being your API) in a truly diligent matter: Consider a package name and its export version as the package’s identity and always make sure this identity (label) refers to the identical exported API.
It’s a pain with unduly large exported packages, but its the only way you can serve your community.
Regards
Felix
Re: Semantic version in Oak
Posted by Francesco Mari <ma...@gmail.com>.
2015-12-07 21:02 GMT+01:00 David Bosschaert <da...@gmail.com>:
> Hi Michael,
>
> I would imagine that one of the following is true:
>
> 1. Typically changes made to maintenance branches are limited to
> bugfixes or implementation details (e.g. optimizations), which would
> not have a bearing on the exported package versions.
>
This kind of change happens quite frequently.
> 2. If there is a change made to API in the 1.4 branch that is backward
> incompatible with the current code, then the same change should really
> be made in the 1.5 branch to keep API users working in the future,
> which keeps the exported package versions in line - you make the same
> version change in the 1.5 branch as in the 1.4 branch.
3. It is hard to think of a situation where an API change is made to
> 1.4 which will not be available in future versions and I've never come
> across such a situation in reality. However if you have such a
> scenario you could change the major version of the package to
> something unusually high, e.g. 10000. Major package versions can be
> thought of as distinct entities so if the current version of your
> package on the maintenance branch is 2.1 and you make a backward
> incompatible change that you won't support in the future, you could
> choose such a high version for it.
>
The more frequent case is a change in the development branch (1.5) that has
to be backported to the maintenance branches (1.4, 1.2, 1.0). If the change
breaks the API of the affected packages, it could be troublesome - see
below.
>
> 1 and 2 above should be pretty simple and 3 should pretty much never
> happen anyway...
>
> Two additional notes. While we're talking about 'Oak 1.4' I assume
> that is the version of the bundle. I assume each package in that
> bundle evolves independently which means that they could have versions
> other than 1.4 (e.g. org.apache.foo = 1.1.2 and org.apache.bar = 1.3
> etc).
>
Packages evolve independently, but they do in potentially divergent
branches. This is the kind of timeline that we usually face:
- Oak 1.4 has a package org.foo.bar 1.0
- Some changes happen on the development branch 1.5
- Oak 1.5 now has a package org.foo.bar 1.1
- A change X happen in the development branch 1.5
- Oak 1.5 now has a package org.foo.bar 1.2
- The change X has to be backported to the maintenance branch 1.4
- Oak 1.4 now should have a package org.foo.bar 1.1
Assuming that the versions were incremented following the semantic
versioning rules, we now have two packages - both called org.foo.bar and
both having version 1.1 - that live on two different branches and contain
different code.
The only obvious solution that comes to my mind is to bump the major
version of every package right after the development branch 1.5 is started,
but I don't like this approach very much because it would break
compatibility with existing clients for no obvious reason.
> Also, the evolution of the *bundle version* does not necessarily have
> to reflect the version of the packages inside the bundle. While some
> people like to reflect some of the API evolution in the bundle version
> number other people simply increase that monotonically or using some
> marketing scheme or something like that. Semantic versions are really
> important for import package ranges so the exported packages should be
> semantically versioned. But people generally don't use ranges to
> require bundles so how these are versioned is a matter of taste...
>
Oak doesn't use bundle versions to convey any kind of useful information to
its users, it's more of a marketing scheme. Versions evolve semantically
for packages, though.
>
> Just my 2c,
>
> David
>
> On 7 December 2015 at 17:24, Michael Dürig <md...@apache.org> wrote:
> >
> > Hi,
> >
> > We started to annotate packages with their export versions (aka. semantic
> > versioning [1]) a while ago. After the 1.2 release we enforced proper API
> > management through the baseline functionality of the Maven bundle plugin.
> > This means that on current trunk the build fails should an API change but
> > the related package export version is not reflecting this.
> >
> > So far this approach worked relatively well. Going forward however I
> presume
> > things to get more complicated. Once Oak 1.4 is out we are going to have
> to
> > maintain a 1.4 branch while continuing development on trunk (i.e. 1.5).
> With
> > this we suddenly will have two potentially diverging branches, which begs
> > the question on how to do release from these branches while at the same
> time
> > maintaining consistent semantic versions of the contained packages across
> > the branches?
> >
> > Michael
> >
> >
> > [1] http://www.osgi.org/wp-content/uploads/SemanticVersioning1.pdf
> > [2] https://issues.apache.org/jira/browse/OAK-2006
>
Re: Semantic version in Oak
Posted by David Bosschaert <da...@gmail.com>.
Hi Michael,
I would imagine that one of the following is true:
1. Typically changes made to maintenance branches are limited to
bugfixes or implementation details (e.g. optimizations), which would
not have a bearing on the exported package versions.
2. If there is a change made to API in the 1.4 branch that is backward
incompatible with the current code, then the same change should really
be made in the 1.5 branch to keep API users working in the future,
which keeps the exported package versions in line - you make the same
version change in the 1.5 branch as in the 1.4 branch.
3. It is hard to think of a situation where an API change is made to
1.4 which will not be available in future versions and I've never come
across such a situation in reality. However if you have such a
scenario you could change the major version of the package to
something unusually high, e.g. 10000. Major package versions can be
thought of as distinct entities so if the current version of your
package on the maintenance branch is 2.1 and you make a backward
incompatible change that you won't support in the future, you could
choose such a high version for it.
1 and 2 above should be pretty simple and 3 should pretty much never
happen anyway...
Two additional notes. While we're talking about 'Oak 1.4' I assume
that is the version of the bundle. I assume each package in that
bundle evolves independently which means that they could have versions
other than 1.4 (e.g. org.apache.foo = 1.1.2 and org.apache.bar = 1.3
etc).
Also, the evolution of the *bundle version* does not necessarily have
to reflect the version of the packages inside the bundle. While some
people like to reflect some of the API evolution in the bundle version
number other people simply increase that monotonically or using some
marketing scheme or something like that. Semantic versions are really
important for import package ranges so the exported packages should be
semantically versioned. But people generally don't use ranges to
require bundles so how these are versioned is a matter of taste...
Just my 2c,
David
On 7 December 2015 at 17:24, Michael Dürig <md...@apache.org> wrote:
>
> Hi,
>
> We started to annotate packages with their export versions (aka. semantic
> versioning [1]) a while ago. After the 1.2 release we enforced proper API
> management through the baseline functionality of the Maven bundle plugin.
> This means that on current trunk the build fails should an API change but
> the related package export version is not reflecting this.
>
> So far this approach worked relatively well. Going forward however I presume
> things to get more complicated. Once Oak 1.4 is out we are going to have to
> maintain a 1.4 branch while continuing development on trunk (i.e. 1.5). With
> this we suddenly will have two potentially diverging branches, which begs
> the question on how to do release from these branches while at the same time
> maintaining consistent semantic versions of the contained packages across
> the branches?
>
> Michael
>
>
> [1] http://www.osgi.org/wp-content/uploads/SemanticVersioning1.pdf
> [2] https://issues.apache.org/jira/browse/OAK-2006