Re: [ALL] Version number(s) for modular components

[Gilles <gi...@harfang.homelinux.org>]

On Wed, 30 Nov 2016 10:23:37 +0000, Stian Soiland-Reyes wrote:
> Yeah, that could be the better way, with a branched off commit for 
> the
> "shrunk" project with a smaller <modules> list in the parent, then no
> particular flags are needed to build from the resulting tag or source 
> repo.
>
> I initially planned to do so within the Taverna project (before we 
> moved to
> ASF), as it could mean smaller releases and avoiding updating OSGi
> coordinates for no-change. It turned out to alienate the rest of the 
> team
> as then releasing become even more special, particularly if more than 
> one
> submodule needed updating. After moving to ASF we had to do release
> artifacts (not just tags and Maven repo) and we abandoned this idea 
> and had
> to bite the bullet to release all modules every time.

IIUC, it then makes it easier for the RM, not the users (especially
those who have to upgrade for "no-change").

I've no problem with that; but let's just say it the way it is.

> As an example of projects doing the partial, look at the Clerezza 
> project
> which practices this using alternative release parents and dists; as 
> an
> outsider I find it a bit difficult to understand what of the project 
> has
> been released where (or at all), and which versions go together.

I was not considering a way to make it more difficult for users:
either they don't touch anything, or they upgrade; in the latter
alternative, a few lines in the release notes tell them what is
the last version of each module, and by convention, only that
combination would be "supported".

> Commons components are luckily small, so I think it could be done - 
> but
> only IF NEEDED, rather than as general practice.

In what cases would you consider it is needed?
In what cases is it allowed?
In what cases is it forbidden?

The discussion triggered by the proposal was meant define
clear rules, with some justification for them in order to curb
speculation.

But the replies did not clearly demonstrate how the proposal was
bad.

In this particular case, I'd have nothing against an argument of
authority, but is should be stated as such.
Obviously, there are limits to consensus-based policy, unless
consensus means that people don't care about the rationale of
their choices.
In that case, it would be more effective to just have a majority
vote to which anyone can then refer, rather than having endless
discussions; some saying it's OK, sometimes, and others saying
they are against, always.

> There is also the consideration of operation system distributions 
> like
> Debian who prefer a single source archive or tag for the whole 
> project (and
> no dual versions) - varying this per release means they are going to 
> not
> update correctly. Several commons component (including math3) are 
> already
> in Debian.

It could be a rationale for restricting what is possible.

But I don't really understand "dual versions".
In the proposal, the project's version would always be unique;
it is supplemented with a version specific to each of its
modules (i.e. a version is attached to a module/artefact); one
of these modules could be, by definition, the "distribution"
(as I've noticed several projects do), whose version would be
incremented in the same way as is done now for the "project".

This would indeed be just an additional convention!
And if it's the same for all "Commons" project, the better:
common build instructions can be published and common tools
defined in the parent project.[1]

Regards,
Gilles

[1] Cf. my questions for creating aggregate (i.e. "distribution")
     artefacts for the release of "Commons RNG"; from the answers
     I got, I still could not generate all the required files!

>
> On 30 Nov 2016 9:04 am, "Gilles" <gi...@harfang.homelinux.org> 
> wrote:
>
>> On Tue, 29 Nov 2016 21:56:34 -0600, Matt Sicker wrote:
>>
>>> What if a feature was added to the maven-release-plugin to release 
>>> a
>>> subset
>>> of submodules? I wonder how feasible that would be.
>>>
>>
>> When I thought of independent module releases, I assumed that
>> it would just be a matter of excluding some of the "<module>"
>> sections in the (parent) POM.  [That seemed to work for making
>> the Jenkins build pass on Java 6 while the "commons-rng-examples"
>> requires Java 7.]
>>
>> Gilles
>>
>> On 28 November 2016 at 19:00, J�rg Schaible <jo...@gmx.de> 
>> wrote:
>>>
>>> Gilles wrote:
>>>>
>>>> > On Mon, 28 Nov 2016 07:31:36 -0700, Apache wrote:
>>>> >> Gilles,
>>>> >>
>>>> >> If you try to do this you are going to get very frustrated with
>>>> >> Maven. You cannot use the Maven Release plugin if all the 
>>>> versions
>>>> >> are
>>>> >> not SNAPSHOTs, and if they always have to be SNAPSHOTs it makes 
>>>> very
>>>> >> little sense to have them be out of sync. If you don\u2019t use the
>>>> >> release
>>>> >> plugin then you will have to come up with some custom release
>>>> >> mechanism that somehow can only release a portion of your 
>>>> project.
>>>> >> This is going to get rather messy as you will constantly be 
>>>> updating
>>>> >> the parent pom to increment versions and require that to be 
>>>> released
>>>> >> along with the modules you are releasing - which means your 
>>>> other
>>>> >> modules really need to be updated to reflect the new parent 
>>>> version.
>>>> >>
>>>> >> To be honest, I did what you are suggesting at a former 
>>>> employer. We
>>>> >> eventually stopped and synchronized the versions of all the 
>>>> modules.
>>>> >> It simply wasn\u2019t worth the effort to have all the versions be
>>>> >> different and the only real cost was releasing components with 
>>>> new
>>>> >> versions that hadn\u2019t changed.
>>>> >
>>>> > Thanks for the testimony.
>>>> > Even if I have no clue how the version string causes a problem,
>>>> > I can readily concede that we can be constrained in how to 
>>>> manage
>>>> > a project because of the shortcomings of some tool.
>>>>
>>>> There is no no short coming, you can do otherwise, but if you 
>>>> follow
>>>> conventions Maven makes your life easy (Maven is all about 
>>>> conventions).
>>>> However, the release process described in rng does not use the 
>>>> release
>>>> plugin, so the point is moot.
>>>>
>>>> > Out of curiosity, is there an alternative (to maven?) that would
>>>> > not suffer from this limitation?
>>>>
>>>> It's not the tool we're discussing.
>>>>
>>>> Cheers,
>>>> J�rg
>>>>
>>>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [ALL] Version number(s) for modular components

[Stian Soiland-Reyes <st...@apache.org>]

I don't think we have to be super rigid, just have a general principle. If
a case comes up, we can consider deviations then, rather than form fixed
rules from hypothetical scenarios and then be bound by them.

Something like RFC-style SHOULD - a component release may deviate from the
general rule, but then needs to be aware of the consequences and kind of
justify it on a case by case basis.

(Example: we released a new  "major" version BCEL 6 but ended up NOT
changing the package/artifact away from bcel5, even if that is Commons
general rule for major releases. This meant many downstream users could
upgrade without breaking their own APIs.)

As for "dual versions" I'm afraid that was a bit of a tangent - if Foo
release 2.0 produced foo-api 2.0 and foo-util 2.0, then in Debian this
would typically be a single source project "commons-foo" that then produces
two jars in each their own Deb.

Now if Foo Util release 2.1 comes out, which don't include foo-api, then
Debian can't use the same build instructions to build a foo-util 2.1 Deb,
as they would then first need to compile foo-api 2.0 from a different
release. (As an open source distro, Debian insists on compiling from source
and don't access Maven Central). That would however also install the old
foo-util 2.0 - but in Debian you can't have two versions of the same
library without renaming to include the version number in the package name.

A possible workaround then is for such a release to also include the old
2.0 foo-api source, the Debian packaging would the only need to deal with
the differentiated version numbers. This would however require some
trickery release-wise or repository-wise (e.g. gitflow model). Trickery and
complexity makes it harder for everyone.

I'm not sure what rational evidence to present as I think this is mainly a
"style" issue. We have heard several anecdotal arguments against
sub-releases as a general principle, but there is no absolute rational case
against it - many Maven users still do it, so it depends on the situation.
We are also "constrained" by ASF policy to make source releases as
archives, Commons' current use of Maven, and a general feeling about what
can be a separate Commons Component.

This can all change over time - I think this discussion is part of that. In
general open source (e.g. on GitHub) I observe a trend towards smaller
projects in smaller git repositories, with faster release cycles, with
fixes going from a pull request to live deployment in less than a day. ASF
have certain infrastructure, policies and practices, Commons modulate with
our own, and each component can tweak those. It's not all fixed in stone,
changes happen when there is  community consensus and someone actively
driving change.

So thank you for raising this point, we should not just assume that
whatever we did yesterday should always be the same way!

I do hope we can agree that the general rule for Commons is simplicity,
both for the user and for developers. While an unnecessary module upgrade
is not as "beautiful" as clean semantic version separations per module, it
is, I would argue, simpler to deal with for both consumers, developers and
release managers.

On 1 Dec 2016 9:05 am, "Gilles" <gi...@harfang.homelinux.org> wrote:

> On Wed, 30 Nov 2016 10:23:37 +0000, Stian Soiland-Reyes wrote:
>
>> Yeah, that could be the better way, with a branched off commit for the
>> "shrunk" project with a smaller <modules> list in the parent, then no
>> particular flags are needed to build from the resulting tag or source
>> repo.
>>
>> I initially planned to do so within the Taverna project (before we moved
>> to
>> ASF), as it could mean smaller releases and avoiding updating OSGi
>> coordinates for no-change. It turned out to alienate the rest of the team
>> as then releasing become even more special, particularly if more than one
>> submodule needed updating. After moving to ASF we had to do release
>> artifacts (not just tags and Maven repo) and we abandoned this idea and
>> had
>> to bite the bullet to release all modules every time.
>>
>
> IIUC, it then makes it easier for the RM, not the users (especially
> those who have to upgrade for "no-change").
>
> I've no problem with that; but let's just say it the way it is.
>
> As an example of projects doing the partial, look at the Clerezza project
>> which practices this using alternative release parents and dists; as an
>> outsider I find it a bit difficult to understand what of the project has
>> been released where (or at all), and which versions go together.
>>
>
> I was not considering a way to make it more difficult for users:
> either they don't touch anything, or they upgrade; in the latter
> alternative, a few lines in the release notes tell them what is
> the last version of each module, and by convention, only that
> combination would be "supported".
>
> Commons components are luckily small, so I think it could be done - but
>> only IF NEEDED, rather than as general practice.
>>
>
> In what cases would you consider it is needed?
> In what cases is it allowed?
> In what cases is it forbidden?
>
> The discussion triggered by the proposal was meant define
> clear rules, with some justification for them in order to curb
> speculation.
>
> But the replies did not clearly demonstrate how the proposal was
> bad.
>
> In this particular case, I'd have nothing against an argument of
> authority, but is should be stated as such.
> Obviously, there are limits to consensus-based policy, unless
> consensus means that people don't care about the rationale of
> their choices.
> In that case, it would be more effective to just have a majority
> vote to which anyone can then refer, rather than having endless
> discussions; some saying it's OK, sometimes, and others saying
> they are against, always.
>
> There is also the consideration of operation system distributions like
>> Debian who prefer a single source archive or tag for the whole project
>> (and
>> no dual versions) - varying this per release means they are going to not
>> update correctly. Several commons component (including math3) are already
>> in Debian.
>>
>
> It could be a rationale for restricting what is possible.
>
> But I don't really understand "dual versions".
> In the proposal, the project's version would always be unique;
> it is supplemented with a version specific to each of its
> modules (i.e. a version is attached to a module/artefact); one
> of these modules could be, by definition, the "distribution"
> (as I've noticed several projects do), whose version would be
> incremented in the same way as is done now for the "project".
>
> This would indeed be just an additional convention!
> And if it's the same for all "Commons" project, the better:
> common build instructions can be published and common tools
> defined in the parent project.[1]
>
> Regards,
> Gilles
>
> [1] Cf. my questions for creating aggregate (i.e. "distribution")
>     artefacts for the release of "Commons RNG"; from the answers
>     I got, I still could not generate all the required files!
>
>
>> On 30 Nov 2016 9:04 am, "Gilles" <gi...@harfang.homelinux.org> wrote:
>>
>> On Tue, 29 Nov 2016 21:56:34 -0600, Matt Sicker wrote:
>>>
>>> What if a feature was added to the maven-release-plugin to release a
>>>> subset
>>>> of submodules? I wonder how feasible that would be.
>>>>
>>>>
>>> When I thought of independent module releases, I assumed that
>>> it would just be a matter of excluding some of the "<module>"
>>> sections in the (parent) POM.  [That seemed to work for making
>>> the Jenkins build pass on Java 6 while the "commons-rng-examples"
>>> requires Java 7.]
>>>
>>> Gilles
>>>
>>> On 28 November 2016 at 19:00, Jörg Schaible <jo...@gmx.de>
>>> wrote:
>>>
>>>>
>>>> Gilles wrote:
>>>>
>>>>>
>>>>> > On Mon, 28 Nov 2016 07:31:36 -0700, Apache wrote:
>>>>> >> Gilles,
>>>>> >>
>>>>> >> If you try to do this you are going to get very frustrated with
>>>>> >> Maven. You cannot use the Maven Release plugin if all the versions
>>>>> >> are
>>>>> >> not SNAPSHOTs, and if they always have to be SNAPSHOTs it makes very
>>>>> >> little sense to have them be out of sync. If you don’t use the
>>>>> >> release
>>>>> >> plugin then you will have to come up with some custom release
>>>>> >> mechanism that somehow can only release a portion of your project.
>>>>> >> This is going to get rather messy as you will constantly be updating
>>>>> >> the parent pom to increment versions and require that to be released
>>>>> >> along with the modules you are releasing - which means your other
>>>>> >> modules really need to be updated to reflect the new parent version.
>>>>> >>
>>>>> >> To be honest, I did what you are suggesting at a former employer. We
>>>>> >> eventually stopped and synchronized the versions of all the modules.
>>>>> >> It simply wasn’t worth the effort to have all the versions be
>>>>> >> different and the only real cost was releasing components with new
>>>>> >> versions that hadn’t changed.
>>>>> >
>>>>> > Thanks for the testimony.
>>>>> > Even if I have no clue how the version string causes a problem,
>>>>> > I can readily concede that we can be constrained in how to manage
>>>>> > a project because of the shortcomings of some tool.
>>>>>
>>>>> There is no no short coming, you can do otherwise, but if you follow
>>>>> conventions Maven makes your life easy (Maven is all about
>>>>> conventions).
>>>>> However, the release process described in rng does not use the release
>>>>> plugin, so the point is moot.
>>>>>
>>>>> > Out of curiosity, is there an alternative (to maven?) that would
>>>>> > not suffer from this limitation?
>>>>>
>>>>> It's not the tool we're discussing.
>>>>>
>>>>> Cheers,
>>>>> Jörg
>>>>>
>>>>>
>>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>