You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sling.apache.org by Robert Munteanu <ro...@apache.org> on 2016/10/03 13:20:31 UTC
Re: Launchpad stable and launchpad unstable
On Wed, 2016-09-28 at 22:06 +0000, Stefan Seifert wrote:
> discussed at the Sling Committer Round Table @ adaptTo() 2016
>
> currently we have only one launchpad in trunk, which contains a
> mixture of release dependencies and snapshot dependencies. this makes
> it difficult to release new versions because one has to release all
> snapshot projects first, and perhaps the last relese version is fine
> as well.
>
> we came up with a new proposal to fulfill the different needs:
>
> - the launchpad maintained in trunk should always be the "stable"
> launchpad, referencing only released dependencies of the sling
> bundles and oft the integration tests project
>
> - the "unstable" launchpad is created dynamically from this stable
> launchpad by using the same bundle list, but for each dependency the
> latest snapshot version that is available in the maven repo,
> including the latest snapshot version oft he integration tests.
> perhaps the slingstart plugin could be extented to generate this.
>
> - i'm not sure what to do with new bundles which are not released yet
> thus are not referenced in the stable launchpad.
I'm +1 on this change, it makes it much simpler to perform releases. If
we make building an unstable launchpad really simple, e.g. an addition
profile or a mojo property, I don't think we lose anything.
Robert
Re: Launchpad stable and launchpad unstable
Posted by Robert Munteanu <ro...@apache.org>.
On Fri, 2016-10-14 at 15:38 +0200, Bertrand Delacretaz wrote:
> I've committed an experimental script [1] which modifies the
> launchpad/builder provisioning model files to remove the version
> numbers for all Sling artifacts. See comments in the script for
> details (and enjoy the sed regexp ;-)
>
> When doing so, the Slingstart plugin uses the latest snapshot instead
> of a specified version, IIUC.
>
> The script doesn't fully work yet (see comments in the script -
> patches or hints welcome) but once it does the process for running
> our
> integration tests on this all-snapshots launchpad would be:
For the record, it doesn't work for me either, but misses more SNAPSHOT
versions:
$ mvn dependency:resolve | grep org.apache.sling | grep -v SNAPSHOT
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
support was removed in 8.0
[INFO] --- maven-dependency-plugin:2.10:resolve (default-cli) @
org.apache.sling.launchpad ---
[INFO]����org.apache.sling:org.apache.sling.servlets.resolver:jar:2.4.6
:provided
[INFO]����org.apache.sling:org.apache.sling.launchpad.base:war:webapp:5
.6.0-2.6.12:provided
[INFO]����org.apache.sling:org.apache.sling.commons.scheduler:jar:2.5.2
:provided
[INFO]����org.apache.sling:org.apache.sling.commons.log.webconsole:jar:
1.0.0:provided
[INFO]����org.apache.sling:org.apache.sling.commons.log:jar:5.0.0:provi
ded
[INFO]����org.apache.sling:org.apache.sling.resourceresolver:jar:1.5.0:
provided
[INFO]����org.apache.sling:org.apache.sling.launchpad.base:jar:5.6.0-
2.6.12:provided
[INFO]����org.apache.sling:org.apache.sling.launchpad.base:jar:app:5.6.
0-2.6.12:provided
[INFO]����org.apache.sling:org.apache.sling.event.dea:jar:1.1.0:provide
d
[INFO]����org.apache.sling:org.apache.sling.api:jar:2.15.0:provided
[INFO]����org.apache.sling:org.apache.sling.installer.core:jar:3.7.0:pr
ovided
Robert
Re: Launchpad stable and launchpad unstable
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Tue, Nov 1, 2016 at 10:08 AM, Robert Munteanu <ro...@apache.org> wrote:
...
> @NeedsBundle("org.apache.sling.commons.json", "2.1.0")
...
> assumeThat(bundleIsPresent("org.apache.sling.commons.json",
...
I like the idea, maybe with names like @NeedsBundleVersion and
bundleVersionPresent(...) to be more explicit.
Maybe we need something similar for package versions, but the
mechanism will be similar.
-Bertrand
Re: Launchpad stable and launchpad unstable
Posted by Robert Munteanu <ro...@apache.org>.
On Wed, 2016-11-02 at 12:02 +0100, Carsten Ziegeler wrote:
> Bertrand Delacretaz wrote
> > On Wed, Nov 2, 2016 at 9:39 AM, Carsten Ziegeler <cziegeler@apache.
> > org> wrote:
> > > Bertrand Delacretaz wrote
> > > > On Wed, Nov 2, 2016 at 6:56 AM, Carsten Ziegeler <cziegeler@apa
> > > > che.org> wrote:
> > > > > ...what problem are we exactly trying to solve?
> > > >
> > > > ...As described in the original post here - having a stable
> > > > launchpad
> > > > that's always ready to release, while also continuously testing
> > > > the
> > > > integration of the latest snapshots.
> > > >
> > >
> > > So if the integration tests always test the latest snapshots then
> > > there
> > > is nothing we have do to, right?...
> >
> > We need to test the launchpad that we release, don't we?
>
> Yeah, sorry that I'm a little bit picky here. All I'm trying that we
> get
> a description of the full picture.
>
> We want to test the *same* integration tests against the current
> launchpad (which has no snapshot deps) and against the same launchpad
> but replacing all sling deps to the latest snapshot versions.
> And that might lead us into the situation you describe below.
>
> >
> > Depending on the combination of bundles that release might have
> > different behavior than combining the latest snapshots, although
> > maybe
> > it's not that likely.
>
> Right, I would argue that it is not very likely.
>
> >
> > As a first step we might manually run the integration tests against
> > the launchpad that's about to be released, that's better than
> > nothing
> > but some tests will fail and we need a manual analysis of that -
> > which
> > is not impossible if the goal is to release say every 3 months.
>
> Instead of coming up with a gigantic approach that will not be used
> or
> only once every two years, I think lets deal with it once we're
> there.
> Something simple like an exclusion list activated by the same
> mechanism
> that switches to snapshot dependencies.
I've tried to summarise the current state of the discussions at
https://cwiki.apache.org/confluence/display/SLING/Moving+the+Sling+La
unchpad+to+use+released+artifacts+only
I think currently we need to decide/finalise:
- how to transform a 'stable' launchpad into an 'unstable' one
- how to exclude tests from running on an 'stable' launchpad but keep
the running on an 'unstable' launchpad.
Comment on the wiki page more than welcome, I have probably missed
something :-)
Thanks,
Robert
>
> �Carsten
>
Re: Launchpad stable and launchpad unstable
Posted by Carsten Ziegeler <cz...@apache.org>.
Bertrand Delacretaz wrote
> On Wed, Nov 2, 2016 at 9:39 AM, Carsten Ziegeler <cz...@apache.org> wrote:
>> Bertrand Delacretaz wrote
>>> On Wed, Nov 2, 2016 at 6:56 AM, Carsten Ziegeler <cz...@apache.org> wrote:
>>>> ...what problem are we exactly trying to solve?
>>> ...As described in the original post here - having a stable launchpad
>>> that's always ready to release, while also continuously testing the
>>> integration of the latest snapshots.
>>>
>> So if the integration tests always test the latest snapshots then there
>> is nothing we have do to, right?...
>
> We need to test the launchpad that we release, don't we?
Yeah, sorry that I'm a little bit picky here. All I'm trying that we get
a description of the full picture.
We want to test the *same* integration tests against the current
launchpad (which has no snapshot deps) and against the same launchpad
but replacing all sling deps to the latest snapshot versions.
And that might lead us into the situation you describe below.
>
> Depending on the combination of bundles that release might have
> different behavior than combining the latest snapshots, although maybe
> it's not that likely.
Right, I would argue that it is not very likely.
>
> As a first step we might manually run the integration tests against
> the launchpad that's about to be released, that's better than nothing
> but some tests will fail and we need a manual analysis of that - which
> is not impossible if the goal is to release say every 3 months.
Instead of coming up with a gigantic approach that will not be used or
only once every two years, I think lets deal with it once we're there.
Something simple like an exclusion list activated by the same mechanism
that switches to snapshot dependencies.
Carsten
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: Launchpad stable and launchpad unstable
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Wed, Nov 2, 2016 at 9:39 AM, Carsten Ziegeler <cz...@apache.org> wrote:
> Bertrand Delacretaz wrote
>> On Wed, Nov 2, 2016 at 6:56 AM, Carsten Ziegeler <cz...@apache.org> wrote:
>>> ...what problem are we exactly trying to solve?
>> ...As described in the original post here - having a stable launchpad
>> that's always ready to release, while also continuously testing the
>> integration of the latest snapshots.
>>
> So if the integration tests always test the latest snapshots then there
> is nothing we have do to, right?...
We need to test the launchpad that we release, don't we?
Depending on the combination of bundles that release might have
different behavior than combining the latest snapshots, although maybe
it's not that likely.
As a first step we might manually run the integration tests against
the launchpad that's about to be released, that's better than nothing
but some tests will fail and we need a manual analysis of that - which
is not impossible if the goal is to release say every 3 months.
-Bertrand
Re: Launchpad stable and launchpad unstable
Posted by Carsten Ziegeler <cz...@apache.org>.
Bertrand Delacretaz wrote
> On Wed, Nov 2, 2016 at 6:56 AM, Carsten Ziegeler <cz...@apache.org> wrote:
>> ...what problem are we exactly trying to solve?
>
> As described in the original post here - having a stable launchpad
> that's always ready to release, while also continuously testing the
> integration of the latest snapshots.
>
So if the integration tests always test the latest snapshots then there
is nothing we have do to, right?
Carsten
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: Launchpad stable and launchpad unstable
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Wed, Nov 2, 2016 at 6:56 AM, Carsten Ziegeler <cz...@apache.org> wrote:
> ...what problem are we exactly trying to solve?
As described in the original post here - having a stable launchpad
that's always ready to release, while also continuously testing the
integration of the latest snapshots.
-Bertrand
Re: Launchpad stable and launchpad unstable
Posted by Carsten Ziegeler <cz...@apache.org>.
Maybe I'm missing something, but what problem are we exactly trying to
solve?
Carsten
Robert Munteanu wrote
> On Mon, 2016-10-31 at 11:17 +0100, Carsten Ziegeler wrote:
>> Bertrand Delacretaz wrote
>>> On Wed, Oct 26, 2016 at 9:08 AM, Robert Munteanu <rombert@apache.or
>>> g> wrote:
>>>> ...How would we handle launchpad tests that rely on functionality
>>>> introduced in a SNAPSHOT dependency?...
>>>
>>> I suppose we'll need an annotation to indicate the requirements for
>>> a
>>> specific test.
>>>
>>
>> Isn't that a little bit complicated? Who removes the annotation when
>> things are released?
>
> How about an annotation or an assumption mechanism which looks for a
> specific bundle + version to be present? That would not need further
> maintenance and can be removed at any time, should we want to remove
> it:
>
> @NeedsBundle("org.apache.sling.commons.json", "2.1.0")
> @Test
> public void validateSomething() { ... }
>
> or
>
> @Test
> public void validateSomething() {
> assumeThat(bundleIsPresent("org.apache.sling.commons.json",
> "2.1.0"));
> ...
> }
>
>>
>> In general, can we come up with a full proposal instead of discussing
>> individual single points? I suggest we setup a wiki where we outline
>> the
>> full process and then can fill in the details
>
> I can come up with a full proposal but there are some things which are
> not totally clear to me, so that's why I'm asking :-)
>
> Robert
>
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: Launchpad stable and launchpad unstable
Posted by Robert Munteanu <ro...@apache.org>.
On Mon, 2016-10-31 at 11:17 +0100, Carsten Ziegeler wrote:
> Bertrand Delacretaz wrote
> > On Wed, Oct 26, 2016 at 9:08 AM, Robert Munteanu <rombert@apache.or
> > g> wrote:
> > > ...How would we handle launchpad tests that rely on functionality
> > > introduced in a SNAPSHOT dependency?...
> >
> > I suppose we'll need an annotation to indicate the requirements for
> > a
> > specific test.
> >
>
> Isn't that a little bit complicated? Who removes the annotation when
> things are released?
How about an annotation or an assumption mechanism which looks for a
specific bundle + version to be present? That would not need further
maintenance and can be removed at any time, should we want to remove
it:
@NeedsBundle("org.apache.sling.commons.json", "2.1.0")
@Test
public void validateSomething() { ... }
or
@Test
public void validateSomething() {
assumeThat(bundleIsPresent("org.apache.sling.commons.json",
"2.1.0"));
...
}
>
> In general, can we come up with a full proposal instead of discussing
> individual single points? I suggest we setup a wiki where we outline
> the
> full process and then can fill in the details
I can come up with a full proposal but there are some things which are
not totally clear to me, so that's why I'm asking :-)
Robert
Re: Launchpad stable and launchpad unstable
Posted by Carsten Ziegeler <cz...@apache.org>.
Bertrand Delacretaz wrote
> On Wed, Oct 26, 2016 at 9:08 AM, Robert Munteanu <ro...@apache.org> wrote:
>> ...How would we handle launchpad tests that rely on functionality
>> introduced in a SNAPSHOT dependency?...
>
> I suppose we'll need an annotation to indicate the requirements for a
> specific test.
>
Isn't that a little bit complicated? Who removes the annotation when
things are released?
In general, can we come up with a full proposal instead of discussing
individual single points? I suggest we setup a wiki where we outline the
full process and then can fill in the details
Carsten
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: Launchpad stable and launchpad unstable
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Wed, Oct 26, 2016 at 9:08 AM, Robert Munteanu <ro...@apache.org> wrote:
> ...How would we handle launchpad tests that rely on functionality
> introduced in a SNAPSHOT dependency?...
I suppose we'll need an annotation to indicate the requirements for a
specific test.
-Bertrand
Re: Launchpad stable and launchpad unstable
Posted by Robert Munteanu <ro...@apache.org>.
On Fri, 2016-10-14 at 15:45 +0200, Carsten Ziegeler wrote:
> Bertrand Delacretaz wrote
> > On Mon, Oct 3, 2016 at 3:20 PM, Robert Munteanu <rombert@apache.org
> > > wrote:
> > > ...If we make building an unstable launchpad really simple, e.g.
> > > an addition
> > > profile or a mojo property, I don't think we lose anything...
> >
> > I've committed an experimental script [1] which modifies the
> > launchpad/builder provisioning model files to remove the version
> > numbers for all Sling artifacts. See comments in the script for
> > details (and enjoy the sed regexp ;-)
> >
> > When doing so, the Slingstart plugin uses the latest snapshot
> > instead
> > of a specified version, IIUC.
> >
> > The script doesn't fully work yet (see comments in the script -
> > patches or hints welcome) but once it does the process for running
> > our
> > integration tests on this all-snapshots launchpad would be:
> >
> > 1) Run this script in launchpad/builder, build and deploy that,
> > with a
> > specific classifier or other marker
> > 2) Run the launchpad/testing build using that special launchpad jar
> > 3) Revert the code changes caused by step 1), unless working in a
> > throwaway folder
> >
> > Would that work for you guys?
> >
>
> Sorry I already replied to your commit as I didn't see this mail :(
>
> I think a script is fine and thanks for taking it up.
> My initial thoughts were to do this in the slingstart maven plugin
> This would give a little bit more control - if required, like
> enabling
> snapshots only for a single feature or something. Or having a
> blacklist.
>
> But maybe that's not needed anyways
How would we handle launchpad tests that rely on functionality
introduced in a SNAPSHOT dependency?
These would fail with the stable launchpad. We could make then run
conditionally just with the unstable launchpad but OTOH we need to
remember to remove that check after the bundle which provides the new
functionality ( or the bug fix ) is released and included in the
launchpad.
Robert
Re: Launchpad stable and launchpad unstable
Posted by Carsten Ziegeler <cz...@apache.org>.
Bertrand Delacretaz wrote
> On Mon, Oct 3, 2016 at 3:20 PM, Robert Munteanu <ro...@apache.org> wrote:
>> ...If we make building an unstable launchpad really simple, e.g. an addition
>> profile or a mojo property, I don't think we lose anything...
>
> I've committed an experimental script [1] which modifies the
> launchpad/builder provisioning model files to remove the version
> numbers for all Sling artifacts. See comments in the script for
> details (and enjoy the sed regexp ;-)
>
> When doing so, the Slingstart plugin uses the latest snapshot instead
> of a specified version, IIUC.
>
> The script doesn't fully work yet (see comments in the script -
> patches or hints welcome) but once it does the process for running our
> integration tests on this all-snapshots launchpad would be:
>
> 1) Run this script in launchpad/builder, build and deploy that, with a
> specific classifier or other marker
> 2) Run the launchpad/testing build using that special launchpad jar
> 3) Revert the code changes caused by step 1), unless working in a
> throwaway folder
>
> Would that work for you guys?
>
Sorry I already replied to your commit as I didn't see this mail :(
I think a script is fine and thanks for taking it up.
My initial thoughts were to do this in the slingstart maven plugin
This would give a little bit more control - if required, like enabling
snapshots only for a single feature or something. Or having a blacklist.
But maybe that's not needed anyways
Carsten
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: Launchpad stable and launchpad unstable
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Mon, Oct 3, 2016 at 3:20 PM, Robert Munteanu <ro...@apache.org> wrote:
>...If we make building an unstable launchpad really simple, e.g. an addition
> profile or a mojo property, I don't think we lose anything...
I've committed an experimental script [1] which modifies the
launchpad/builder provisioning model files to remove the version
numbers for all Sling artifacts. See comments in the script for
details (and enjoy the sed regexp ;-)
When doing so, the Slingstart plugin uses the latest snapshot instead
of a specified version, IIUC.
The script doesn't fully work yet (see comments in the script -
patches or hints welcome) but once it does the process for running our
integration tests on this all-snapshots launchpad would be:
1) Run this script in launchpad/builder, build and deploy that, with a
specific classifier or other marker
2) Run the launchpad/testing build using that special launchpad jar
3) Revert the code changes caused by step 1), unless working in a
throwaway folder
Would that work for you guys?
-Bertrand
(BTW the resulting launchpad with all snapshots doesn't start, might
be a good indicator that we need this!
[1] https://svn.apache.org/repos/asf/sling/trunk/launchpad/builder/set-sling-snapshots.sh