You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sling.apache.org by Robert Munteanu <ro...@apache.org> on 2022/04/29 14:30:06 UTC
Running all Sling ITs with the Starter module - SLING-7935
Hi,
We have discussed making testing a bit easier with the Sling Starter.
Right now we have the Starter itself and the launchpad-testing module
that is used for executing the tests.
This can in my opinion be improved by moving the test execution in the
Starter module itself.
I have created two PRs that achieve this goal:
-
https://github.com/apache/sling-org-apache-sling-launchpad-integration-tests/pull/4
- https://github.com/apache/sling-org-apache-sling-starter/pull/66
After we merge them, with a `mvn clean verify` we can run the full set
of ITs in ~2,5 minutes.
I plan to merge these changes on Wednesday. Please let me know if you
see any reason for this not to proceed.
Thanks,
Robert
Re: Running all Sling ITs with the Starter module - SLING-7935
Posted by Oliver Lietz <ap...@oliverlietz.de>.
On Tuesday, 3 May 2022 18:20:14 CEST Stefan Seifert wrote:
> way 2 would be a really great solution, not sure how much effort it is to
> get there.
> it would also eliminate the few "satellite" IT projects we currently have
> e.g. for models and caconfig, and some other projects that have self-baked
> integration tests mostly based on the older sling packagings could be
> migrated as well. it's really a pain to have the ITs in separate GIT repos,
> they often get overlooked and break the whole PR-review-flow.
Running ITs in the same module was one of the initial goals for Testing
PaxExam¹. So this is already possible for a long time.
1) https://sling.apache.org/documentation/development/testing-paxexam.html#features
O.
> stefan
>
>
>
> >-----Original Message-----
> >From: Robert Munteanu <ro...@apache.org>
> >Sent: Tuesday, May 3, 2022 5:35 PM
> >To: dev@sling.apache.org
> >Subject: Re: Running all Sling ITs with the Starter module - SLING-7935
> >
> >On Tue, 2022-05-03 at 14:20 +0200, Carsten Ziegeler wrote:
> >
> >> Thanks, sounds good to me.
> >>
> >>
> >>
> >> Would it make sense to also move the tests into the starter, so tests
> >> can be updated/maintained with the starter?
> >
> >
> >
> >Until we get there, I plan to make at least one incremental
> >improvement, and that is the ability to run the Starter ITs from other
> >projects, with overriden bundle versions.
> >
> >We should be able to make change (for instance) in the Servlets POST
> >bundle and run the Starter ITs with only the Servlets POST bundle built
> >locally. That would IMO make testing PRs much simpler as regressions
> >will be visible without rebuilding the Starter.
> >
> >After that, we can take things in two directions:
> >
> >1. Move all the test-related classes in the starter project.
> >2. Move test-related classes into the bundle they are testing, for
> >example [1] into the Servlets POST bundle. At the same time, we should
> >also be running those tests in the Starter module.
> >
> >I see 1. as considerably simplifying builds and reducing the number
> >projects, but not improving the flow of submitting tests with the
> >bundles.
> >
> >On the other hand 2. will complicate a little how we aggregate and
> >launch the tests in the Starter module but will have the advantage of
> >being able to submit integration tests with changes in a single PR.
> >
> >_If_ we decide to go with 2, we will probably have a hybrid approach
> >where some tests get moved in the starter module as they are not really
> >specific to a single module and the rest get moved to bundle modules.
> >
> >I haven't really given it too much though, so feedback is welcome.
> >
> >Thanks,
> >Robert
> >
> >[1]:
> >https://github.com/apache/sling-org-apache-sling-launchpad-integration-> >tests/tree/master/src/main/java/org/apache/sling/launchpad/webapp/integrati
> >
ontest/servlets/post
> >
> >
> >>
> >>
> >> Regards
> >> Carsten
> >>
> >>
> >>
> >> Am 29.04.2022 um 16:30 schrieb Robert Munteanu:
> >>
> >> > Hi,
> >> >
> >> >
> >> >
> >> > We have discussed making testing a bit easier with the Sling
> >> > Starter.
> >> > Right now we have the Starter itself and the launchpad-testing
> >> > module
> >> > that is used for executing the tests.
> >> >
> >> >
> >> >
> >> > This can in my opinion be improved by moving the test execution in
> >> > the
> >> > Starter module itself.
> >> >
> >> >
> >> >
> >> > I have created two PRs that achieve this goal:
> >> >
> >> >
> >> >
> >> > -
> >> > https://github.com/apache/sling-org-apache-sling-launchpad-integration-> >> >
> >
> >tests/pull/4
> >
> >> > - https://github.com/apache/sling-org-apache-sling-starter/pull/66
> >> >
> >> >
> >> >
> >> > After we merge them, with a `mvn clean verify` we can run the full
> >> > set
> >> > of ITs in ~2,5 minutes.
> >> >
> >> >
> >> >
> >> > I plan to merge these changes on Wednesday. Please let me know if
> >> > you
> >> > see any reason for this not to proceed.
> >> >
> >> >
> >> >
> >> > Thanks,
> >> > Robert
> >>
> >>
>
>
Re: Running all Sling ITs with the Starter module - SLING-7935
Posted by Robert Munteanu <ro...@apache.org>.
On Tue, 2022-05-03 at 16:20 +0000, Stefan Seifert wrote:
> way 2 would be a really great solution, not sure how much effort it
> is to get there.
>
> it would also eliminate the few "satellite" IT projects we currently
> have e.g. for models and caconfig, and some other projects that have
> self-baked integration tests mostly based on the older sling
> packagings could be migrated as well. it's really a pain to have the
> ITs in separate GIT repos, they often get overlooked and break the
> whole PR-review-flow.
Right, I don't think it's going to be easy. I have a rough idea about
the end goal:
1. Add Maven tooling that is able to launch a certain version of the
starter ( fix number or 'latest SNAPSHOT' ) from other projects
2. Add Maven tooling to override a bundle's version directly by
synthesizing additional feature model defintions behind the scenes
3. Move launchpad integration tests to the module they are testing
4. Convert launchpad integration tests so that they follow a certain
convention ( JUnit runner?) so they work with an external Sling app
5. Deploy module integration tests as maven artifacts
6. Maven tooling to launch module integration tests during the starter
build, probably by looking up bundles that have associated integration
test artifacts.
With this set into place, we should be able to run ITs in the following
manager:
a. Module runs its specific ITs against a Sling Starter instance (
current snapshot or release )
b. Module runs all Starter ITs against a Sling Starter instance (
current snapshot or release )
c. Sling Starter runs all ITs - both those defined in its repository
and those collected from modules
I think that from my outline above point 4 will be the hardest - we
have lots of JUnit 3 tests and we need to find a good solution to make
sure they can run in both contexts - module ITs and Starter ITs.
Thanks,
Robert
RE: Running all Sling ITs with the Starter module - SLING-7935
Posted by Stefan Seifert <St...@diva-e.com.INVALID>.
way 2 would be a really great solution, not sure how much effort it is to get there.
it would also eliminate the few "satellite" IT projects we currently have e.g. for models and caconfig, and some other projects that have self-baked integration tests mostly based on the older sling packagings could be migrated as well. it's really a pain to have the ITs in separate GIT repos, they often get overlooked and break the whole PR-review-flow.
stefan
>-----Original Message-----
>From: Robert Munteanu <ro...@apache.org>
>Sent: Tuesday, May 3, 2022 5:35 PM
>To: dev@sling.apache.org
>Subject: Re: Running all Sling ITs with the Starter module - SLING-7935
>
>On Tue, 2022-05-03 at 14:20 +0200, Carsten Ziegeler wrote:
>> Thanks, sounds good to me.
>>
>> Would it make sense to also move the tests into the starter, so tests
>> can be updated/maintained with the starter?
>
>
>Until we get there, I plan to make at least one incremental
>improvement, and that is the ability to run the Starter ITs from other
>projects, with overriden bundle versions.
>
>We should be able to make change (for instance) in the Servlets POST
>bundle and run the Starter ITs with only the Servlets POST bundle built
>locally. That would IMO make testing PRs much simpler as regressions
>will be visible without rebuilding the Starter.
>
>After that, we can take things in two directions:
>
>1. Move all the test-related classes in the starter project.
>2. Move test-related classes into the bundle they are testing, for
>example [1] into the Servlets POST bundle. At the same time, we should
>also be running those tests in the Starter module.
>
>I see 1. as considerably simplifying builds and reducing the number
>projects, but not improving the flow of submitting tests with the
>bundles.
>
>On the other hand 2. will complicate a little how we aggregate and
>launch the tests in the Starter module but will have the advantage of
>being able to submit integration tests with changes in a single PR.
>
>_If_ we decide to go with 2, we will probably have a hybrid approach
>where some tests get moved in the starter module as they are not really
>specific to a single module and the rest get moved to bundle modules.
>
>I haven't really given it too much though, so feedback is welcome.
>
>Thanks,
>Robert
>
>[1]:
>https://github.com/apache/sling-org-apache-sling-launchpad-integration-
>tests/tree/master/src/main/java/org/apache/sling/launchpad/webapp/integrati
>ontest/servlets/post
>
>>
>> Regards
>> Carsten
>>
>> Am 29.04.2022 um 16:30 schrieb Robert Munteanu:
>> > Hi,
>> >
>> > We have discussed making testing a bit easier with the Sling
>> > Starter.
>> > Right now we have the Starter itself and the launchpad-testing
>> > module
>> > that is used for executing the tests.
>> >
>> > This can in my opinion be improved by moving the test execution in
>> > the
>> > Starter module itself.
>> >
>> > I have created two PRs that achieve this goal:
>> >
>> > -
>> > https://github.com/apache/sling-org-apache-sling-launchpad-integration-
>tests/pull/4
>> > - https://github.com/apache/sling-org-apache-sling-starter/pull/66
>> >
>> > After we merge them, with a `mvn clean verify` we can run the full
>> > set
>> > of ITs in ~2,5 minutes.
>> >
>> > I plan to merge these changes on Wednesday. Please let me know if
>> > you
>> > see any reason for this not to proceed.
>> >
>> > Thanks,
>> > Robert
>>
Re: Running all Sling ITs with the Starter module - SLING-7935
Posted by Robert Munteanu <ro...@apache.org>.
On Tue, 2022-05-03 at 14:20 +0200, Carsten Ziegeler wrote:
> Thanks, sounds good to me.
>
> Would it make sense to also move the tests into the starter, so tests
> can be updated/maintained with the starter?
Until we get there, I plan to make at least one incremental
improvement, and that is the ability to run the Starter ITs from other
projects, with overriden bundle versions.
We should be able to make change (for instance) in the Servlets POST
bundle and run the Starter ITs with only the Servlets POST bundle built
locally. That would IMO make testing PRs much simpler as regressions
will be visible without rebuilding the Starter.
After that, we can take things in two directions:
1. Move all the test-related classes in the starter project.
2. Move test-related classes into the bundle they are testing, for
example [1] into the Servlets POST bundle. At the same time, we should
also be running those tests in the Starter module.
I see 1. as considerably simplifying builds and reducing the number
projects, but not improving the flow of submitting tests with the
bundles.
On the other hand 2. will complicate a little how we aggregate and
launch the tests in the Starter module but will have the advantage of
being able to submit integration tests with changes in a single PR.
_If_ we decide to go with 2, we will probably have a hybrid approach
where some tests get moved in the starter module as they are not really
specific to a single module and the rest get moved to bundle modules.
I haven't really given it too much though, so feedback is welcome.
Thanks,
Robert
[1]:
https://github.com/apache/sling-org-apache-sling-launchpad-integration-tests/tree/master/src/main/java/org/apache/sling/launchpad/webapp/integrationtest/servlets/post
>
> Regards
> Carsten
>
> Am 29.04.2022 um 16:30 schrieb Robert Munteanu:
> > Hi,
> >
> > We have discussed making testing a bit easier with the Sling
> > Starter.
> > Right now we have the Starter itself and the launchpad-testing
> > module
> > that is used for executing the tests.
> >
> > This can in my opinion be improved by moving the test execution in
> > the
> > Starter module itself.
> >
> > I have created two PRs that achieve this goal:
> >
> > -
> > https://github.com/apache/sling-org-apache-sling-launchpad-integration-tests/pull/4
> > - https://github.com/apache/sling-org-apache-sling-starter/pull/66
> >
> > After we merge them, with a `mvn clean verify` we can run the full
> > set
> > of ITs in ~2,5 minutes.
> >
> > I plan to merge these changes on Wednesday. Please let me know if
> > you
> > see any reason for this not to proceed.
> >
> > Thanks,
> > Robert
>
Re: Running all Sling ITs with the Starter module - SLING-7935
Posted by Carsten Ziegeler <cz...@apache.org>.
Thanks, sounds good to me.
Would it make sense to also move the tests into the starter, so tests
can be updated/maintained with the starter?
Regards
Carsten
Am 29.04.2022 um 16:30 schrieb Robert Munteanu:
> Hi,
>
> We have discussed making testing a bit easier with the Sling Starter.
> Right now we have the Starter itself and the launchpad-testing module
> that is used for executing the tests.
>
> This can in my opinion be improved by moving the test execution in the
> Starter module itself.
>
> I have created two PRs that achieve this goal:
>
> -
> https://github.com/apache/sling-org-apache-sling-launchpad-integration-tests/pull/4
> - https://github.com/apache/sling-org-apache-sling-starter/pull/66
>
> After we merge them, with a `mvn clean verify` we can run the full set
> of ITs in ~2,5 minutes.
>
> I plan to merge these changes on Wednesday. Please let me know if you
> see any reason for this not to proceed.
>
> Thanks,
> Robert
--
Carsten Ziegeler
Adobe
cziegeler@apache.org