You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sling.apache.org by Bertrand Delacretaz <bd...@apache.org> on 2016/03/11 10:19:13 UTC
The Sling build takes too long, how to fix?
Hi,
Here are typical times on my box for a full "mvn clean install" build,
first a few modules that take more than a minute:
[INFO] Apache Sling Discovery Commons ..................... SUCCESS [03:30 min]
[INFO] Apache Sling Health Check Core ..................... SUCCESS [01:08 min]
[INFO] Apache Sling Sample Integration Tests .............. SUCCESS [01:21 min]
[INFO] Apache Sling JCR Installer ......................... SUCCESS [01:53 min]
[INFO] Apache Sling Validation Framework Integration Tests FAILURE [01:01 min]
[INFO] Apache Sling Launchpad Testing ..................... SUCCESS [02:43 min]
And then the really bad ones which add up to enough time for a (quick) barbecue:
[INFO] Apache Sling Resource-Based Discovery Service ...... SUCCESS [21:12 min]
[INFO] Apache Sling Oak-Based Discovery Service ........... FAILURE [57:18 min]
[INFO] Apache Sling Event Support ......................... SUCCESS [09:02 min]
Do we agree that this second category is bad?
I suppose the result is that people rarely or never run a full build
with tests - IMO the full build should be coffee break compatible, so
around 10-15 minutes.
I haven't looked in detail yet at the second category above, does
someone familiar with those tests have suggestions?
Reduce the number of iterations unless a specific Maven profile is active?
Create a JUnit SlowTests category?
-Bertrand
Re: The Sling build takes too long, how to fix?
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Fri, Mar 11, 2016 at 2:08 PM, Stefan Egli <st...@apache.org> wrote:
> ...Alternatively we could use the @Category junit annotation and exclude them
> by default via an 'excludedGroups' via surefire
I'm not sure if that works with JUnit 3 style tests, but maybe all the
affected tests are JUnit 4 anyway, I haven't checked.
We have used those categories before, svn.apache.org/r1538994 - that
worked well.
> ...(and include them in the integration test phase)...
No, we need more granularity than that.
I think we can get a 10-15 minutes full build while keeping most of
our integration tests active, we just need to optionally disable a few
problematic tests which can be either unit or integration tests (*).
Or maybe reduce them to run less testing cycles where that's possible,
and run all cycles based on a specific profile.
-Bertrand
(*) but might need to be renamed to IT to make it clearer that they
take a long time - that's a different concern
Re: The Sling build takes too long, how to fix?
Posted by Stefan Egli <st...@apache.org>.
Hi,
Coming back to this topic, I'd like to go ahead with the Junit @Category
variant and would thus suggest to introduce a 'Slow' category marker
interface to the sling.commons.testing bundle, which would be excluded
from surefire/failsafe by default (via excludedGroups) - probably in the
parent/pom ?
There are then various ways of how to enable/disable this Slow category,
but one simple approach would be to introduce a new profile eg
'includeSlowTests' which fiddles with this excludedGroups eg via
properties. We should also ensure that the Slow tests are nevertheless run
when you do a release.. (no cheating)
I've implemented the category/profile variant for infamously slow
discovery, which would bring down the total from ca 39min down to ca 8min.
A patch with the suggested change (except parent/pom changes..) is in
SLING-5598 - pls let me know what you think. Unless there's -1 I'd at
least introduce the category to sling.commons.testing and use it in
discovery - doing this in general for all Sling bundles, ie in the
parent/pom, could be perhaps done in a separate step.
Cheers,
Stefan
--
https://issues.apache.org/jira/browse/SLING-5598
On 11/03/16 16:49, "Bertrand Delacretaz" <bd...@apache.org> wrote:
>On Fri, Mar 11, 2016 at 4:40 PM, Stefan Egli <st...@apache.org>
>wrote:
>> We could try to overwrite PaxExam as suggested in
>> https://vzurczak.wordpress.com/2015/01/19/skip-tests-with-pax-exam/ ..
>
>Don't we use different versions of that runner in places, 3.x and 4.x ?
>
>After a quick search for "Category" at [1] I think JUnit categories
>should work for all runners, it looks like it's the Maven surefire and
>failsafe plugin which handle them.
>
>And if not there's always the less elegant package or class name tricks.
>
>-Bertrand
>
>[1]
>http://svn.apache.org/repos/asf/maven/surefire/trunk/surefire-providers
Re: The Sling build takes too long, how to fix?
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Fri, Mar 11, 2016 at 4:40 PM, Stefan Egli <st...@apache.org> wrote:
> We could try to overwrite PaxExam as suggested in
> https://vzurczak.wordpress.com/2015/01/19/skip-tests-with-pax-exam/ ..
Don't we use different versions of that runner in places, 3.x and 4.x ?
After a quick search for "Category" at [1] I think JUnit categories
should work for all runners, it looks like it's the Maven surefire and
failsafe plugin which handle them.
And if not there's always the less elegant package or class name tricks.
-Bertrand
[1] http://svn.apache.org/repos/asf/maven/surefire/trunk/surefire-providers
Re: The Sling build takes too long, how to fix?
Posted by Stefan Egli <st...@apache.org>.
We could try to overwrite PaxExam as suggested in
https://vzurczak.wordpress.com/2015/01/19/skip-tests-with-pax-exam/
Cheers,
Stefan
On 11/03/16 15:47, "Bertrand Delacretaz" <bd...@apache.org> wrote:
>On Fri, Mar 11, 2016 at 3:15 PM, Stefan Egli <st...@apache.org>
>wrote:
>>>On 11/03/16 14:47, "Bertrand Delacretaz" <bd...@apache.org> wrote:
>>>> @Before
>>>> public void checkSlowTests() {
>>>> assumeNotNull(System.getProperty("sling.slow.tests.enabled"));
>>>> }
>>>
>>>+1, simple and effective...
>
>Unfortunately that's not effective with tests that use the PaxRunner,
>as the @Before method is called after setting up the test OSGi
>framework. So tests disabled in this way still take a few seconds to
>run, and PaxRunner doesn't support @BeforeClass.
>
>I guess JUnit classes is the clean way to handle this.
>
>-Bertrand
Re: The Sling build takes too long, how to fix?
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Fri, Mar 11, 2016 at 3:15 PM, Stefan Egli <st...@apache.org> wrote:
>>On 11/03/16 14:47, "Bertrand Delacretaz" <bd...@apache.org> wrote:
>>> @Before
>>> public void checkSlowTests() {
>>> assumeNotNull(System.getProperty("sling.slow.tests.enabled"));
>>> }
>>
>>+1, simple and effective...
Unfortunately that's not effective with tests that use the PaxRunner,
as the @Before method is called after setting up the test OSGi
framework. So tests disabled in this way still take a few seconds to
run, and PaxRunner doesn't support @BeforeClass.
I guess JUnit classes is the clean way to handle this.
-Bertrand
Re: The Sling build takes too long, how to fix?
Posted by Stefan Egli <st...@apache.org>.
On 11/03/16 14:56, "Stefan Egli" <st...@apache.org> wrote:
>On 11/03/16 14:47, "Bertrand Delacretaz" <bd...@apache.org> wrote:
>
>> @Before
>> public void checkSlowTests() {
>> assumeNotNull(System.getProperty("sling.slow.tests.enabled"));
>> }
>
>+1, simple and effective
created https://issues.apache.org/jira/browse/SLING-5598 to track this for
discovery
Cheers,
Stefan
Re: The Sling build takes too long, how to fix?
Posted by Stefan Egli <st...@apache.org>.
On 11/03/16 14:47, "Bertrand Delacretaz" <bd...@apache.org> wrote:
> @Before
> public void checkSlowTests() {
> assumeNotNull(System.getProperty("sling.slow.tests.enabled"));
> }
+1, simple and effective
Cheers,
Stefan
Re: The Sling build takes too long, how to fix?
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Fri, Mar 11, 2016 at 2:08 PM, Stefan Egli <st...@apache.org> wrote:
> ...Alternatively we could use the @Category junit annotation...
Or the Assume mechanism [1]...
That sounds a bit wrong semantically but doing something like
@Before
public void checkSlowTests() {
assumeNotNull(System.getProperty("sling.slow.tests.enabled"));
}
Requires very little changes, and if needed we can refine "categories"
with more property names.
-Bertrand
[1] http://junit.sourceforge.net/javadoc/org/junit/Assume.html
Re: The Sling build takes too long, how to fix?
Posted by Stefan Egli <st...@apache.org>.
Hi,
On 11/03/16 13:56, "Bertrand Delacretaz" <bd...@apache.org> wrote:
>Hi,
>
>On Fri, Mar 11, 2016 at 1:46 PM, Stefan Egli <st...@apache.org>
>wrote:
>> ...But short term, I think we could exclude them and put them in a
>>separate
>> integration tests category...
>
>As I said I don't want to exclude all integration tests by default, we
>need to be more granular.
>
>A cheap way of marking those tests as slow is to move them so that
>their Java package name includes the word "slow" with dots around it.
>
>We can then configure the testing plugins to ignore such tests by
>default, this should work regardless of which version of JUnit or the
>plugins is used (to be verified).
>
>That's assuming those tests work if moved to different packages,
>otherwise we might define a class naming convention like FooTestSlow
>and FooITSlow instead, but that's a bit uglier.
Alternatively we could use the @Category junit annotation and exclude them
by default via an 'excludedGroups' via surefire (and include them in the
integration test phase) as described in [0] and [1]? (Or we could perhaps
even introduce our own annotation..)
Cheers,
Stefan
--
[0]
http://stackoverflow.com/questions/14132174/how-to-exclude-all-junit4-tests
-with-a-given-category-using-maven-surefire
[1]
http://www.agile-engineering.net/2012/04/unit-and-integration-tests-with-ma
ven.html
Re: The Sling build takes too long, how to fix?
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi,
On Fri, Mar 11, 2016 at 1:46 PM, Stefan Egli <st...@apache.org> wrote:
> ...But short term, I think we could exclude them and put them in a separate
> integration tests category...
As I said I don't want to exclude all integration tests by default, we
need to be more granular.
A cheap way of marking those tests as slow is to move them so that
their Java package name includes the word "slow" with dots around it.
We can then configure the testing plugins to ignore such tests by
default, this should work regardless of which version of JUnit or the
plugins is used (to be verified).
That's assuming those tests work if moved to different packages,
otherwise we might define a class naming convention like FooTestSlow
and FooITSlow instead, but that's a bit uglier.
> ...renaming them to IT
> excludes them from 'mvn test' but also from 'mvn verify' it seems..
That's probably due to an incomplete or incorrect
maven-failsafe-plugin configuration.
That config is correct in the testing/samples/bundle-with-it module,
for example.
-Bertrand
Re: The Sling build takes too long, how to fix?
Posted by Stefan Egli <st...@apache.org>.
Hi
On 11/03/16 13:08, "Julian Sedding" <js...@gmail.com> wrote:
>In case I'm stating the obvious, I think there's an elephant in the room.
>
>Could we try to make the slow tests faster? If the indeed sleep a lot,
>maybe there are ways around that. IMHO sleeping tests also have the
>tendency to fail "randomly", i.e. are not deterministic. So a
>side-benefit *could* be more stable builds?
Yes, the discovery tests could be improved to be faster - even though
that's not too easily possible as it involves the repository too where
delays can happen. So it's not clear how much faster they can be done, but
it should be tried at least.
But short term, I think we could exclude them and put them in a separate
integration tests category (how do we mark those? eg renaming them to IT
excludes them from 'mvn test' but also from 'mvn verify' it seems..)
Cheers,
Stefan
Re: The Sling build takes too long, how to fix?
Posted by Carsten Ziegeler <cz...@apache.org>.
Julian Sedding wrote
> Hi
>
> In case I'm stating the obvious, I think there's an elephant in the room.
>
> Could we try to make the slow tests faster? If the indeed sleep a lot,
> maybe there are ways around that. IMHO sleeping tests also have the
> tendency to fail "randomly", i.e. are not deterministic. So a
> side-benefit *could* be more stable builds?
>
I can only speak for the eventing tests, but they only have a few sleeps
in it, most of the time is really testing the job handling.
Carsten
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: The Sling build takes too long, how to fix?
Posted by Julian Sedding <js...@gmail.com>.
Hi
In case I'm stating the obvious, I think there's an elephant in the room.
Could we try to make the slow tests faster? If the indeed sleep a lot,
maybe there are ways around that. IMHO sleeping tests also have the
tendency to fail "randomly", i.e. are not deterministic. So a
side-benefit *could* be more stable builds?
Regards
Julian
On Fri, Mar 11, 2016 at 11:20 AM, Stefan Egli <st...@apache.org> wrote:
> Hi,
>
> On 11/03/16 10:37, "Oliver Lietz" <ap...@oliverlietz.de> wrote:
>
>>On Friday 11 March 2016 10:19:13 Bertrand Delacretaz wrote:
>>>
>>>
>>> Do we agree that this second category is bad?
>>>
>>> I suppose the result is that people rarely or never run a full build
>>> with tests - IMO the full build should be coffee break compatible, so
>>> around 10-15 minutes.
>>>
>>> I haven't looked in detail yet at the second category above, does
>>> someone familiar with those tests have suggestions?
>>> Reduce the number of iterations unless a specific Maven profile is
>>>active?
>>> Create a JUnit SlowTests category?
>>
>>are these slow tests really unit tests or integration tests? We already
>>have a
>>profile integrationTests which could be used to run (more) slow tests.
>>IMHO we should stop doing full builds and only build modules which have
>>changed.
>
> Speaking for the largest chunk of time consumed - my discovery tests -
> those are probably more unit tests like.
>
> But yes, they should probably be in a 'slow' category of tests to avoid
> consuming time of ppl. Even if that'll mean they will not be run by most
> as few would volunteer to run the slow tests.
>
> So if we agree on how to categorize them as slow I can look into
> extracting a useful part as normal and the rest as slow.
>
> Cheers,
> Stefan
>
>
Re: The Sling build takes too long, how to fix?
Posted by Stefan Egli <st...@apache.org>.
Hi,
On 11/03/16 10:37, "Oliver Lietz" <ap...@oliverlietz.de> wrote:
>On Friday 11 March 2016 10:19:13 Bertrand Delacretaz wrote:
>>
>>
>> Do we agree that this second category is bad?
>>
>> I suppose the result is that people rarely or never run a full build
>> with tests - IMO the full build should be coffee break compatible, so
>> around 10-15 minutes.
>>
>> I haven't looked in detail yet at the second category above, does
>> someone familiar with those tests have suggestions?
>> Reduce the number of iterations unless a specific Maven profile is
>>active?
>> Create a JUnit SlowTests category?
>
>are these slow tests really unit tests or integration tests? We already
>have a
>profile integrationTests which could be used to run (more) slow tests.
>IMHO we should stop doing full builds and only build modules which have
>changed.
Speaking for the largest chunk of time consumed - my discovery tests -
those are probably more unit tests like.
But yes, they should probably be in a 'slow' category of tests to avoid
consuming time of ppl. Even if that'll mean they will not be run by most
as few would volunteer to run the slow tests.
So if we agree on how to categorize them as slow I can look into
extracting a useful part as normal and the rest as slow.
Cheers,
Stefan
Re: The Sling build takes too long, how to fix?
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi,
On Fri, Mar 11, 2016 at 1:32 PM, Oliver Lietz <ap...@oliverlietz.de> wrote:
> ...Sometimes it feels no one cares about the tests at all and Jenkins is only
> "spamming"...
I'm not looking at Jenkins all the time but when I do I usually dig in
to improve things a few small steps at a time.
Like with this thread and a few commits today that should fix flaky tests.
> ...And we have the launchpad integration tests (using snapshots instead of
> releases as most modules do anyway) which should find such issues, no?...
We've been moving many integration tests closer to their modules
lately, so even if launchpad/testing should stay enabled by default
IMO it's not sufficient.
-Bertrand
Re: The Sling build takes too long, how to fix?
Posted by Oliver Lietz <ap...@oliverlietz.de>.
On Friday 11 March 2016 09:46:11 Radu Cotescu wrote:
> Hi Olli,
Hi Radu,
> I think it's dangerous to build only the changed modules as most of them
> export API and provide implementations consumed by the other bundles from
> the launchpad. Sometimes, although the API stays compatible, the
> implementation might subtly change and produce havoc.
right, but does it really make any difference in our current situation?
Sometimes it feels no one cares about the tests at all and Jenkins is only
"spamming".
And we have the launchpad integration tests (using snapshots instead of
releases as most modules do anyway) which should find such issues, no?
Don't get me wrong: I'm more than happy if we can improve the current testing
situation. And any extra step which has to be taken (e.g. activating a
profile) leads to tests _not_ run in daily business.
Regards,
O.
> I guess the suggestion to bind all integration tests (which are definitely
> slower than unit tests) to the integrationTests profile, that's not
> activated by default, makes more sense.
>
> Cheers,
> Radu
>
> On Fri, 11 Mar 2016 at 10:37 Oliver Lietz <ap...@oliverlietz.de> wrote:
> > On Friday 11 March 2016 10:19:13 Bertrand Delacretaz wrote:
> > > Hi,
> >
> > Hi,
> >
> > > Here are typical times on my box for a full "mvn clean install" build,
> > > first a few modules that take more than a minute:
> > >
> > > [INFO] Apache Sling Discovery Commons ..................... SUCCESS
> >
> > [03:30
> >
> > > min] [INFO] Apache Sling Health Check Core ..................... SUCCESS
> > > [01:08 min] [INFO] Apache Sling Sample Integration Tests ..............
> > > SUCCESS [01:21 min] [INFO] Apache Sling JCR Installer
> > > ......................... SUCCESS [01:53 min] [INFO] Apache Sling
> > > Validation Framework Integration Tests FAILURE [01:01 min] [INFO]
> > > Apache
> > > Sling Launchpad Testing ..................... SUCCESS [02:43 min]
> > >
> > > And then the really bad ones which add up to enough time for a (quick)
> > > barbecue:
> > >
> > > [INFO] Apache Sling Resource-Based Discovery Service ...... SUCCESS
> >
> > [21:12
> >
> > > min] [INFO] Apache Sling Oak-Based Discovery Service ........... FAILURE
> > > [57:18 min] [INFO] Apache Sling Event Support .........................
> > > SUCCESS [09:02 min]
> > >
> > > Do we agree that this second category is bad?
> > >
> > > I suppose the result is that people rarely or never run a full build
> > > with tests - IMO the full build should be coffee break compatible, so
> > > around 10-15 minutes.
> > >
> > > I haven't looked in detail yet at the second category above, does
> > > someone familiar with those tests have suggestions?
> > > Reduce the number of iterations unless a specific Maven profile is
> >
> > active?
> >
> > > Create a JUnit SlowTests category?
> >
> > are these slow tests really unit tests or integration tests? We already
> > have a
> > profile integrationTests which could be used to run (more) slow tests.
> > IMHO we should stop doing full builds and only build modules which have
> > changed.
> >
> > Regards,
> > O.
> >
> > > -Bertrand
Re: The Sling build takes too long, how to fix?
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi,
On Fri, Mar 11, 2016 at 10:47 AM, Ian Boston <ie...@tfd.co.uk> wrote:
> ...Another approach is to do something similar to the workflow in most GitHub
> projects where Travis builds almost everything on every trigger including
> pull requests. eg [1]...
This looks possible on Apache infrastructure:
https://blogs.apache.org/infra/entry/github_pull_request_builds_now
And also https://issues.apache.org/jira/browse/INFRA-11065 which
points to https://builds.apache.org/job/kafka-trunk-git-pr-jdk7/ for a
detailed configuration example.
-Bertrand
Re: The Sling build takes too long, how to fix?
Posted by Ian Boston <ie...@tfd.co.uk>.
Hi,
I think the tests are not integration tests, but are extensive in their
coverage, often with sleeps.
If the build system/code structure could be setup in a way that when we
make a change only those modules that depend on the change get built and
tested, that would be much better. I don't think Maven can currently do
that in a multi-project (?).
Another approach is to do something similar to the workflow in most GitHub
projects where Travis builds almost everything on every trigger including
pull requests. eg [1]. Not sure if that's possible using the Apache SVN
infra, or if it fits the CTR workflow, but it does save local build cycles.
Best Regards
Ian
1 https://github.com/swagger-api/swagger-codegen/pull/2351
On 11 March 2016 at 09:46, Radu Cotescu <ra...@apache.org> wrote:
> Hi Olli,
>
> I think it's dangerous to build only the changed modules as most of them
> export API and provide implementations consumed by the other bundles from
> the launchpad. Sometimes, although the API stays compatible, the
> implementation might subtly change and produce havoc.
>
> I guess the suggestion to bind all integration tests (which are definitely
> slower than unit tests) to the integrationTests profile, that's not
> activated by default, makes more sense.
>
> Cheers,
> Radu
>
> On Fri, 11 Mar 2016 at 10:37 Oliver Lietz <ap...@oliverlietz.de> wrote:
>
> > On Friday 11 March 2016 10:19:13 Bertrand Delacretaz wrote:
> > > Hi,
> >
> > Hi,
> >
> > > Here are typical times on my box for a full "mvn clean install" build,
> > > first a few modules that take more than a minute:
> > >
> > > [INFO] Apache Sling Discovery Commons ..................... SUCCESS
> > [03:30
> > > min] [INFO] Apache Sling Health Check Core .....................
> SUCCESS
> > > [01:08 min] [INFO] Apache Sling Sample Integration Tests ..............
> > > SUCCESS [01:21 min] [INFO] Apache Sling JCR Installer
> > > ......................... SUCCESS [01:53 min] [INFO] Apache Sling
> > > Validation Framework Integration Tests FAILURE [01:01 min] [INFO]
> Apache
> > > Sling Launchpad Testing ..................... SUCCESS [02:43 min]
> > >
> > > And then the really bad ones which add up to enough time for a (quick)
> > > barbecue:
> > >
> > > [INFO] Apache Sling Resource-Based Discovery Service ...... SUCCESS
> > [21:12
> > > min] [INFO] Apache Sling Oak-Based Discovery Service ...........
> FAILURE
> > > [57:18 min] [INFO] Apache Sling Event Support .........................
> > > SUCCESS [09:02 min]
> > >
> > > Do we agree that this second category is bad?
> > >
> > > I suppose the result is that people rarely or never run a full build
> > > with tests - IMO the full build should be coffee break compatible, so
> > > around 10-15 minutes.
> > >
> > > I haven't looked in detail yet at the second category above, does
> > > someone familiar with those tests have suggestions?
> > > Reduce the number of iterations unless a specific Maven profile is
> > active?
> > > Create a JUnit SlowTests category?
> >
> > are these slow tests really unit tests or integration tests? We already
> > have a
> > profile integrationTests which could be used to run (more) slow tests.
> > IMHO we should stop doing full builds and only build modules which have
> > changed.
> >
> > Regards,
> > O.
> >
> > > -Bertrand
> >
> >
>
Re: The Sling build takes too long, how to fix?
Posted by Radu Cotescu <ra...@apache.org>.
Hi Olli,
I think it's dangerous to build only the changed modules as most of them
export API and provide implementations consumed by the other bundles from
the launchpad. Sometimes, although the API stays compatible, the
implementation might subtly change and produce havoc.
I guess the suggestion to bind all integration tests (which are definitely
slower than unit tests) to the integrationTests profile, that's not
activated by default, makes more sense.
Cheers,
Radu
On Fri, 11 Mar 2016 at 10:37 Oliver Lietz <ap...@oliverlietz.de> wrote:
> On Friday 11 March 2016 10:19:13 Bertrand Delacretaz wrote:
> > Hi,
>
> Hi,
>
> > Here are typical times on my box for a full "mvn clean install" build,
> > first a few modules that take more than a minute:
> >
> > [INFO] Apache Sling Discovery Commons ..................... SUCCESS
> [03:30
> > min] [INFO] Apache Sling Health Check Core ..................... SUCCESS
> > [01:08 min] [INFO] Apache Sling Sample Integration Tests ..............
> > SUCCESS [01:21 min] [INFO] Apache Sling JCR Installer
> > ......................... SUCCESS [01:53 min] [INFO] Apache Sling
> > Validation Framework Integration Tests FAILURE [01:01 min] [INFO] Apache
> > Sling Launchpad Testing ..................... SUCCESS [02:43 min]
> >
> > And then the really bad ones which add up to enough time for a (quick)
> > barbecue:
> >
> > [INFO] Apache Sling Resource-Based Discovery Service ...... SUCCESS
> [21:12
> > min] [INFO] Apache Sling Oak-Based Discovery Service ........... FAILURE
> > [57:18 min] [INFO] Apache Sling Event Support .........................
> > SUCCESS [09:02 min]
> >
> > Do we agree that this second category is bad?
> >
> > I suppose the result is that people rarely or never run a full build
> > with tests - IMO the full build should be coffee break compatible, so
> > around 10-15 minutes.
> >
> > I haven't looked in detail yet at the second category above, does
> > someone familiar with those tests have suggestions?
> > Reduce the number of iterations unless a specific Maven profile is
> active?
> > Create a JUnit SlowTests category?
>
> are these slow tests really unit tests or integration tests? We already
> have a
> profile integrationTests which could be used to run (more) slow tests.
> IMHO we should stop doing full builds and only build modules which have
> changed.
>
> Regards,
> O.
>
> > -Bertrand
>
>
Re: The Sling build takes too long, how to fix?
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Fri, Mar 11, 2016 at 10:37 AM, Oliver Lietz <ap...@oliverlietz.de> wrote:
> ...are these slow tests really unit tests or integration tests?...
I haven't looked in detail so far.
But even if they are technically integration tests, I would like to
have a profile that aims for a 10-15 minutes full build that includes
reasonably fast integration tests. The launchpad/testing module, for
example, is very important, runs about 560 integration tests and takes
less than 3 minutes on my box so I wouldn't disable it in that coffee
break compatible profile.
Or alternatively have a specific profile to enable those slow tests,
enable that on Jenkins only and also have the coffee break build
active on Jenkins to get quick feedback.
IMO we want people to run tests as often as possible - developers
often have subtly different environments which help uncover more
issues.
-Bertrand
Re: The Sling build takes too long, how to fix?
Posted by Oliver Lietz <ap...@oliverlietz.de>.
On Friday 11 March 2016 10:19:13 Bertrand Delacretaz wrote:
> Hi,
Hi,
> Here are typical times on my box for a full "mvn clean install" build,
> first a few modules that take more than a minute:
>
> [INFO] Apache Sling Discovery Commons ..................... SUCCESS [03:30
> min] [INFO] Apache Sling Health Check Core ..................... SUCCESS
> [01:08 min] [INFO] Apache Sling Sample Integration Tests ..............
> SUCCESS [01:21 min] [INFO] Apache Sling JCR Installer
> ......................... SUCCESS [01:53 min] [INFO] Apache Sling
> Validation Framework Integration Tests FAILURE [01:01 min] [INFO] Apache
> Sling Launchpad Testing ..................... SUCCESS [02:43 min]
>
> And then the really bad ones which add up to enough time for a (quick)
> barbecue:
>
> [INFO] Apache Sling Resource-Based Discovery Service ...... SUCCESS [21:12
> min] [INFO] Apache Sling Oak-Based Discovery Service ........... FAILURE
> [57:18 min] [INFO] Apache Sling Event Support .........................
> SUCCESS [09:02 min]
>
> Do we agree that this second category is bad?
>
> I suppose the result is that people rarely or never run a full build
> with tests - IMO the full build should be coffee break compatible, so
> around 10-15 minutes.
>
> I haven't looked in detail yet at the second category above, does
> someone familiar with those tests have suggestions?
> Reduce the number of iterations unless a specific Maven profile is active?
> Create a JUnit SlowTests category?
are these slow tests really unit tests or integration tests? We already have a
profile integrationTests which could be used to run (more) slow tests.
IMHO we should stop doing full builds and only build modules which have
changed.
Regards,
O.
> -Bertrand