You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sling.apache.org by Ian Boston <ie...@tfd.co.uk> on 2016/07/12 13:50:54 UTC
Build stability/time.
Hi,
According to jenkins 1 in 10 builds fail, and some times randomly. For
instance, [2] succeeded but [3] failed, due to [4], a patch to a README.md
According to [1] its not been discussed.
How are others dealing with this ? (-Dmaven.test.skip=true or not bothering
to build all before committing)
I assume only the test methodology is unreliable, and not the released
artifacts ?
Could the unreliable tests be moved to ITs ?
Could the the long running tests also be moved to ITs ?
Best Regards
ian
1
http://markmail.org/search/?q=sling%20build%20stability#query:sling%20build%20stability%20list%3Aorg.apache.incubator.sling-dev+page:1+state:facets
2 https://builds.apache.org/job/sling-trunk-1.7/4270/
3 https://builds.apache.org/job/sling-trunk-1.7/4271/
4 https://builds.apache.org/job/sling-trunk-1.7/4271/changes#detail0
Re: Build stability/time.
Posted by Ian Boston <ie...@tfd.co.uk>.
Hi,
I upgraded to mvn 3.3.9 using JDK 8 and now the earlier tests pass first
time, so there was something there with mvn 3.0.x
but Sling Event Support fails after just under 1h. The bulk of projects
upto that point take < 1min to build. Next longest is 3:15 min
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.648 sec
- in org.apache.sling.event.it.UnorderedQueueTest
Results
Tests in error:
ChaosTest.testDoChaos:390->AbstractJobHandlingTest.sleep:209 » test
timed out...
....
[INFO] Apache Sling Scripting Sightly Integration Tests ... SUCCESS [03:15
min]
.....
[INFO] Apache Sling Event Support ......................... FAILURE [25:10
min]
....
[INFO] Total time: 53:29 min
My machine is old, but not that old.
OSX 10.11.5, 8GB RAM, SATA2 SSD benchmarking at about 50K IOPS with a 4
core 2.5GHz processor. Nothing else running.
Best Regards
Ian
On 12 July 2016 at 16:47, Robert Munteanu <ro...@apache.org> wrote:
> On Tue, 2016-07-12 at 15:23 +0100, Ian Boston wrote:
> > Hi,
> >
> > After 3 mvn clean install -rf :org.apache.sling.commons.threads it
> > passed
> > the unit tests.
> >
> > x43543:sling ieb$ mvn -version
> > Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-
> > 02-19
> > 13:51:28+0000)
> > Maven home: /usr/local/maven
> > Java version: 1.7.0_71, vendor: Oracle Corporation
> > Java home:
> > /Library/Java/JavaVirtualMachines/jdk1.7.0_71.jdk/Contents/Home/jre
> > Default locale: en_US, platform encoding: UTF-8
> > OS name: "mac os x", version: "10.11.5", arch: "x86_64", family:
> > "mac"
> > x43543:sling ieb$
> >
> > Does maven need to be 3.3 ?
>
> I don't think it's the Maven version, but maybe some tests are
> (slightly) time-sensitive. I think that the various combinations of
> OS/Memory/Disk speed that we have as developers, plus the fact that
> usually the Jenkins setup has lower performance leads to flakiness in
> test executions.
>
> Robert
>
> >
> > Best Regards
> >
> > Ian
> >
> >
> >
> > On 12 July 2016 at 15:13, Carsten Ziegeler <cz...@apache.org>
> > wrote:
> >
> > > No errors here, Java 7, maven 3.3.9
> > >
> > > Carsten
> > >
> > > > Hi,
> > > > Latest fail.
> > > > If I do mvn clean install -rf :org.apache.sling.commons.threads I
> > > > get a
> > > > different fail.
> > > >
> > > > Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> > > > 0.212 sec
> > > > <<< FAILURE! - in
> > > > org.apache.sling.commons.threads.impl.ThreadExpiringThreadPoolTes
> > > > t
> > > >
> > > >
> > > shouldCreateNewThreadAfterExpiryForFailingTasks(org.apache.sling.co
> > > mmons.threads.impl.ThreadExpiringThreadPoolTest)
> > > > Time elapsed: 0.035 sec <<< FAILURE!
> > > >
> > > > java.lang.AssertionError:
> > > >
> > > > Expected: is <1>
> > > >
> > > > but: was <2>
> > > >
> > > > at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
> > > >
> > > > at org.junit.Assert.assertThat(Assert.java:956)
> > > >
> > > > at org.junit.Assert.assertThat(Assert.java:923)
> > > >
> > > > at
> > > >
> > > org.apache.sling.commons.threads.impl.ThreadExpiringThreadPoolTest.
> > > shouldCreateNewThreadAfterExpiryForFailingTasks(ThreadExpiringThrea
> > > dPoolTest.java:92)
> > > >
> > > >
> > > >
> > > > Results :
> > > >
> > > >
> > > > Failed tests:
> > > >
> > > >
> > > >
> > > ThreadExpiringThreadPoolTest.shouldCreateNewThreadAfterExpiryForFai
> > > lingTasks:92
> > > >
> > > > Expected: is <1>
> > > >
> > > > but: was <2>
> > > >
> > > >
> > > > Tests run: 8, Failures: 1, Errors: 0, Skipped: 0
> > > >
> > > >
> > > >
> > > > On 12 July 2016 at 15:02, Ian Boston <ie...@tfd.co.uk> wrote:
> > > >
> > > > > Hi,
> > > > > I am seeing a local build is unstable.
> > > > >
> > > > > Been trying for 2 days now. Updating from trunk and recently
> > > > > trying to
> > > > > take "working" build commits from Jenkins.
> > > > >
> > > > > Perhaps something is wrong with my maven/jdk install, but
> > > > > normally when
> > > a
> > > > > build fails in a Unit test, its because the unit tests has
> > > > > failed and
> > > not a
> > > > > maven/JDK fail. maven/JDK fails a generally complete failures
> > > > > in
> > > starting
> > > > > maven or compile.
> > > > > Best Regards
> > > > > Ian
> > > > >
> > > > >
> > > > > On 12 July 2016 at 14:55, Carsten Ziegeler <cziegeler@apache.or
> > > > > g>
> > > wrote:
> > > > >
> > > > > > I can only speak for myself, but usually everything builds
> > > > > > totally fine
> > > > > > locally. Too often something on the build infrastructure goes
> > > > > > wrong
> > > > > >
> > > > > > Carsten
> > > > > > > Hi,
> > > > > > >
> > > > > > >
> > > > > > > According to jenkins 1 in 10 builds fail, and some times
> > > > > > > randomly. For
> > > > > > > instance, [2] succeeded but [3] failed, due to [4], a patch
> > > > > > > to a
> > > > > > README.md
> > > > > > >
> > > > > > > According to [1] its not been discussed.
> > > > > > > How are others dealing with this ? (-Dmaven.test.skip=true
> > > > > > > or not
> > > > > > bothering
> > > > > > > to build all before committing)
> > > > > > >
> > > > > > > I assume only the test methodology is unreliable, and not
> > > > > > > the released
> > > > > > > artifacts ?
> > > > > > > Could the unreliable tests be moved to ITs ?
> > > > > > > Could the the long running tests also be moved to ITs ?
> > > > > > >
> > > > > > > Best Regards
> > > > > > > ian
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > 1
> > > > > > >
> > > > > >
> > > http://markmail.org/search/?q=sling%20build%20stability#query:sling
> > > %20build%20stability%20list%3Aorg.apache.incubator.sling-
> > > dev+page:1+state:facets
> > > > > > > 2 https://builds.apache.org/job/sling-trunk-1.7/4270/
> > > > > > > 3 https://builds.apache.org/job/sling-trunk-1.7/4271/
> > > > > > > 4 https://builds.apache.org/job/sling-trunk-1.7/4271/change
> > > > > > > s#detail0
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Carsten Ziegeler
> > > > > > Adobe Research Switzerland
> > > > > > cziegeler@apache.org
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > >
> > > --
> > > Carsten Ziegeler
> > > Adobe Research Switzerland
> > > cziegeler@apache.org
> > >
> > >
>
>
Re: Build stability/time.
Posted by Robert Munteanu <ro...@apache.org>.
On Tue, 2016-07-12 at 15:23 +0100, Ian Boston wrote:
> Hi,
>
> After 3 mvn clean install -rf :org.apache.sling.commons.threads it
> passed
> the unit tests.
>
> x43543:sling ieb$ mvn -version
> Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-
> 02-19
> 13:51:28+0000)
> Maven home: /usr/local/maven
> Java version: 1.7.0_71, vendor: Oracle Corporation
> Java home:
> /Library/Java/JavaVirtualMachines/jdk1.7.0_71.jdk/Contents/Home/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.11.5", arch: "x86_64", family:
> "mac"
> x43543:sling ieb$
>
> Does maven need to be 3.3 ?
I don't think it's the Maven version, but maybe some tests are
(slightly) time-sensitive. I think that the various combinations of
OS/Memory/Disk speed that we have as developers, plus the fact that
usually the Jenkins setup has lower performance leads to flakiness in
test executions.
Robert
>
> Best Regards
>
> Ian
>
>
>
> On 12 July 2016 at 15:13, Carsten Ziegeler <cz...@apache.org>
> wrote:
>
> > No errors here, Java 7, maven 3.3.9
> >
> > Carsten
> >
> > > Hi,
> > > Latest fail.
> > > If I do mvn clean install -rf :org.apache.sling.commons.threads I
> > > get a
> > > different fail.
> > >
> > > Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> > > 0.212 sec
> > > <<< FAILURE! - in
> > > org.apache.sling.commons.threads.impl.ThreadExpiringThreadPoolTes
> > > t
> > >
> > >
> > shouldCreateNewThreadAfterExpiryForFailingTasks(org.apache.sling.co
> > mmons.threads.impl.ThreadExpiringThreadPoolTest)
> > > Time elapsed: 0.035 sec��<<< FAILURE!
> > >
> > > java.lang.AssertionError:
> > >
> > > Expected: is <1>
> > >
> > > �����but: was <2>
> > >
> > > at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
> > >
> > > at org.junit.Assert.assertThat(Assert.java:956)
> > >
> > > at org.junit.Assert.assertThat(Assert.java:923)
> > >
> > > at
> > >
> > org.apache.sling.commons.threads.impl.ThreadExpiringThreadPoolTest.
> > shouldCreateNewThreadAfterExpiryForFailingTasks(ThreadExpiringThrea
> > dPoolTest.java:92)
> > >
> > >
> > >
> > > Results :
> > >
> > >
> > > Failed tests:
> > >
> > >
> > >
> > ThreadExpiringThreadPoolTest.shouldCreateNewThreadAfterExpiryForFai
> > lingTasks:92
> > >
> > > Expected: is <1>
> > >
> > > �����but: was <2>
> > >
> > >
> > > Tests run: 8, Failures: 1, Errors: 0, Skipped: 0
> > >
> > >
> > >
> > > On 12 July 2016 at 15:02, Ian Boston <ie...@tfd.co.uk> wrote:
> > >
> > > > Hi,
> > > > I am seeing a local build is unstable.
> > > >
> > > > Been trying for 2 days now. Updating from trunk and recently
> > > > trying to
> > > > take "working" build commits from Jenkins.
> > > >
> > > > Perhaps something is wrong with my maven/jdk install, but
> > > > normally when
> > a
> > > > build fails in a Unit test, its because the unit tests has
> > > > failed and
> > not a
> > > > maven/JDK fail. maven/JDK fails a generally complete failures
> > > > in
> > starting
> > > > maven or compile.
> > > > Best Regards
> > > > Ian
> > > >
> > > >
> > > > On 12 July 2016 at 14:55, Carsten Ziegeler <cziegeler@apache.or
> > > > g>
> > wrote:
> > > >
> > > > > I can only speak for myself, but usually everything builds
> > > > > totally fine
> > > > > locally. Too often something on the build infrastructure goes
> > > > > wrong
> > > > >
> > > > > Carsten
> > > > > > Hi,
> > > > > >
> > > > > >
> > > > > > According to jenkins 1 in 10 builds fail, and some times
> > > > > > randomly. For
> > > > > > instance, [2] succeeded but [3] failed, due to [4], a patch
> > > > > > to a
> > > > > README.md
> > > > > >
> > > > > > According to [1] its not been discussed.
> > > > > > How are others dealing with this ? (-Dmaven.test.skip=true
> > > > > > or not
> > > > > bothering
> > > > > > to build all before committing)
> > > > > >
> > > > > > I assume only the test methodology is unreliable, and not
> > > > > > the released
> > > > > > artifacts ?
> > > > > > Could the unreliable tests be moved to ITs ?
> > > > > > Could the the long running tests also be moved to ITs ?
> > > > > >
> > > > > > Best Regards
> > > > > > ian
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > 1
> > > > > >
> > > > >
> > http://markmail.org/search/?q=sling%20build%20stability#query:sling
> > %20build%20stability%20list%3Aorg.apache.incubator.sling-
> > dev+page:1+state:facets
> > > > > > 2 https://builds.apache.org/job/sling-trunk-1.7/4270/
> > > > > > 3 https://builds.apache.org/job/sling-trunk-1.7/4271/
> > > > > > 4 https://builds.apache.org/job/sling-trunk-1.7/4271/change
> > > > > > s#detail0
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Carsten Ziegeler
> > > > > Adobe Research Switzerland
> > > > > cziegeler@apache.org
> > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> >
> > --
> > Carsten Ziegeler
> > Adobe Research Switzerland
> > cziegeler@apache.org
> >
> >
Re: Build stability/time.
Posted by Ian Boston <ie...@tfd.co.uk>.
Hi,
After 3 mvn clean install -rf :org.apache.sling.commons.threads it passed
the unit tests.
x43543:sling ieb$ mvn -version
Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19
13:51:28+0000)
Maven home: /usr/local/maven
Java version: 1.7.0_71, vendor: Oracle Corporation
Java home:
/Library/Java/JavaVirtualMachines/jdk1.7.0_71.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.11.5", arch: "x86_64", family: "mac"
x43543:sling ieb$
Does maven need to be 3.3 ?
Best Regards
Ian
On 12 July 2016 at 15:13, Carsten Ziegeler <cz...@apache.org> wrote:
> No errors here, Java 7, maven 3.3.9
>
> Carsten
>
> > Hi,
> > Latest fail.
> > If I do mvn clean install -rf :org.apache.sling.commons.threads I get a
> > different fail.
> >
> > Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.212 sec
> > <<< FAILURE! - in
> > org.apache.sling.commons.threads.impl.ThreadExpiringThreadPoolTest
> >
> >
> shouldCreateNewThreadAfterExpiryForFailingTasks(org.apache.sling.commons.threads.impl.ThreadExpiringThreadPoolTest)
> > Time elapsed: 0.035 sec <<< FAILURE!
> >
> > java.lang.AssertionError:
> >
> > Expected: is <1>
> >
> > but: was <2>
> >
> > at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
> >
> > at org.junit.Assert.assertThat(Assert.java:956)
> >
> > at org.junit.Assert.assertThat(Assert.java:923)
> >
> > at
> >
> org.apache.sling.commons.threads.impl.ThreadExpiringThreadPoolTest.shouldCreateNewThreadAfterExpiryForFailingTasks(ThreadExpiringThreadPoolTest.java:92)
> >
> >
> >
> > Results :
> >
> >
> > Failed tests:
> >
> >
> >
> ThreadExpiringThreadPoolTest.shouldCreateNewThreadAfterExpiryForFailingTasks:92
> >
> > Expected: is <1>
> >
> > but: was <2>
> >
> >
> > Tests run: 8, Failures: 1, Errors: 0, Skipped: 0
> >
> >
> >
> > On 12 July 2016 at 15:02, Ian Boston <ie...@tfd.co.uk> wrote:
> >
> >> Hi,
> >> I am seeing a local build is unstable.
> >>
> >> Been trying for 2 days now. Updating from trunk and recently trying to
> >> take "working" build commits from Jenkins.
> >>
> >> Perhaps something is wrong with my maven/jdk install, but normally when
> a
> >> build fails in a Unit test, its because the unit tests has failed and
> not a
> >> maven/JDK fail. maven/JDK fails a generally complete failures in
> starting
> >> maven or compile.
> >> Best Regards
> >> Ian
> >>
> >>
> >> On 12 July 2016 at 14:55, Carsten Ziegeler <cz...@apache.org>
> wrote:
> >>
> >>> I can only speak for myself, but usually everything builds totally fine
> >>> locally. Too often something on the build infrastructure goes wrong
> >>>
> >>> Carsten
> >>>> Hi,
> >>>>
> >>>>
> >>>> According to jenkins 1 in 10 builds fail, and some times randomly. For
> >>>> instance, [2] succeeded but [3] failed, due to [4], a patch to a
> >>> README.md
> >>>>
> >>>> According to [1] its not been discussed.
> >>>> How are others dealing with this ? (-Dmaven.test.skip=true or not
> >>> bothering
> >>>> to build all before committing)
> >>>>
> >>>> I assume only the test methodology is unreliable, and not the released
> >>>> artifacts ?
> >>>> Could the unreliable tests be moved to ITs ?
> >>>> Could the the long running tests also be moved to ITs ?
> >>>>
> >>>> Best Regards
> >>>> ian
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> 1
> >>>>
> >>>
> http://markmail.org/search/?q=sling%20build%20stability#query:sling%20build%20stability%20list%3Aorg.apache.incubator.sling-dev+page:1+state:facets
> >>>> 2 https://builds.apache.org/job/sling-trunk-1.7/4270/
> >>>> 3 https://builds.apache.org/job/sling-trunk-1.7/4271/
> >>>> 4 https://builds.apache.org/job/sling-trunk-1.7/4271/changes#detail0
> >>>>
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> Carsten Ziegeler
> >>> Adobe Research Switzerland
> >>> cziegeler@apache.org
> >>>
> >>>
> >>
> >
>
>
>
>
> --
> Carsten Ziegeler
> Adobe Research Switzerland
> cziegeler@apache.org
>
>
Re: Build stability/time.
Posted by Carsten Ziegeler <cz...@apache.org>.
No errors here, Java 7, maven 3.3.9
Carsten
> Hi,
> Latest fail.
> If I do mvn clean install -rf :org.apache.sling.commons.threads I get a
> different fail.
>
> Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.212 sec
> <<< FAILURE! - in
> org.apache.sling.commons.threads.impl.ThreadExpiringThreadPoolTest
>
> shouldCreateNewThreadAfterExpiryForFailingTasks(org.apache.sling.commons.threads.impl.ThreadExpiringThreadPoolTest)
> Time elapsed: 0.035 sec <<< FAILURE!
>
> java.lang.AssertionError:
>
> Expected: is <1>
>
> but: was <2>
>
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
>
> at org.junit.Assert.assertThat(Assert.java:956)
>
> at org.junit.Assert.assertThat(Assert.java:923)
>
> at
> org.apache.sling.commons.threads.impl.ThreadExpiringThreadPoolTest.shouldCreateNewThreadAfterExpiryForFailingTasks(ThreadExpiringThreadPoolTest.java:92)
>
>
>
> Results :
>
>
> Failed tests:
>
>
> ThreadExpiringThreadPoolTest.shouldCreateNewThreadAfterExpiryForFailingTasks:92
>
> Expected: is <1>
>
> but: was <2>
>
>
> Tests run: 8, Failures: 1, Errors: 0, Skipped: 0
>
>
>
> On 12 July 2016 at 15:02, Ian Boston <ie...@tfd.co.uk> wrote:
>
>> Hi,
>> I am seeing a local build is unstable.
>>
>> Been trying for 2 days now. Updating from trunk and recently trying to
>> take "working" build commits from Jenkins.
>>
>> Perhaps something is wrong with my maven/jdk install, but normally when a
>> build fails in a Unit test, its because the unit tests has failed and not a
>> maven/JDK fail. maven/JDK fails a generally complete failures in starting
>> maven or compile.
>> Best Regards
>> Ian
>>
>>
>> On 12 July 2016 at 14:55, Carsten Ziegeler <cz...@apache.org> wrote:
>>
>>> I can only speak for myself, but usually everything builds totally fine
>>> locally. Too often something on the build infrastructure goes wrong
>>>
>>> Carsten
>>>> Hi,
>>>>
>>>>
>>>> According to jenkins 1 in 10 builds fail, and some times randomly. For
>>>> instance, [2] succeeded but [3] failed, due to [4], a patch to a
>>> README.md
>>>>
>>>> According to [1] its not been discussed.
>>>> How are others dealing with this ? (-Dmaven.test.skip=true or not
>>> bothering
>>>> to build all before committing)
>>>>
>>>> I assume only the test methodology is unreliable, and not the released
>>>> artifacts ?
>>>> Could the unreliable tests be moved to ITs ?
>>>> Could the the long running tests also be moved to ITs ?
>>>>
>>>> Best Regards
>>>> ian
>>>>
>>>>
>>>>
>>>>
>>>> 1
>>>>
>>> http://markmail.org/search/?q=sling%20build%20stability#query:sling%20build%20stability%20list%3Aorg.apache.incubator.sling-dev+page:1+state:facets
>>>> 2 https://builds.apache.org/job/sling-trunk-1.7/4270/
>>>> 3 https://builds.apache.org/job/sling-trunk-1.7/4271/
>>>> 4 https://builds.apache.org/job/sling-trunk-1.7/4271/changes#detail0
>>>>
>>>
>>>
>>>
>>>
>>> --
>>> Carsten Ziegeler
>>> Adobe Research Switzerland
>>> cziegeler@apache.org
>>>
>>>
>>
>
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: Build stability/time.
Posted by Ian Boston <ie...@tfd.co.uk>.
Hi,
Latest fail.
If I do mvn clean install -rf :org.apache.sling.commons.threads I get a
different fail.
Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.212 sec
<<< FAILURE! - in
org.apache.sling.commons.threads.impl.ThreadExpiringThreadPoolTest
shouldCreateNewThreadAfterExpiryForFailingTasks(org.apache.sling.commons.threads.impl.ThreadExpiringThreadPoolTest)
Time elapsed: 0.035 sec <<< FAILURE!
java.lang.AssertionError:
Expected: is <1>
but: was <2>
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
at org.junit.Assert.assertThat(Assert.java:956)
at org.junit.Assert.assertThat(Assert.java:923)
at
org.apache.sling.commons.threads.impl.ThreadExpiringThreadPoolTest.shouldCreateNewThreadAfterExpiryForFailingTasks(ThreadExpiringThreadPoolTest.java:92)
Results :
Failed tests:
ThreadExpiringThreadPoolTest.shouldCreateNewThreadAfterExpiryForFailingTasks:92
Expected: is <1>
but: was <2>
Tests run: 8, Failures: 1, Errors: 0, Skipped: 0
On 12 July 2016 at 15:02, Ian Boston <ie...@tfd.co.uk> wrote:
> Hi,
> I am seeing a local build is unstable.
>
> Been trying for 2 days now. Updating from trunk and recently trying to
> take "working" build commits from Jenkins.
>
> Perhaps something is wrong with my maven/jdk install, but normally when a
> build fails in a Unit test, its because the unit tests has failed and not a
> maven/JDK fail. maven/JDK fails a generally complete failures in starting
> maven or compile.
> Best Regards
> Ian
>
>
> On 12 July 2016 at 14:55, Carsten Ziegeler <cz...@apache.org> wrote:
>
>> I can only speak for myself, but usually everything builds totally fine
>> locally. Too often something on the build infrastructure goes wrong
>>
>> Carsten
>> > Hi,
>> >
>> >
>> > According to jenkins 1 in 10 builds fail, and some times randomly. For
>> > instance, [2] succeeded but [3] failed, due to [4], a patch to a
>> README.md
>> >
>> > According to [1] its not been discussed.
>> > How are others dealing with this ? (-Dmaven.test.skip=true or not
>> bothering
>> > to build all before committing)
>> >
>> > I assume only the test methodology is unreliable, and not the released
>> > artifacts ?
>> > Could the unreliable tests be moved to ITs ?
>> > Could the the long running tests also be moved to ITs ?
>> >
>> > Best Regards
>> > ian
>> >
>> >
>> >
>> >
>> > 1
>> >
>> http://markmail.org/search/?q=sling%20build%20stability#query:sling%20build%20stability%20list%3Aorg.apache.incubator.sling-dev+page:1+state:facets
>> > 2 https://builds.apache.org/job/sling-trunk-1.7/4270/
>> > 3 https://builds.apache.org/job/sling-trunk-1.7/4271/
>> > 4 https://builds.apache.org/job/sling-trunk-1.7/4271/changes#detail0
>> >
>>
>>
>>
>>
>> --
>> Carsten Ziegeler
>> Adobe Research Switzerland
>> cziegeler@apache.org
>>
>>
>
Re: Build stability/time.
Posted by Ian Boston <ie...@tfd.co.uk>.
Hi,
I am seeing a local build is unstable.
Been trying for 2 days now. Updating from trunk and recently trying to take
"working" build commits from Jenkins.
Perhaps something is wrong with my maven/jdk install, but normally when a
build fails in a Unit test, its because the unit tests has failed and not a
maven/JDK fail. maven/JDK fails a generally complete failures in starting
maven or compile.
Best Regards
Ian
On 12 July 2016 at 14:55, Carsten Ziegeler <cz...@apache.org> wrote:
> I can only speak for myself, but usually everything builds totally fine
> locally. Too often something on the build infrastructure goes wrong
>
> Carsten
> > Hi,
> >
> >
> > According to jenkins 1 in 10 builds fail, and some times randomly. For
> > instance, [2] succeeded but [3] failed, due to [4], a patch to a
> README.md
> >
> > According to [1] its not been discussed.
> > How are others dealing with this ? (-Dmaven.test.skip=true or not
> bothering
> > to build all before committing)
> >
> > I assume only the test methodology is unreliable, and not the released
> > artifacts ?
> > Could the unreliable tests be moved to ITs ?
> > Could the the long running tests also be moved to ITs ?
> >
> > Best Regards
> > ian
> >
> >
> >
> >
> > 1
> >
> http://markmail.org/search/?q=sling%20build%20stability#query:sling%20build%20stability%20list%3Aorg.apache.incubator.sling-dev+page:1+state:facets
> > 2 https://builds.apache.org/job/sling-trunk-1.7/4270/
> > 3 https://builds.apache.org/job/sling-trunk-1.7/4271/
> > 4 https://builds.apache.org/job/sling-trunk-1.7/4271/changes#detail0
> >
>
>
>
>
> --
> Carsten Ziegeler
> Adobe Research Switzerland
> cziegeler@apache.org
>
>
Re: Build stability/time.
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Tue, Jul 12, 2016 at 3:55 PM, Carsten Ziegeler <cz...@apache.org> wrote:
> ...I can only speak for myself, but usually everything builds totally fine
> locally....
I often see this as well.
Tests that fail semi-randomly should be tracked with the jira sling-IT
label, https://issues.apache.org/jira/issues/?jql=labels%20%3D%20sling-IT
I'm fine with moving slow (whatever that means) tests to ITs.
-Bertrand
Re: Build stability/time.
Posted by Carsten Ziegeler <cz...@apache.org>.
I can only speak for myself, but usually everything builds totally fine
locally. Too often something on the build infrastructure goes wrong
Carsten
> Hi,
>
>
> According to jenkins 1 in 10 builds fail, and some times randomly. For
> instance, [2] succeeded but [3] failed, due to [4], a patch to a README.md
>
> According to [1] its not been discussed.
> How are others dealing with this ? (-Dmaven.test.skip=true or not bothering
> to build all before committing)
>
> I assume only the test methodology is unreliable, and not the released
> artifacts ?
> Could the unreliable tests be moved to ITs ?
> Could the the long running tests also be moved to ITs ?
>
> Best Regards
> ian
>
>
>
>
> 1
> http://markmail.org/search/?q=sling%20build%20stability#query:sling%20build%20stability%20list%3Aorg.apache.incubator.sling-dev+page:1+state:facets
> 2 https://builds.apache.org/job/sling-trunk-1.7/4270/
> 3 https://builds.apache.org/job/sling-trunk-1.7/4271/
> 4 https://builds.apache.org/job/sling-trunk-1.7/4271/changes#detail0
>
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org