You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cayenne.apache.org by Andrus Adamchik <an...@objectstyle.org> on 2014/10/23 13:09:19 UTC
Pull requests on Jenkins
https://builds.apache.org/job/Cayenne%20pull%20requests/
Didn't know we had those setup on Jenkins. Was it us or was it done by infra?
And too bad we have ongoing Jenkins flakiness with regular builds; otherwise this tool would have been very useful.
Andrus
Re: Pull requests on Jenkins
Posted by Andrus Adamchik <an...@objectstyle.org>.
On Oct 24, 2014, at 1:14 AM, Aristedes Maniatis <ar...@maniatis.org> wrote:
> I assume you mean adding setUp where needed.
It is already there. The DB cleanup algorithm in it is not always reliable.
> In the meantime we can upgrade to junit 4 pretty easily. I started writing some regex just now, and then stumbled on this:
Sorry, I didn't explain well - we are already on Junit 4. We just need to take advantage of the features.
Andrus
Re: Pull requests on Jenkins
Posted by Mike Kienenberger <mk...@gmail.com>.
I still think testng is a better choice.
And it will run junit3 tests as well.
On Thu, Oct 23, 2014 at 6:14 PM, Aristedes Maniatis <ar...@maniatis.org> wrote:
> On 24/10/2014 3:58am, Andrus Adamchik wrote:
>> No. Resetting the DB is currently hardcoded per test, so due to subtle bugs in copy/paste of the boilerplate code it is order-dependent. Redoing this (esp. in the view of Junit 4 upgrade) is on the list. Until then we'd chase them one by one. Considering that we have hundreds of tests, this is a losing battle.
>
> I assume you mean adding setUp where needed.
>
> In the meantime we can upgrade to junit 4 pretty easily. I started writing some regex just now, and then stumbled on this:
>
> http://stackoverflow.com/questions/264680/best-way-to-automagically-migrate-tests-from-junit-3-to-junit-4
>
> Unless anyone objects, I might knock this one over when I'm not busy one day.
>
>
> Ari
>
> --
> -------------------------->
> Aristedes Maniatis
> GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
Re: Pull requests on Jenkins
Posted by Aristedes Maniatis <ar...@maniatis.org>.
On 24/10/2014 3:58am, Andrus Adamchik wrote:
> No. Resetting the DB is currently hardcoded per test, so due to subtle bugs in copy/paste of the boilerplate code it is order-dependent. Redoing this (esp. in the view of Junit 4 upgrade) is on the list. Until then we'd chase them one by one. Considering that we have hundreds of tests, this is a losing battle.
I assume you mean adding setUp where needed.
In the meantime we can upgrade to junit 4 pretty easily. I started writing some regex just now, and then stumbled on this:
http://stackoverflow.com/questions/264680/best-way-to-automagically-migrate-tests-from-junit-3-to-junit-4
Unless anyone objects, I might knock this one over when I'm not busy one day.
Ari
--
-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
Re: Pull requests on Jenkins
Posted by Andrus Adamchik <an...@objectstyle.org>.
> OK, what I did:
Awesome.
> This error looks a lot like a race condition between the setup of the test database and the order the tests run in.
Most certainly.
> Are we sure:
>
> * the tests always run in the same order
No. Resetting the DB is currently hardcoded per test, so due to subtle bugs in copy/paste of the boilerplate code it is order-dependent. Redoing this (esp. in the view of Junit 4 upgrade) is on the list. Until then we'd chase them one by one. Considering that we have hundreds of tests, this is a losing battle.
> * the tests aren't running in parallel
At least within one mvn execution they are sequential. And I don't think hsql/derby in-memory would allow multiple processes to connect to the same DB.
Andrus
On Oct 23, 2014, at 7:13 PM, Aristedes Maniatis <ar...@maniatis.org> wrote:
> OK, what I did:
>
> 1. Added Java 8 to our builds
> https://builds.apache.org/job/cayenne-master/
>
> 2. Set up junit test failures to be reported more clearly
> https://builds.apache.org/job/Cayenne%20pull%20requests/lastBuild/testReport/
>
> 3. Derby/Java 8 is now always built first and if it fails, then the other matrix jobs are not attempted
>
> 4. Set up a job to build every pull request at github. That's just Java 8 with the default db (is that derby?)
> https://builds.apache.org/job/Cayenne%20pull%20requests
>
>
> For test failures like this one:
>
> https://builds.apache.org/job/cayenne-master/53/cayenneTestConnection=derby,jdk=jdk-1.8.0,label=Ubuntu/testReport/org.apache.cayenne.access/DataContextClobTest/testNullClob/
>
> Are we sure:
>
> * the tests always run in the same order
> * the tests aren't running in parallel
>
> This error looks a lot like a race condition between the setup of the test database and the order the tests run in.
>
>
> Ari
>
>
>
> On 24/10/2014 2:06am, Aristedes Maniatis wrote:
>> I just did that a few hours ago. Hadn't quite got around to telling everyone :-)
>>
>> I also added Java 8 to our regular builds and set up the unit tests to report more clearly. I was hoping to see why we get so many failures. I don't know if it is Jenkins or just race conditions in our tests.
>>
>> Ari
>>
>>
>> On 23/10/2014 10:09pm, Andrus Adamchik wrote:
>>> https://builds.apache.org/job/Cayenne%20pull%20requests/
>>>
>>> Didn't know we had those setup on Jenkins. Was it us or was it done by infra?
>>>
>>> And too bad we have ongoing Jenkins flakiness with regular builds; otherwise this tool would have been very useful.
>>>
>>> Andrus
>>>
>>
>
> --
> -------------------------->
> Aristedes Maniatis
> GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
>
Re: Pull requests on Jenkins
Posted by Aristedes Maniatis <ar...@maniatis.org>.
OK, what I did:
1. Added Java 8 to our builds
https://builds.apache.org/job/cayenne-master/
2. Set up junit test failures to be reported more clearly
https://builds.apache.org/job/Cayenne%20pull%20requests/lastBuild/testReport/
3. Derby/Java 8 is now always built first and if it fails, then the other matrix jobs are not attempted
4. Set up a job to build every pull request at github. That's just Java 8 with the default db (is that derby?)
https://builds.apache.org/job/Cayenne%20pull%20requests
For test failures like this one:
https://builds.apache.org/job/cayenne-master/53/cayenneTestConnection=derby,jdk=jdk-1.8.0,label=Ubuntu/testReport/org.apache.cayenne.access/DataContextClobTest/testNullClob/
Are we sure:
* the tests always run in the same order
* the tests aren't running in parallel
This error looks a lot like a race condition between the setup of the test database and the order the tests run in.
Ari
On 24/10/2014 2:06am, Aristedes Maniatis wrote:
> I just did that a few hours ago. Hadn't quite got around to telling everyone :-)
>
> I also added Java 8 to our regular builds and set up the unit tests to report more clearly. I was hoping to see why we get so many failures. I don't know if it is Jenkins or just race conditions in our tests.
>
> Ari
>
>
> On 23/10/2014 10:09pm, Andrus Adamchik wrote:
>> https://builds.apache.org/job/Cayenne%20pull%20requests/
>>
>> Didn't know we had those setup on Jenkins. Was it us or was it done by infra?
>>
>> And too bad we have ongoing Jenkins flakiness with regular builds; otherwise this tool would have been very useful.
>>
>> Andrus
>>
>
--
-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
Re: Pull requests on Jenkins
Posted by Aristedes Maniatis <ar...@maniatis.org>.
I just did that a few hours ago. Hadn't quite got around to telling everyone :-)
I also added Java 8 to our regular builds and set up the unit tests to report more clearly. I was hoping to see why we get so many failures. I don't know if it is Jenkins or just race conditions in our tests.
Ari
On 23/10/2014 10:09pm, Andrus Adamchik wrote:
> https://builds.apache.org/job/Cayenne%20pull%20requests/
>
> Didn't know we had those setup on Jenkins. Was it us or was it done by infra?
>
> And too bad we have ongoing Jenkins flakiness with regular builds; otherwise this tool would have been very useful.
>
> Andrus
>
--
-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A