You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by Zoran Regvart <zo...@regvart.com> on 2018/05/09 09:45:14 UTC
Daily build to test with JDK 9 and 10
Hello Cameleers,
I've created a build that will run once every day to run tests on JDK
9 and 10[1], the configuration for the build is in
`Jenkinsfile.daily`[2].
Suggestions and contributions are welcome :)
zoran
[1] https://builds.apache.org/view/A-D/view/Camel/job/Camel.daily/
[2] https://gitbox.apache.org/repos/asf?p=camel.git;a=blob;f=Jenkinsfile.daily;hb=HEAD
--
Zoran Regvart
Re: Daily build to test with JDK 9 and 10
Posted by Willem Jiang <wi...@gmail.com>.
Not sure if we can leverage travis to do this kind of work.
As camel build is a little bit longer one by one, if we can run the
component tests in parallel that you could save us lot of time.
Willem Jiang
Blog: http://willemjiang.blogspot.com (English)
http://jnn.iteye.com (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On Thu, May 10, 2018 at 6:15 AM, Zoran Regvart <zo...@regvart.com> wrote:
> Hi Cameleers,
> so it's not that bad[1] 77 tests failed in total. I think we need to
> play with opening up some modules a bit. I'm a bit sad that it takes 9
> hours for the job to run, when we add JDK 11 I would expect it to take
> 13 hours, would be nice if there was a way to just run the tests in
> parallel on multiple JDKs.
>
> zoran
>
> [1] https://builds.apache.org/view/C/view/Apache%20Camel/job/Camel.daily/
> lastCompletedBuild/testReport/
>
> On Wed, May 9, 2018 at 11:45 AM, Zoran Regvart <zo...@regvart.com> wrote:
> > Hello Cameleers,
> > I've created a build that will run once every day to run tests on JDK
> > 9 and 10[1], the configuration for the build is in
> > `Jenkinsfile.daily`[2].
> >
> > Suggestions and contributions are welcome :)
> >
> > zoran
> >
> > [1] https://builds.apache.org/view/A-D/view/Camel/job/Camel.daily/
> > [2] https://gitbox.apache.org/repos/asf?p=camel.git;a=blob;
> f=Jenkinsfile.daily;hb=HEAD
> > --
> > Zoran Regvart
>
>
>
> --
> Zoran Regvart
>
Re: Daily build to test with JDK 9 and 10
Posted by Pascal Schumacher <pa...@gmx.net>.
Hi Zoran,
it is exactly as you said. Just parallelizing the test part of the build
did not work for the reasons you mentioned. Therefore I have reverted
the change.
- Pascal
Am 11.05.2018 um 00:05 schrieb Zoran Regvart:
> Hi Pascal,
> thank you for looking into this. I don't think that using parallel
> will help us, here's why: we're building on JDK 8 and we need to use
> binaries built on JDK 8. Running same tests in parallel will cause a
> lot of flakiness as they might use the same network ports or touch the
> same files. So that leaves us with copying the build workspace for
> each JDK we want to additionally test before the parallel step -- and
> that would be a lot of IO. That's why I mentioned a copy-on-write
> filesystem. With something like that, and Luca mentioned using docker,
> we could cheaply fork the file system to run the two stages in
> parallel.
>
> Hope that makes sense...
>
> zoran
>
> On Thu, May 10, 2018 at 4:10 PM, Pascal Schumacher
> <pa...@gmx.net> wrote:
>> It is great to have a build for JDK 9 and 10!
>>
>> (Next step is JDK 11, which is already available according to
>> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
>> :))
>>
>> Making the job executing the test for JDK 9 and JDK 10 in parallel was easy:
>>
>> https://github.com/apache/camel/commit/1c26b28145553e8db45f5b5412903d16ef1f6948
>>
>> The job is still running
>> (https://builds.apache.org/view/A-D/view/Camel/job/Camel.daily/17/flowGraphTable/),
>> but peeking into the logout I guess this easy solution will not work. Both
>> stages are executed on the same node and are therefore using the same file
>> system which causes problems/conflicts with e.g. Apache Derby.
>>
>> - Pascal
>>
>>
>> Am 10.05.2018 um 00:15 schrieb Zoran Regvart:
>>> Hi Cameleers,
>>> so it's not that bad[1] 77 tests failed in total. I think we need to
>>> play with opening up some modules a bit. I'm a bit sad that it takes 9
>>> hours for the job to run, when we add JDK 11 I would expect it to take
>>> 13 hours, would be nice if there was a way to just run the tests in
>>> parallel on multiple JDKs.
>>>
>>> zoran
>>>
>>> [1]
>>> https://builds.apache.org/view/C/view/Apache%20Camel/job/Camel.daily/lastCompletedBuild/testReport/
>>
>
>
Re: Daily build to test with JDK 9 and 10
Posted by Zoran Regvart <zo...@regvart.com>.
Hi Pascal,
thank you for looking into this. I don't think that using parallel
will help us, here's why: we're building on JDK 8 and we need to use
binaries built on JDK 8. Running same tests in parallel will cause a
lot of flakiness as they might use the same network ports or touch the
same files. So that leaves us with copying the build workspace for
each JDK we want to additionally test before the parallel step -- and
that would be a lot of IO. That's why I mentioned a copy-on-write
filesystem. With something like that, and Luca mentioned using docker,
we could cheaply fork the file system to run the two stages in
parallel.
Hope that makes sense...
zoran
On Thu, May 10, 2018 at 4:10 PM, Pascal Schumacher
<pa...@gmx.net> wrote:
> It is great to have a build for JDK 9 and 10!
>
> (Next step is JDK 11, which is already available according to
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> :))
>
> Making the job executing the test for JDK 9 and JDK 10 in parallel was easy:
>
> https://github.com/apache/camel/commit/1c26b28145553e8db45f5b5412903d16ef1f6948
>
> The job is still running
> (https://builds.apache.org/view/A-D/view/Camel/job/Camel.daily/17/flowGraphTable/),
> but peeking into the logout I guess this easy solution will not work. Both
> stages are executed on the same node and are therefore using the same file
> system which causes problems/conflicts with e.g. Apache Derby.
>
> - Pascal
>
>
> Am 10.05.2018 um 00:15 schrieb Zoran Regvart:
>>
>> Hi Cameleers,
>> so it's not that bad[1] 77 tests failed in total. I think we need to
>> play with opening up some modules a bit. I'm a bit sad that it takes 9
>> hours for the job to run, when we add JDK 11 I would expect it to take
>> 13 hours, would be nice if there was a way to just run the tests in
>> parallel on multiple JDKs.
>>
>> zoran
>>
>> [1]
>> https://builds.apache.org/view/C/view/Apache%20Camel/job/Camel.daily/lastCompletedBuild/testReport/
>
>
--
Zoran Regvart
Re: Daily build to test with JDK 9 and 10
Posted by Pascal Schumacher <pa...@gmx.net>.
It is great to have a build for JDK 9 and 10!
(Next step is JDK 11, which is already available according to
https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
:))
Making the job executing the test for JDK 9 and JDK 10 in parallel was easy:
https://github.com/apache/camel/commit/1c26b28145553e8db45f5b5412903d16ef1f6948
The job is still running
(https://builds.apache.org/view/A-D/view/Camel/job/Camel.daily/17/flowGraphTable/),
but peeking into the logout I guess this easy solution will not work.
Both stages are executed on the same node and are therefore using the
same file system which causes problems/conflicts with e.g. Apache Derby.
- Pascal
Am 10.05.2018 um 00:15 schrieb Zoran Regvart:
> Hi Cameleers,
> so it's not that bad[1] 77 tests failed in total. I think we need to
> play with opening up some modules a bit. I'm a bit sad that it takes 9
> hours for the job to run, when we add JDK 11 I would expect it to take
> 13 hours, would be nice if there was a way to just run the tests in
> parallel on multiple JDKs.
>
> zoran
>
> [1] https://builds.apache.org/view/C/view/Apache%20Camel/job/Camel.daily/lastCompletedBuild/testReport/
Re: Daily build to test with JDK 9 and 10
Posted by Zoran Regvart <zo...@regvart.com>.
Hi Cameleers,
so it's not that bad[1] 77 tests failed in total. I think we need to
play with opening up some modules a bit. I'm a bit sad that it takes 9
hours for the job to run, when we add JDK 11 I would expect it to take
13 hours, would be nice if there was a way to just run the tests in
parallel on multiple JDKs.
zoran
[1] https://builds.apache.org/view/C/view/Apache%20Camel/job/Camel.daily/lastCompletedBuild/testReport/
On Wed, May 9, 2018 at 11:45 AM, Zoran Regvart <zo...@regvart.com> wrote:
> Hello Cameleers,
> I've created a build that will run once every day to run tests on JDK
> 9 and 10[1], the configuration for the build is in
> `Jenkinsfile.daily`[2].
>
> Suggestions and contributions are welcome :)
>
> zoran
>
> [1] https://builds.apache.org/view/A-D/view/Camel/job/Camel.daily/
> [2] https://gitbox.apache.org/repos/asf?p=camel.git;a=blob;f=Jenkinsfile.daily;hb=HEAD
> --
> Zoran Regvart
--
Zoran Regvart
Re: Daily build to test with JDK 9 and 10
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Zoran,
it sounds good to me.
Thanks !
Regards
JB
On 09/05/2018 11:45, Zoran Regvart wrote:
> Hello Cameleers,
> I've created a build that will run once every day to run tests on JDK
> 9 and 10[1], the configuration for the build is in
> `Jenkinsfile.daily`[2].
>
> Suggestions and contributions are welcome :)
>
> zoran
>
> [1] https://builds.apache.org/view/A-D/view/Camel/job/Camel.daily/
> [2] https://gitbox.apache.org/repos/asf?p=camel.git;a=blob;f=Jenkinsfile.daily;hb=HEAD
>