Automated testing of pull requests?

[Pascal Schumacher <pa...@gmx.net>]

Hi everybody,

what about setting up automated testing of pull request?

The build server automatically build the pull request and updates it 
with the results: e.g. https://github.com/apache/commons-lang/pull/119 
https://github.com/apache/groovy/pull/214

This gives the person submitting the pull request fast feedback and 
allows him to fix any errors.

Also the person merging the pull request can see if the merge will cause 
any problems.

Setting this up is easy.

You can either use builds.apache.org, where you set up job like this 
one: https://builds.apache.org/view/Groovy/job/Groovy%20Github%20PRs/ 
and create a infra ticket to activate the github integration.

Or you can use travis-ci, if you add a .travis.yml file to the repo and 
create an infra ticket to enable travis-ci integration.

What do you thinks?

Cheers,
Pascal

Re: Automated testing of pull requests?

[Kevin Earls <ke...@redhat.com>]

Hi Claus,

I have been looking at doing something similar on the Fuse Jenkins
(see camel-2.15.0.redhat-6-3-x-checkin-incremental and ENTESB-4481) for our
next release as the newer versions of Jenkins support incremental builds.
I haven't spent a lot of time on it yet but so far it looks like most Camel
build take 1-1.5 hours instead of the usual 4 or so.

I'm not sure yet if we can turn this into a PR job but if I get it
straightened out the same thing could be done upstream.

Cheers,

Kevin

On Mon, Dec 21, 2015 at 7:47 AM, Claus Ibsen <cl...@gmail.com> wrote:

> Hi
>
> There is about 16.000+ unit tests in the Camel source code. And it
> takes like 4-6 hours to run depending on computer power. And a of the
> test may fail due "port number in use" issues etc.
>
> However if the PR is only changing files in a component (which most of
> them really are) then we could run the tests only for that component -
> which the person doing the PR should really also have done. Then the
> testing is feasible to run. I wonder if that would be possible, eg if
> it can somehow detect which camel component the PR is only affected,
> and only do the test/build of that, so if you chance camel-quartz2,
> then it does a
>
> cd components
> cd camel-quartz2
> mvn clean install
>
> or something to only unit test that.
>
> Not sure if it would need to build SNAPSHOT dependencies of the entire
> project first to have them up to date, eg so camel-core etc is build
> prior? As we have so many components that takes like 30 min to do,
>
> If so its 2 steps
>
> mvn install -Pfastinstall
> cd components
>
> cd camel-quartz2
> mvn clean install
>
>
>
> And we can also run the checkstyle rule before hand also to catch code
> formatting issues.
>
>
> On Sun, Dec 20, 2015 at 11:38 AM, Pascal Schumacher
> <pa...@gmx.net> wrote:
> > Hi everybody,
> >
> > what about setting up automated testing of pull request?
> >
> > The build server automatically build the pull request and updates it with
> > the results: e.g. https://github.com/apache/commons-lang/pull/119
> > https://github.com/apache/groovy/pull/214
> >
> > This gives the person submitting the pull request fast feedback and
> allows
> > him to fix any errors.
> >
> > Also the person merging the pull request can see if the merge will cause
> any
> > problems.
> >
> > Setting this up is easy.
> >
> > You can either use builds.apache.org, where you set up job like this
> one:
> > https://builds.apache.org/view/Groovy/job/Groovy%20Github%20PRs/ and
> create
> > a infra ticket to activate the github integration.
> >
> > Or you can use travis-ci, if you add a .travis.yml file to the repo and
> > create an infra ticket to enable travis-ci integration.
> >
> > What do you thinks?
> >
> > Cheers,
> > Pascal
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>

Re: Automated testing of pull requests?

[Claus Ibsen <cl...@gmail.com>]

Hi

There is about 16.000+ unit tests in the Camel source code. And it
takes like 4-6 hours to run depending on computer power. And a of the
test may fail due "port number in use" issues etc.

However if the PR is only changing files in a component (which most of
them really are) then we could run the tests only for that component -
which the person doing the PR should really also have done. Then the
testing is feasible to run. I wonder if that would be possible, eg if
it can somehow detect which camel component the PR is only affected,
and only do the test/build of that, so if you chance camel-quartz2,
then it does a

cd components
cd camel-quartz2
mvn clean install

or something to only unit test that.

Not sure if it would need to build SNAPSHOT dependencies of the entire
project first to have them up to date, eg so camel-core etc is build
prior? As we have so many components that takes like 30 min to do,

If so its 2 steps

mvn install -Pfastinstall
cd components

cd camel-quartz2
mvn clean install



And we can also run the checkstyle rule before hand also to catch code
formatting issues.


On Sun, Dec 20, 2015 at 11:38 AM, Pascal Schumacher
<pa...@gmx.net> wrote:
> Hi everybody,
>
> what about setting up automated testing of pull request?
>
> The build server automatically build the pull request and updates it with
> the results: e.g. https://github.com/apache/commons-lang/pull/119
> https://github.com/apache/groovy/pull/214
>
> This gives the person submitting the pull request fast feedback and allows
> him to fix any errors.
>
> Also the person merging the pull request can see if the merge will cause any
> problems.
>
> Setting this up is easy.
>
> You can either use builds.apache.org, where you set up job like this one:
> https://builds.apache.org/view/Groovy/job/Groovy%20Github%20PRs/ and create
> a infra ticket to activate the github integration.
>
> Or you can use travis-ci, if you add a .travis.yml file to the repo and
> create an infra ticket to enable travis-ci integration.
>
> What do you thinks?
>
> Cheers,
> Pascal



-- 
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2