You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Hugo Trippaers <hu...@trippaers.nl> on 2013/07/19 18:51:29 UTC
Reviewboard and jenkins
Hey all,
I've been working on some scripting to get patches submitted on review board automagically tested with Jenkins.
The way it works is that a script will check if the jenkins user on review board (catofong) has commented on the review. If it hasn't it will submit a job to jenkins.cloudstack.org with a patch parameter. When the job it ready it will add a comment to the review with the result of the build and the url of that particular build. If the build succeeds cato will set the ship it flag to indicate that it's ok.
I'm finishing up the tests with this script now, so before i move it to jenkins.cloudstack.org i would like to feedback if we think this would use useful.
An result of the test run can be seen here: https://reviews.apache.org/r/12762/
Cheers,
Hugo
Re: Reviewboard and jenkins
Posted by Hugo <hu...@trippaers.nl>.
Hey Chip,
Working on it! :-)
Cheers,
Hugo
On Friday, July 26, 2013, Chip Childers wrote:
> Hugo,
>
> Any chance you could add some logic to pick the right branch for the build
> test?
>
>
> On Mon, Jul 22, 2013 at 6:06 AM, Hugo Trippaers <hugo@trippaers.nl<javascript:;>>
> wrote:
>
> > The job is running on jenkins.cloudstack.org
> >
> > This is de job doing the work:
> >
> >
> http://jenkins.cloudstack.org/view/management/job/mgmt-build-reviewboard-requests/
> >
> > This is the job being executed with the patch:
> >
> >
> http://jenkins.cloudstack.org/view/master/job/cloudstack-master-with-patch/
> >
> > I really like the pipeline ideas, but we need to be careful with the
> > timing and the external factors influencing builds. If we run complex
> > chains there is a possibility that jobs will fail due to external
> factors.
> > This would reflect badly on reviewboard as that review would be marked as
> > "bad" for a reason that has nothing to do with the patch itself.
> >
> > For now i would like to stick with just the unit tests (master build) and
> > when this proves stable we can add other builds in the chain?
> >
> > Cheers,
> >
> > Hugo
> >
> >
> > On Jul 20, 2013, at 6:58 PM, Prasanna Santhanam <tsp@apache.org<javascript:;>>
> wrote:
> >
> > > On Sat, Jul 20, 2013 at 09:46:59AM -0400, David Nalley wrote:
> > >> On Sat, Jul 20, 2013 at 5:53 AM, Hugo Trippaers <trippie@gmail.com<javascript:;>
> >
> > wrote:
> > >>>
> > >>>
> > >>> Sent from my iPhone
> > >>>
> > >>> On 20 jul. 2013, at 10:09, Prasanna Santhanam <tsp@apache.org<javascript:;>>
> wrote:
> > >>>
> > >>>> On Sat, Jul 20, 2013 at 12:20:11AM +0200, Hugo Trippaers wrote:
> > >>>>>
> > >>>>> On Jul 19, 2013, at 7:15 PM, Daan Hoogland <
> daan.hoogland@gmail.com <javascript:;>>
> > wrote:
> > >>>>>
> > >>>>>> good stuff, does it run the risk of running while another instance
> > is busy?
> > >>>>>
> > >>>>> Probably not, it's a single threaded script. I intend to use
> jenkins
> > >>>>> to schedule it and i can tell jenkins not to start another instance
> > >>>>> of a job once one is running.
> > >>>> should set a "quiet period" for the job so that it doesn't kick up
> > >>>> jobs for each checkin but a group instead. we'll run out of
> executors
> > >>>> fast if that happens.
> > >>>
> > >>> Every time the script runs it will look at the newest 20 submissions
> > and process them in order. That should be ok for now.
> > >>>
> > >>
> > >> I actually wonder if we shouldn't pipeline this - e.g. - does the
> > >> patch apply
> > >> If it applies does it build (and have unit tests pass)
> > >> If it builds does it package
> > >> If it packages, do the integration tests pass (or given the long
> > >> running nature, perhaps we have stages of that as well)
> > >>
> > >> The first three are pretty easy to scale using on-demand resources.
> > >
> > > Yup that's what I'd like to do with this as well. The simulator
> > > pipeline that Ian is helping build will tie in very well with this.
> > >
> > >>
> > >>
> > >>>>
> > >>>>> There are probably some bug still in the script, but hey we got to
> > >>>>> start somewhere. ;-)
> > >>>>
> > >>>> Absolutely, could you put it up on cso-infra/github so we can hack
> it
> > >>>> and customize?
> > >>>
> > >>> Absolutely, will put it there this weekend if I have some time. I
> > don't have enough permissions to setup a new repo in CloudStack extras
> yet.
> > >>>
> > >>
> > >>
> > >> You do now.
> > >
> > > --
> > > Prasanna.,
> > >
> > > ------------------------
> > > Powered by BigRock.com
> > >
> >
> >
> >
>
--
Tsune ni ite, kyu ni awasu.
Hugo
hugo@strocamp.net
Re: Reviewboard and jenkins
Posted by Chip Childers <ch...@sungard.com>.
Hugo,
Any chance you could add some logic to pick the right branch for the build
test?
On Mon, Jul 22, 2013 at 6:06 AM, Hugo Trippaers <hu...@trippaers.nl> wrote:
> The job is running on jenkins.cloudstack.org
>
> This is de job doing the work:
>
> http://jenkins.cloudstack.org/view/management/job/mgmt-build-reviewboard-requests/
>
> This is the job being executed with the patch:
>
> http://jenkins.cloudstack.org/view/master/job/cloudstack-master-with-patch/
>
> I really like the pipeline ideas, but we need to be careful with the
> timing and the external factors influencing builds. If we run complex
> chains there is a possibility that jobs will fail due to external factors.
> This would reflect badly on reviewboard as that review would be marked as
> "bad" for a reason that has nothing to do with the patch itself.
>
> For now i would like to stick with just the unit tests (master build) and
> when this proves stable we can add other builds in the chain?
>
> Cheers,
>
> Hugo
>
>
> On Jul 20, 2013, at 6:58 PM, Prasanna Santhanam <ts...@apache.org> wrote:
>
> > On Sat, Jul 20, 2013 at 09:46:59AM -0400, David Nalley wrote:
> >> On Sat, Jul 20, 2013 at 5:53 AM, Hugo Trippaers <tr...@gmail.com>
> wrote:
> >>>
> >>>
> >>> Sent from my iPhone
> >>>
> >>> On 20 jul. 2013, at 10:09, Prasanna Santhanam <ts...@apache.org> wrote:
> >>>
> >>>> On Sat, Jul 20, 2013 at 12:20:11AM +0200, Hugo Trippaers wrote:
> >>>>>
> >>>>> On Jul 19, 2013, at 7:15 PM, Daan Hoogland <da...@gmail.com>
> wrote:
> >>>>>
> >>>>>> good stuff, does it run the risk of running while another instance
> is busy?
> >>>>>
> >>>>> Probably not, it's a single threaded script. I intend to use jenkins
> >>>>> to schedule it and i can tell jenkins not to start another instance
> >>>>> of a job once one is running.
> >>>> should set a "quiet period" for the job so that it doesn't kick up
> >>>> jobs for each checkin but a group instead. we'll run out of executors
> >>>> fast if that happens.
> >>>
> >>> Every time the script runs it will look at the newest 20 submissions
> and process them in order. That should be ok for now.
> >>>
> >>
> >> I actually wonder if we shouldn't pipeline this - e.g. - does the
> >> patch apply
> >> If it applies does it build (and have unit tests pass)
> >> If it builds does it package
> >> If it packages, do the integration tests pass (or given the long
> >> running nature, perhaps we have stages of that as well)
> >>
> >> The first three are pretty easy to scale using on-demand resources.
> >
> > Yup that's what I'd like to do with this as well. The simulator
> > pipeline that Ian is helping build will tie in very well with this.
> >
> >>
> >>
> >>>>
> >>>>> There are probably some bug still in the script, but hey we got to
> >>>>> start somewhere. ;-)
> >>>>
> >>>> Absolutely, could you put it up on cso-infra/github so we can hack it
> >>>> and customize?
> >>>
> >>> Absolutely, will put it there this weekend if I have some time. I
> don't have enough permissions to setup a new repo in CloudStack extras yet.
> >>>
> >>
> >>
> >> You do now.
> >
> > --
> > Prasanna.,
> >
> > ------------------------
> > Powered by BigRock.com
> >
>
>
>
Re: Reviewboard and jenkins
Posted by Prasanna Santhanam <ts...@apache.org>.
On Mon, Jul 22, 2013 at 12:06:31PM +0200, Hugo Trippaers wrote:
> The job is running on jenkins.cloudstack.org
>
> This is de job doing the work:
> http://jenkins.cloudstack.org/view/management/job/mgmt-build-reviewboard-requests/
>
> This is the job being executed with the patch:
> http://jenkins.cloudstack.org/view/master/job/cloudstack-master-with-patch/
>
> I really like the pipeline ideas, but we need to be careful with the
> timing and the external factors influencing builds. If we run
> complex chains there is a possibility that jobs will fail due to
> external factors. This would reflect badly on reviewboard as that
> review would be marked as "bad" for a reason that has nothing to do
> with the patch itself.
>
> For now i would like to stick with just the unit tests (master
> build) and when this proves stable we can add other builds in the
> chain?
>
Works for me. I'm wondering how long it is taking right now for a
patch to undergo full build + unittests on it. Saw quite a few
patches being PASSED by jenkins on rb.
Also I want to ensure that we don't 'Ship' the patch based on the
jenkins 'Ship it'. Only that the patch compiles successfully and is
good to be reviewed by a human reviewer. I know that's common sense,
but just wanted to spell it out.
Thanks,
--
Prasanna.,
------------------------
Powered by BigRock.com
Re: Reviewboard and jenkins
Posted by Hugo Trippaers <hu...@trippaers.nl>.
The job is running on jenkins.cloudstack.org
This is de job doing the work:
http://jenkins.cloudstack.org/view/management/job/mgmt-build-reviewboard-requests/
This is the job being executed with the patch:
http://jenkins.cloudstack.org/view/master/job/cloudstack-master-with-patch/
I really like the pipeline ideas, but we need to be careful with the timing and the external factors influencing builds. If we run complex chains there is a possibility that jobs will fail due to external factors. This would reflect badly on reviewboard as that review would be marked as "bad" for a reason that has nothing to do with the patch itself.
For now i would like to stick with just the unit tests (master build) and when this proves stable we can add other builds in the chain?
Cheers,
Hugo
On Jul 20, 2013, at 6:58 PM, Prasanna Santhanam <ts...@apache.org> wrote:
> On Sat, Jul 20, 2013 at 09:46:59AM -0400, David Nalley wrote:
>> On Sat, Jul 20, 2013 at 5:53 AM, Hugo Trippaers <tr...@gmail.com> wrote:
>>>
>>>
>>> Sent from my iPhone
>>>
>>> On 20 jul. 2013, at 10:09, Prasanna Santhanam <ts...@apache.org> wrote:
>>>
>>>> On Sat, Jul 20, 2013 at 12:20:11AM +0200, Hugo Trippaers wrote:
>>>>>
>>>>> On Jul 19, 2013, at 7:15 PM, Daan Hoogland <da...@gmail.com> wrote:
>>>>>
>>>>>> good stuff, does it run the risk of running while another instance is busy?
>>>>>
>>>>> Probably not, it's a single threaded script. I intend to use jenkins
>>>>> to schedule it and i can tell jenkins not to start another instance
>>>>> of a job once one is running.
>>>> should set a "quiet period" for the job so that it doesn't kick up
>>>> jobs for each checkin but a group instead. we'll run out of executors
>>>> fast if that happens.
>>>
>>> Every time the script runs it will look at the newest 20 submissions and process them in order. That should be ok for now.
>>>
>>
>> I actually wonder if we shouldn't pipeline this - e.g. - does the
>> patch apply
>> If it applies does it build (and have unit tests pass)
>> If it builds does it package
>> If it packages, do the integration tests pass (or given the long
>> running nature, perhaps we have stages of that as well)
>>
>> The first three are pretty easy to scale using on-demand resources.
>
> Yup that's what I'd like to do with this as well. The simulator
> pipeline that Ian is helping build will tie in very well with this.
>
>>
>>
>>>>
>>>>> There are probably some bug still in the script, but hey we got to
>>>>> start somewhere. ;-)
>>>>
>>>> Absolutely, could you put it up on cso-infra/github so we can hack it
>>>> and customize?
>>>
>>> Absolutely, will put it there this weekend if I have some time. I don't have enough permissions to setup a new repo in CloudStack extras yet.
>>>
>>
>>
>> You do now.
>
> --
> Prasanna.,
>
> ------------------------
> Powered by BigRock.com
>
Re: Reviewboard and jenkins
Posted by Prasanna Santhanam <ts...@apache.org>.
On Sat, Jul 20, 2013 at 09:46:59AM -0400, David Nalley wrote:
> On Sat, Jul 20, 2013 at 5:53 AM, Hugo Trippaers <tr...@gmail.com> wrote:
> >
> >
> > Sent from my iPhone
> >
> > On 20 jul. 2013, at 10:09, Prasanna Santhanam <ts...@apache.org> wrote:
> >
> >> On Sat, Jul 20, 2013 at 12:20:11AM +0200, Hugo Trippaers wrote:
> >>>
> >>> On Jul 19, 2013, at 7:15 PM, Daan Hoogland <da...@gmail.com> wrote:
> >>>
> >>>> good stuff, does it run the risk of running while another instance is busy?
> >>>
> >>> Probably not, it's a single threaded script. I intend to use jenkins
> >>> to schedule it and i can tell jenkins not to start another instance
> >>> of a job once one is running.
> >> should set a "quiet period" for the job so that it doesn't kick up
> >> jobs for each checkin but a group instead. we'll run out of executors
> >> fast if that happens.
> >
> > Every time the script runs it will look at the newest 20 submissions and process them in order. That should be ok for now.
> >
>
> I actually wonder if we shouldn't pipeline this - e.g. - does the
> patch apply
> If it applies does it build (and have unit tests pass)
> If it builds does it package
> If it packages, do the integration tests pass (or given the long
> running nature, perhaps we have stages of that as well)
>
> The first three are pretty easy to scale using on-demand resources.
Yup that's what I'd like to do with this as well. The simulator
pipeline that Ian is helping build will tie in very well with this.
>
>
> >>
> >>> There are probably some bug still in the script, but hey we got to
> >>> start somewhere. ;-)
> >>
> >> Absolutely, could you put it up on cso-infra/github so we can hack it
> >> and customize?
> >
> > Absolutely, will put it there this weekend if I have some time. I don't have enough permissions to setup a new repo in CloudStack extras yet.
> >
>
>
> You do now.
--
Prasanna.,
------------------------
Powered by BigRock.com
Re: Reviewboard and jenkins
Posted by David Nalley <da...@gnsa.us>.
On Sat, Jul 20, 2013 at 5:53 AM, Hugo Trippaers <tr...@gmail.com> wrote:
>
>
> Sent from my iPhone
>
> On 20 jul. 2013, at 10:09, Prasanna Santhanam <ts...@apache.org> wrote:
>
>> On Sat, Jul 20, 2013 at 12:20:11AM +0200, Hugo Trippaers wrote:
>>>
>>> On Jul 19, 2013, at 7:15 PM, Daan Hoogland <da...@gmail.com> wrote:
>>>
>>>> good stuff, does it run the risk of running while another instance is busy?
>>>
>>> Probably not, it's a single threaded script. I intend to use jenkins
>>> to schedule it and i can tell jenkins not to start another instance
>>> of a job once one is running.
>> should set a "quiet period" for the job so that it doesn't kick up
>> jobs for each checkin but a group instead. we'll run out of executors
>> fast if that happens.
>
> Every time the script runs it will look at the newest 20 submissions and process them in order. That should be ok for now.
>
I actually wonder if we shouldn't pipeline this - e.g. - does the patch apply
If it applies does it build (and have unit tests pass)
If it builds does it package
If it packages, do the integration tests pass (or given the long
running nature, perhaps we have stages of that as well)
The first three are pretty easy to scale using on-demand resources.
>>
>>> There are probably some bug still in the script, but hey we got to
>>> start somewhere. ;-)
>>
>> Absolutely, could you put it up on cso-infra/github so we can hack it
>> and customize?
>
> Absolutely, will put it there this weekend if I have some time. I don't have enough permissions to setup a new repo in CloudStack extras yet.
>
You do now.
Re: Reviewboard and jenkins
Posted by Hugo Trippaers <tr...@gmail.com>.
Sent from my iPhone
On 20 jul. 2013, at 10:09, Prasanna Santhanam <ts...@apache.org> wrote:
> On Sat, Jul 20, 2013 at 12:20:11AM +0200, Hugo Trippaers wrote:
>>
>> On Jul 19, 2013, at 7:15 PM, Daan Hoogland <da...@gmail.com> wrote:
>>
>>> good stuff, does it run the risk of running while another instance is busy?
>>
>> Probably not, it's a single threaded script. I intend to use jenkins
>> to schedule it and i can tell jenkins not to start another instance
>> of a job once one is running.
> should set a "quiet period" for the job so that it doesn't kick up
> jobs for each checkin but a group instead. we'll run out of executors
> fast if that happens.
Every time the script runs it will look at the newest 20 submissions and process them in order. That should be ok for now.
>
>> There are probably some bug still in the script, but hey we got to
>> start somewhere. ;-)
>
> Absolutely, could you put it up on cso-infra/github so we can hack it
> and customize?
Absolutely, will put it there this weekend if I have some time. I don't have enough permissions to setup a new repo in CloudStack extras yet.
Cheers,
Hugo
>
> --
> Prasanna.,
>
> ------------------------
> Powered by BigRock.com
>
Re: Reviewboard and jenkins
Posted by Prasanna Santhanam <ts...@apache.org>.
On Sat, Jul 20, 2013 at 12:20:11AM +0200, Hugo Trippaers wrote:
>
> On Jul 19, 2013, at 7:15 PM, Daan Hoogland <da...@gmail.com> wrote:
>
> > good stuff, does it run the risk of running while another instance is busy?
>
> Probably not, it's a single threaded script. I intend to use jenkins
> to schedule it and i can tell jenkins not to start another instance
> of a job once one is running.
>
should set a "quiet period" for the job so that it doesn't kick up
jobs for each checkin but a group instead. we'll run out of executors
fast if that happens.
> There are probably some bug still in the script, but hey we got to
> start somewhere. ;-)
>
Absolutely, could you put it up on cso-infra/github so we can hack it
and customize?
--
Prasanna.,
------------------------
Powered by BigRock.com
Re: Reviewboard and jenkins
Posted by Hugo Trippaers <hu...@trippaers.nl>.
On Jul 19, 2013, at 7:15 PM, Daan Hoogland <da...@gmail.com> wrote:
> good stuff, does it run the risk of running while another instance is busy?
Probably not, it's a single threaded script. I intend to use jenkins to schedule it and i can tell jenkins not to start another instance of a job once one is running.
There are probably some bug still in the script, but hey we got to start somewhere. ;-)
Cheers,
Hugo
>
>
> On Fri, Jul 19, 2013 at 6:57 PM, Prasanna Santhanam <ts...@apache.org> wrote:
>
>> On Fri, Jul 19, 2013 at 06:51:29PM +0200, Hugo Trippaers wrote:
>>> Hey all,
>>>
>>> I've been working on some scripting to get patches submitted on review
>> board automagically tested with Jenkins.
>>>
>>> The way it works is that a script will check if the jenkins user on
>>> review board (catofong) has commented on the review. If it hasn't it
>>> will submit a job to jenkins.cloudstack.org with a patch parameter.
>>> When the job it ready it will add a comment to the review with the
>>> result of the build and the url of that particular build. If the
>>> build succeeds cato will set the ship it flag to indicate that it's
>>> ok.
>>>
>>> I'm finishing up the tests with this script now, so before i move it
>>> to jenkins.cloudstack.org i would like to feedback if we think this
>>> would use useful.
>>>
>>> An result of the test run can be seen here:
>>> https://reviews.apache.org/r/12762/
>>>
>>> Cheers,
>>>
>>> Hugo
>>
>> YES! YES! YES! and link it to the checkin tests coming up in the
>> `simulator` view on jenkins.cloudstack.org.
>>
>> --
>> Prasanna.,
>>
>> ------------------------
>> Powered by BigRock.com
>>
>>
Re: Reviewboard and jenkins
Posted by Daan Hoogland <da...@gmail.com>.
good stuff, does it run the risk of running while another instance is busy?
On Fri, Jul 19, 2013 at 6:57 PM, Prasanna Santhanam <ts...@apache.org> wrote:
> On Fri, Jul 19, 2013 at 06:51:29PM +0200, Hugo Trippaers wrote:
> > Hey all,
> >
> > I've been working on some scripting to get patches submitted on review
> board automagically tested with Jenkins.
> >
> > The way it works is that a script will check if the jenkins user on
> > review board (catofong) has commented on the review. If it hasn't it
> > will submit a job to jenkins.cloudstack.org with a patch parameter.
> > When the job it ready it will add a comment to the review with the
> > result of the build and the url of that particular build. If the
> > build succeeds cato will set the ship it flag to indicate that it's
> > ok.
> >
> > I'm finishing up the tests with this script now, so before i move it
> > to jenkins.cloudstack.org i would like to feedback if we think this
> > would use useful.
> >
> > An result of the test run can be seen here:
> > https://reviews.apache.org/r/12762/
> >
> > Cheers,
> >
> > Hugo
>
> YES! YES! YES! and link it to the checkin tests coming up in the
> `simulator` view on jenkins.cloudstack.org.
>
> --
> Prasanna.,
>
> ------------------------
> Powered by BigRock.com
>
>
Re: Reviewboard and jenkins
Posted by Prasanna Santhanam <ts...@apache.org>.
On Fri, Jul 19, 2013 at 06:51:29PM +0200, Hugo Trippaers wrote:
> Hey all,
>
> I've been working on some scripting to get patches submitted on review board automagically tested with Jenkins.
>
> The way it works is that a script will check if the jenkins user on
> review board (catofong) has commented on the review. If it hasn't it
> will submit a job to jenkins.cloudstack.org with a patch parameter.
> When the job it ready it will add a comment to the review with the
> result of the build and the url of that particular build. If the
> build succeeds cato will set the ship it flag to indicate that it's
> ok.
>
> I'm finishing up the tests with this script now, so before i move it
> to jenkins.cloudstack.org i would like to feedback if we think this
> would use useful.
>
> An result of the test run can be seen here:
> https://reviews.apache.org/r/12762/
>
> Cheers,
>
> Hugo
YES! YES! YES! and link it to the checkin tests coming up in the
`simulator` view on jenkins.cloudstack.org.
--
Prasanna.,
------------------------
Powered by BigRock.com
Re: Reviewboard and jenkins
Posted by Chip Childers <ch...@sungard.com>.
Freaking awesome.
I saw that hit the ML, and was just about to ask about it.
On Fri, Jul 19, 2013 at 12:51 PM, Hugo Trippaers <hu...@trippaers.nl> wrote:
> Hey all,
>
> I've been working on some scripting to get patches submitted on review
> board automagically tested with Jenkins.
>
> The way it works is that a script will check if the jenkins user on review
> board (catofong) has commented on the review. If it hasn't it will submit a
> job to jenkins.cloudstack.org with a patch parameter. When the job it
> ready it will add a comment to the review with the result of the build and
> the url of that particular build. If the build succeeds cato will set the
> ship it flag to indicate that it's ok.
>
> I'm finishing up the tests with this script now, so before i move it to
> jenkins.cloudstack.org i would like to feedback if we think this would
> use useful.
>
> An result of the test run can be seen here:
> https://reviews.apache.org/r/12762/
>
> Cheers,
>
> Hugo
>
>
>