You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by Konstantin Boudnik <co...@apache.org> on 2015/11/02 23:36:29 UTC
The state of new CI
Just in case it was missed before, I'd like to thank Evans for this amazing
work on our new CI infrastructure
http://ci.bigtop.apache.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-packages/
OMG! This is so great! I know Evans was doing a lot of it for sure, but I want
to thank everyone who spent time working on this or contributed otherwise!
Cos
Re: The state of new CI
Posted by Konstantin Boudnik <co...@apache.org>.
On Mon, Nov 02, 2015 at 02:51PM, Jonathan Kelly wrote:
> Very cool. Thanks a lot, Evans!
>
> BTW, two things:
>
> 1) Would it be possible to run this on port 80 instead of port 8080?
> Amazon's corporate firewall blocks outbound port 8080, so we can't view
> this CI while on the corporate network.
Shouldn't be a big deal. Evans, what do you think?
> 2) Is the setup/configuration of this Jenkins server documented/scripted
> somewhere in such a way that we might be able to duplicate it internally?
> We on EMR would really love to be able to have a Jenkins setup that more
> closely mirrors what the community is doing so that we can be sure that
> patches that we would like to contribute back are not going to break
> upstream.
We have an initial scripting work done under bigtop-ci/jenkins/, but I believe
it is quite behind of what we have today. Perhaps, the faster route is to copy
the jobs and Jenkins configs from ci.bigtop.a.o to your internal server?
Ideally, it'd be great to finish the groovy generation script to reflect the
current CI setup.
Cos
> ~ Jonathan
>
> On Mon, Nov 2, 2015 at 2:36 PM, Konstantin Boudnik <co...@apache.org> wrote:
>
> > Just in case it was missed before, I'd like to thank Evans for this amazing
> > work on our new CI infrastructure
> >
> >
> > http://ci.bigtop.apache.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-packages/
> >
> > OMG! This is so great! I know Evans was doing a lot of it for sure, but I
> > want
> > to thank everyone who spent time working on this or contributed otherwise!
> >
> > Cos
> >
Re: The state of new CI
Posted by Konstantin Boudnik <co...@apache.org>.
On Mon, Nov 02, 2015 at 02:51PM, Jonathan Kelly wrote:
> Very cool. Thanks a lot, Evans!
>
> BTW, two things:
>
> 1) Would it be possible to run this on port 80 instead of port 8080?
> Amazon's corporate firewall blocks outbound port 8080, so we can't view
> this CI while on the corporate network.
Shouldn't be a big deal. Evans, what do you think?
> 2) Is the setup/configuration of this Jenkins server documented/scripted
> somewhere in such a way that we might be able to duplicate it internally?
> We on EMR would really love to be able to have a Jenkins setup that more
> closely mirrors what the community is doing so that we can be sure that
> patches that we would like to contribute back are not going to break
> upstream.
We have an initial scripting work done under bigtop-ci/jenkins/, but I believe
it is quite behind of what we have today. Perhaps, the faster route is to copy
the jobs and Jenkins configs from ci.bigtop.a.o to your internal server?
Ideally, it'd be great to finish the groovy generation script to reflect the
current CI setup.
Cos
> ~ Jonathan
>
> On Mon, Nov 2, 2015 at 2:36 PM, Konstantin Boudnik <co...@apache.org> wrote:
>
> > Just in case it was missed before, I'd like to thank Evans for this amazing
> > work on our new CI infrastructure
> >
> >
> > http://ci.bigtop.apache.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-packages/
> >
> > OMG! This is so great! I know Evans was doing a lot of it for sure, but I
> > want
> > to thank everyone who spent time working on this or contributed otherwise!
> >
> > Cos
> >
Re: The state of new CI
Posted by Jonathan Kelly <jo...@gmail.com>.
Very cool. Thanks a lot, Evans!
BTW, two things:
1) Would it be possible to run this on port 80 instead of port 8080?
Amazon's corporate firewall blocks outbound port 8080, so we can't view
this CI while on the corporate network.
2) Is the setup/configuration of this Jenkins server documented/scripted
somewhere in such a way that we might be able to duplicate it internally?
We on EMR would really love to be able to have a Jenkins setup that more
closely mirrors what the community is doing so that we can be sure that
patches that we would like to contribute back are not going to break
upstream.
~ Jonathan
On Mon, Nov 2, 2015 at 2:36 PM, Konstantin Boudnik <co...@apache.org> wrote:
> Just in case it was missed before, I'd like to thank Evans for this amazing
> work on our new CI infrastructure
>
>
> http://ci.bigtop.apache.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-packages/
>
> OMG! This is so great! I know Evans was doing a lot of it for sure, but I
> want
> to thank everyone who spent time working on this or contributed otherwise!
>
> Cos
>
Re: The state of new CI
Posted by Andrew Purtell <an...@gmail.com>.
This is really great Evans.
I have occasion to set up a Bigtop based CI again soon and rather than do it by hand will try out these instructions!
> On Nov 6, 2015, at 11:09 AM, Evans Ye <ev...@apache.org> wrote:
>
> I wrote the Bigtop CI Setup Guide here:
>
> https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide
>
> It's brief, but it should give you all the intel you need to setup Bigtop
> CI.
> if you find something unclear, just let me know and I'll then polish it. :)
>
> 2015-11-05 15:21 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
>
>>> On Wed, Nov 04, 2015 at 01:28AM, Evans Ye wrote:
>>> Thank you all for the kind and warm words!
>>>
>>>> 1) Would it be possible to run this on port 80 instead of port 8080?
>>>
>>> This is piece of cake by leveraging Docker. Its on 80 now. :)
>>> http://ci.bigtop.apache.org/view/Bigtop-trunk/job/Bigtop-trunk-packages/
>>>
>>>> 2) Is the setup/configuration of this Jenkins server
>> documented/scripted
>>>> somewhere ...
>>>
>>> Yup, I really should get this done!
>>> Sorry I haven't got time to put a polished doc for jenkins setup on our
>>> wiki.
>>> But I did log down all the steps in my evernote.
>>> Should be easy to roll our a version for you. :)
>>>
>>> And Cos is right, what I've done have nothing to do with ci/jenkins
>> groovy
>>> scripts.
>>> I don't have time to investigate them because of time limitation set by
>> old
>>> CI master retirement...
>>> Despite that, I tend to use simple commands for jobs setup in new CI,
>> hence
>>> it's not to hard for you to replicate by looking into job definition
>>> directly. I'll write details in the doc.
>>> Anyhow, I think it's much better to use our gradle system for automation
>> to
>>> run any bigtop functions. With this simplicity users won't need to write
>>> any hard-to read code in Jenkins commands.
>>>
>>> Cos,
>>> I'm still stuck on BIGTOP-2110, which is to fix packages build random
>>> failure.
>>> Since you're much familiar to gradle, do you have a better idea to cache
>>> gradle packages inside container?
>>
>> I am out the whole day tomorrow, but would be happy to look at it Friday or
>> over the weekend. From the top of my head - shall we go with the dockerized
>> repo server approach like we've discussed earlier?
>>
>> Cos
>>
>>> Evans
>>>
>>>
>>> 2015-11-03 9:20 GMT+08:00 Andrew Purtell <ap...@apache.org>:
>>>
>>>> Thanks so much for doing this Evans!
>>>>
>>>>
>>>> On Mon, Nov 2, 2015 at 2:36 PM, Konstantin Boudnik <co...@apache.org>
>> wrote:
>>>>
>>>>> Just in case it was missed before, I'd like to thank Evans for this
>>>> amazing
>>>>> work on our new CI infrastructure
>> http://ci.bigtop.apache.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-packages/
>>>>>
>>>>> OMG! This is so great! I know Evans was doing a lot of it for sure,
>> but I
>>>>> want
>>>>> to thank everyone who spent time working on this or contributed
>>>> otherwise!
>>>>>
>>>>> Cos
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>>
>>>> - Andy
>>>>
>>>> Problems worthy of attack prove their worth by hitting back. - Piet
>> Hein
>>>> (via Tom White)
>>
Re: The state of new CI
Posted by jay vyas <ja...@gmail.com>.
I really like the fact that we're making our CI deployment a first class
product that others can leverage.
great work evans!
On Fri, Nov 6, 2015 at 2:46 PM, Olaf Flebbe <of...@oflebbe.de> wrote:
> Evans, let me thank you for your work on the CI! Unfortunately I am very
> busy with other things this and next week, so I cannot contribute right now.
>
> -Olaf
>
>
--
jay vyas
Re: The state of new CI
Posted by Olaf Flebbe <of...@oflebbe.de>.
Evans, let me thank you for your work on the CI! Unfortunately I am very busy with other things this and next week, so I cannot contribute right now.
-Olaf
Re: The state of new CI
Posted by Evans Ye <ev...@apache.org>.
I wrote the Bigtop CI Setup Guide here:
https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+CI+Setup+Guide
It's brief, but it should give you all the intel you need to setup Bigtop
CI.
if you find something unclear, just let me know and I'll then polish it. :)
2015-11-05 15:21 GMT+08:00 Konstantin Boudnik <co...@apache.org>:
> On Wed, Nov 04, 2015 at 01:28AM, Evans Ye wrote:
> > Thank you all for the kind and warm words!
> >
> > > 1) Would it be possible to run this on port 80 instead of port 8080?
> >
> > This is piece of cake by leveraging Docker. Its on 80 now. :)
> > http://ci.bigtop.apache.org/view/Bigtop-trunk/job/Bigtop-trunk-packages/
> >
> > > 2) Is the setup/configuration of this Jenkins server
> documented/scripted
> > > somewhere ...
> >
> > Yup, I really should get this done!
> > Sorry I haven't got time to put a polished doc for jenkins setup on our
> > wiki.
> > But I did log down all the steps in my evernote.
> > Should be easy to roll our a version for you. :)
> >
> > And Cos is right, what I've done have nothing to do with ci/jenkins
> groovy
> > scripts.
> > I don't have time to investigate them because of time limitation set by
> old
> > CI master retirement...
> > Despite that, I tend to use simple commands for jobs setup in new CI,
> hence
> > it's not to hard for you to replicate by looking into job definition
> > directly. I'll write details in the doc.
> > Anyhow, I think it's much better to use our gradle system for automation
> to
> > run any bigtop functions. With this simplicity users won't need to write
> > any hard-to read code in Jenkins commands.
> >
> > Cos,
> > I'm still stuck on BIGTOP-2110, which is to fix packages build random
> > failure.
> > Since you're much familiar to gradle, do you have a better idea to cache
> > gradle packages inside container?
>
> I am out the whole day tomorrow, but would be happy to look at it Friday or
> over the weekend. From the top of my head - shall we go with the dockerized
> repo server approach like we've discussed earlier?
>
> Cos
>
> > Evans
> >
> >
> > 2015-11-03 9:20 GMT+08:00 Andrew Purtell <ap...@apache.org>:
> >
> > > Thanks so much for doing this Evans!
> > >
> > >
> > > On Mon, Nov 2, 2015 at 2:36 PM, Konstantin Boudnik <co...@apache.org>
> wrote:
> > >
> > > > Just in case it was missed before, I'd like to thank Evans for this
> > > amazing
> > > > work on our new CI infrastructure
> > > >
> > > >
> > > >
> > >
> http://ci.bigtop.apache.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-packages/
> > > >
> > > > OMG! This is so great! I know Evans was doing a lot of it for sure,
> but I
> > > > want
> > > > to thank everyone who spent time working on this or contributed
> > > otherwise!
> > > >
> > > > Cos
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > > - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
>
Re: The state of new CI
Posted by Konstantin Boudnik <co...@apache.org>.
On Wed, Nov 04, 2015 at 01:28AM, Evans Ye wrote:
> Thank you all for the kind and warm words!
>
> > 1) Would it be possible to run this on port 80 instead of port 8080?
>
> This is piece of cake by leveraging Docker. Its on 80 now. :)
> http://ci.bigtop.apache.org/view/Bigtop-trunk/job/Bigtop-trunk-packages/
>
> > 2) Is the setup/configuration of this Jenkins server documented/scripted
> > somewhere ...
>
> Yup, I really should get this done!
> Sorry I haven't got time to put a polished doc for jenkins setup on our
> wiki.
> But I did log down all the steps in my evernote.
> Should be easy to roll our a version for you. :)
>
> And Cos is right, what I've done have nothing to do with ci/jenkins groovy
> scripts.
> I don't have time to investigate them because of time limitation set by old
> CI master retirement...
> Despite that, I tend to use simple commands for jobs setup in new CI, hence
> it's not to hard for you to replicate by looking into job definition
> directly. I'll write details in the doc.
> Anyhow, I think it's much better to use our gradle system for automation to
> run any bigtop functions. With this simplicity users won't need to write
> any hard-to read code in Jenkins commands.
>
> Cos,
> I'm still stuck on BIGTOP-2110, which is to fix packages build random
> failure.
> Since you're much familiar to gradle, do you have a better idea to cache
> gradle packages inside container?
I am out the whole day tomorrow, but would be happy to look at it Friday or
over the weekend. From the top of my head - shall we go with the dockerized
repo server approach like we've discussed earlier?
Cos
> Evans
>
>
> 2015-11-03 9:20 GMT+08:00 Andrew Purtell <ap...@apache.org>:
>
> > Thanks so much for doing this Evans!
> >
> >
> > On Mon, Nov 2, 2015 at 2:36 PM, Konstantin Boudnik <co...@apache.org> wrote:
> >
> > > Just in case it was missed before, I'd like to thank Evans for this
> > amazing
> > > work on our new CI infrastructure
> > >
> > >
> > >
> > http://ci.bigtop.apache.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-packages/
> > >
> > > OMG! This is so great! I know Evans was doing a lot of it for sure, but I
> > > want
> > > to thank everyone who spent time working on this or contributed
> > otherwise!
> > >
> > > Cos
> > >
> >
> >
> >
> > --
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
Re: The state of new CI
Posted by Evans Ye <ev...@apache.org>.
Thank you all for the kind and warm words!
> 1) Would it be possible to run this on port 80 instead of port 8080?
This is piece of cake by leveraging Docker. Its on 80 now. :)
http://ci.bigtop.apache.org/view/Bigtop-trunk/job/Bigtop-trunk-packages/
> 2) Is the setup/configuration of this Jenkins server documented/scripted
> somewhere ...
Yup, I really should get this done!
Sorry I haven't got time to put a polished doc for jenkins setup on our
wiki.
But I did log down all the steps in my evernote.
Should be easy to roll our a version for you. :)
And Cos is right, what I've done have nothing to do with ci/jenkins groovy
scripts.
I don't have time to investigate them because of time limitation set by old
CI master retirement...
Despite that, I tend to use simple commands for jobs setup in new CI, hence
it's not to hard for you to replicate by looking into job definition
directly. I'll write details in the doc.
Anyhow, I think it's much better to use our gradle system for automation to
run any bigtop functions. With this simplicity users won't need to write
any hard-to read code in Jenkins commands.
Cos,
I'm still stuck on BIGTOP-2110, which is to fix packages build random
failure.
Since you're much familiar to gradle, do you have a better idea to cache
gradle packages inside container?
Evans
2015-11-03 9:20 GMT+08:00 Andrew Purtell <ap...@apache.org>:
> Thanks so much for doing this Evans!
>
>
> On Mon, Nov 2, 2015 at 2:36 PM, Konstantin Boudnik <co...@apache.org> wrote:
>
> > Just in case it was missed before, I'd like to thank Evans for this
> amazing
> > work on our new CI infrastructure
> >
> >
> >
> http://ci.bigtop.apache.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-packages/
> >
> > OMG! This is so great! I know Evans was doing a lot of it for sure, but I
> > want
> > to thank everyone who spent time working on this or contributed
> otherwise!
> >
> > Cos
> >
>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: The state of new CI
Posted by Evans Ye <ev...@apache.org>.
Thank you all for the kind and warm words!
> 1) Would it be possible to run this on port 80 instead of port 8080?
This is piece of cake by leveraging Docker. Its on 80 now. :)
http://ci.bigtop.apache.org/view/Bigtop-trunk/job/Bigtop-trunk-packages/
> 2) Is the setup/configuration of this Jenkins server documented/scripted
> somewhere ...
Yup, I really should get this done!
Sorry I haven't got time to put a polished doc for jenkins setup on our
wiki.
But I did log down all the steps in my evernote.
Should be easy to roll our a version for you. :)
And Cos is right, what I've done have nothing to do with ci/jenkins groovy
scripts.
I don't have time to investigate them because of time limitation set by old
CI master retirement...
Despite that, I tend to use simple commands for jobs setup in new CI, hence
it's not to hard for you to replicate by looking into job definition
directly. I'll write details in the doc.
Anyhow, I think it's much better to use our gradle system for automation to
run any bigtop functions. With this simplicity users won't need to write
any hard-to read code in Jenkins commands.
Cos,
I'm still stuck on BIGTOP-2110, which is to fix packages build random
failure.
Since you're much familiar to gradle, do you have a better idea to cache
gradle packages inside container?
Evans
2015-11-03 9:20 GMT+08:00 Andrew Purtell <ap...@apache.org>:
> Thanks so much for doing this Evans!
>
>
> On Mon, Nov 2, 2015 at 2:36 PM, Konstantin Boudnik <co...@apache.org> wrote:
>
> > Just in case it was missed before, I'd like to thank Evans for this
> amazing
> > work on our new CI infrastructure
> >
> >
> >
> http://ci.bigtop.apache.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-packages/
> >
> > OMG! This is so great! I know Evans was doing a lot of it for sure, but I
> > want
> > to thank everyone who spent time working on this or contributed
> otherwise!
> >
> > Cos
> >
>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: The state of new CI
Posted by Andrew Purtell <ap...@apache.org>.
Thanks so much for doing this Evans!
On Mon, Nov 2, 2015 at 2:36 PM, Konstantin Boudnik <co...@apache.org> wrote:
> Just in case it was missed before, I'd like to thank Evans for this amazing
> work on our new CI infrastructure
>
>
> http://ci.bigtop.apache.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-packages/
>
> OMG! This is so great! I know Evans was doing a lot of it for sure, but I
> want
> to thank everyone who spent time working on this or contributed otherwise!
>
> Cos
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: The state of new CI
Posted by Andrew Purtell <ap...@apache.org>.
Thanks so much for doing this Evans!
On Mon, Nov 2, 2015 at 2:36 PM, Konstantin Boudnik <co...@apache.org> wrote:
> Just in case it was missed before, I'd like to thank Evans for this amazing
> work on our new CI infrastructure
>
>
> http://ci.bigtop.apache.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-packages/
>
> OMG! This is so great! I know Evans was doing a lot of it for sure, but I
> want
> to thank everyone who spent time working on this or contributed otherwise!
>
> Cos
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: The state of new CI
Posted by Jonathan Kelly <jo...@gmail.com>.
Very cool. Thanks a lot, Evans!
BTW, two things:
1) Would it be possible to run this on port 80 instead of port 8080?
Amazon's corporate firewall blocks outbound port 8080, so we can't view
this CI while on the corporate network.
2) Is the setup/configuration of this Jenkins server documented/scripted
somewhere in such a way that we might be able to duplicate it internally?
We on EMR would really love to be able to have a Jenkins setup that more
closely mirrors what the community is doing so that we can be sure that
patches that we would like to contribute back are not going to break
upstream.
~ Jonathan
On Mon, Nov 2, 2015 at 2:36 PM, Konstantin Boudnik <co...@apache.org> wrote:
> Just in case it was missed before, I'd like to thank Evans for this amazing
> work on our new CI infrastructure
>
>
> http://ci.bigtop.apache.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-packages/
>
> OMG! This is so great! I know Evans was doing a lot of it for sure, but I
> want
> to thank everyone who spent time working on this or contributed otherwise!
>
> Cos
>