You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by shane knapp <sk...@berkeley.edu> on 2014/10/09 00:01:04 UTC
new jenkins update + tentative release date
greetings!
i've got some updates regarding our new jenkins infrastructure, as well as
the initial date and plan for rolling things out:
*** current testing/build break whack-a-mole:
a lot of out of date artifacts are cached in the current jenkins, which has
caused a few builds during my testing to break due to dependency resolution
failure[1][2].
bumping these versions can cause your builds to fail, due to public api
changes and the like. consider yourself warned that some projects might
require some debugging... :)
tomorrow, i will be at databricks working w/@joshrosen to make sure that
the spark builds have any bugs hammered out.
*** deployment plan:
unless something completely horrible happens, THE NEW JENKINS WILL GO LIVE
ON MONDAY (october 13th).
all jenkins infrastructure will be DOWN for the entirety of the day
(starting at ~8am). this means no builds, period. i'm hoping that the
downtime will be much shorter than this, but we'll have to see how
everything goes.
all test/build history WILL BE PRESERVED. i will be rsyncing the jenkins
jobs/ directory over, complete w/history as part of the deployment.
once i'm feeling good about the state of things, i'll point the original
url to the new instances and send out an all clear.
if you are a student at UC berkeley, you can log in to jenkins using your
LDAP login, and (by default) view but not change plans. if you do not have
a UC berkeley LDAP login, you can still view plans anonymously.
IF YOU ARE A PLAN ADMIN, THEN PLEASE REACH OUT, ASAP, PRIVATELY AND I WILL
SET UP ADMIN ACCESS TO YOUR BUILDS.
*** post deployment plan:
fix all of the things that break!
i will be keeping a VERY close eye on the builds, checking for breaks, and
helping out where i can. if the situation is dire, i can always roll back
to the old jenkins infra... but i hope we never get to that point! :)
i'm hoping that things will go smoothly, but please be patient as i'm
certain we'll hit a few bumps in the road.
please let me know if you guys have any comments/questions/concerns... :)
shane
1 - https://github.com/bigdatagenomics/bdg-services/pull/18
2 - https://github.com/bigdatagenomics/avocado/pull/111
Re: new jenkins update + tentative release date
Posted by Nicholas Chammas <ni...@gmail.com>.
*fingers crossed*
On Mon, Oct 13, 2014 at 5:54 PM, shane knapp <sk...@berkeley.edu> wrote:
> ok, i found something that may help:
>
> https://issues.jenkins-ci.org/browse/JENKINS-20445?focusedCommentId=195638&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-195638
>
> i set this to 20 minutes... let's see if that helps.
>
> On Mon, Oct 13, 2014 at 2:48 PM, Nicholas Chammas <
> nicholas.chammas@gmail.com> wrote:
>
>> Ah, that sucks. Thank you for looking into this.
>>
>> On Mon, Oct 13, 2014 at 5:43 PM, shane knapp <sk...@berkeley.edu> wrote:
>>
>>> On Mon, Oct 13, 2014 at 2:28 PM, Nicholas Chammas <
>>> nicholas.chammas@gmail.com> wrote:
>>>
>>>> Thanks for doing this work Shane.
>>>>
>>>> So is Jenkins in the new datacenter now? Do you know if the problems
>>>> with checking out patches from GitHub should be resolved now? Here's an
>>>> example from the past hour
>>>> <https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21702/console>
>>>> .
>>>>
>>>>
>>>> yeah, i just noticed that we're still having the checkout issues. i
>>> was really hoping that the better network would just make this go away...
>>> guess i'll be doing a deeper dive now.
>>>
>>> i would just up the timeout, but that's not coming out for a little
>>> while yet:
>>> https://issues.jenkins-ci.org/browse/JENKINS-20387
>>>
>>> (we are currently running the latest -- 2.2.7, and the timeout field is
>>> coming in 2.3, whenever that is)
>>>
>>> i'll try and strace/replicate it locally as well.
>>>
>>>
>>
>>
>
Re: new jenkins update + tentative release date
Posted by shane knapp <sk...@berkeley.edu>.
ok, i found something that may help:
https://issues.jenkins-ci.org/browse/JENKINS-20445?focusedCommentId=195638&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-195638
i set this to 20 minutes... let's see if that helps.
On Mon, Oct 13, 2014 at 2:48 PM, Nicholas Chammas <
nicholas.chammas@gmail.com> wrote:
> Ah, that sucks. Thank you for looking into this.
>
> On Mon, Oct 13, 2014 at 5:43 PM, shane knapp <sk...@berkeley.edu> wrote:
>
>> On Mon, Oct 13, 2014 at 2:28 PM, Nicholas Chammas <
>> nicholas.chammas@gmail.com> wrote:
>>
>>> Thanks for doing this work Shane.
>>>
>>> So is Jenkins in the new datacenter now? Do you know if the problems
>>> with checking out patches from GitHub should be resolved now? Here's an
>>> example from the past hour
>>> <https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21702/console>
>>> .
>>>
>>>
>>> yeah, i just noticed that we're still having the checkout issues. i was
>> really hoping that the better network would just make this go away...
>> guess i'll be doing a deeper dive now.
>>
>> i would just up the timeout, but that's not coming out for a little while
>> yet:
>> https://issues.jenkins-ci.org/browse/JENKINS-20387
>>
>> (we are currently running the latest -- 2.2.7, and the timeout field is
>> coming in 2.3, whenever that is)
>>
>> i'll try and strace/replicate it locally as well.
>>
>>
>
>
Re: new jenkins update + tentative release date
Posted by Nicholas Chammas <ni...@gmail.com>.
Ah, that sucks. Thank you for looking into this.
On Mon, Oct 13, 2014 at 5:43 PM, shane knapp <sk...@berkeley.edu> wrote:
> On Mon, Oct 13, 2014 at 2:28 PM, Nicholas Chammas <
> nicholas.chammas@gmail.com> wrote:
>
>> Thanks for doing this work Shane.
>>
>> So is Jenkins in the new datacenter now? Do you know if the problems with
>> checking out patches from GitHub should be resolved now? Here's an
>> example from the past hour
>> <https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21702/console>
>> .
>>
>>
>> yeah, i just noticed that we're still having the checkout issues. i was
> really hoping that the better network would just make this go away...
> guess i'll be doing a deeper dive now.
>
> i would just up the timeout, but that's not coming out for a little while
> yet:
> https://issues.jenkins-ci.org/browse/JENKINS-20387
>
> (we are currently running the latest -- 2.2.7, and the timeout field is
> coming in 2.3, whenever that is)
>
> i'll try and strace/replicate it locally as well.
>
>
Re: new jenkins update + tentative release date
Posted by shane knapp <sk...@berkeley.edu>.
On Mon, Oct 13, 2014 at 2:28 PM, Nicholas Chammas <
nicholas.chammas@gmail.com> wrote:
> Thanks for doing this work Shane.
>
> So is Jenkins in the new datacenter now? Do you know if the problems with
> checking out patches from GitHub should be resolved now? Here's an
> example from the past hour
> <https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21702/console>
> .
>
>
> yeah, i just noticed that we're still having the checkout issues. i was
really hoping that the better network would just make this go away...
guess i'll be doing a deeper dive now.
i would just up the timeout, but that's not coming out for a little while
yet:
https://issues.jenkins-ci.org/browse/JENKINS-20387
(we are currently running the latest -- 2.2.7, and the timeout field is
coming in 2.3, whenever that is)
i'll try and strace/replicate it locally as well.
Re: new jenkins update + tentative release date
Posted by Nicholas Chammas <ni...@gmail.com>.
Thanks for doing this work Shane.
So is Jenkins in the new datacenter now? Do you know if the problems with
checking out patches from GitHub should be resolved now? Here's an example
from the past hour
<https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21702/console>
.
Nick
On Mon, Oct 13, 2014 at 1:35 PM, shane knapp <sk...@berkeley.edu> wrote:
> AND WE ARE LIIIIIIIVE!
>
> https://amplab.cs.berkeley.edu/jenkins/
>
> have at it, folks!
>
> On Mon, Oct 13, 2014 at 10:15 AM, shane knapp <sk...@berkeley.edu> wrote:
>
> > quick update: we should be back up and running in the next ~60mins.
> >
> > On Mon, Oct 13, 2014 at 7:54 AM, shane knapp <sk...@berkeley.edu>
> wrote:
> >
> >> Jenkins is in quiet mode and the move will be starting after i have my
> >> coffee. :)
> >>
> >> On Sun, Oct 12, 2014 at 11:26 PM, Josh Rosen <ro...@gmail.com>
> >> wrote:
> >>
> >>> Reminder: this Jenkins migration is happening tomorrow morning
> (Monday).
> >>>
> >>> On Fri, Oct 10, 2014 at 1:01 PM, shane knapp <sk...@berkeley.edu>
> >>> wrote:
> >>>
> >>>> reminder: this IS happening, first thing monday morning PDT. :)
> >>>>
> >>>> On Wed, Oct 8, 2014 at 3:01 PM, shane knapp <sk...@berkeley.edu>
> >>>> wrote:
> >>>>
> >>>> > greetings!
> >>>> >
> >>>> > i've got some updates regarding our new jenkins infrastructure, as
> >>>> well as
> >>>> > the initial date and plan for rolling things out:
> >>>> >
> >>>> > *** current testing/build break whack-a-mole:
> >>>> > a lot of out of date artifacts are cached in the current jenkins,
> >>>> which
> >>>> > has caused a few builds during my testing to break due to dependency
> >>>> > resolution failure[1][2].
> >>>> >
> >>>> > bumping these versions can cause your builds to fail, due to public
> >>>> api
> >>>> > changes and the like. consider yourself warned that some projects
> >>>> might
> >>>> > require some debugging... :)
> >>>> >
> >>>> > tomorrow, i will be at databricks working w/@joshrosen to make sure
> >>>> that
> >>>> > the spark builds have any bugs hammered out.
> >>>> >
> >>>> > *** deployment plan:
> >>>> > unless something completely horrible happens, THE NEW JENKINS WILL
> GO
> >>>> LIVE
> >>>> > ON MONDAY (october 13th).
> >>>> >
> >>>> > all jenkins infrastructure will be DOWN for the entirety of the day
> >>>> > (starting at ~8am). this means no builds, period. i'm hoping that
> >>>> the
> >>>> > downtime will be much shorter than this, but we'll have to see how
> >>>> > everything goes.
> >>>> >
> >>>> > all test/build history WILL BE PRESERVED. i will be rsyncing the
> >>>> jenkins
> >>>> > jobs/ directory over, complete w/history as part of the deployment.
> >>>> >
> >>>> > once i'm feeling good about the state of things, i'll point the
> >>>> original
> >>>> > url to the new instances and send out an all clear.
> >>>> >
> >>>> > if you are a student at UC berkeley, you can log in to jenkins using
> >>>> your
> >>>> > LDAP login, and (by default) view but not change plans. if you do
> >>>> not have
> >>>> > a UC berkeley LDAP login, you can still view plans anonymously.
> >>>> >
> >>>> > IF YOU ARE A PLAN ADMIN, THEN PLEASE REACH OUT, ASAP, PRIVATELY AND
> I
> >>>> WILL
> >>>> > SET UP ADMIN ACCESS TO YOUR BUILDS.
> >>>> >
> >>>> > *** post deployment plan:
> >>>> > fix all of the things that break!
> >>>> >
> >>>> > i will be keeping a VERY close eye on the builds, checking for
> >>>> breaks, and
> >>>> > helping out where i can. if the situation is dire, i can always
> roll
> >>>> back
> >>>> > to the old jenkins infra... but i hope we never get to that point!
> >>>> :)
> >>>> >
> >>>> > i'm hoping that things will go smoothly, but please be patient as
> i'm
> >>>> > certain we'll hit a few bumps in the road.
> >>>> >
> >>>> > please let me know if you guys have any
> >>>> comments/questions/concerns... :)
> >>>> >
> >>>> > shane
> >>>> >
> >>>> > 1 - https://github.com/bigdatagenomics/bdg-services/pull/18
> >>>> > 2 - https://github.com/bigdatagenomics/avocado/pull/111
> >>>> >
> >>>>
> >>>
> >>>
> >>
> >
>
Re: new jenkins update + tentative release date
Posted by shane knapp <sk...@berkeley.edu>.
AND WE ARE LIIIIIIIVE!
https://amplab.cs.berkeley.edu/jenkins/
have at it, folks!
On Mon, Oct 13, 2014 at 10:15 AM, shane knapp <sk...@berkeley.edu> wrote:
> quick update: we should be back up and running in the next ~60mins.
>
> On Mon, Oct 13, 2014 at 7:54 AM, shane knapp <sk...@berkeley.edu> wrote:
>
>> Jenkins is in quiet mode and the move will be starting after i have my
>> coffee. :)
>>
>> On Sun, Oct 12, 2014 at 11:26 PM, Josh Rosen <ro...@gmail.com>
>> wrote:
>>
>>> Reminder: this Jenkins migration is happening tomorrow morning (Monday).
>>>
>>> On Fri, Oct 10, 2014 at 1:01 PM, shane knapp <sk...@berkeley.edu>
>>> wrote:
>>>
>>>> reminder: this IS happening, first thing monday morning PDT. :)
>>>>
>>>> On Wed, Oct 8, 2014 at 3:01 PM, shane knapp <sk...@berkeley.edu>
>>>> wrote:
>>>>
>>>> > greetings!
>>>> >
>>>> > i've got some updates regarding our new jenkins infrastructure, as
>>>> well as
>>>> > the initial date and plan for rolling things out:
>>>> >
>>>> > *** current testing/build break whack-a-mole:
>>>> > a lot of out of date artifacts are cached in the current jenkins,
>>>> which
>>>> > has caused a few builds during my testing to break due to dependency
>>>> > resolution failure[1][2].
>>>> >
>>>> > bumping these versions can cause your builds to fail, due to public
>>>> api
>>>> > changes and the like. consider yourself warned that some projects
>>>> might
>>>> > require some debugging... :)
>>>> >
>>>> > tomorrow, i will be at databricks working w/@joshrosen to make sure
>>>> that
>>>> > the spark builds have any bugs hammered out.
>>>> >
>>>> > *** deployment plan:
>>>> > unless something completely horrible happens, THE NEW JENKINS WILL GO
>>>> LIVE
>>>> > ON MONDAY (october 13th).
>>>> >
>>>> > all jenkins infrastructure will be DOWN for the entirety of the day
>>>> > (starting at ~8am). this means no builds, period. i'm hoping that
>>>> the
>>>> > downtime will be much shorter than this, but we'll have to see how
>>>> > everything goes.
>>>> >
>>>> > all test/build history WILL BE PRESERVED. i will be rsyncing the
>>>> jenkins
>>>> > jobs/ directory over, complete w/history as part of the deployment.
>>>> >
>>>> > once i'm feeling good about the state of things, i'll point the
>>>> original
>>>> > url to the new instances and send out an all clear.
>>>> >
>>>> > if you are a student at UC berkeley, you can log in to jenkins using
>>>> your
>>>> > LDAP login, and (by default) view but not change plans. if you do
>>>> not have
>>>> > a UC berkeley LDAP login, you can still view plans anonymously.
>>>> >
>>>> > IF YOU ARE A PLAN ADMIN, THEN PLEASE REACH OUT, ASAP, PRIVATELY AND I
>>>> WILL
>>>> > SET UP ADMIN ACCESS TO YOUR BUILDS.
>>>> >
>>>> > *** post deployment plan:
>>>> > fix all of the things that break!
>>>> >
>>>> > i will be keeping a VERY close eye on the builds, checking for
>>>> breaks, and
>>>> > helping out where i can. if the situation is dire, i can always roll
>>>> back
>>>> > to the old jenkins infra... but i hope we never get to that point!
>>>> :)
>>>> >
>>>> > i'm hoping that things will go smoothly, but please be patient as i'm
>>>> > certain we'll hit a few bumps in the road.
>>>> >
>>>> > please let me know if you guys have any
>>>> comments/questions/concerns... :)
>>>> >
>>>> > shane
>>>> >
>>>> > 1 - https://github.com/bigdatagenomics/bdg-services/pull/18
>>>> > 2 - https://github.com/bigdatagenomics/avocado/pull/111
>>>> >
>>>>
>>>
>>>
>>
>
Re: new jenkins update + tentative release date
Posted by shane knapp <sk...@berkeley.edu>.
quick update: we should be back up and running in the next ~60mins.
On Mon, Oct 13, 2014 at 7:54 AM, shane knapp <sk...@berkeley.edu> wrote:
> Jenkins is in quiet mode and the move will be starting after i have my
> coffee. :)
>
> On Sun, Oct 12, 2014 at 11:26 PM, Josh Rosen <ro...@gmail.com> wrote:
>
>> Reminder: this Jenkins migration is happening tomorrow morning (Monday).
>>
>> On Fri, Oct 10, 2014 at 1:01 PM, shane knapp <sk...@berkeley.edu> wrote:
>>
>>> reminder: this IS happening, first thing monday morning PDT. :)
>>>
>>> On Wed, Oct 8, 2014 at 3:01 PM, shane knapp <sk...@berkeley.edu> wrote:
>>>
>>> > greetings!
>>> >
>>> > i've got some updates regarding our new jenkins infrastructure, as
>>> well as
>>> > the initial date and plan for rolling things out:
>>> >
>>> > *** current testing/build break whack-a-mole:
>>> > a lot of out of date artifacts are cached in the current jenkins, which
>>> > has caused a few builds during my testing to break due to dependency
>>> > resolution failure[1][2].
>>> >
>>> > bumping these versions can cause your builds to fail, due to public api
>>> > changes and the like. consider yourself warned that some projects
>>> might
>>> > require some debugging... :)
>>> >
>>> > tomorrow, i will be at databricks working w/@joshrosen to make sure
>>> that
>>> > the spark builds have any bugs hammered out.
>>> >
>>> > *** deployment plan:
>>> > unless something completely horrible happens, THE NEW JENKINS WILL GO
>>> LIVE
>>> > ON MONDAY (october 13th).
>>> >
>>> > all jenkins infrastructure will be DOWN for the entirety of the day
>>> > (starting at ~8am). this means no builds, period. i'm hoping that the
>>> > downtime will be much shorter than this, but we'll have to see how
>>> > everything goes.
>>> >
>>> > all test/build history WILL BE PRESERVED. i will be rsyncing the
>>> jenkins
>>> > jobs/ directory over, complete w/history as part of the deployment.
>>> >
>>> > once i'm feeling good about the state of things, i'll point the
>>> original
>>> > url to the new instances and send out an all clear.
>>> >
>>> > if you are a student at UC berkeley, you can log in to jenkins using
>>> your
>>> > LDAP login, and (by default) view but not change plans. if you do not
>>> have
>>> > a UC berkeley LDAP login, you can still view plans anonymously.
>>> >
>>> > IF YOU ARE A PLAN ADMIN, THEN PLEASE REACH OUT, ASAP, PRIVATELY AND I
>>> WILL
>>> > SET UP ADMIN ACCESS TO YOUR BUILDS.
>>> >
>>> > *** post deployment plan:
>>> > fix all of the things that break!
>>> >
>>> > i will be keeping a VERY close eye on the builds, checking for breaks,
>>> and
>>> > helping out where i can. if the situation is dire, i can always roll
>>> back
>>> > to the old jenkins infra... but i hope we never get to that point! :)
>>> >
>>> > i'm hoping that things will go smoothly, but please be patient as i'm
>>> > certain we'll hit a few bumps in the road.
>>> >
>>> > please let me know if you guys have any
>>> comments/questions/concerns... :)
>>> >
>>> > shane
>>> >
>>> > 1 - https://github.com/bigdatagenomics/bdg-services/pull/18
>>> > 2 - https://github.com/bigdatagenomics/avocado/pull/111
>>> >
>>>
>>
>>
>
Re: new jenkins update + tentative release date
Posted by shane knapp <sk...@berkeley.edu>.
Jenkins is in quiet mode and the move will be starting after i have my
coffee. :)
On Sun, Oct 12, 2014 at 11:26 PM, Josh Rosen <ro...@gmail.com> wrote:
> Reminder: this Jenkins migration is happening tomorrow morning (Monday).
>
> On Fri, Oct 10, 2014 at 1:01 PM, shane knapp <sk...@berkeley.edu> wrote:
>
>> reminder: this IS happening, first thing monday morning PDT. :)
>>
>> On Wed, Oct 8, 2014 at 3:01 PM, shane knapp <sk...@berkeley.edu> wrote:
>>
>> > greetings!
>> >
>> > i've got some updates regarding our new jenkins infrastructure, as well
>> as
>> > the initial date and plan for rolling things out:
>> >
>> > *** current testing/build break whack-a-mole:
>> > a lot of out of date artifacts are cached in the current jenkins, which
>> > has caused a few builds during my testing to break due to dependency
>> > resolution failure[1][2].
>> >
>> > bumping these versions can cause your builds to fail, due to public api
>> > changes and the like. consider yourself warned that some projects might
>> > require some debugging... :)
>> >
>> > tomorrow, i will be at databricks working w/@joshrosen to make sure that
>> > the spark builds have any bugs hammered out.
>> >
>> > *** deployment plan:
>> > unless something completely horrible happens, THE NEW JENKINS WILL GO
>> LIVE
>> > ON MONDAY (october 13th).
>> >
>> > all jenkins infrastructure will be DOWN for the entirety of the day
>> > (starting at ~8am). this means no builds, period. i'm hoping that the
>> > downtime will be much shorter than this, but we'll have to see how
>> > everything goes.
>> >
>> > all test/build history WILL BE PRESERVED. i will be rsyncing the
>> jenkins
>> > jobs/ directory over, complete w/history as part of the deployment.
>> >
>> > once i'm feeling good about the state of things, i'll point the original
>> > url to the new instances and send out an all clear.
>> >
>> > if you are a student at UC berkeley, you can log in to jenkins using
>> your
>> > LDAP login, and (by default) view but not change plans. if you do not
>> have
>> > a UC berkeley LDAP login, you can still view plans anonymously.
>> >
>> > IF YOU ARE A PLAN ADMIN, THEN PLEASE REACH OUT, ASAP, PRIVATELY AND I
>> WILL
>> > SET UP ADMIN ACCESS TO YOUR BUILDS.
>> >
>> > *** post deployment plan:
>> > fix all of the things that break!
>> >
>> > i will be keeping a VERY close eye on the builds, checking for breaks,
>> and
>> > helping out where i can. if the situation is dire, i can always roll
>> back
>> > to the old jenkins infra... but i hope we never get to that point! :)
>> >
>> > i'm hoping that things will go smoothly, but please be patient as i'm
>> > certain we'll hit a few bumps in the road.
>> >
>> > please let me know if you guys have any comments/questions/concerns...
>> :)
>> >
>> > shane
>> >
>> > 1 - https://github.com/bigdatagenomics/bdg-services/pull/18
>> > 2 - https://github.com/bigdatagenomics/avocado/pull/111
>> >
>>
>
>
Re: new jenkins update + tentative release date
Posted by Josh Rosen <ro...@gmail.com>.
Reminder: this Jenkins migration is happening tomorrow morning (Monday).
On Fri, Oct 10, 2014 at 1:01 PM, shane knapp <sk...@berkeley.edu> wrote:
> reminder: this IS happening, first thing monday morning PDT. :)
>
> On Wed, Oct 8, 2014 at 3:01 PM, shane knapp <sk...@berkeley.edu> wrote:
>
> > greetings!
> >
> > i've got some updates regarding our new jenkins infrastructure, as well
> as
> > the initial date and plan for rolling things out:
> >
> > *** current testing/build break whack-a-mole:
> > a lot of out of date artifacts are cached in the current jenkins, which
> > has caused a few builds during my testing to break due to dependency
> > resolution failure[1][2].
> >
> > bumping these versions can cause your builds to fail, due to public api
> > changes and the like. consider yourself warned that some projects might
> > require some debugging... :)
> >
> > tomorrow, i will be at databricks working w/@joshrosen to make sure that
> > the spark builds have any bugs hammered out.
> >
> > *** deployment plan:
> > unless something completely horrible happens, THE NEW JENKINS WILL GO
> LIVE
> > ON MONDAY (october 13th).
> >
> > all jenkins infrastructure will be DOWN for the entirety of the day
> > (starting at ~8am). this means no builds, period. i'm hoping that the
> > downtime will be much shorter than this, but we'll have to see how
> > everything goes.
> >
> > all test/build history WILL BE PRESERVED. i will be rsyncing the jenkins
> > jobs/ directory over, complete w/history as part of the deployment.
> >
> > once i'm feeling good about the state of things, i'll point the original
> > url to the new instances and send out an all clear.
> >
> > if you are a student at UC berkeley, you can log in to jenkins using your
> > LDAP login, and (by default) view but not change plans. if you do not
> have
> > a UC berkeley LDAP login, you can still view plans anonymously.
> >
> > IF YOU ARE A PLAN ADMIN, THEN PLEASE REACH OUT, ASAP, PRIVATELY AND I
> WILL
> > SET UP ADMIN ACCESS TO YOUR BUILDS.
> >
> > *** post deployment plan:
> > fix all of the things that break!
> >
> > i will be keeping a VERY close eye on the builds, checking for breaks,
> and
> > helping out where i can. if the situation is dire, i can always roll
> back
> > to the old jenkins infra... but i hope we never get to that point! :)
> >
> > i'm hoping that things will go smoothly, but please be patient as i'm
> > certain we'll hit a few bumps in the road.
> >
> > please let me know if you guys have any comments/questions/concerns...
> :)
> >
> > shane
> >
> > 1 - https://github.com/bigdatagenomics/bdg-services/pull/18
> > 2 - https://github.com/bigdatagenomics/avocado/pull/111
> >
>
Re: new jenkins update + tentative release date
Posted by shane knapp <sk...@berkeley.edu>.
reminder: this IS happening, first thing monday morning PDT. :)
On Wed, Oct 8, 2014 at 3:01 PM, shane knapp <sk...@berkeley.edu> wrote:
> greetings!
>
> i've got some updates regarding our new jenkins infrastructure, as well as
> the initial date and plan for rolling things out:
>
> *** current testing/build break whack-a-mole:
> a lot of out of date artifacts are cached in the current jenkins, which
> has caused a few builds during my testing to break due to dependency
> resolution failure[1][2].
>
> bumping these versions can cause your builds to fail, due to public api
> changes and the like. consider yourself warned that some projects might
> require some debugging... :)
>
> tomorrow, i will be at databricks working w/@joshrosen to make sure that
> the spark builds have any bugs hammered out.
>
> *** deployment plan:
> unless something completely horrible happens, THE NEW JENKINS WILL GO LIVE
> ON MONDAY (october 13th).
>
> all jenkins infrastructure will be DOWN for the entirety of the day
> (starting at ~8am). this means no builds, period. i'm hoping that the
> downtime will be much shorter than this, but we'll have to see how
> everything goes.
>
> all test/build history WILL BE PRESERVED. i will be rsyncing the jenkins
> jobs/ directory over, complete w/history as part of the deployment.
>
> once i'm feeling good about the state of things, i'll point the original
> url to the new instances and send out an all clear.
>
> if you are a student at UC berkeley, you can log in to jenkins using your
> LDAP login, and (by default) view but not change plans. if you do not have
> a UC berkeley LDAP login, you can still view plans anonymously.
>
> IF YOU ARE A PLAN ADMIN, THEN PLEASE REACH OUT, ASAP, PRIVATELY AND I WILL
> SET UP ADMIN ACCESS TO YOUR BUILDS.
>
> *** post deployment plan:
> fix all of the things that break!
>
> i will be keeping a VERY close eye on the builds, checking for breaks, and
> helping out where i can. if the situation is dire, i can always roll back
> to the old jenkins infra... but i hope we never get to that point! :)
>
> i'm hoping that things will go smoothly, but please be patient as i'm
> certain we'll hit a few bumps in the road.
>
> please let me know if you guys have any comments/questions/concerns... :)
>
> shane
>
> 1 - https://github.com/bigdatagenomics/bdg-services/pull/18
> 2 - https://github.com/bigdatagenomics/avocado/pull/111
>