You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by shane knapp <sk...@berkeley.edu> on 2015/05/14 02:12:31 UTC
[build system] scheduled datacenter downtime, sunday may 17th
our datacenter is rejiggering our network (read: fully re-engineering large
portions from the ground up) and has downtime scheduled from 9am-3pm PDT,
this sunday may17th.
this means our jenkins instance will not be available to the outside world,
and i will be putting jenkins in to quiet mode the night before. this will
allow any running builds to finish, and to save me from getting up @ 6am on
my day off. :)
once things are back up and running (~3pm or earlier), i will purge the
build queue and bring jenkins out of quiet mode.
of course, stay tuned to this bat-channel for future, and potentially
riveting updates!
Re: [build system] scheduled datacenter downtime, sunday may 17th
Posted by shane knapp <sk...@berkeley.edu>.
our sysadmins fixed the auth issue about an hour ago... /etc/shadow's
perms got borked somehow and that was breaking logins for local (non-ldap)
accounts.
we're all green.
On Sun, May 17, 2015 at 2:46 PM, shane knapp <sk...@berkeley.edu> wrote:
> ...and we're back up. it looks like things are going to be flaky for the
> next couple of hours, so i'm going to let this stabilize before i dig in to
> the auth problems.
>
> On Sun, May 17, 2015 at 2:34 PM, shane knapp <sk...@berkeley.edu> wrote:
>
>> ...and we've lost network connectivity again.
>>
>> things are still very flaky. more updates as they come.
>>
>> On Sun, May 17, 2015 at 2:32 PM, shane knapp <sk...@berkeley.edu> wrote:
>>
>>> actually, LDAP auth is fixed, but if you have a local account that i've
>>> created for you, it's not letting you log in to jenkins' UI.
>>>
>>> looking at this now.
>>>
>>> On Sun, May 17, 2015 at 2:13 PM, shane knapp <sk...@berkeley.edu>
>>> wrote:
>>>
>>>> auth is fixed, and jenkins is out of quiet mode and now building.
>>>> sorry for the delay!
>>>>
>>>> On Sun, May 17, 2015 at 2:06 PM, shane knapp <sk...@berkeley.edu>
>>>> wrote:
>>>>
>>>>> machine rebooted, but auth is completely broken (web and CLI on the
>>>>> server). i'm trying to fix this now.
>>>>>
>>>>> On Sun, May 17, 2015 at 1:51 PM, shane knapp <sk...@berkeley.edu>
>>>>> wrote:
>>>>>
>>>>>> ok, i think it's time to reboot the jenkins master.
>>>>>>
>>>>>> On Sun, May 17, 2015 at 1:44 PM, shane knapp <sk...@berkeley.edu>
>>>>>> wrote:
>>>>>>
>>>>>>> jenkins is being a little recalcitrant and i'm looking at logs to
>>>>>>> see why it won't start.
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: [build system] scheduled datacenter downtime, sunday may 17th
Posted by shane knapp <sk...@berkeley.edu>.
...and we're back up. it looks like things are going to be flaky for the
next couple of hours, so i'm going to let this stabilize before i dig in to
the auth problems.
On Sun, May 17, 2015 at 2:34 PM, shane knapp <sk...@berkeley.edu> wrote:
> ...and we've lost network connectivity again.
>
> things are still very flaky. more updates as they come.
>
> On Sun, May 17, 2015 at 2:32 PM, shane knapp <sk...@berkeley.edu> wrote:
>
>> actually, LDAP auth is fixed, but if you have a local account that i've
>> created for you, it's not letting you log in to jenkins' UI.
>>
>> looking at this now.
>>
>> On Sun, May 17, 2015 at 2:13 PM, shane knapp <sk...@berkeley.edu> wrote:
>>
>>> auth is fixed, and jenkins is out of quiet mode and now building. sorry
>>> for the delay!
>>>
>>> On Sun, May 17, 2015 at 2:06 PM, shane knapp <sk...@berkeley.edu>
>>> wrote:
>>>
>>>> machine rebooted, but auth is completely broken (web and CLI on the
>>>> server). i'm trying to fix this now.
>>>>
>>>> On Sun, May 17, 2015 at 1:51 PM, shane knapp <sk...@berkeley.edu>
>>>> wrote:
>>>>
>>>>> ok, i think it's time to reboot the jenkins master.
>>>>>
>>>>> On Sun, May 17, 2015 at 1:44 PM, shane knapp <sk...@berkeley.edu>
>>>>> wrote:
>>>>>
>>>>>> jenkins is being a little recalcitrant and i'm looking at logs to see
>>>>>> why it won't start.
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: [build system] scheduled datacenter downtime, sunday may 17th
Posted by shane knapp <sk...@berkeley.edu>.
...and we've lost network connectivity again.
things are still very flaky. more updates as they come.
On Sun, May 17, 2015 at 2:32 PM, shane knapp <sk...@berkeley.edu> wrote:
> actually, LDAP auth is fixed, but if you have a local account that i've
> created for you, it's not letting you log in to jenkins' UI.
>
> looking at this now.
>
> On Sun, May 17, 2015 at 2:13 PM, shane knapp <sk...@berkeley.edu> wrote:
>
>> auth is fixed, and jenkins is out of quiet mode and now building. sorry
>> for the delay!
>>
>> On Sun, May 17, 2015 at 2:06 PM, shane knapp <sk...@berkeley.edu> wrote:
>>
>>> machine rebooted, but auth is completely broken (web and CLI on the
>>> server). i'm trying to fix this now.
>>>
>>> On Sun, May 17, 2015 at 1:51 PM, shane knapp <sk...@berkeley.edu>
>>> wrote:
>>>
>>>> ok, i think it's time to reboot the jenkins master.
>>>>
>>>> On Sun, May 17, 2015 at 1:44 PM, shane knapp <sk...@berkeley.edu>
>>>> wrote:
>>>>
>>>>> jenkins is being a little recalcitrant and i'm looking at logs to see
>>>>> why it won't start.
>>>>>
>>>>
>>>>
>>>
>>
>
Re: [build system] scheduled datacenter downtime, sunday may 17th
Posted by shane knapp <sk...@berkeley.edu>.
actually, LDAP auth is fixed, but if you have a local account that i've
created for you, it's not letting you log in to jenkins' UI.
looking at this now.
On Sun, May 17, 2015 at 2:13 PM, shane knapp <sk...@berkeley.edu> wrote:
> auth is fixed, and jenkins is out of quiet mode and now building. sorry
> for the delay!
>
> On Sun, May 17, 2015 at 2:06 PM, shane knapp <sk...@berkeley.edu> wrote:
>
>> machine rebooted, but auth is completely broken (web and CLI on the
>> server). i'm trying to fix this now.
>>
>> On Sun, May 17, 2015 at 1:51 PM, shane knapp <sk...@berkeley.edu> wrote:
>>
>>> ok, i think it's time to reboot the jenkins master.
>>>
>>> On Sun, May 17, 2015 at 1:44 PM, shane knapp <sk...@berkeley.edu>
>>> wrote:
>>>
>>>> jenkins is being a little recalcitrant and i'm looking at logs to see
>>>> why it won't start.
>>>>
>>>
>>>
>>
>
Re: [build system] scheduled datacenter downtime, sunday may 17th
Posted by shane knapp <sk...@berkeley.edu>.
auth is fixed, and jenkins is out of quiet mode and now building. sorry
for the delay!
On Sun, May 17, 2015 at 2:06 PM, shane knapp <sk...@berkeley.edu> wrote:
> machine rebooted, but auth is completely broken (web and CLI on the
> server). i'm trying to fix this now.
>
> On Sun, May 17, 2015 at 1:51 PM, shane knapp <sk...@berkeley.edu> wrote:
>
>> ok, i think it's time to reboot the jenkins master.
>>
>> On Sun, May 17, 2015 at 1:44 PM, shane knapp <sk...@berkeley.edu> wrote:
>>
>>> jenkins is being a little recalcitrant and i'm looking at logs to see
>>> why it won't start.
>>>
>>
>>
>
Re: [build system] scheduled datacenter downtime, sunday may 17th
Posted by shane knapp <sk...@berkeley.edu>.
machine rebooted, but auth is completely broken (web and CLI on the
server). i'm trying to fix this now.
On Sun, May 17, 2015 at 1:51 PM, shane knapp <sk...@berkeley.edu> wrote:
> ok, i think it's time to reboot the jenkins master.
>
> On Sun, May 17, 2015 at 1:44 PM, shane knapp <sk...@berkeley.edu> wrote:
>
>> jenkins is being a little recalcitrant and i'm looking at logs to see why
>> it won't start.
>>
>
>
Re: [build system] scheduled datacenter downtime, sunday may 17th
Posted by shane knapp <sk...@berkeley.edu>.
ok, i think it's time to reboot the jenkins master.
On Sun, May 17, 2015 at 1:44 PM, shane knapp <sk...@berkeley.edu> wrote:
> jenkins is being a little recalcitrant and i'm looking at logs to see why
> it won't start.
>
Re: [build system] scheduled datacenter downtime, sunday may 17th
Posted by shane knapp <sk...@berkeley.edu>.
jenkins is being a little recalcitrant and i'm looking at logs to see why
it won't start.
Re: [build system] scheduled datacenter downtime, sunday may 17th
Posted by Josh Rosen <ro...@gmail.com>.
Reminder: the network migration has started this morning, so Jenkins is
currently down.
Status updates on the migration are being published at
http://ucbsystems.org/
On Wed, May 13, 2015 at 5:12 PM, shane knapp <sk...@berkeley.edu> wrote:
> our datacenter is rejiggering our network (read: fully re-engineering large
> portions from the ground up) and has downtime scheduled from 9am-3pm PDT,
> this sunday may17th.
>
> this means our jenkins instance will not be available to the outside world,
> and i will be putting jenkins in to quiet mode the night before. this will
> allow any running builds to finish, and to save me from getting up @ 6am on
> my day off. :)
>
> once things are back up and running (~3pm or earlier), i will purge the
> build queue and bring jenkins out of quiet mode.
>
> of course, stay tuned to this bat-channel for future, and potentially
> riveting updates!
>