You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Shane Knapp <sk...@berkeley.edu> on 2019/08/14 17:22:47 UTC
[build system] colo maintenance & outage tomorrow, 10am-2pm PDT
the berkeley colo had a major power distribution breaker fail, and they've
scheduled an emergency repair for tomorrow (thursday) @ 10am.
they expect this to take ~4 hours.
i will be shutting down the machines (again) ~9am, and bringing them all
back up once i get the all-clear.
sorry for the inconvenience...
shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
Re: [build system] colo maintenance & outage tomorrow, 10am-2pm PDT
Posted by Shane Knapp <sk...@berkeley.edu>.
a couple of workers needed a bit more time to finish booting up, so no need
for my excursion tomorrow. :)
builds be building, things look happy.
On Thu, Aug 15, 2019 at 6:46 PM Shane Knapp <sk...@berkeley.edu> wrote:
> it's back up! some of the workers didn't come back cleanly, so i'll have
> to hit up the colo tomorrow and persuade them in person.
>
> On Thu, Aug 15, 2019 at 6:45 PM Wenchen Fan <cl...@gmail.com> wrote:
>
>> Thanks for tracking it Shane!
>>
>> On Fri, Aug 16, 2019 at 7:41 AM Shane Knapp <sk...@berkeley.edu> wrote:
>>
>>> just got an update:
>>>
>>> there was a problem w/the replacement part, and they're trying to fix
>>> it. if that's successful, the expect to have power restored within the
>>> hour.
>>>
>>> if that doesn't work, a new (new) replacement part is scheduled to
>>> arrive at 8am tomorrow.
>>>
>>> shane
>>>
>>> On Thu, Aug 15, 2019 at 2:07 PM Shane Knapp <sk...@berkeley.edu> wrote:
>>>
>>>> quick update:
>>>>
>>>> it's been 4 hours, the colo is still down, and i haven't gotten any
>>>> news yet as to when they're planning on getting power restored.
>>>>
>>>> once i hear something i will let everyone know what's up.
>>>>
>>>> On Wed, Aug 14, 2019 at 10:22 AM Shane Knapp <sk...@berkeley.edu>
>>>> wrote:
>>>>
>>>>> the berkeley colo had a major power distribution breaker fail, and
>>>>> they've scheduled an emergency repair for tomorrow (thursday) @ 10am.
>>>>>
>>>>> they expect this to take ~4 hours.
>>>>>
>>>>> i will be shutting down the machines (again) ~9am, and bringing them
>>>>> all back up once i get the all-clear.
>>>>>
>>>>> sorry for the inconvenience...
>>>>>
>>>>> shane
>>>>> --
>>>>> Shane Knapp
>>>>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>>>>> https://rise.cs.berkeley.edu
>>>>>
>>>>
>>>>
>>>> --
>>>> Shane Knapp
>>>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>>>> https://rise.cs.berkeley.edu
>>>>
>>>
>>>
>>> --
>>> Shane Knapp
>>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>>> https://rise.cs.berkeley.edu
>>>
>>
>
> --
> Shane Knapp
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
>
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
Re: [build system] colo maintenance & outage tomorrow, 10am-2pm PDT
Posted by Shane Knapp <sk...@berkeley.edu>.
it's back up! some of the workers didn't come back cleanly, so i'll have
to hit up the colo tomorrow and persuade them in person.
On Thu, Aug 15, 2019 at 6:45 PM Wenchen Fan <cl...@gmail.com> wrote:
> Thanks for tracking it Shane!
>
> On Fri, Aug 16, 2019 at 7:41 AM Shane Knapp <sk...@berkeley.edu> wrote:
>
>> just got an update:
>>
>> there was a problem w/the replacement part, and they're trying to fix
>> it. if that's successful, the expect to have power restored within the
>> hour.
>>
>> if that doesn't work, a new (new) replacement part is scheduled to arrive
>> at 8am tomorrow.
>>
>> shane
>>
>> On Thu, Aug 15, 2019 at 2:07 PM Shane Knapp <sk...@berkeley.edu> wrote:
>>
>>> quick update:
>>>
>>> it's been 4 hours, the colo is still down, and i haven't gotten any news
>>> yet as to when they're planning on getting power restored.
>>>
>>> once i hear something i will let everyone know what's up.
>>>
>>> On Wed, Aug 14, 2019 at 10:22 AM Shane Knapp <sk...@berkeley.edu>
>>> wrote:
>>>
>>>> the berkeley colo had a major power distribution breaker fail, and
>>>> they've scheduled an emergency repair for tomorrow (thursday) @ 10am.
>>>>
>>>> they expect this to take ~4 hours.
>>>>
>>>> i will be shutting down the machines (again) ~9am, and bringing them
>>>> all back up once i get the all-clear.
>>>>
>>>> sorry for the inconvenience...
>>>>
>>>> shane
>>>> --
>>>> Shane Knapp
>>>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>>>> https://rise.cs.berkeley.edu
>>>>
>>>
>>>
>>> --
>>> Shane Knapp
>>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>>> https://rise.cs.berkeley.edu
>>>
>>
>>
>> --
>> Shane Knapp
>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>> https://rise.cs.berkeley.edu
>>
>
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
Re: [build system] colo maintenance & outage tomorrow, 10am-2pm PDT
Posted by Wenchen Fan <cl...@gmail.com>.
Thanks for tracking it Shane!
On Fri, Aug 16, 2019 at 7:41 AM Shane Knapp <sk...@berkeley.edu> wrote:
> just got an update:
>
> there was a problem w/the replacement part, and they're trying to fix it.
> if that's successful, the expect to have power restored within the hour.
>
> if that doesn't work, a new (new) replacement part is scheduled to arrive
> at 8am tomorrow.
>
> shane
>
> On Thu, Aug 15, 2019 at 2:07 PM Shane Knapp <sk...@berkeley.edu> wrote:
>
>> quick update:
>>
>> it's been 4 hours, the colo is still down, and i haven't gotten any news
>> yet as to when they're planning on getting power restored.
>>
>> once i hear something i will let everyone know what's up.
>>
>> On Wed, Aug 14, 2019 at 10:22 AM Shane Knapp <sk...@berkeley.edu> wrote:
>>
>>> the berkeley colo had a major power distribution breaker fail, and
>>> they've scheduled an emergency repair for tomorrow (thursday) @ 10am.
>>>
>>> they expect this to take ~4 hours.
>>>
>>> i will be shutting down the machines (again) ~9am, and bringing them all
>>> back up once i get the all-clear.
>>>
>>> sorry for the inconvenience...
>>>
>>> shane
>>> --
>>> Shane Knapp
>>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>>> https://rise.cs.berkeley.edu
>>>
>>
>>
>> --
>> Shane Knapp
>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>> https://rise.cs.berkeley.edu
>>
>
>
> --
> Shane Knapp
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
>
Re: [build system] colo maintenance & outage tomorrow, 10am-2pm PDT
Posted by Shane Knapp <sk...@berkeley.edu>.
just got an update:
there was a problem w/the replacement part, and they're trying to fix it.
if that's successful, the expect to have power restored within the hour.
if that doesn't work, a new (new) replacement part is scheduled to arrive
at 8am tomorrow.
shane
On Thu, Aug 15, 2019 at 2:07 PM Shane Knapp <sk...@berkeley.edu> wrote:
> quick update:
>
> it's been 4 hours, the colo is still down, and i haven't gotten any news
> yet as to when they're planning on getting power restored.
>
> once i hear something i will let everyone know what's up.
>
> On Wed, Aug 14, 2019 at 10:22 AM Shane Knapp <sk...@berkeley.edu> wrote:
>
>> the berkeley colo had a major power distribution breaker fail, and
>> they've scheduled an emergency repair for tomorrow (thursday) @ 10am.
>>
>> they expect this to take ~4 hours.
>>
>> i will be shutting down the machines (again) ~9am, and bringing them all
>> back up once i get the all-clear.
>>
>> sorry for the inconvenience...
>>
>> shane
>> --
>> Shane Knapp
>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>> https://rise.cs.berkeley.edu
>>
>
>
> --
> Shane Knapp
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
>
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
Re: [build system] colo maintenance & outage tomorrow, 10am-2pm PDT
Posted by Shane Knapp <sk...@berkeley.edu>.
quick update:
it's been 4 hours, the colo is still down, and i haven't gotten any news
yet as to when they're planning on getting power restored.
once i hear something i will let everyone know what's up.
On Wed, Aug 14, 2019 at 10:22 AM Shane Knapp <sk...@berkeley.edu> wrote:
> the berkeley colo had a major power distribution breaker fail, and they've
> scheduled an emergency repair for tomorrow (thursday) @ 10am.
>
> they expect this to take ~4 hours.
>
> i will be shutting down the machines (again) ~9am, and bringing them all
> back up once i get the all-clear.
>
> sorry for the inconvenience...
>
> shane
> --
> Shane Knapp
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
>
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu