You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@apache.org by Lance Albertson <la...@osuosl.org> on 2021/07/26 22:06:18 UTC
[osuosl-openpower] Backend storage issues
All,
We encountered a momentary issue with the backend storage while rebooting
some of the nodes. This has caused several VMs to be stuck in an unstable
state. We're going through all of the affected VMs and forcing a reboot on
them to resolve the issue.
Apologies for outages this may have caused.
Thanks!
--
Lance Albertson
Director
Oregon State University | Open Source Lab
Re: [osuosl-openpower] Backend storage issues
Posted by Lance Albertson <la...@osuosl.org>.
I just completed the reboots. All the systems should be back online. Please
let me know if you have any issue with your VM(s).
Thanks!
On Mon, Jul 26, 2021 at 5:28 PM Lance Albertson <la...@osuosl.org> wrote:
> It looks as though all of the VMs were affected by this issue and will
> require a hard reset. Since I need to reboot the hypervisors anyway, I'm
> going to go ahead and do that on each node and let it shutdown the VMs
> instead of doing a live migrations. This will impact every VM on the
> cluster. I will try and spot check the VMs to ensure they're back online
> after the reboot.
>
> Again, sorry for the issues this is causing.
>
> Thanks-
>
> On Mon, Jul 26, 2021 at 3:06 PM Lance Albertson <la...@osuosl.org> wrote:
>
>> All,
>>
>> We encountered a momentary issue with the backend storage while rebooting
>> some of the nodes. This has caused several VMs to be stuck in an unstable
>> state. We're going through all of the affected VMs and forcing a reboot on
>> them to resolve the issue.
>>
>> Apologies for outages this may have caused.
>>
>> Thanks!
>>
>> --
>> Lance Albertson
>> Director
>> Oregon State University | Open Source Lab
>>
>
>
> --
> Lance Albertson
> Director
> Oregon State University | Open Source Lab
>
--
Lance Albertson
Director
Oregon State University | Open Source Lab
Re: [osuosl-openpower] Backend storage issues
Posted by Lance Albertson <la...@osuosl.org>.
It looks as though all of the VMs were affected by this issue and will
require a hard reset. Since I need to reboot the hypervisors anyway, I'm
going to go ahead and do that on each node and let it shutdown the VMs
instead of doing a live migrations. This will impact every VM on the
cluster. I will try and spot check the VMs to ensure they're back online
after the reboot.
Again, sorry for the issues this is causing.
Thanks-
On Mon, Jul 26, 2021 at 3:06 PM Lance Albertson <la...@osuosl.org> wrote:
> All,
>
> We encountered a momentary issue with the backend storage while rebooting
> some of the nodes. This has caused several VMs to be stuck in an unstable
> state. We're going through all of the affected VMs and forcing a reboot on
> them to resolve the issue.
>
> Apologies for outages this may have caused.
>
> Thanks!
>
> --
> Lance Albertson
> Director
> Oregon State University | Open Source Lab
>
--
Lance Albertson
Director
Oregon State University | Open Source Lab