[osuosl-openpower] Service disruption on Ceph storage cluster

[Lance Albertson <la...@osuosl.org>]

All,

While rebooting one of the Ceph cluster nodes this morning, the cluster got
into an inconsistent state blocking I/O requests for some VMs. This started
at around 9:20AM PDT (1620 UTC) and was resolved around 9:40AM PDT (1640).
Prior to rebooting the machine I was performing an upgrade on the Ceph
cluster from the Nautilus to Mimic release. The cluster seemed to be in an
OK state after performing an upgrade, however after rebooting one of the
nodes I ran into this issue.

I'm going to continue rebooting the remaining nodes one at a time and
hopefully the same issue doesn't happen again.

Sorry for any issues this may have caused. I'll send any further updates as
they are needed.

Thanks-

-- 
Lance Albertson
Director
Oregon State University | Open Source Lab