ha check running with disabled ha

[me...@swen.io]

Hi all,

 

we encountered a strange issue today in our lab installation. We are running
CS 4.19.0 upgraded from CS 4.18.1 with linstor as primary storage. We are
using a linstor jar file provided by Linbit which is not the default one in
4.19.0!

This plugin already implemented the feature to use linstor storage for host
ha. I provide this only for information reasons.

 

The issue we encountered was that even we have do not have HA enabled on
cluster and host level, cloudstack agent on our KVM hosts triggered HA
actions and rebooted our hosts. We found this on our agent.log:

 

Feb 19 11:53:05 pc-kvm-2 java[6617]: WARN  [kvm.resource.KVMHAMonitor]
(Thread-1:) (logid:) Write heartbeat for pool
[71c272d3-b180-4b18-a0fc-cfc1dc5b86c9] failed: Down; try: 2 of 5.

Feb 19 11:58:58 pc-kvm-2 java[9465]: WARN  [kvm.resource.KVMHAMonitor]
(Thread-1:) (logid:) Write heartbeat for pool
[71c272d3-b180-4b18-a0fc-cfc1dc5b86c9] failed: Down; try: 3 of 5.

Feb 19 12:00:08 pc-kvm-2 java[9465]: WARN  [kvm.resource.KVMHAMonitor]
(Thread-1:) (logid:) Write heartbeat for pool
[71c272d3-b180-4b18-a0fc-cfc1dc5b86c9] failed: Down; try: 4 of 5.

Feb 19 12:01:08 pc-kvm-2 java[9465]: WARN  [kvm.resource.KVMHAMonitor]
(Thread-1:) (logid:) Write heartbeat for pool
[71c272d3-b180-4b18-a0fc-cfc1dc5b86c9] failed: Down; try: 5 of 5.

Feb 19 12:01:08 pc-kvm-2 java[9465]: WARN  [kvm.resource.KVMHAMonitor]
(Thread-1:) (logid:) Write heartbeat for pool
[71c272d3-b180-4b18-a0fc-cfc1dc5b86c9] failed: Down; stopping
cloudstack-agent.

Feb 19 12:02:08 pc-kvm-2 heartbeat: kvmspheartbeat.sh will reboot system
because it was unable to write the heartbeat to the storage.

 

We and Linbit did some debugging but we were unable to find any  code which
checks if host HA is enabled or not and will not perform HA tasks if HA is
disabled. Can somebody please double-check this?

Thank you very much!

 

Regards,

Swen