You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "shwstppr (via GitHub)" <gi...@apache.org> on 2025/02/11 07:35:16 UTC
[I] Multiple alerts for HA on VM and host [cloudstack]
shwstppr opened a new issue, #10367:
URL: https://github.com/apache/cloudstack/issues/10367
### problem
HA jobs for VM on host disconnection/down are not synchronized. CloudStack creates multiple HA jobs for the same VM when host goes into the Down state
### versions
4.19.2.0-SNAPSHOT
### The steps to reproduce the bug
1. Create a cluster with 2 hypervisor hosts
2. Create a HA enabled VM
3. Make the host running VM go into down state (used https://linuxconfig.org/how-to-crash-your-linux-system-with-fork-bomb)
4. After sometime when VM is started on a different hosts observe multiple HA alerts are generated for the same
### What to do about it?
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multiple alerts for HA on VM and host [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on issue #10367:
URL: https://github.com/apache/cloudstack/issues/10367#issuecomment-2854130353
fixed in #10624
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multiple alerts for HA on VM and host [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on issue #10367:
URL: https://github.com/apache/cloudstack/issues/10367#issuecomment-2703406590
reproduced both outputs now
I am noticing some serious changes in the alert manager between 4.19 and main, so not sure on which to address this yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multiple alerts for HA on VM and host [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on issue #10367:
URL: https://github.com/apache/cloudstack/issues/10367#issuecomment-2684823596
> One extra thing I noticed is that the host going to disconnected state does not yield an alert, though I think that is worth one as well. We could include that in this work or create a new issue for it.
I spoke too early: 
it actually happens multiple times. I did not get that for the VM HA job (yet)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multiple alerts for HA on VM and host [cloudstack]
Posted by "btzq (via GitHub)" <gi...@apache.org>.
btzq commented on issue #10367:
URL: https://github.com/apache/cloudstack/issues/10367#issuecomment-2654030164
We actually face this alot too! Ive noticed this but wasnt sure if this was normal.
In your test, does CS creating multiple jobs also reduce the likelihood of a sucessful HA?
Asking cause we often encounter situations where:
- In a large hypervisor, it it dies and HA is started, only a portion if VMs start up normally. The rest fails.
- When CS is busy, i encounter situations where a VM starts normally, but CS thinks its timedout, and stops the VM. It causes the VM to flap on and off for 10-15 minutes until it stops.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multiple alerts for HA on VM and host [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on issue #10367:
URL: https://github.com/apache/cloudstack/issues/10367#issuecomment-2684798631
@shwstppr , I tried to reproduce both with your fork-bomb method and with boldly powering off the host. I did get HA to kick in but only one job and one alert.
One extra thing I noticed is that the host going to disconnected state does not yield an alert, though I think that is worth one as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multiple alerts for HA on VM and host [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland closed issue #10367: Multiple alerts for HA on VM and host
URL: https://github.com/apache/cloudstack/issues/10367
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multiple alerts for HA on VM and host [cloudstack]
Posted by "btzq (via GitHub)" <gi...@apache.org>.
btzq commented on issue #10367:
URL: https://github.com/apache/cloudstack/issues/10367#issuecomment-2655188792
I am also now wondering. In CS i remember theres a global setting for max worker threads or ha threads.
Will fixing the duplicate HA jobs also improve the efficiency of these threads? 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multiple alerts for HA on VM and host [cloudstack]
Posted by "shwstppr (via GitHub)" <gi...@apache.org>.
shwstppr commented on issue #10367:
URL: https://github.com/apache/cloudstack/issues/10367#issuecomment-2656290371
@btzq In my testing with 2x KVM hosts and limited VMs, it didn't fail HA for those VMs.
Fixing this should help the performance of HA in my opinion as it would spend less time working on duplicate jobs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org