You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Florian Noel <f....@webetsolutions.com> on 2021/09/29 12:00:01 UTC
HA behaviour with Cloudstack / XCP-NG
Hi everyone,
We are using Cloudstack 4.15.1 with a cluster of hypervisors XCP 8.2
We are testing High Availability.
I have read that XCP (Xenserver) is responsible to elect a new pool master if the pool master fails.
And Cloudstack is responsible to restart virtual machines if a hypervisor fails.
We have virtual machines with compute offering inclued HA capability and virtual routers on the pool master.
If we disconnect the management cable from the pool master, HA-XCP electes a new pool master, the old pool master restarts.
This is expected behaviour.
However, Cloudstack never launches the virtual machines and the virtual routers on our other nodes XCP that were on the pool master.
Is this the expected behaviour ? We would have liked Cloudstack to be able to launch the VMs on the other nodes.
I'm not sure Cloudstack knows the pool master have changed. Why ? How can I solve this ?
We have also tested disconnect storage cables from the pool master and in this case, Cloudstack is able to launch virtual machines and virtual routers on our other nodes of the cluster after few minutes.
Thanks for your help.
Best regards.
Florian
[Logo Web et Solutions]<https://cloud.letsignit.com/collect/bc/60e5c62f48323abd316580a3?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDzCgMKxZAoqvlt4NqCVlovo0bn9PcMUWFMak1jGIGRgGg==>
[Facebook]<https://cloud.letsignit.com/collect/bc/60e5c62f48323abd316580a3?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDyIo6EwBskR6pg3M12nuwExu8D-tkYDv5BE1h2dA1rTOfbHIEta8XTaUC0Et-KgDBM=>
[Twitter]<https://cloud.letsignit.com/collect/bc/60e5c62f48323abd316580a3?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDxVGISVA_RnJl21WVuzHCTH_v3e4PfK5YBq_Q228Kqxog==>
[LinkedIn]<https://cloud.letsignit.com/collect/bc/60e5c62f48323abd316580a3?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDz5UNyOTEm_EvRFXdshn5-xaylm0Ysa1fuL9vCg5uDKfouGPQSgwbQq28Nl7_fXFIA=>
[Youtube]<https://cloud.letsignit.com/collect/bc/60e5c62f48323abd316580a3?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDyEop3qI2i2HFrm2U65Sd5oXm55IjnZsXt1s4eREvsJGMpsgNaX2L3OdByrUM3b4Xg=>
Florian Noel
Administrateur Systèmes Et Réseaux
[https://storage.letsignit.com/icons/designer/v2/phone-1.png] 02 35 78 11 90
705 Avenue Isaac Newton
76800 Saint-Etienne-Du-Rouvray
[Payneo]<https://cloud.letsignit.com/collect/b/60ed92296e8c02bf93d4f9aa?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDx4rIKe6rk374sFS07v0YLIvIF68SXTHzNmGDb3XO6dLQ==>
Re: HA behaviour with Cloudstack / XCP-NG
Posted by Daan Hoogland <da...@gmail.com>.
>
> However, Cloudstack never launches the virtual machines and the virtual
> routers on our other nodes XCP that were on the pool master.
> Is this the expected behaviour ?
>
Yes, this is expected behaviour, cloudstack doesn't know if the VMs are
still running in this 'splitbrain' scenario and thus cannot risk restarting
them.
> We would have liked Cloudstack to be able to launch the VMs on the other
> nodes.
>
Only when cloudstack is sure the VMs are stopped (e.g. the old pool master
reboots and comes back online with only stoppen VMs) will it restart the
VMs.
On Wed, Sep 29, 2021 at 2:13 PM Florian Noel <f....@webetsolutions.com>
wrote:
> Hi everyone,
>
> We are using Cloudstack 4.15.1 with a cluster of hypervisors XCP 8.2
> We are testing High Availability.
>
> I have read that XCP (Xenserver) is responsible to elect a new pool master
> if the pool master fails.
> And Cloudstack is responsible to restart virtual machines if a hypervisor
> fails.
>
> We have virtual machines with compute offering inclued HA capability and
> virtual routers on the pool master.
> If we disconnect the management cable from the pool master, HA-XCP electes
> a new pool master, the old pool master restarts.
> This is expected behaviour.
>
>
> However, Cloudstack never launches the virtual machines and the virtual
> routers on our other nodes XCP that were on the pool master.
> Is this the expected behaviour ? We would have liked Cloudstack to be able
> to launch the VMs on the other nodes.
> I'm not sure Cloudstack knows the pool master have changed. Why ? How can
> I solve this ?
>
>
> We have also tested disconnect storage cables from the pool master and in
> this case, Cloudstack is able to launch virtual machines and virtual
> routers on our other nodes of the cluster after few minutes.
>
> Thanks for your help.
>
> Best regards.
>
> Florian
>
>
> [Logo Web et Solutions]<
> https://cloud.letsignit.com/collect/bc/60e5c62f48323abd316580a3?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDzCgMKxZAoqvlt4NqCVlovo0bn9PcMUWFMak1jGIGRgGg==
> >
>
> [Facebook]<
> https://cloud.letsignit.com/collect/bc/60e5c62f48323abd316580a3?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDyIo6EwBskR6pg3M12nuwExu8D-tkYDv5BE1h2dA1rTOfbHIEta8XTaUC0Et-KgDBM=
> >
>
> [Twitter]<
> https://cloud.letsignit.com/collect/bc/60e5c62f48323abd316580a3?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDxVGISVA_RnJl21WVuzHCTH_v3e4PfK5YBq_Q228Kqxog==
> >
>
> [LinkedIn]<
> https://cloud.letsignit.com/collect/bc/60e5c62f48323abd316580a3?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDz5UNyOTEm_EvRFXdshn5-xaylm0Ysa1fuL9vCg5uDKfouGPQSgwbQq28Nl7_fXFIA=
> >
>
> [Youtube]<
> https://cloud.letsignit.com/collect/bc/60e5c62f48323abd316580a3?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDyEop3qI2i2HFrm2U65Sd5oXm55IjnZsXt1s4eREvsJGMpsgNaX2L3OdByrUM3b4Xg=
> >
>
> Florian Noel
>
> Administrateur Systèmes Et Réseaux
>
> [https://storage.letsignit.com/icons/designer/v2/phone-1.png] 02 35 78 11
> 90
>
> 705 Avenue Isaac Newton
>
> 76800 Saint-Etienne-Du-Rouvray
>
> [Payneo]<
> https://cloud.letsignit.com/collect/b/60ed92296e8c02bf93d4f9aa?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDx4rIKe6rk374sFS07v0YLIvIF68SXTHzNmGDb3XO6dLQ==
> >
>
>
>
>
--
Daan
Re: HA behaviour with Cloudstack / XCP-NG
Posted by Daan Hoogland <da...@gmail.com>.
>
> However, Cloudstack never launches the virtual machines and the virtual
> routers on our other nodes XCP that were on the pool master.
> Is this the expected behaviour ?
>
Yes, this is expected behaviour, cloudstack doesn't know if the VMs are
still running in this 'splitbrain' scenario and thus cannot risk restarting
them.
> We would have liked Cloudstack to be able to launch the VMs on the other
> nodes.
>
Only when cloudstack is sure the VMs are stopped (e.g. the old pool master
reboots and comes back online with only stoppen VMs) will it restart the
VMs.
On Wed, Sep 29, 2021 at 2:13 PM Florian Noel <f....@webetsolutions.com>
wrote:
> Hi everyone,
>
> We are using Cloudstack 4.15.1 with a cluster of hypervisors XCP 8.2
> We are testing High Availability.
>
> I have read that XCP (Xenserver) is responsible to elect a new pool master
> if the pool master fails.
> And Cloudstack is responsible to restart virtual machines if a hypervisor
> fails.
>
> We have virtual machines with compute offering inclued HA capability and
> virtual routers on the pool master.
> If we disconnect the management cable from the pool master, HA-XCP electes
> a new pool master, the old pool master restarts.
> This is expected behaviour.
>
>
> However, Cloudstack never launches the virtual machines and the virtual
> routers on our other nodes XCP that were on the pool master.
> Is this the expected behaviour ? We would have liked Cloudstack to be able
> to launch the VMs on the other nodes.
> I'm not sure Cloudstack knows the pool master have changed. Why ? How can
> I solve this ?
>
>
> We have also tested disconnect storage cables from the pool master and in
> this case, Cloudstack is able to launch virtual machines and virtual
> routers on our other nodes of the cluster after few minutes.
>
> Thanks for your help.
>
> Best regards.
>
> Florian
>
>
> [Logo Web et Solutions]<
> https://cloud.letsignit.com/collect/bc/60e5c62f48323abd316580a3?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDzCgMKxZAoqvlt4NqCVlovo0bn9PcMUWFMak1jGIGRgGg==
> >
>
> [Facebook]<
> https://cloud.letsignit.com/collect/bc/60e5c62f48323abd316580a3?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDyIo6EwBskR6pg3M12nuwExu8D-tkYDv5BE1h2dA1rTOfbHIEta8XTaUC0Et-KgDBM=
> >
>
> [Twitter]<
> https://cloud.letsignit.com/collect/bc/60e5c62f48323abd316580a3?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDxVGISVA_RnJl21WVuzHCTH_v3e4PfK5YBq_Q228Kqxog==
> >
>
> [LinkedIn]<
> https://cloud.letsignit.com/collect/bc/60e5c62f48323abd316580a3?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDz5UNyOTEm_EvRFXdshn5-xaylm0Ysa1fuL9vCg5uDKfouGPQSgwbQq28Nl7_fXFIA=
> >
>
> [Youtube]<
> https://cloud.letsignit.com/collect/bc/60e5c62f48323abd316580a3?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDyEop3qI2i2HFrm2U65Sd5oXm55IjnZsXt1s4eREvsJGMpsgNaX2L3OdByrUM3b4Xg=
> >
>
> Florian Noel
>
> Administrateur Systèmes Et Réseaux
>
> [https://storage.letsignit.com/icons/designer/v2/phone-1.png] 02 35 78 11
> 90
>
> 705 Avenue Isaac Newton
>
> 76800 Saint-Etienne-Du-Rouvray
>
> [Payneo]<
> https://cloud.letsignit.com/collect/b/60ed92296e8c02bf93d4f9aa?p=NCQXXscJv3N-mDjmqdZzYH59ppVbYP3afFkR7SxQ1JaS_ev9TYs06R5yG_cSPe6tLuS3Bgn1EjTO39P6hIWtNhqUZ5n-wh878kG0mKc-TDx4rIKe6rk374sFS07v0YLIvIF68SXTHzNmGDb3XO6dLQ==
> >
>
>
>
>
--
Daan