Dissimilar host OS within the same cluster - Not allowed?

["S.Fuller" <st...@gmail.com>]

I am working to transition the host OS for my Cloudstack 4.11.3 hosts from
CentOS to Ubuntu. I was able successfully bring up a new Ubuntu host with
Cloudstack and wanted to have it be part of an existing cluster, but after
attempting to add the server I'm noting the following warning in the agent
log file

"Can't add host: XX.XX.XX.XX with hostOS: Ubuntu into a cluster,in which
there are CentOS hosts added"

Is this really the case? I did not see anything obvious in the
documentation about this.  I was able to successfully add the new Ubuntu
server into a new cluster within the same POD, and have it see storage,
networking, etc, so the host itself appears to be configured correctly.

-- 
Steve Fuller
stevefuller@gmail.com

Re: Dissimilar host OS within the same cluster - Not allowed?

["S.Fuller" <st...@gmail.com>]

Yeah. Started playing around with it a little bit in a test setup and while
adding the host to the cluster by modifying the script worked, it appears
that there's a number of things like binaries and their locations that are
keyed off of the host OS type. Appears that it might be easier to realign
expectations vs moving down this particular road.

On Thu, Oct 13, 2022 at 1:26 PM Eric Green <er...@gmail.com> wrote:

> I just stood up a Ubuntu cluster and started adding hosts to it while
> decommissioning hosts from the Centos cluster and restarting virtual
> machines onto it. The whole point of a cluster is that it is homogeneous
> so that things like virtual machine migration can be assured to work.
> There's no way that Centos and Ubuntu are going to have identical KVM
> versions guaranteed to make things like virtual machine migration work,
> nevermind all the other environmental stuff that needs to be identical
> in order to make virtual machine migration actually work.
>
> In short, you can do it using the workarounds others have provided, but
> the more reliable way is to stand up a new cluster with the new OS and
> manually shut down VM's and then start them up on the new cluster.
> Painful if you don't have the compute resources to do the move all in
> one service outage, but (shrug). It's the reliable way to do things.
>
> On 10/13/2022 7:28 AM, S.Fuller wrote:
> > I am working to transition the host OS for my Cloudstack 4.11.3 hosts
> from
> > CentOS to Ubuntu. I was able successfully bring up a new Ubuntu host with
> > Cloudstack and wanted to have it be part of an existing cluster, but
> after
> > attempting to add the server I'm noting the following warning in the
> agent
> > log file
> >
> > "Can't add host: XX.XX.XX.XX with hostOS: Ubuntu into a cluster,in which
> > there are CentOS hosts added"
> >
> > Is this really the case? I did not see anything obvious in the
> > documentation about this.  I was able to successfully add the new Ubuntu
> > server into a new cluster within the same POD, and have it see storage,
> > networking, etc, so the host itself appears to be configured correctly.
> >
>


-- 
Steve Fuller
stevefuller@gmail.com

Re: Dissimilar host OS within the same cluster - Not allowed?

[Eric Green <er...@gmail.com>]

I just stood up a Ubuntu cluster and started adding hosts to it while 
decommissioning hosts from the Centos cluster and restarting virtual 
machines onto it. The whole point of a cluster is that it is homogeneous 
so that things like virtual machine migration can be assured to work. 
There's no way that Centos and Ubuntu are going to have identical KVM 
versions guaranteed to make things like virtual machine migration work, 
nevermind all the other environmental stuff that needs to be identical 
in order to make virtual machine migration actually work.

In short, you can do it using the workarounds others have provided, but 
the more reliable way is to stand up a new cluster with the new OS and 
manually shut down VM's and then start them up on the new cluster. 
Painful if you don't have the compute resources to do the move all in 
one service outage, but (shrug). It's the reliable way to do things.

On 10/13/2022 7:28 AM, S.Fuller wrote:
> I am working to transition the host OS for my Cloudstack 4.11.3 hosts from
> CentOS to Ubuntu. I was able successfully bring up a new Ubuntu host with
> Cloudstack and wanted to have it be part of an existing cluster, but after
> attempting to add the server I'm noting the following warning in the agent
> log file
>
> "Can't add host: XX.XX.XX.XX with hostOS: Ubuntu into a cluster,in which
> there are CentOS hosts added"
>
> Is this really the case? I did not see anything obvious in the
> documentation about this.  I was able to successfully add the new Ubuntu
> server into a new cluster within the same POD, and have it see storage,
> networking, etc, so the host itself appears to be configured correctly.
>

Re: Dissimilar host OS within the same cluster - Not allowed?

["S.Fuller" <st...@gmail.com>]

...and of course 30 seconds after I hit send on this message, I find the
very obvious line in the documentation about this. I was hoping I could add
these new servers to the same cluster and use maintenance mode to migrate
the KVM workloads over to the new hosts. Unfor, it appears my only
choice is going to be a much more gradual migration and coordinating
workload shutdowns, unless anyone has developed a better method?

On Thu, Oct 13, 2022 at 9:52 AM Nux <nu...@li.nux.ro> wrote:

> Hi,
>
> Cloudstack tries to keep that demarcation between OSes so as to maintain
> some sort of compatibility where software versions and capabilities are
> concerned, for stability's sake.
> That said, it can be bypassed in code, but it's best to do it like you
> have already shown by using another pod.
> ---
> Nux
> www.nux.ro
>
> On 2022-10-13 15:28, S.Fuller wrote:
> > I am working to transition the host OS for my Cloudstack 4.11.3 hosts
> > from
> > CentOS to Ubuntu. I was able successfully bring up a new Ubuntu host
> > with
> > Cloudstack and wanted to have it be part of an existing cluster, but
> > after
> > attempting to add the server I'm noting the following warning in the
> > agent
> > log file
> >
> > "Can't add host: XX.XX.XX.XX with hostOS: Ubuntu into a cluster,in
> > which
> > there are CentOS hosts added"
> >
> > Is this really the case? I did not see anything obvious in the
> > documentation about this.  I was able to successfully add the new
> > Ubuntu
> > server into a new cluster within the same POD, and have it see storage,
> > networking, etc, so the host itself appears to be configured correctly.
>


-- 
Steve Fuller
stevefuller@gmail.com

Re: Dissimilar host OS within the same cluster - Not allowed?

[Nux <nu...@li.nux.ro>]

OR you can modify the existing entries in the table to say "Ubuntu", 
this might be easier actually as it doesn't require messing with the 
shell script.

Be careful though as upon restart cloudstack-agent might check the OS 
again and update the table, so keep an eye on it.

---
Nux
www.nux.ro

On 2022-10-13 16:02, Nux wrote:
> If you really know what you are doing and are confident can get
> yourself out of a problem, then you can force Cloudstack to register
> your Ubuntu new server as CentOS by editing this script on the hv
> (prior to host addition):
> /usr/share/cloudstack-common/scripts/vm/hypervisor/versions.sh
> 
> Once you have migrated all your CentOS servers, you can edit the
> "host_details" DB table and replace CentOS with Ubuntu.
> 
> ---
> Nux
> www.nux.ro
> 
> On 2022-10-13 15:51, Nux wrote:
>> Hi,
>> 
>> Cloudstack tries to keep that demarcation between OSes so as to
>> maintain some sort of compatibility where software versions and
>> capabilities are concerned, for stability's sake.
>> That said, it can be bypassed in code, but it's best to do it like you
>> have already shown by using another pod.
>> ---
>> Nux
>> www.nux.ro
>> 
>> On 2022-10-13 15:28, S.Fuller wrote:
>>> I am working to transition the host OS for my Cloudstack 4.11.3 hosts 
>>> from
>>> CentOS to Ubuntu. I was able successfully bring up a new Ubuntu host 
>>> with
>>> Cloudstack and wanted to have it be part of an existing cluster, but 
>>> after
>>> attempting to add the server I'm noting the following warning in the 
>>> agent
>>> log file
>>> 
>>> "Can't add host: XX.XX.XX.XX with hostOS: Ubuntu into a cluster,in 
>>> which
>>> there are CentOS hosts added"
>>> 
>>> Is this really the case? I did not see anything obvious in the
>>> documentation about this.  I was able to successfully add the new 
>>> Ubuntu
>>> server into a new cluster within the same POD, and have it see 
>>> storage,
>>> networking, etc, so the host itself appears to be configured 
>>> correctly.

Re: Dissimilar host OS within the same cluster - Not allowed?

[Nux <nu...@li.nux.ro>]

If you really know what you are doing and are confident can get yourself 
out of a problem, then you can force Cloudstack to register your Ubuntu 
new server as CentOS by editing this script on the hv (prior to host 
addition):
/usr/share/cloudstack-common/scripts/vm/hypervisor/versions.sh

Once you have migrated all your CentOS servers, you can edit the 
"host_details" DB table and replace CentOS with Ubuntu.

---
Nux
www.nux.ro

On 2022-10-13 15:51, Nux wrote:
> Hi,
> 
> Cloudstack tries to keep that demarcation between OSes so as to
> maintain some sort of compatibility where software versions and
> capabilities are concerned, for stability's sake.
> That said, it can be bypassed in code, but it's best to do it like you
> have already shown by using another pod.
> ---
> Nux
> www.nux.ro
> 
> On 2022-10-13 15:28, S.Fuller wrote:
>> I am working to transition the host OS for my Cloudstack 4.11.3 hosts 
>> from
>> CentOS to Ubuntu. I was able successfully bring up a new Ubuntu host 
>> with
>> Cloudstack and wanted to have it be part of an existing cluster, but 
>> after
>> attempting to add the server I'm noting the following warning in the 
>> agent
>> log file
>> 
>> "Can't add host: XX.XX.XX.XX with hostOS: Ubuntu into a cluster,in 
>> which
>> there are CentOS hosts added"
>> 
>> Is this really the case? I did not see anything obvious in the
>> documentation about this.  I was able to successfully add the new 
>> Ubuntu
>> server into a new cluster within the same POD, and have it see 
>> storage,
>> networking, etc, so the host itself appears to be configured 
>> correctly.

Re: Dissimilar host OS within the same cluster - Not allowed?

[Nux <nu...@li.nux.ro>]

Hi,

Cloudstack tries to keep that demarcation between OSes so as to maintain 
some sort of compatibility where software versions and capabilities are 
concerned, for stability's sake.
That said, it can be bypassed in code, but it's best to do it like you 
have already shown by using another pod.
---
Nux
www.nux.ro

On 2022-10-13 15:28, S.Fuller wrote:
> I am working to transition the host OS for my Cloudstack 4.11.3 hosts 
> from
> CentOS to Ubuntu. I was able successfully bring up a new Ubuntu host 
> with
> Cloudstack and wanted to have it be part of an existing cluster, but 
> after
> attempting to add the server I'm noting the following warning in the 
> agent
> log file
> 
> "Can't add host: XX.XX.XX.XX with hostOS: Ubuntu into a cluster,in 
> which
> there are CentOS hosts added"
> 
> Is this really the case? I did not see anything obvious in the
> documentation about this.  I was able to successfully add the new 
> Ubuntu
> server into a new cluster within the same POD, and have it see storage,
> networking, etc, so the host itself appears to be configured correctly.