System VMs and bridge connections

[Jon Marshall <jm...@hotmail.co.uk>]

Can someone tell me where I am going wrong or if this is possible (apologies for the long post)



I have configured the management server as per installation instructions with just an interface in the management network using subnet 172.16.7.0/27



I then configured a host with 3 separate NICs –



  1.  Management interface with IP from same subnet as management server IP
  2.  Second NIC using a subnet of 172.16.6.128/25. This is meant to be the subnet for the VMs.
  3.  Third NIC with an IP from the 172.16.232.0/28 subnet which is where the NFS server is.



I am using KVM so configured Linux bridges eg cloudbr0 for 1), cloudbr1 for 2) and cloudbr2 for 3).



I then connected to the UI and did the basic setup.



It worked in that the host showed as up and the system VMs came up but neither system VM was working properly so I logged into both and saw the same problem.



The VMs had picked up an IP from both the management network and the VM subnet eg.



Eth1 – 172.16.7.10

Eth2 – 172.16.6.177



The default gateway was 172.16.6.129 ie. From the VM subnet. But neither VM could ping that default gateway.



When I looked at the bridges on the host the mac address of eth2 was seen on cloudbr0 which is the management subnet. When I then logged into the physical L3 switch I could see eth2’s mac address in the management vlan and not the VM subnet vlan.



So it seems like the bridging between the VMs and the physical NICs is not working properly or more likely there is something basic I am not understanding.



Should I be looking to use advanced networking or is the above setup possible with just basic network.



I am using cloudstack v4.10 and feel a bit of an idiot as all the docs say setting up basic networking is really easy 😊  (I did do an install where it is all on the same server and that worked fine).



Any pointers much appreciated.



PS.  I cannot console to the system VMs because of the above and the SSVM does not have interface in the NFS network even though there is a physical NIC on the host.