You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by "bradh352 (via GitHub)" <gi...@apache.org> on 2025/06/10 13:15:51 UTC
[D] How to join an LXC container host? [cloudstack]
GitHub user bradh352 created a discussion: How to join an LXC container host?
I've just spun up a cloudstack 4.20.1 8-node environment. All hosts are configured identically (regardless if their purpose is for KVM or LXC).
I joined the first 5 hosts as KVM hosts and that works fine.
Then I created a new cluster **under the same zone** via the UI for my LXC hosts with the intention of joining the remaining 3 nodes as LXC nodes but it fails (after a long wait).
Looking at the agent logs, all I see is:
```
DEBUG:root:execute:uname -r
DEBUG:root:execute:uname -m
DEBUG:root:execute:hostname -f
DEBUG:root:execute:kvm-ok
DEBUG:root:execute:awk '/MemTotal/ { printf "%.3f \n", $2/1024 }' /proc/meminfo
DEBUG:root:execute:ip a | grep "^\w" | grep -iv "^lo" | wc -l
DEBUG:root:execute:service apparmor status
DEBUG:root:execute:apparmor_status |grep libvirt
DEBUG:root:Failed to execute:
DEBUG:root:cloudbr0 is not a network device, is it down?
DEBUG:root:execute:sudo /usr/sbin/service network-manager status
DEBUG:root:Failed to execute:Unit network-manager.service could not be found.
DEBUG:root:execute:ip route show default | awk '{print $3,$5}'
DEBUG:root:execute:ifconfig public
DEBUG:root:Failed to execute:/bin/sh: 1: ifconfig: not found
DEBUG:root:Failed to get address from ifconfig
DEBUG:root:execute:sudo update-rc.d -f apparmor remove
DEBUG:root:execute:sudo update-rc.d -f apparmor defaults
DEBUG:root:execute:sudo /usr/sbin/service apparmor status
DEBUG:root:execute:sudo /usr/sbin/service apparmor start
DEBUG:root:execute:sudo /usr/sbin/service apparmor status
DEBUG:root:execute:sudo /usr/sbin/service apparmor start
DEBUG:root:execute:sudo update-rc.d -f network-manager remove
DEBUG:root:execute:sudo update-rc.d -f network-manager defaults
DEBUG:root:Failed to execute:update-rc.d: error: unable to read /etc/init.d/network-manager
DEBUG:root:execute:sudo /usr/sbin/service network-manager status
DEBUG:root:Failed to execute:Unit network-manager.service could not be found.
DEBUG:root:execute:sudo /usr/sbin/service network-manager start
DEBUG:root:Failed to execute:Failed to start network-manager.service: Unit network-manager.service not found.
DEBUG:root:execute:sudo /usr/sbin/service network-manager status
DEBUG:root:Failed to execute:Unit network-manager.service could not be found.
DEBUG:root:execute:sudo /usr/sbin/service network-manager start
DEBUG:root:Failed to execute:Failed to start network-manager.service: Unit network-manager.service not found.
DEBUG:root:execute:/etc/init.d/networking stop
DEBUG:root:Failed to execute:/bin/sh: 1: /etc/init.d/networking: not found
DEBUG:root:execute:/etc/init.d/networking start
DEBUG:root:Failed to execute:/bin/sh: 1: /etc/init.d/networking: not found
```
Looking at the command history, I see:
```
Jun 9 21:05:52 node7.testenv.bradhouse.dev sudo: cloudstack : PWD=/home/cloudstack ; USER=root ; COMMAND=/usr/bin/cloudstack-setup-agent -m 10.10.100.2,10.10.100.3,10.10.100.4 -z 2 -p 2 -c 5 -g 78999d33-6584-340b-a4bc-8c19b52aa195 -a -s --pubNic=cloudbr0 --prvNic=cloudbr0 --guestNic=cloudbr0 --hypervisor=lxc
```
compared to KVM:
```
Jun 9 20:43:31 node4.testenv.bradhouse.dev sudo: cloudstack : PWD=/home/cloudstack ; USER=root ; COMMAND=/usr/bin/cloudstack-setup-agent -m 10.10.100.2,10.10.100.3,10.10.100.4 -z 2 -p 2 -c 2 -g 044ed14f-03ec-31b4-a02a-c842cdcbba1b -a -s --pubNic=public --prvNic=hypervisor --guestNic=public --hypervisor=kvm
```
The thing that stands out at me is the pubNic, prvNic and guestNic ... the LXC should match what is passed for the KVM nodes, but instead are reverting to the default 'cloudbr0' which I don't have as I named my bridges more useful names. Any idea why these aren't passed properly?
Perhaps I missed something in the new cluster? I thought the network settings were zone-wide.
GitHub link: https://github.com/apache/cloudstack/discussions/10999
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] How to join an LXC container host? [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
GitHub user weizhouapache added a comment to the discussion: How to join an LXC container host?
I would say , it is not tested and well-maintained
GitHub link: https://github.com/apache/cloudstack/discussions/10999#discussioncomment-13425113
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] How to join an LXC container host? [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
GitHub user weizhouapache added a comment to the discussion: How to join an LXC container host?
@bradh352
I do not remember anyone else uses LXC
maybe add the host to a KVM host to see if it works ?
GitHub link: https://github.com/apache/cloudstack/discussions/10999#discussioncomment-13422286
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] How to join an LXC container host? [cloudstack]
Posted by "bradh352 (via GitHub)" <gi...@apache.org>.
GitHub user bradh352 added a comment to the discussion: How to join an LXC container host?
yes, if I join it to the kvm cluster it joins fine. this is an lxc-specific issue. are you saying lxc is no longer supported?
GitHub link: https://github.com/apache/cloudstack/discussions/10999#discussioncomment-13424913
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] How to join an LXC container host? [cloudstack]
Posted by "bradh352 (via GitHub)" <gi...@apache.org>.
GitHub user bradh352 closed a discussion: How to join an LXC container host?
I've just spun up a cloudstack 4.20.1 8-node environment. All hosts are configured identically (regardless if their purpose is for KVM or LXC).
I joined the first 5 hosts as KVM hosts and that works fine.
Then I created a new cluster **under the same zone** via the UI for my LXC hosts with the intention of joining the remaining 3 nodes as LXC nodes but it fails (after a long wait).
Looking at the agent logs, all I see is:
```
DEBUG:root:execute:uname -r
DEBUG:root:execute:uname -m
DEBUG:root:execute:hostname -f
DEBUG:root:execute:kvm-ok
DEBUG:root:execute:awk '/MemTotal/ { printf "%.3f \n", $2/1024 }' /proc/meminfo
DEBUG:root:execute:ip a | grep "^\w" | grep -iv "^lo" | wc -l
DEBUG:root:execute:service apparmor status
DEBUG:root:execute:apparmor_status |grep libvirt
DEBUG:root:Failed to execute:
DEBUG:root:cloudbr0 is not a network device, is it down?
DEBUG:root:execute:sudo /usr/sbin/service network-manager status
DEBUG:root:Failed to execute:Unit network-manager.service could not be found.
DEBUG:root:execute:ip route show default | awk '{print $3,$5}'
DEBUG:root:execute:ifconfig public
DEBUG:root:Failed to execute:/bin/sh: 1: ifconfig: not found
DEBUG:root:Failed to get address from ifconfig
DEBUG:root:execute:sudo update-rc.d -f apparmor remove
DEBUG:root:execute:sudo update-rc.d -f apparmor defaults
DEBUG:root:execute:sudo /usr/sbin/service apparmor status
DEBUG:root:execute:sudo /usr/sbin/service apparmor start
DEBUG:root:execute:sudo /usr/sbin/service apparmor status
DEBUG:root:execute:sudo /usr/sbin/service apparmor start
DEBUG:root:execute:sudo update-rc.d -f network-manager remove
DEBUG:root:execute:sudo update-rc.d -f network-manager defaults
DEBUG:root:Failed to execute:update-rc.d: error: unable to read /etc/init.d/network-manager
DEBUG:root:execute:sudo /usr/sbin/service network-manager status
DEBUG:root:Failed to execute:Unit network-manager.service could not be found.
DEBUG:root:execute:sudo /usr/sbin/service network-manager start
DEBUG:root:Failed to execute:Failed to start network-manager.service: Unit network-manager.service not found.
DEBUG:root:execute:sudo /usr/sbin/service network-manager status
DEBUG:root:Failed to execute:Unit network-manager.service could not be found.
DEBUG:root:execute:sudo /usr/sbin/service network-manager start
DEBUG:root:Failed to execute:Failed to start network-manager.service: Unit network-manager.service not found.
DEBUG:root:execute:/etc/init.d/networking stop
DEBUG:root:Failed to execute:/bin/sh: 1: /etc/init.d/networking: not found
DEBUG:root:execute:/etc/init.d/networking start
DEBUG:root:Failed to execute:/bin/sh: 1: /etc/init.d/networking: not found
```
Looking at the command history, I see:
```
Jun 9 21:05:52 node7.testenv.bradhouse.dev sudo: cloudstack : PWD=/home/cloudstack ; USER=root ; COMMAND=/usr/bin/cloudstack-setup-agent -m 10.10.100.2,10.10.100.3,10.10.100.4 -z 2 -p 2 -c 5 -g 78999d33-6584-340b-a4bc-8c19b52aa195 -a -s --pubNic=cloudbr0 --prvNic=cloudbr0 --guestNic=cloudbr0 --hypervisor=lxc
```
compared to KVM:
```
Jun 9 20:43:31 node4.testenv.bradhouse.dev sudo: cloudstack : PWD=/home/cloudstack ; USER=root ; COMMAND=/usr/bin/cloudstack-setup-agent -m 10.10.100.2,10.10.100.3,10.10.100.4 -z 2 -p 2 -c 2 -g 044ed14f-03ec-31b4-a02a-c842cdcbba1b -a -s --pubNic=public --prvNic=hypervisor --guestNic=public --hypervisor=kvm
```
The thing that stands out at me is the pubNic, prvNic and guestNic ... the LXC should match what is passed for the KVM nodes, but instead are reverting to the default 'cloudbr0' which I don't have as I named my bridges more useful names. Any idea why these aren't passed properly?
Perhaps I missed something in the new cluster? I thought the network settings were zone-wide.
GitHub link: https://github.com/apache/cloudstack/discussions/10999
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] How to join an LXC container host? [cloudstack]
Posted by "bradh352 (via GitHub)" <gi...@apache.org>.
GitHub user bradh352 added a comment to the discussion: How to join an LXC container host?
In case its relevant, here's some cloudmonkey output with settings:
```
root@node3:~# cmk list clusters
{
"cluster": [
{
"allocationstate": "Enabled",
"arch": "x86_64",
"clustertype": "CloudManaged",
"cpuovercommitratio": "4.0",
"hasannotations": false,
"hypervisortype": "KVM",
"id": "2c2425fe-a9cc-41e7-a60a-8f806fd0712a",
"managedstate": "Managed",
"memoryovercommitratio": "2.0",
"name": "vm",
"podid": "c758118b-1503-4d28-a10e-5d314760ec44",
"podname": "pod",
"resourcedetails": {
"cpuOvercommitRatio": "4.0",
"memoryOvercommitRatio": "2.0"
},
"zoneid": "e3b58c99-67bf-4dde-958d-bdda7e2ab667",
"zonename": "testlab"
},
{
"allocationstate": "Enabled",
"arch": "x86_64",
"clustertype": "CloudManaged",
"cpuovercommitratio": "4.0",
"hasannotations": false,
"hypervisortype": "LXC",
"id": "2cd66273-5698-48c5-9c20-0e21e4d174cc",
"managedstate": "Managed",
"memoryovercommitratio": "2.0",
"name": "lxc",
"podid": "c758118b-1503-4d28-a10e-5d314760ec44",
"podname": "pod",
"resourcedetails": {
"cpuOvercommitRatio": "4.0",
"memoryOvercommitRatio": "2.0"
},
"zoneid": "e3b58c99-67bf-4dde-958d-bdda7e2ab667",
"zonename": "testlab"
}
],
"count": 2
}
```
```
root@node3:~# cmk list physicalnetworks
{
"count": 3,
"physicalnetwork": [
{
"broadcastdomainrange": "ZONE",
"id": "a2feeaae-abe0-40c0-9501-55b5fbc4556f",
"isolationmethods": "VLAN",
"name": "mgmt-net",
"state": "Enabled",
"zoneid": "e3b58c99-67bf-4dde-958d-bdda7e2ab667",
"zonename": "testlab"
},
{
"broadcastdomainrange": "ZONE",
"id": "e27bd3b0-da2b-4638-8ada-0d89e778c673",
"isolationmethods": "VLAN",
"name": "public-net",
"state": "Enabled",
"zoneid": "e3b58c99-67bf-4dde-958d-bdda7e2ab667",
"zonename": "testlab"
},
{
"broadcastdomainrange": "ZONE",
"id": "afcfc718-06e9-44c4-9645-85b1e44c5506",
"isolationmethods": "VXLAN",
"name": "guest-net",
"state": "Enabled",
"vlan": "5000-10000",
"zoneid": "e3b58c99-67bf-4dde-958d-bdda7e2ab667",
"zonename": "testlab"
}
]
}
```
```
root@node3:~# cmk list traffictypes physicalnetworkid=a2feeaae-abe0-40c0-9501-55b5fbc4556f
{
"count": 2,
"traffictype": [
{
"id": "798b1608-be32-4c6d-819b-629ea9331164",
"kvmnetworklabel": "hypervisor",
"physicalnetworkid": "a2feeaae-abe0-40c0-9501-55b5fbc4556f",
"traffictype": "Management"
},
{
"id": "31853d60-5ac4-495b-85fe-0546a3c46a35",
"kvmnetworklabel": "hypervisor",
"physicalnetworkid": "a2feeaae-abe0-40c0-9501-55b5fbc4556f",
"traffictype": "Storage"
}
]
}
root@node3:~# cmk list traffictypes physicalnetworkid=e27bd3b0-da2b-4638-8ada-0d89e778c673
{
"count": 1,
"traffictype": [
{
"id": "ff2303c6-7374-402f-a0b2-322b89da621e",
"kvmnetworklabel": "public",
"physicalnetworkid": "e27bd3b0-da2b-4638-8ada-0d89e778c673",
"traffictype": "Public"
}
]
}
root@node3:~# cmk list traffictypes physicalnetworkid=afcfc718-06e9-44c4-9645-85b1e44c5506
{
"count": 1,
"traffictype": [
{
"id": "65b0f74c-4f7a-42b2-98d5-514c5988d154",
"kvmnetworklabel": "public",
"physicalnetworkid": "afcfc718-06e9-44c4-9645-85b1e44c5506",
"traffictype": "Guest"
}
]
}
```
GitHub link: https://github.com/apache/cloudstack/discussions/10999#discussioncomment-13421505
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] How to join an LXC container host? [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
GitHub user DaanHoogland added a comment to the discussion: How to join an LXC container host?
@bradh352 there is a deploy anything feature in the making, maybe you can use that for LXC? (see #9752)
GitHub link: https://github.com/apache/cloudstack/discussions/10999#discussioncomment-13641248
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org
Re: [D] How to join an LXC container host? [cloudstack]
Posted by "bradh352 (via GitHub)" <gi...@apache.org>.
GitHub user bradh352 added a comment to the discussion: How to join an LXC container host?
Oh, and a secondary odd thing, when adding a host for LXC, the option for the ssh key is gone, it seems like it can only use a username/password? When looking at the API itself I'm not seeing that limitation, and not finding any discussion on that in the docs.
GitHub link: https://github.com/apache/cloudstack/discussions/10999#discussioncomment-13421433
----
This is an automatically sent email for users@cloudstack.apache.org.
To unsubscribe, please send an email to: users-unsubscribe@cloudstack.apache.org