You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Seth Lyons <sl...@automateio.com> on 2021/12/01 23:51:13 UTC
RE: Cloudstack Kubernetes Issue
Hello,
I have my endpoint api (https domain) working as tested with cloudmonkey. I want to say the issue is with networking in my basic guest network. My virtual router, ssvm, and cpvm has connection to the internet (can ping google.com) and can be pinged from my local pc. When I create a guest vm it does not have internet access and cannot be pinged by my local pc or the virtual router. I have added 0.0.0.0/0 as both ingress and egress rules in my security group for all ports. I am using two different IP ranges and gateways for the management network and guest network. I am using KVM as my host.
Here is my network report from the host machine:
[slyons@ch0 network-scripts]$ ifconfig cloudbr1
cloudbr1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 92:6c:1c:b1:85:d8 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[slyons@ch0 network-scripts]$ brctl show
bridge name bridge id STP enabled interfaces
cloud0 8000.fe00a9fe5cbc no vnet1
vnet2
vnet5
cloudbr0 8000.002590db2cdc yes enp3s0
vnet0
vnet11
vnet3
vnet4
vnet6
vnet7
cloudbr1 8000.000000000000 yes
I'd be happy to provide other logs that might be helpful with this.
Thanks,
Seth
-----Original Message-----
From: Rohit Yadav <ro...@shapeblue.com>
Sent: Tuesday, November 23, 2021 5:58 AM
To: users@cloudstack.apache.org
Subject: Re: Cloudstack Kubernetes Issue
Hi Seth,
It depends what failures you're getting, if it's related to endpointe pl fix the global setting with your API URL (for ex. I'm using http://192.168.1.10:8080/client/api). If you're using a domain name, ensure that's globally resolvable or at least by the DNS server configured for your zone/network. After you fix and try again, it may help if share your errors and logs.
If the URL is https enabled, avoid letsencrypt whose root authority X3 has expired few months and may give security errors.
Regards.
________________________________
From: Seth Lyons <sl...@automateio.com>
Sent: Tuesday, November 23, 2021 01:40
To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
Subject: RE: Cloudstack Kubernetes Issue
Hello, I am deploying 1.22.2 using link http://download.cloudstack.org/cks/setup-1.22.2.iso to create the template.
Sorry for the late response I had reinstalled everything and still the same issue.
Maybe it is the endpointe.url ? Does this need to be configured for my SSL encryption in the global settings? I have it set to my management dns name which resolves correctly.
Right now it looks like http://cloudstack.website.com:8080/client/api
Seth
-----Original Message-----
From: Pearl d'Silva <pe...@shapeblue.com>
Sent: Wednesday, November 17, 2021 3:12 AM
To: users@cloudstack.apache.org
Subject: Re: Cloudstack Kubernetes Issue
Hi,
Could you please share which version of Kubernetes you are trying to deploy. From ACS 4.16 onward, k8s versions > 1.20 are only supported.
Thanks,
Pearl
________________________________
From: Seth Lyons <sl...@automateio.com>
Sent: Wednesday, November 17, 2021 2:51 AM
To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
Subject: Cloudstack Kubernetes Issue
Hello,
I am running Cloudstacks 4.16 and trying to deploy the Kubernetes add on. When I add the service it is stuck in the Starting state, but I see that two instances are created and seem to be running when viewed through the console proxy.
I see this error in the logs which seems related:
2021-11-16 15:29:24,718 WARN [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-12:ctx-aeed885c job-550 ctx-9066b68b) (logid:ac518757) Unable to retrieve VMs for Kubernetes cluster : Seth
2021-11-16 15:29:24,718 WARN [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-12:ctx-aeed885c job-550 ctx-9066b68b) (logid:ac518757) Unable to retrieve control VM for Kubernetes cluster : Seth
2021-11-16 15:29:26,912 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-23:ctx-1628641a job-550/job-551 ctx-defc38ee) (logid:ac518757) STATS: Failed to alloc resource from host: 11 reservedCpu: 0, requested cpu: 4000, reservedMem: (0 bytes) 0, requested mem: (2.00 GB) 2147483648
2021-11-16 15:29:34,893 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-24:ctx-efb62ae9 job-550/job-552 ctx-b435021d) (logid:ac518757) STATS: Failed to alloc resource from host: 11 reservedCpu: 0, requested cpu: 4000, reservedMem: (0 bytes) 0, requested mem: (2.00 GB) 2147483648
[Photo]
[Logo]<http://www.automateio.com/>
Seth Lyons
DevOps Engineer | Automate io
[phone-icon] (415) 757-4696
[phone-icon] slyons@automateio.com<ma...@automateio.com>
[phone-icon] www.automateio.com<http://www.automateio.com/>
[Banner]<http://www.automateio.com/>
RE: Cloudstack Kubernetes Issue
Posted by Seth Lyons <sl...@automateio.com>.
Hello Wei,
Here is the report of security group rules:
{
"account": "admin",
"description": "Default Security Group",
"domain": "ROOT",
"domainid": "e83a1b32-496c-11ec-9600-0cc47a41ae34",
"egressrule": [
{
"cidr": "0.0.0.0/0",
"endport": 0,
"protocol": "all",
"ruleid": "aa49fff1-7abf-4407-85a9-eb809c77fcd0",
"startport": 0,
"tags": []
}
],
"id": "31841e89-496d-11ec-9600-0cc47a41ae34",
"ingressrule": [
{
"cidr": "0.0.0.0/0",
"endport": 0,
"protocol": "all",
"ruleid": "cd9314e2-f6c9-4c8e-a79b-5931d4af05f0",
"startport": 0,
"tags": []
}
],
"name": "default",
"tags": [],
"virtualmachinecount": 1,
"virtualmachineids": [
"bb1e7173-6ddd-4162-a14f-9e856e9167a3"
]
Thanks,
Seth
-----Original Message-----
From: Wei ZHOU <us...@gmail.com>
Sent: Thursday, December 2, 2021 2:14 AM
To: users <us...@cloudstack.apache.org>
Subject: Re: Cloudstack Kubernetes Issue
Hi Seth,
Could you please check the vm security groups ?
-Wei
On Thu, 2 Dec 2021 at 00:51, Seth Lyons <sl...@automateio.com> wrote:
> Hello,
> I have my endpoint api (https domain) working as tested with cloudmonkey.
> I want to say the issue is with networking in my basic guest network.
> My virtual router, ssvm, and cpvm has connection to the internet (can
> ping
> google.com) and can be pinged from my local pc. When I create a guest
> vm it does not have internet access and cannot be pinged by my local
> pc or the virtual router. I have added 0.0.0.0/0 as both ingress and
> egress rules in my security group for all ports. I am using two
> different IP ranges and gateways for the management network and guest
> network. I am using KVM as my host.
>
> Here is my network report from the host machine:
>
> [slyons@ch0 network-scripts]$ ifconfig cloudbr1
> cloudbr1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
> ether 92:6c:1c:b1:85:d8 txqueuelen 1000 (Ethernet)
> RX packets 0 bytes 0 (0.0 B)
> RX errors 0 dropped 0 overruns 0 frame 0
> TX packets 0 bytes 0 (0.0 B)
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
> [slyons@ch0 network-scripts]$ brctl show
> bridge name bridge id STP enabled interfaces
> cloud0 8000.fe00a9fe5cbc no vnet1
> vnet2
> vnet5
> cloudbr0 8000.002590db2cdc yes enp3s0
> vnet0
> vnet11
> vnet3
> vnet4
> vnet6
> vnet7
> cloudbr1 8000.000000000000 yes
>
>
>
> I'd be happy to provide other logs that might be helpful with this.
>
> Thanks,
> Seth
>
>
>
> -----Original Message-----
> From: Rohit Yadav <ro...@shapeblue.com>
> Sent: Tuesday, November 23, 2021 5:58 AM
> To: users@cloudstack.apache.org
> Subject: Re: Cloudstack Kubernetes Issue
>
> Hi Seth,
>
> It depends what failures you're getting, if it's related to endpointe
> pl fix the global setting with your API URL (for ex. I'm using
> http://192.168.1.10:8080/client/api). If you're using a domain name,
> ensure that's globally resolvable or at least by the DNS server
> configured for your zone/network. After you fix and try again, it may
> help if share your errors and logs.
>
> If the URL is https enabled, avoid letsencrypt whose root authority X3
> has expired few months and may give security errors.
>
>
> Regards.
>
> ________________________________
> From: Seth Lyons <sl...@automateio.com>
> Sent: Tuesday, November 23, 2021 01:40
> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
> Subject: RE: Cloudstack Kubernetes Issue
>
> Hello, I am deploying 1.22.2 using link
> http://download.cloudstack.org/cks/setup-1.22.2.iso to create the
> template.
>
> Sorry for the late response I had reinstalled everything and still the
> same issue.
>
> Maybe it is the endpointe.url ? Does this need to be configured for my
> SSL encryption in the global settings? I have it set to my management
> dns name which resolves correctly.
>
> Right now it looks like http://cloudstack.website.com:8080/client/api
>
>
>
>
>
> Seth
>
>
>
>
>
> -----Original Message-----
> From: Pearl d'Silva <pe...@shapeblue.com>
> Sent: Wednesday, November 17, 2021 3:12 AM
> To: users@cloudstack.apache.org
> Subject: Re: Cloudstack Kubernetes Issue
>
> Hi,
>
> Could you please share which version of Kubernetes you are trying to
> deploy. From ACS 4.16 onward, k8s versions > 1.20 are only supported.
>
>
> Thanks,
> Pearl
>
> ________________________________
> From: Seth Lyons <sl...@automateio.com>
> Sent: Wednesday, November 17, 2021 2:51 AM
> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
> Subject: Cloudstack Kubernetes Issue
>
> Hello,
> I am running Cloudstacks 4.16 and trying to deploy the Kubernetes add on.
> When I add the service it is stuck in the Starting state, but I see
> that two instances are created and seem to be running when viewed
> through the console proxy.
> I see this error in the logs which seems related:
>
> 2021-11-16 15:29:24,718 WARN
> [c.c.k.c.a.KubernetesClusterActionWorker]
> (API-Job-Executor-12:ctx-aeed885c job-550 ctx-9066b68b)
> (logid:ac518757) Unable to retrieve VMs for Kubernetes cluster : Seth
> 2021-11-16 15:29:24,718 WARN
> [c.c.k.c.a.KubernetesClusterActionWorker]
> (API-Job-Executor-12:ctx-aeed885c job-550 ctx-9066b68b)
> (logid:ac518757) Unable to retrieve control VM for Kubernetes cluster
> : Seth
> 2021-11-16 15:29:26,912 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-23:ctx-1628641a job-550/job-551 ctx-defc38ee)
> (logid:ac518757) STATS: Failed to alloc resource from host: 11 reservedCpu:
> 0, requested cpu: 4000, reservedMem: (0 bytes) 0, requested mem: (2.00
> GB)
> 2147483648
> 2021-11-16 15:29:34,893 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-24:ctx-efb62ae9 job-550/job-552 ctx-b435021d)
> (logid:ac518757) STATS: Failed to alloc resource from host: 11 reservedCpu:
> 0, requested cpu: 4000, reservedMem: (0 bytes) 0, requested mem: (2.00
> GB)
> 2147483648
>
>
>
> [Photo]
> [Logo]<http://www.automateio.com/>
> Seth Lyons
> DevOps Engineer | Automate io
>
> [phone-icon] (415) 757-4696
> [phone-icon] slyons@automateio.com<ma...@automateio.com>
> [phone-icon] www.automateio.com<http://www.automateio.com/>
> [Banner]<http://www.automateio.com/>
>
>
>
>
>
>
Re: Cloudstack Kubernetes Issue
Posted by Wei ZHOU <us...@gmail.com>.
Hi Seth,
Could you please check the vm security groups ?
-Wei
On Thu, 2 Dec 2021 at 00:51, Seth Lyons <sl...@automateio.com> wrote:
> Hello,
> I have my endpoint api (https domain) working as tested with cloudmonkey.
> I want to say the issue is with networking in my basic guest network. My
> virtual router, ssvm, and cpvm has connection to the internet (can ping
> google.com) and can be pinged from my local pc. When I create a guest vm
> it does not have internet access and cannot be pinged by my local pc or the
> virtual router. I have added 0.0.0.0/0 as both ingress and egress rules
> in my security group for all ports. I am using two different IP ranges and
> gateways for the management network and guest network. I am using KVM as
> my host.
>
> Here is my network report from the host machine:
>
> [slyons@ch0 network-scripts]$ ifconfig cloudbr1
> cloudbr1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
> ether 92:6c:1c:b1:85:d8 txqueuelen 1000 (Ethernet)
> RX packets 0 bytes 0 (0.0 B)
> RX errors 0 dropped 0 overruns 0 frame 0
> TX packets 0 bytes 0 (0.0 B)
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
> [slyons@ch0 network-scripts]$ brctl show
> bridge name bridge id STP enabled interfaces
> cloud0 8000.fe00a9fe5cbc no vnet1
> vnet2
> vnet5
> cloudbr0 8000.002590db2cdc yes enp3s0
> vnet0
> vnet11
> vnet3
> vnet4
> vnet6
> vnet7
> cloudbr1 8000.000000000000 yes
>
>
>
> I'd be happy to provide other logs that might be helpful with this.
>
> Thanks,
> Seth
>
>
>
> -----Original Message-----
> From: Rohit Yadav <ro...@shapeblue.com>
> Sent: Tuesday, November 23, 2021 5:58 AM
> To: users@cloudstack.apache.org
> Subject: Re: Cloudstack Kubernetes Issue
>
> Hi Seth,
>
> It depends what failures you're getting, if it's related to endpointe pl
> fix the global setting with your API URL (for ex. I'm using
> http://192.168.1.10:8080/client/api). If you're using a domain name,
> ensure that's globally resolvable or at least by the DNS server configured
> for your zone/network. After you fix and try again, it may help if share
> your errors and logs.
>
> If the URL is https enabled, avoid letsencrypt whose root authority X3 has
> expired few months and may give security errors.
>
>
> Regards.
>
> ________________________________
> From: Seth Lyons <sl...@automateio.com>
> Sent: Tuesday, November 23, 2021 01:40
> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
> Subject: RE: Cloudstack Kubernetes Issue
>
> Hello, I am deploying 1.22.2 using link
> http://download.cloudstack.org/cks/setup-1.22.2.iso to create the
> template.
>
> Sorry for the late response I had reinstalled everything and still the
> same issue.
>
> Maybe it is the endpointe.url ? Does this need to be configured for my SSL
> encryption in the global settings? I have it set to my management dns name
> which resolves correctly.
>
> Right now it looks like http://cloudstack.website.com:8080/client/api
>
>
>
>
>
> Seth
>
>
>
>
>
> -----Original Message-----
> From: Pearl d'Silva <pe...@shapeblue.com>
> Sent: Wednesday, November 17, 2021 3:12 AM
> To: users@cloudstack.apache.org
> Subject: Re: Cloudstack Kubernetes Issue
>
> Hi,
>
> Could you please share which version of Kubernetes you are trying to
> deploy. From ACS 4.16 onward, k8s versions > 1.20 are only supported.
>
>
> Thanks,
> Pearl
>
> ________________________________
> From: Seth Lyons <sl...@automateio.com>
> Sent: Wednesday, November 17, 2021 2:51 AM
> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
> Subject: Cloudstack Kubernetes Issue
>
> Hello,
> I am running Cloudstacks 4.16 and trying to deploy the Kubernetes add on.
> When I add the service it is stuck in the Starting state, but I see that
> two instances are created and seem to be running when viewed through the
> console proxy.
> I see this error in the logs which seems related:
>
> 2021-11-16 15:29:24,718 WARN [c.c.k.c.a.KubernetesClusterActionWorker]
> (API-Job-Executor-12:ctx-aeed885c job-550 ctx-9066b68b) (logid:ac518757)
> Unable to retrieve VMs for Kubernetes cluster : Seth
> 2021-11-16 15:29:24,718 WARN [c.c.k.c.a.KubernetesClusterActionWorker]
> (API-Job-Executor-12:ctx-aeed885c job-550 ctx-9066b68b) (logid:ac518757)
> Unable to retrieve control VM for Kubernetes cluster : Seth
> 2021-11-16 15:29:26,912 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-23:ctx-1628641a job-550/job-551 ctx-defc38ee)
> (logid:ac518757) STATS: Failed to alloc resource from host: 11 reservedCpu:
> 0, requested cpu: 4000, reservedMem: (0 bytes) 0, requested mem: (2.00 GB)
> 2147483648
> 2021-11-16 15:29:34,893 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-24:ctx-efb62ae9 job-550/job-552 ctx-b435021d)
> (logid:ac518757) STATS: Failed to alloc resource from host: 11 reservedCpu:
> 0, requested cpu: 4000, reservedMem: (0 bytes) 0, requested mem: (2.00 GB)
> 2147483648
>
>
>
> [Photo]
> [Logo]<http://www.automateio.com/>
> Seth Lyons
> DevOps Engineer | Automate io
>
> [phone-icon] (415) 757-4696
> [phone-icon] slyons@automateio.com<ma...@automateio.com>
> [phone-icon] www.automateio.com<http://www.automateio.com/>
> [Banner]<http://www.automateio.com/>
>
>
>
>
>
>