You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by benoit lair <ku...@gmail.com> on 2024/04/08 14:52:26 UTC
ACS 4.16 - Change SystemVM template for CKS
Hello Folks,
I am trying to install K8s cluster with community iso 1.28.4
I am with a ACS 4.16.0 environment
It seems K8S is not working out of the box with v > 1.23.3 due to
containerd.io version
I would like to release a template who will work with K8s > 1.23.3 on acs
4.16
How can i tell CS to take another system vm template, avoiding to mess
normal features with VR and VPC VR keeping the systemvm template issued
with CS 4.16 ?
I am with XCP-NG 8.2.1 in production
Thanks for your help or advises
Best regards,
Benoit
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by benoit lair <ku...@gmail.com>.
@Embedded, did you take a look on
https://docs.cloudstack.apache.org/en/latest/plugins/cloudstack-kubernetes-service.html
?
You have registered k8s iso image ?
What is your error message ?
you can connect to your control node via pub.key hosted on mgmt server with
"ssh -i rsa_test.cloud -p 2222 core@
<public_ip_of_vr_attributed_to_k8s_cluster>
Take a look at logs from control node (via port 2222)
Le jeu. 11 avr. 2024 à 12:22, Embedded <li...@optimcloud.com.invalid> a
écrit :
>
>
>
> On Thursday 11 April 2024 03:46:49 PM (+07:00), benoit lair wrote:
>
> > Any advices ?
> >
> > Best regards
>
> Im headed down the exact same path with debian manager and xcp-ng 8.2.1
> so i can see how the CKS works, only im on 4.19, i also used shapeblues
> image
>
> happy to help once im at that point, its a fresh install
>
> >
> > Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a
> écrit :
> >
> > > I am opened to every alternatives of changing system vm templates
> > > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
> > >
> > > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a
> écrit :
> > >
> > >> Hello Folks,
> > >>
> > >> I am trying to install K8s cluster with community iso 1.28.4
> > >> I am with a ACS 4.16.0 environment
> > >> It seems K8S is not working out of the box with v > 1.23.3 due to
> > >> containerd.io version
> > >>
> > >> I would like to release a template who will work with K8s > 1.23.3 on
> acs
> > >> 4.16
> > >> How can i tell CS to take another system vm template, avoiding to mess
> > >> normal features with VR and VPC VR keeping the systemvm template
> issued
> > >> with CS 4.16 ?
> > >>
> > >> I am with XCP-NG 8.2.1 in production
> > >>
> > >> Thanks for your help or advises
> > >> Best regards,
> > >> Benoit
> > >>
> > >
> >
>
> --
> Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by Embedded <li...@optimcloud.com.INVALID>.
On Thursday 11 April 2024 03:46:49 PM (+07:00), benoit lair wrote:
> Any advices ?
>
> Best regards
Im headed down the exact same path with debian manager and xcp-ng 8.2.1
so i can see how the CKS works, only im on 4.19, i also used shapeblues image
happy to help once im at that point, its a fresh install
>
> Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a écrit :
>
> > I am opened to every alternatives of changing system vm templates
> > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
> >
> > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a écrit :
> >
> >> Hello Folks,
> >>
> >> I am trying to install K8s cluster with community iso 1.28.4
> >> I am with a ACS 4.16.0 environment
> >> It seems K8S is not working out of the box with v > 1.23.3 due to
> >> containerd.io version
> >>
> >> I would like to release a template who will work with K8s > 1.23.3 on acs
> >> 4.16
> >> How can i tell CS to take another system vm template, avoiding to mess
> >> normal features with VR and VPC VR keeping the systemvm template issued
> >> with CS 4.16 ?
> >>
> >> I am with XCP-NG 8.2.1 in production
> >>
> >> Thanks for your help or advises
> >> Best regards,
> >> Benoit
> >>
> >
>
--
Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by Rohit Yadav <ro...@shapeblue.com>.
Benoit,
I haven't tested it but could work - worth testing. CAPC was tested on 4.17/4.18+.
Regards.
________________________________
From: benoit lair <ku...@gmail.com>
Sent: Thursday, April 11, 2024 17:09
To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
Cc: dev <de...@cloudstack.apache.org>
Subject: Re: ACS 4.16 - Change SystemVM template for CKS
Rohit,
It is possible to install CAPC on a CS 4.16 ?
Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <ro...@shapeblue.com> a
écrit :
> Hi Benoit,
>
> The CKS feature has been improving over versions, I don't know if what
> you're trying to achieve is possible with it. Maybe try a different version
> of the data iso:
> http://download.cloudstack.org/cks/
>
> Alternatively, you can also have a look at the CAPC project:
> https://cluster-api-cloudstack.sigs.k8s.io
>
>
> Regards.
>
>
>
>
> ________________________________
> From: benoit lair <ku...@gmail.com>
> Sent: Thursday, April 11, 2024 14:16
> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <
> dev@cloudstack.apache.org>
> Subject: Re: ACS 4.16 - Change SystemVM template for CKS
>
> Any advices ?
>
> Best regards
>
> Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a écrit :
>
> > I am opened to every alternatives of changing system vm templates
> > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
> >
> > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a
> écrit :
> >
> >> Hello Folks,
> >>
> >> I am trying to install K8s cluster with community iso 1.28.4
> >> I am with a ACS 4.16.0 environment
> >> It seems K8S is not working out of the box with v > 1.23.3 due to
> >> containerd.io version
> >>
> >> I would like to release a template who will work with K8s > 1.23.3 on
> acs
> >> 4.16
> >> How can i tell CS to take another system vm template, avoiding to mess
> >> normal features with VR and VPC VR keeping the systemvm template issued
> >> with CS 4.16 ?
> >>
> >> I am with XCP-NG 8.2.1 in production
> >>
> >> Thanks for your help or advises
> >> Best regards,
> >> Benoit
> >>
> >
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by Rohit Yadav <ro...@shapeblue.com>.
Benoit,
I haven't tested it but could work - worth testing. CAPC was tested on 4.17/4.18+.
Regards.
________________________________
From: benoit lair <ku...@gmail.com>
Sent: Thursday, April 11, 2024 17:09
To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
Cc: dev <de...@cloudstack.apache.org>
Subject: Re: ACS 4.16 - Change SystemVM template for CKS
Rohit,
It is possible to install CAPC on a CS 4.16 ?
Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <ro...@shapeblue.com> a
écrit :
> Hi Benoit,
>
> The CKS feature has been improving over versions, I don't know if what
> you're trying to achieve is possible with it. Maybe try a different version
> of the data iso:
> http://download.cloudstack.org/cks/
>
> Alternatively, you can also have a look at the CAPC project:
> https://cluster-api-cloudstack.sigs.k8s.io
>
>
> Regards.
>
>
>
>
> ________________________________
> From: benoit lair <ku...@gmail.com>
> Sent: Thursday, April 11, 2024 14:16
> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <
> dev@cloudstack.apache.org>
> Subject: Re: ACS 4.16 - Change SystemVM template for CKS
>
> Any advices ?
>
> Best regards
>
> Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a écrit :
>
> > I am opened to every alternatives of changing system vm templates
> > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
> >
> > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a
> écrit :
> >
> >> Hello Folks,
> >>
> >> I am trying to install K8s cluster with community iso 1.28.4
> >> I am with a ACS 4.16.0 environment
> >> It seems K8S is not working out of the box with v > 1.23.3 due to
> >> containerd.io version
> >>
> >> I would like to release a template who will work with K8s > 1.23.3 on
> acs
> >> 4.16
> >> How can i tell CS to take another system vm template, avoiding to mess
> >> normal features with VR and VPC VR keeping the systemvm template issued
> >> with CS 4.16 ?
> >>
> >> I am with XCP-NG 8.2.1 in production
> >>
> >> Thanks for your help or advises
> >> Best regards,
> >> Benoit
> >>
> >
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by benoit lair <ku...@gmail.com>.
Rohit, does it mean i can overload the integrated kubernetes provider
plugin of cs 4.16 with CAPC ? i still can manage it via CS UI ? or just
with api calls ?
Le jeu. 11 avr. 2024 à 13:39, benoit lair <ku...@gmail.com> a écrit :
> Rohit,
>
> It is possible to install CAPC on a CS 4.16 ?
>
> Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <ro...@shapeblue.com> a
> écrit :
>
>> Hi Benoit,
>>
>> The CKS feature has been improving over versions, I don't know if what
>> you're trying to achieve is possible with it. Maybe try a different version
>> of the data iso:
>> http://download.cloudstack.org/cks/
>>
>> Alternatively, you can also have a look at the CAPC project:
>> https://cluster-api-cloudstack.sigs.k8s.io
>>
>>
>> Regards.
>>
>>
>>
>>
>> ________________________________
>> From: benoit lair <ku...@gmail.com>
>> Sent: Thursday, April 11, 2024 14:16
>> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <
>> dev@cloudstack.apache.org>
>> Subject: Re: ACS 4.16 - Change SystemVM template for CKS
>>
>> Any advices ?
>>
>> Best regards
>>
>> Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a écrit
>> :
>>
>> > I am opened to every alternatives of changing system vm templates
>> > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
>> >
>> > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a
>> écrit :
>> >
>> >> Hello Folks,
>> >>
>> >> I am trying to install K8s cluster with community iso 1.28.4
>> >> I am with a ACS 4.16.0 environment
>> >> It seems K8S is not working out of the box with v > 1.23.3 due to
>> >> containerd.io version
>> >>
>> >> I would like to release a template who will work with K8s > 1.23.3 on
>> acs
>> >> 4.16
>> >> How can i tell CS to take another system vm template, avoiding to mess
>> >> normal features with VR and VPC VR keeping the systemvm template issued
>> >> with CS 4.16 ?
>> >>
>> >> I am with XCP-NG 8.2.1 in production
>> >>
>> >> Thanks for your help or advises
>> >> Best regards,
>> >> Benoit
>> >>
>> >
>>
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by benoit lair <ku...@gmail.com>.
Rohit,
It is possible to install CAPC on a CS 4.16 ?
Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <ro...@shapeblue.com> a
écrit :
> Hi Benoit,
>
> The CKS feature has been improving over versions, I don't know if what
> you're trying to achieve is possible with it. Maybe try a different version
> of the data iso:
> http://download.cloudstack.org/cks/
>
> Alternatively, you can also have a look at the CAPC project:
> https://cluster-api-cloudstack.sigs.k8s.io
>
>
> Regards.
>
>
>
>
> ________________________________
> From: benoit lair <ku...@gmail.com>
> Sent: Thursday, April 11, 2024 14:16
> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <
> dev@cloudstack.apache.org>
> Subject: Re: ACS 4.16 - Change SystemVM template for CKS
>
> Any advices ?
>
> Best regards
>
> Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a écrit :
>
> > I am opened to every alternatives of changing system vm templates
> > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
> >
> > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a
> écrit :
> >
> >> Hello Folks,
> >>
> >> I am trying to install K8s cluster with community iso 1.28.4
> >> I am with a ACS 4.16.0 environment
> >> It seems K8S is not working out of the box with v > 1.23.3 due to
> >> containerd.io version
> >>
> >> I would like to release a template who will work with K8s > 1.23.3 on
> acs
> >> 4.16
> >> How can i tell CS to take another system vm template, avoiding to mess
> >> normal features with VR and VPC VR keeping the systemvm template issued
> >> with CS 4.16 ?
> >>
> >> I am with XCP-NG 8.2.1 in production
> >>
> >> Thanks for your help or advises
> >> Best regards,
> >> Benoit
> >>
> >
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by Embedded <li...@optimcloud.com.INVALID>.
is docker required, pretty sure k8s runs wth containerd or crio installed, no docker needed
On Friday 12 April 2024 07:25:19 PM (+07:00), benoit lair wrote:
> i succeded to install systemvm template 4.19 as serving template for
> control and worker nodes with cks community iso 1.28.4
> when cluster saying "starting", i connect to node controller via p2222, saw
> that docker was not installed, but containerd.io yes
>
> I've done the following :
>
> apt install docker-ce
> cp /etc/containerd/config.toml /etc/containerd/config.toml.bck
> containerd config default | tee /etc/containerd/config.toml
> /opt/bin/setup-kube-system
> /opt/bin/deploy-kube-system
>
> on to control node and same after on worker node
> CS UI show kubernetes yaml
> a "kubectl.exe --kubeconfig kube.conf.conf get nodes"
>
> NAME STATUS ROLES AGE VERSION
> k8s-cks-cl19-control-18ed18311c9 Ready control-plane 3h1m v1.28.4
> k8s-cks-cl19-node-18ed1850433 Ready <none> 170m v1.28.4
>
> However CS says the cluster is in alert state
> And dashboad is not working
>
> Any advice ?
>
> when executing this on my laptop :
> kubectl.exe --kubeconfig cl19_k8s_1.28.4.conf proxy
>
> and opening this :
> http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
>
> I have this result :
> {
> "kind": "Status",
> "apiVersion": "v1",
> "metadata": {},
> "status": "Failure",
> "message": "no endpoints available for service \"kubernetes-dashboard\"",
> "reason": "ServiceUnavailable",
> "code": 503
> }
>
> Somebody got this problem with dashboard ?
>
> Did i miss something installing the cluster manually with acs stuff ?
>
> Le jeu. 11 avr. 2024 à 17:40, benoit lair <ku...@gmail.com> a écrit :
>
> > Hi Wei,
> >
> > Thanks for the sharing, i also tried to install the systemvm 4.19
> > I have control node and worker node under template systemvm 4.19 (
> > http://download.cloudstack.org/systemvm/4.19/systemvmtemplate-4.19.0-xen.vhd.bz2
> > )
> > I tried with community cks iso 1.25.0 and 1.28.9
> > On systemvm 4.19 docker was not present by default, just containerd in
> > 1.6.x version
> > I installed docker-ce and docker-ce-cli :
> >
> > apt install -y apt-transport-https ca-certificates curl
> > software-properties-common
> > curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg
> > --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
> > echo "deb [arch=$(dpkg --print-architecture)
> > signed-by=/usr/share/keyrings/docker-archive-keyring.gpg]
> > https://download.docker.com/linux/debian $(lsb_release -cs) stable" |
> > sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
> > remove line about docker in double in /etc/apt/sources.list
> > apt update
> > apt install -y docker-ce
> > apt install -y docker-cli
> > i mounted the 1.28.4 cks community iso in control vm
> >
> > and /opt/bin/setup-kube-system
> > i have these errors :
> >
> > Is it the good way to manually install kube system
> >
> > root@k8s-cks-cl17-control-18ecdc1e5bd:/home/core#
> > /opt/bin/setup-kube-system
> > Installing binaries from /mnt/k8sdisk/
> > 5b1fa8e3e100: Loading layer
> > [==================================================>] 803.8kB/803.8kB
> > 39c831b1aa26: Loading layer
> > [==================================================>] 26.25MB/26.25MB
> > Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
> > 417cb9b79ade: Loading layer
> > [==================================================>] 657.7kB/657.7kB
> > 8d323b160d65: Loading layer
> > [==================================================>] 24.95MB/24.95MB
> > Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
> > 6a4a177e62f3: Loading layer
> > [==================================================>] 115.2kB/115.2kB
> > 398c9baff0ce: Loading layer
> > [==================================================>] 16.07MB/16.07MB
> > Loaded image: registry.k8s.io/coredns/coredns:v1.10.1
> > bd8a70623766: Loading layer
> > [==================================================>] 75.78MB/75.78MB
> > c88361932af5: Loading layer
> > [==================================================>] 508B/508B
> > Loaded image: kubernetesui/dashboard:v2.7.0
> > e023e0e48e6e: Loading layer
> > [==================================================>] 103.7kB/103.7kB
> > 6fbdf253bbc2: Loading layer
> > [==================================================>] 21.2kB/21.2kB
> > 7bea6b893187: Loading layer
> > [==================================================>] 716.5kB/716.5kB
> > ff5700ec5418: Loading layer
> > [==================================================>] 317B/317B
> > d52f02c6501c: Loading layer
> > [==================================================>] 198B/198B
> > e624a5370eca: Loading layer
> > [==================================================>] 113B/113B
> > 1a73b54f556b: Loading layer
> > [==================================================>] 385B/385B
> > d2d7ec0f6756: Loading layer
> > [==================================================>] 355B/355B
> > 4cb10dd2545b: Loading layer
> > [==================================================>] 130.6kB/130.6kB
> > ba9afb2b3e0c: Loading layer
> > [==================================================>] 1.076MB/1.076MB
> > 22bba3da6b0d: Loading layer
> > [==================================================>] 91.29MB/91.29MB
> > a4563151d59b: Loading layer
> > [==================================================>] 9.544MB/9.544MB
> > Loaded image: registry.k8s.io/etcd:3.5.9-0
> > 54ad2ec71039: Loading layer
> > [==================================================>] 103.7kB/103.7kB
> > 6fbdf253bbc2: Loading layer
> > [==================================================>] 21.2kB/21.2kB
> > 7bea6b893187: Loading layer
> > [==================================================>] 716.5kB/716.5kB
> > ff5700ec5418: Loading layer
> > [==================================================>] 317B/317B
> > d52f02c6501c: Loading layer
> > [==================================================>] 198B/198B
> > e624a5370eca: Loading layer
> > [==================================================>] 113B/113B
> > 1a73b54f556b: Loading layer
> > [==================================================>] 385B/385B
> > d2d7ec0f6756: Loading layer
> > [==================================================>] 355B/355B
> > 4cb10dd2545b: Loading layer
> > [==================================================>] 130.6kB/130.6kB
> > 3f3c83f9a2fa: Loading layer
> > [==================================================>] 724.8kB/724.8kB
> > 6a2c4bedb8de: Loading layer
> > [==================================================>] 32.98MB/32.98MB
> > Loaded image: registry.k8s.io/kube-apiserver:v1.28.4
> > e0b89915180d: Loading layer
> > [==================================================>] 31.72MB/31.72MB
> > Loaded image: registry.k8s.io/kube-controller-manager:v1.28.4
> > d2a4b045f3d0: Loading layer
> > [==================================================>] 7.73MB/7.73MB
> > 3905b20a2df2: Loading layer
> > [==================================================>] 16.85MB/16.85MB
> > Loaded image: registry.k8s.io/kube-proxy:v1.28.4
> > f35f4c1ae44f: Loading layer
> > [==================================================>] 17.13MB/17.13MB
> > Loaded image: registry.k8s.io/kube-scheduler:v1.28.4
> > d01384fea991: Loading layer
> > [==================================================>] 19.74MB/19.74MB
> > bcec7eb9e567: Loading layer
> > [==================================================>] 530B/530B
> > Loaded image: kubernetesui/metrics-scraper:v1.0.8
> > e3e5579ddd43: Loading layer
> > [==================================================>] 317.6kB/317.6kB
> > Loaded image: registry.k8s.io/pause:3.9
> > 1b3ee35aacca: Loading layer
> > [==================================================>] 2.796MB/2.796MB
> > 910ce076f504: Loading layer
> > [==================================================>] 7.425MB/7.425MB
> > a8e8b7b8e08a: Loading layer
> > [==================================================>] 8.101MB/8.101MB
> > 084e56a9c24b: Loading layer
> > [==================================================>] 4.376MB/4.376MB
> > 334d70dc85ec: Loading layer
> > [==================================================>] 159B/159B
> > a13197d8bda5: Loading layer
> > [==================================================>] 8.216MB/8.216MB
> > Loaded image: weaveworks/weave-kube:2.8.1
> > 998efb010df6: Loading layer
> > [==================================================>] 728.2kB/728.2kB
> > ed37391def99: Loading layer
> > [==================================================>] 9.28MB/9.28MB
> > 175a472c5f77: Loading layer
> > [==================================================>] 299B/299B
> > a8764e32e9fe: Loading layer
> > [==================================================>] 613B/613B
> > Loaded image: weaveworks/weave-npc:2.8.1
> > net.bridge.bridge-nf-call-iptables = 1
> > [init] Using Kubernetes version: v1.28.4
> > [preflight] Running pre-flight checks
> > error execution phase preflight: [preflight] Some fatal errors occurred:
> > [ERROR CRI]: container runtime is not running: output:
> > time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection:
> > validate CRI v1 runtime API for endpoint
> > \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> > Unimplemented desc = unknown service runtime.v1.RuntimeService"
> > , error: exit status 1
> > [preflight] If you know what you are doing, you can make a check non-fatal
> > with `--ignore-preflight-errors=...`
> > To see the stack trace of this error execute with --v=5 or higher
> > [init] Using Kubernetes version: v1.28.4
> > [preflight] Running pre-flight checks
> > error execution phase preflight: [preflight] Some fatal errors occurred:
> > [ERROR CRI]: container runtime is not running: output:
> > time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection:
> > validate CRI v1 runtime API for endpoint
> > \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> > Unimplemented desc = unknown service runtime.v1.RuntimeService"
> > , error: exit status 1
> > [preflight] If you know what you are doing, you can make a check non-fatal
> > with `--ignore-preflight-errors=...`
> > To see the stack trace of this error execute with --v=5 or higher
> > [init] Using Kubernetes version: v1.28.4
> > [preflight] Running pre-flight checks
> > error execution phase preflight: [preflight] Some fatal errors occurred:
> > [ERROR CRI]: container runtime is not running: output:
> > time="2024-04-11T15:33:27Z" level=fatal msg="validate service connection:
> > validate CRI v1 runtime API for endpoint
> > \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> > Unimplemented desc = unknown service runtime.v1.RuntimeService"
> > , error: exit status 1
> > [preflight] If you know what you are doing, you can make a check non-fatal
> > with `--ignore-preflight-errors=...`
> > To see the stack trace of this error execute with --v=5 or higher
> > Error: kubeadm init failed!
> >
> >
> >
> >
> >
> > Le jeu. 11 avr. 2024 à 17:19, Wei ZHOU <us...@gmail.com> a écrit :
> >
> >> Hi,
> >>
> >> Please refer to
> >> https://github.com/apache/cloudstack/issues/8681#issuecomment-1999083241
> >>
> >> The containerd in 4.16/4.17 sysyemvm template is too old.
> >>
> >> -Wei
> >>
> >>
> >> On Thursday, April 11, 2024, benoit lair <ku...@gmail.com> wrote:
> >>
> >> > Hi Rohit,
> >> >
> >> > I already tested with different iso from cloudstack from CKS repos,
> >> > It is ok with ACS 4.16 with community iso 1.23.3 but it fails from
> >> 1.24.0
> >> > and following others versions
> >> >
> >> > I tried to do an upgrade from 1.23.3 with 1.24.0, it fails,
> >> > I tried to bootstrap a k8s cluster with 1.24.0, also with 1.25 or 1.27.3
> >> > and 1.28.4
> >> >
> >> > Last try, i tested k8s community iso 1.25.0
> >> > in CS UI it saying "Create Kubernetes Cluster k8s-cks-cl16 in
> >> progress"
> >> > and it fails
> >> > If i try to ssh on control node with VR_Pub_IP port 2222
> >> > i have nothing in execution
> >> >
> >> > If i try manual install :
> >> > core@k8s-cks-cl16-control-18eccd0c12c:~$ sudo
> >> /opt/bin/setup-kube-system
> >> > mount: /mnt/k8sdisk: /dev/sr0 already mounted on /mnt/k8sdisk.
> >> > Installing binaries from /mnt/k8sdisk/
> >> > 5b1fa8e3e100: Loading layer
> >> > [==================================================>] 803.8kB/803.8kB
> >> > 39c831b1aa26: Loading layer
> >> > [==================================================>] 26.25MB/26.25MB
> >> > Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
> >> > 417cb9b79ade: Loading layer
> >> > [==================================================>] 657.7kB/657.7kB
> >> > 8d323b160d65: Loading layer
> >> > [==================================================>] 24.95MB/24.95MB
> >> > Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
> >> > 256bc5c338a6: Loading layer
> >> > [==================================================>] 122.2kB/122.2kB
> >> > df1818f16337: Loading layer
> >> > [==================================================>] 14.71MB/14.71MB
> >> > Loaded image: registry.k8s.io/coredns/coredns:v1.9.3
> >> > bd8a70623766: Loading layer
> >> > [==================================================>] 75.78MB/75.78MB
> >> > c88361932af5: Loading layer
> >> > [==================================================>] 508B/508B
> >> > Loaded image: kubernetesui/dashboard:v2.7.0
> >> > 0b031aac6569: Loading layer
> >> > [==================================================>] 804kB/804kB
> >> > cc0597072b83: Loading layer
> >> > [==================================================>] 1.076MB/1.076MB
> >> > 5a8045f731fc: Loading layer
> >> > [==================================================>] 90.58MB/90.58MB
> >> > f5a5ef1a005f: Loading layer
> >> > [==================================================>] 1.188MB/1.188MB
> >> > 5cf3bfe5417d: Loading layer
> >> > [==================================================>] 8.506MB/8.506MB
> >> > Loaded image: registry.k8s.io/etcd:3.5.4-0
> >> > 8d7366c22fd8: Loading layer
> >> > [==================================================>] 804.1kB/804.1kB
> >> > 7d5f5bba7c45: Loading layer
> >> > [==================================================>] 694.9kB/694.9kB
> >> > 8154c9483a6c: Loading layer
> >> > [==================================================>] 32.72MB/32.72MB
> >> > Loaded image: registry.k8s.io/kube-apiserver:v1.25.0
> >> > c5f1a47b47bc: Loading layer
> >> > [==================================================>] 29.76MB/29.76MB
> >> > Loaded image: registry.k8s.io/kube-controller-manager:v1.25.0
> >> > e4d19dd12df2: Loading layer
> >> > [==================================================>] 7.631MB/7.631MB
> >> > ba3637ccd78a: Loading layer
> >> > [==================================================>] 12.63MB/12.63MB
> >> > Loaded image: registry.k8s.io/kube-proxy:v1.25.0
> >> > 026969d859f7: Loading layer
> >> > [==================================================>] 14.29MB/14.29MB
> >> > Loaded image: registry.k8s.io/kube-scheduler:v1.25.0
> >> > d01384fea991: Loading layer
> >> > [==================================================>] 19.74MB/19.74MB
> >> > bcec7eb9e567: Loading layer
> >> > [==================================================>] 530B/530B
> >> > Loaded image: kubernetesui/metrics-scraper:v1.0.8
> >> > 961e93cda9dd: Loading layer
> >> > [==================================================>] 307kB/307kB
> >> > Loaded image: registry.k8s.io/pause:3.8
> >> > 1b3ee35aacca: Loading layer
> >> > [==================================================>] 2.796MB/2.796MB
> >> > 17d89e2a5199: Loading layer
> >> > [==================================================>] 7.425MB/7.425MB
> >> > 148b94859892: Loading layer
> >> > [==================================================>] 8.1MB/8.1MB
> >> > dbea472c1ced: Loading layer
> >> > [==================================================>] 4.376MB/4.376MB
> >> > e81f1a367c17: Loading layer
> >> > [==================================================>] 131B/131B
> >> > 351848f6b295: Loading layer
> >> > [==================================================>] 8.216MB/8.216MB
> >> > Loaded image: weaveworks/weave-kube:latest
> >> > b20f7ff12b24: Loading layer
> >> > [==================================================>] 727.9kB/727.9kB
> >> > 1d49e386162e: Loading layer
> >> > [==================================================>] 9.281MB/9.281MB
> >> > 20145eeac146: Loading layer
> >> > [==================================================>] 297B/297B
> >> > d4cc72748587: Loading layer
> >> > [==================================================>] 600B/600B
> >> > Loaded image: weaveworks/weave-npc:latest
> >> > net.bridge.bridge-nf-call-iptables = 1
> >> > [init] Using Kubernetes version: v1.25.0
> >> > [preflight] Running pre-flight checks
> >> > [WARNING SystemVerification]: missing optional cgroups: blkio
> >> > error execution phase preflight: [preflight] Some fatal errors occurred:
> >> > [ERROR CRI]: container runtime is not running: output: E0411
> >> > 11:38:11.935458 12988 remote_runtime.go:948] "Status from runtime
> >> service
> >> > failed" err="rpc error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > time="2024-04-11T11:38:11Z" level=fatal msg="getting status of runtime:
> >> rpc
> >> > error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > , error: exit status 1
> >> > [preflight] If you know what you are doing, you can make a check
> >> non-fatal
> >> > with `--ignore-preflight-errors=...`
> >> > To see the stack trace of this error execute with --v=5 or higher
> >> > [init] Using Kubernetes version: v1.25.0
> >> > [preflight] Running pre-flight checks
> >> > [WARNING SystemVerification]: missing optional cgroups: blkio
> >> > error execution phase preflight: [preflight] Some fatal errors occurred:
> >> > [ERROR CRI]: container runtime is not running: output: E0411
> >> > 11:38:12.079422 13015 remote_runtime.go:948] "Status from runtime
> >> service
> >> > failed" err="rpc error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime:
> >> rpc
> >> > error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > , error: exit status 1
> >> > [preflight] If you know what you are doing, you can make a check
> >> non-fatal
> >> > with `--ignore-preflight-errors=...`
> >> > To see the stack trace of this error execute with --v=5 or higher
> >> > [init] Using Kubernetes version: v1.25.0
> >> > [preflight] Running pre-flight checks
> >> > [WARNING SystemVerification]: missing optional cgroups: blkio
> >> > error execution phase preflight: [preflight] Some fatal errors occurred:
> >> > [ERROR CRI]: container runtime is not running: output: E0411
> >> > 11:38:12.228669 13043 remote_runtime.go:948] "Status from runtime
> >> service
> >> > failed" err="rpc error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime:
> >> rpc
> >> > error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > , error: exit status 1
> >> > [preflight] If you know what you are doing, you can make a check
> >> non-fatal
> >> > with `--ignore-preflight-errors=...`
> >> > To see the stack trace of this error execute with --v=5 or higher
> >> > Error: kubeadm init failed!
> >> >
> >> >
> >> > Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <ro...@shapeblue.com> a
> >> > écrit :
> >> >
> >> > > Hi Benoit,
> >> > >
> >> > > The CKS feature has been improving over versions, I don't know if what
> >> > > you're trying to achieve is possible with it. Maybe try a different
> >> > version
> >> > > of the data iso:
> >> > > http://download.cloudstack.org/cks/
> >> > >
> >> > > Alternatively, you can also have a look at the CAPC project:
> >> > > https://cluster-api-cloudstack.sigs.k8s.io
> >> > >
> >> > >
> >> > > Regards.
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > ________________________________
> >> > > From: benoit lair <ku...@gmail.com>
> >> > > Sent: Thursday, April 11, 2024 14:16
> >> > > To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <
> >> > > dev@cloudstack.apache.org>
> >> > > Subject: Re: ACS 4.16 - Change SystemVM template for CKS
> >> > >
> >> > > Any advices ?
> >> > >
> >> > > Best regards
> >> > >
> >> > > Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a
> >> > écrit :
> >> > >
> >> > > > I am opened to every alternatives of changing system vm templates
> >> > > > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
> >> > > >
> >> > > > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a
> >> > > écrit :
> >> > > >
> >> > > >> Hello Folks,
> >> > > >>
> >> > > >> I am trying to install K8s cluster with community iso 1.28.4
> >> > > >> I am with a ACS 4.16.0 environment
> >> > > >> It seems K8S is not working out of the box with v > 1.23.3 due to
> >> > > >> containerd.io version
> >> > > >>
> >> > > >> I would like to release a template who will work with K8s > 1.23.3
> >> on
> >> > > acs
> >> > > >> 4.16
> >> > > >> How can i tell CS to take another system vm template, avoiding to
> >> mess
> >> > > >> normal features with VR and VPC VR keeping the systemvm template
> >> > issued
> >> > > >> with CS 4.16 ?
> >> > > >>
> >> > > >> I am with XCP-NG 8.2.1 in production
> >> > > >>
> >> > > >> Thanks for your help or advises
> >> > > >> Best regards,
> >> > > >> Benoit
> >> > > >>
> >> > > >
> >> > >
> >> >
> >>
> >
>
--
Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by Wei ZHOU <us...@gmail.com>.
See inline
On Tuesday, April 16, 2024, benoit lair <ku...@gmail.com> wrote:
> Ok and so there's no interaction with CS for firewalling and public ips
> access par example ?
CAPC automatically adds some lb rules and firewall rules for the k8s
apiserver.
To support k8s services with Type=LoadBalancer, you can deploy
https://github.com/apache/cloudstack-kubernetes-provider
getting it in a vpc tier for internal dmz use ?
No. vpc is not supported.
Refer to
https://github.com/kubernetes-sigs/cluster-api-provider-cloudstack/issues/314
> Le mar. 16 avr. 2024 à 11:12, Wei ZHOU <us...@gmail.com> a écrit :
>
> > Hi benoit,
> >
> > You can try CAPC, it should work with 4.16
> > https://github.com/kubernetes-sigs/cluster-api-provider-cloudstack
> >
> > You need a local kind cluster to manage the CAPC clusters. They are
> > not managed by CloudStack.
> >
> >
> > -Wei
> >
> > On Fri, Apr 12, 2024 at 2:43 PM benoit lair <ku...@gmail.com>
> wrote:
> > >
> > > Hi Wei,
> > >
> > > upgrading from 4.16 to 4.19 will be a huge work for me
> > > I have to test vpc/shared network, vrrp vpc with public lb Netscaler
> and
> > > withou it on several VPCs, i added support for VPX 12 and VPX 13 on it
> > with
> > > a nginx middleware rewriting rules from CS
> > > I am on xcp-ng 8.2.1
> > > i kept during years with a CS 4.3, restarting from 4.16, i would like
> > this
> > > time to keep LTS in production
> > > Is there a specific upgrade path to respect ? a specific process to
> > observe
> > > ? what rollback mechanism is possible ?
> > >
> > > For my K8S needs, I would like to upgrade it, but it was looking first
> > for
> > > a preproduction K8s working on 1.28 in order integrating CI/CD
> pipelines
> > > So this would not too be a problem upgrading CS after installing a k8s
> > > cluster for CI/CD
> > >
> > > Rohit was talking about CAPC, i can use it independently of the version
> > of
> > > ACS i use ?
> > >
> >
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by benoit lair <ku...@gmail.com>.
Ok and so there's no interaction with CS for firewalling and public ips
access par example ?
getting it in a vpc tier for internal dmz use ?
Le mar. 16 avr. 2024 à 11:12, Wei ZHOU <us...@gmail.com> a écrit :
> Hi benoit,
>
> You can try CAPC, it should work with 4.16
> https://github.com/kubernetes-sigs/cluster-api-provider-cloudstack
>
> You need a local kind cluster to manage the CAPC clusters. They are
> not managed by CloudStack.
>
>
> -Wei
>
> On Fri, Apr 12, 2024 at 2:43 PM benoit lair <ku...@gmail.com> wrote:
> >
> > Hi Wei,
> >
> > upgrading from 4.16 to 4.19 will be a huge work for me
> > I have to test vpc/shared network, vrrp vpc with public lb Netscaler and
> > withou it on several VPCs, i added support for VPX 12 and VPX 13 on it
> with
> > a nginx middleware rewriting rules from CS
> > I am on xcp-ng 8.2.1
> > i kept during years with a CS 4.3, restarting from 4.16, i would like
> this
> > time to keep LTS in production
> > Is there a specific upgrade path to respect ? a specific process to
> observe
> > ? what rollback mechanism is possible ?
> >
> > For my K8S needs, I would like to upgrade it, but it was looking first
> for
> > a preproduction K8s working on 1.28 in order integrating CI/CD pipelines
> > So this would not too be a problem upgrading CS after installing a k8s
> > cluster for CI/CD
> >
> > Rohit was talking about CAPC, i can use it independently of the version
> of
> > ACS i use ?
> >
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by Wei ZHOU <us...@gmail.com>.
Hi benoit,
You can try CAPC, it should work with 4.16
https://github.com/kubernetes-sigs/cluster-api-provider-cloudstack
You need a local kind cluster to manage the CAPC clusters. They are
not managed by CloudStack.
-Wei
On Fri, Apr 12, 2024 at 2:43 PM benoit lair <ku...@gmail.com> wrote:
>
> Hi Wei,
>
> upgrading from 4.16 to 4.19 will be a huge work for me
> I have to test vpc/shared network, vrrp vpc with public lb Netscaler and
> withou it on several VPCs, i added support for VPX 12 and VPX 13 on it with
> a nginx middleware rewriting rules from CS
> I am on xcp-ng 8.2.1
> i kept during years with a CS 4.3, restarting from 4.16, i would like this
> time to keep LTS in production
> Is there a specific upgrade path to respect ? a specific process to observe
> ? what rollback mechanism is possible ?
>
> For my K8S needs, I would like to upgrade it, but it was looking first for
> a preproduction K8s working on 1.28 in order integrating CI/CD pipelines
> So this would not too be a problem upgrading CS after installing a k8s
> cluster for CI/CD
>
> Rohit was talking about CAPC, i can use it independently of the version of
> ACS i use ?
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by benoit lair <ku...@gmail.com>.
Hi Wei,
upgrading from 4.16 to 4.19 will be a huge work for me
I have to test vpc/shared network, vrrp vpc with public lb Netscaler and
withou it on several VPCs, i added support for VPX 12 and VPX 13 on it with
a nginx middleware rewriting rules from CS
I am on xcp-ng 8.2.1
i kept during years with a CS 4.3, restarting from 4.16, i would like this
time to keep LTS in production
Is there a specific upgrade path to respect ? a specific process to observe
? what rollback mechanism is possible ?
For my K8S needs, I would like to upgrade it, but it was looking first for
a preproduction K8s working on 1.28 in order integrating CI/CD pipelines
So this would not too be a problem upgrading CS after installing a k8s
cluster for CI/CD
Rohit was talking about CAPC, i can use it independently of the version of
ACS i use ?
Thanks
Le ven. 12 avr. 2024 à 14:30, Wei ZHOU <us...@gmail.com> a écrit :
> Hi,
>
> I would suggest you not to spend time on this... 4.16 is EOL
>
> Please upgrade to cloudstack 4.18/4.19, or use old kubernetes versions ..
>
>
> -Wei
>
>
> On Fri, Apr 12, 2024 at 2:25 PM benoit lair <ku...@gmail.com> wrote:
> >
> > i succeded to install systemvm template 4.19 as serving template for
> > control and worker nodes with cks community iso 1.28.4
> > when cluster saying "starting", i connect to node controller via p2222,
> saw
> > that docker was not installed, but containerd.io yes
> >
> > I've done the following :
> >
> > apt install docker-ce
> > cp /etc/containerd/config.toml /etc/containerd/config.toml.bck
> > containerd config default | tee /etc/containerd/config.toml
> > /opt/bin/setup-kube-system
> > /opt/bin/deploy-kube-system
> >
> > on to control node and same after on worker node
> > CS UI show kubernetes yaml
> > a "kubectl.exe --kubeconfig kube.conf.conf get nodes"
> >
> > NAME STATUS ROLES AGE
> VERSION
> > k8s-cks-cl19-control-18ed18311c9 Ready control-plane 3h1m
> v1.28.4
> > k8s-cks-cl19-node-18ed1850433 Ready <none> 170m
> v1.28.4
> >
> > However CS says the cluster is in alert state
> > And dashboad is not working
> >
> > Any advice ?
> >
> > when executing this on my laptop :
> > kubectl.exe --kubeconfig cl19_k8s_1.28.4.conf proxy
> >
> > and opening this :
> >
> http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
> >
> > I have this result :
> > {
> > "kind": "Status",
> > "apiVersion": "v1",
> > "metadata": {},
> > "status": "Failure",
> > "message": "no endpoints available for service
> \"kubernetes-dashboard\"",
> > "reason": "ServiceUnavailable",
> > "code": 503
> > }
> >
> > Somebody got this problem with dashboard ?
> >
> > Did i miss something installing the cluster manually with acs stuff ?
> >
> > Le jeu. 11 avr. 2024 à 17:40, benoit lair <ku...@gmail.com> a
> écrit :
> >
> > > Hi Wei,
> > >
> > > Thanks for the sharing, i also tried to install the systemvm 4.19
> > > I have control node and worker node under template systemvm 4.19 (
> > >
> http://download.cloudstack.org/systemvm/4.19/systemvmtemplate-4.19.0-xen.vhd.bz2
> > > )
> > > I tried with community cks iso 1.25.0 and 1.28.9
> > > On systemvm 4.19 docker was not present by default, just containerd in
> > > 1.6.x version
> > > I installed docker-ce and docker-ce-cli :
> > >
> > > apt install -y apt-transport-https ca-certificates curl
> > > software-properties-common
> > > curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg
> > > --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
> > > echo "deb [arch=$(dpkg --print-architecture)
> > > signed-by=/usr/share/keyrings/docker-archive-keyring.gpg]
> > > https://download.docker.com/linux/debian $(lsb_release -cs) stable" |
> > > sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
> > > remove line about docker in double in /etc/apt/sources.list
> > > apt update
> > > apt install -y docker-ce
> > > apt install -y docker-cli
> > > i mounted the 1.28.4 cks community iso in control vm
> > >
> > > and /opt/bin/setup-kube-system
> > > i have these errors :
> > >
> > > Is it the good way to manually install kube system
> > >
> > > root@k8s-cks-cl17-control-18ecdc1e5bd:/home/core#
> > > /opt/bin/setup-kube-system
> > > Installing binaries from /mnt/k8sdisk/
> > > 5b1fa8e3e100: Loading layer
> > > [==================================================>] 803.8kB/803.8kB
> > > 39c831b1aa26: Loading layer
> > > [==================================================>] 26.25MB/26.25MB
> > > Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
> > > 417cb9b79ade: Loading layer
> > > [==================================================>] 657.7kB/657.7kB
> > > 8d323b160d65: Loading layer
> > > [==================================================>] 24.95MB/24.95MB
> > > Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
> > > 6a4a177e62f3: Loading layer
> > > [==================================================>] 115.2kB/115.2kB
> > > 398c9baff0ce: Loading layer
> > > [==================================================>] 16.07MB/16.07MB
> > > Loaded image: registry.k8s.io/coredns/coredns:v1.10.1
> > > bd8a70623766: Loading layer
> > > [==================================================>] 75.78MB/75.78MB
> > > c88361932af5: Loading layer
> > > [==================================================>] 508B/508B
> > > Loaded image: kubernetesui/dashboard:v2.7.0
> > > e023e0e48e6e: Loading layer
> > > [==================================================>] 103.7kB/103.7kB
> > > 6fbdf253bbc2: Loading layer
> > > [==================================================>] 21.2kB/21.2kB
> > > 7bea6b893187: Loading layer
> > > [==================================================>] 716.5kB/716.5kB
> > > ff5700ec5418: Loading layer
> > > [==================================================>] 317B/317B
> > > d52f02c6501c: Loading layer
> > > [==================================================>] 198B/198B
> > > e624a5370eca: Loading layer
> > > [==================================================>] 113B/113B
> > > 1a73b54f556b: Loading layer
> > > [==================================================>] 385B/385B
> > > d2d7ec0f6756: Loading layer
> > > [==================================================>] 355B/355B
> > > 4cb10dd2545b: Loading layer
> > > [==================================================>] 130.6kB/130.6kB
> > > ba9afb2b3e0c: Loading layer
> > > [==================================================>] 1.076MB/1.076MB
> > > 22bba3da6b0d: Loading layer
> > > [==================================================>] 91.29MB/91.29MB
> > > a4563151d59b: Loading layer
> > > [==================================================>] 9.544MB/9.544MB
> > > Loaded image: registry.k8s.io/etcd:3.5.9-0
> > > 54ad2ec71039: Loading layer
> > > [==================================================>] 103.7kB/103.7kB
> > > 6fbdf253bbc2: Loading layer
> > > [==================================================>] 21.2kB/21.2kB
> > > 7bea6b893187: Loading layer
> > > [==================================================>] 716.5kB/716.5kB
> > > ff5700ec5418: Loading layer
> > > [==================================================>] 317B/317B
> > > d52f02c6501c: Loading layer
> > > [==================================================>] 198B/198B
> > > e624a5370eca: Loading layer
> > > [==================================================>] 113B/113B
> > > 1a73b54f556b: Loading layer
> > > [==================================================>] 385B/385B
> > > d2d7ec0f6756: Loading layer
> > > [==================================================>] 355B/355B
> > > 4cb10dd2545b: Loading layer
> > > [==================================================>] 130.6kB/130.6kB
> > > 3f3c83f9a2fa: Loading layer
> > > [==================================================>] 724.8kB/724.8kB
> > > 6a2c4bedb8de: Loading layer
> > > [==================================================>] 32.98MB/32.98MB
> > > Loaded image: registry.k8s.io/kube-apiserver:v1.28.4
> > > e0b89915180d: Loading layer
> > > [==================================================>] 31.72MB/31.72MB
> > > Loaded image: registry.k8s.io/kube-controller-manager:v1.28.4
> > > d2a4b045f3d0: Loading layer
> > > [==================================================>] 7.73MB/7.73MB
> > > 3905b20a2df2: Loading layer
> > > [==================================================>] 16.85MB/16.85MB
> > > Loaded image: registry.k8s.io/kube-proxy:v1.28.4
> > > f35f4c1ae44f: Loading layer
> > > [==================================================>] 17.13MB/17.13MB
> > > Loaded image: registry.k8s.io/kube-scheduler:v1.28.4
> > > d01384fea991: Loading layer
> > > [==================================================>] 19.74MB/19.74MB
> > > bcec7eb9e567: Loading layer
> > > [==================================================>] 530B/530B
> > > Loaded image: kubernetesui/metrics-scraper:v1.0.8
> > > e3e5579ddd43: Loading layer
> > > [==================================================>] 317.6kB/317.6kB
> > > Loaded image: registry.k8s.io/pause:3.9
> > > 1b3ee35aacca: Loading layer
> > > [==================================================>] 2.796MB/2.796MB
> > > 910ce076f504: Loading layer
> > > [==================================================>] 7.425MB/7.425MB
> > > a8e8b7b8e08a: Loading layer
> > > [==================================================>] 8.101MB/8.101MB
> > > 084e56a9c24b: Loading layer
> > > [==================================================>] 4.376MB/4.376MB
> > > 334d70dc85ec: Loading layer
> > > [==================================================>] 159B/159B
> > > a13197d8bda5: Loading layer
> > > [==================================================>] 8.216MB/8.216MB
> > > Loaded image: weaveworks/weave-kube:2.8.1
> > > 998efb010df6: Loading layer
> > > [==================================================>] 728.2kB/728.2kB
> > > ed37391def99: Loading layer
> > > [==================================================>] 9.28MB/9.28MB
> > > 175a472c5f77: Loading layer
> > > [==================================================>] 299B/299B
> > > a8764e32e9fe: Loading layer
> > > [==================================================>] 613B/613B
> > > Loaded image: weaveworks/weave-npc:2.8.1
> > > net.bridge.bridge-nf-call-iptables = 1
> > > [init] Using Kubernetes version: v1.28.4
> > > [preflight] Running pre-flight checks
> > > error execution phase preflight: [preflight] Some fatal errors
> occurred:
> > > [ERROR CRI]: container runtime is not running: output:
> > > time="2024-04-11T15:33:26Z" level=fatal msg="validate service
> connection:
> > > validate CRI v1 runtime API for endpoint
> > > \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> > > Unimplemented desc = unknown service runtime.v1.RuntimeService"
> > > , error: exit status 1
> > > [preflight] If you know what you are doing, you can make a check
> non-fatal
> > > with `--ignore-preflight-errors=...`
> > > To see the stack trace of this error execute with --v=5 or higher
> > > [init] Using Kubernetes version: v1.28.4
> > > [preflight] Running pre-flight checks
> > > error execution phase preflight: [preflight] Some fatal errors
> occurred:
> > > [ERROR CRI]: container runtime is not running: output:
> > > time="2024-04-11T15:33:26Z" level=fatal msg="validate service
> connection:
> > > validate CRI v1 runtime API for endpoint
> > > \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> > > Unimplemented desc = unknown service runtime.v1.RuntimeService"
> > > , error: exit status 1
> > > [preflight] If you know what you are doing, you can make a check
> non-fatal
> > > with `--ignore-preflight-errors=...`
> > > To see the stack trace of this error execute with --v=5 or higher
> > > [init] Using Kubernetes version: v1.28.4
> > > [preflight] Running pre-flight checks
> > > error execution phase preflight: [preflight] Some fatal errors
> occurred:
> > > [ERROR CRI]: container runtime is not running: output:
> > > time="2024-04-11T15:33:27Z" level=fatal msg="validate service
> connection:
> > > validate CRI v1 runtime API for endpoint
> > > \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> > > Unimplemented desc = unknown service runtime.v1.RuntimeService"
> > > , error: exit status 1
> > > [preflight] If you know what you are doing, you can make a check
> non-fatal
> > > with `--ignore-preflight-errors=...`
> > > To see the stack trace of this error execute with --v=5 or higher
> > > Error: kubeadm init failed!
> > >
> > >
> > >
> > >
> > >
> > > Le jeu. 11 avr. 2024 à 17:19, Wei ZHOU <us...@gmail.com> a
> écrit :
> > >
> > >> Hi,
> > >>
> > >> Please refer to
> > >>
> https://github.com/apache/cloudstack/issues/8681#issuecomment-1999083241
> > >>
> > >> The containerd in 4.16/4.17 sysyemvm template is too old.
> > >>
> > >> -Wei
> > >>
> > >>
> > >> On Thursday, April 11, 2024, benoit lair <ku...@gmail.com>
> wrote:
> > >>
> > >> > Hi Rohit,
> > >> >
> > >> > I already tested with different iso from cloudstack from CKS repos,
> > >> > It is ok with ACS 4.16 with community iso 1.23.3 but it fails from
> > >> 1.24.0
> > >> > and following others versions
> > >> >
> > >> > I tried to do an upgrade from 1.23.3 with 1.24.0, it fails,
> > >> > I tried to bootstrap a k8s cluster with 1.24.0, also with 1.25 or
> 1.27.3
> > >> > and 1.28.4
> > >> >
> > >> > Last try, i tested k8s community iso 1.25.0
> > >> > in CS UI it saying "Create Kubernetes Cluster k8s-cks-cl16 in
> > >> progress"
> > >> > and it fails
> > >> > If i try to ssh on control node with VR_Pub_IP port 2222
> > >> > i have nothing in execution
> > >> >
> > >> > If i try manual install :
> > >> > core@k8s-cks-cl16-control-18eccd0c12c:~$ sudo
> > >> /opt/bin/setup-kube-system
> > >> > mount: /mnt/k8sdisk: /dev/sr0 already mounted on /mnt/k8sdisk.
> > >> > Installing binaries from /mnt/k8sdisk/
> > >> > 5b1fa8e3e100: Loading layer
> > >> > [==================================================>]
> 803.8kB/803.8kB
> > >> > 39c831b1aa26: Loading layer
> > >> > [==================================================>]
> 26.25MB/26.25MB
> > >> > Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
> > >> > 417cb9b79ade: Loading layer
> > >> > [==================================================>]
> 657.7kB/657.7kB
> > >> > 8d323b160d65: Loading layer
> > >> > [==================================================>]
> 24.95MB/24.95MB
> > >> > Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
> > >> > 256bc5c338a6: Loading layer
> > >> > [==================================================>]
> 122.2kB/122.2kB
> > >> > df1818f16337: Loading layer
> > >> > [==================================================>]
> 14.71MB/14.71MB
> > >> > Loaded image: registry.k8s.io/coredns/coredns:v1.9.3
> > >> > bd8a70623766: Loading layer
> > >> > [==================================================>]
> 75.78MB/75.78MB
> > >> > c88361932af5: Loading layer
> > >> > [==================================================>] 508B/508B
> > >> > Loaded image: kubernetesui/dashboard:v2.7.0
> > >> > 0b031aac6569: Loading layer
> > >> > [==================================================>] 804kB/804kB
> > >> > cc0597072b83: Loading layer
> > >> > [==================================================>]
> 1.076MB/1.076MB
> > >> > 5a8045f731fc: Loading layer
> > >> > [==================================================>]
> 90.58MB/90.58MB
> > >> > f5a5ef1a005f: Loading layer
> > >> > [==================================================>]
> 1.188MB/1.188MB
> > >> > 5cf3bfe5417d: Loading layer
> > >> > [==================================================>]
> 8.506MB/8.506MB
> > >> > Loaded image: registry.k8s.io/etcd:3.5.4-0
> > >> > 8d7366c22fd8: Loading layer
> > >> > [==================================================>]
> 804.1kB/804.1kB
> > >> > 7d5f5bba7c45: Loading layer
> > >> > [==================================================>]
> 694.9kB/694.9kB
> > >> > 8154c9483a6c: Loading layer
> > >> > [==================================================>]
> 32.72MB/32.72MB
> > >> > Loaded image: registry.k8s.io/kube-apiserver:v1.25.0
> > >> > c5f1a47b47bc: Loading layer
> > >> > [==================================================>]
> 29.76MB/29.76MB
> > >> > Loaded image: registry.k8s.io/kube-controller-manager:v1.25.0
> > >> > e4d19dd12df2: Loading layer
> > >> > [==================================================>]
> 7.631MB/7.631MB
> > >> > ba3637ccd78a: Loading layer
> > >> > [==================================================>]
> 12.63MB/12.63MB
> > >> > Loaded image: registry.k8s.io/kube-proxy:v1.25.0
> > >> > 026969d859f7: Loading layer
> > >> > [==================================================>]
> 14.29MB/14.29MB
> > >> > Loaded image: registry.k8s.io/kube-scheduler:v1.25.0
> > >> > d01384fea991: Loading layer
> > >> > [==================================================>]
> 19.74MB/19.74MB
> > >> > bcec7eb9e567: Loading layer
> > >> > [==================================================>] 530B/530B
> > >> > Loaded image: kubernetesui/metrics-scraper:v1.0.8
> > >> > 961e93cda9dd: Loading layer
> > >> > [==================================================>] 307kB/307kB
> > >> > Loaded image: registry.k8s.io/pause:3.8
> > >> > 1b3ee35aacca: Loading layer
> > >> > [==================================================>]
> 2.796MB/2.796MB
> > >> > 17d89e2a5199: Loading layer
> > >> > [==================================================>]
> 7.425MB/7.425MB
> > >> > 148b94859892: Loading layer
> > >> > [==================================================>] 8.1MB/8.1MB
> > >> > dbea472c1ced: Loading layer
> > >> > [==================================================>]
> 4.376MB/4.376MB
> > >> > e81f1a367c17: Loading layer
> > >> > [==================================================>] 131B/131B
> > >> > 351848f6b295: Loading layer
> > >> > [==================================================>]
> 8.216MB/8.216MB
> > >> > Loaded image: weaveworks/weave-kube:latest
> > >> > b20f7ff12b24: Loading layer
> > >> > [==================================================>]
> 727.9kB/727.9kB
> > >> > 1d49e386162e: Loading layer
> > >> > [==================================================>]
> 9.281MB/9.281MB
> > >> > 20145eeac146: Loading layer
> > >> > [==================================================>] 297B/297B
> > >> > d4cc72748587: Loading layer
> > >> > [==================================================>] 600B/600B
> > >> > Loaded image: weaveworks/weave-npc:latest
> > >> > net.bridge.bridge-nf-call-iptables = 1
> > >> > [init] Using Kubernetes version: v1.25.0
> > >> > [preflight] Running pre-flight checks
> > >> > [WARNING SystemVerification]: missing optional cgroups:
> blkio
> > >> > error execution phase preflight: [preflight] Some fatal errors
> occurred:
> > >> > [ERROR CRI]: container runtime is not running: output: E0411
> > >> > 11:38:11.935458 12988 remote_runtime.go:948] "Status from runtime
> > >> service
> > >> > failed" err="rpc error: code = Unimplemented desc = unknown service
> > >> > runtime.v1alpha2.RuntimeService"
> > >> > time="2024-04-11T11:38:11Z" level=fatal msg="getting status of
> runtime:
> > >> rpc
> > >> > error: code = Unimplemented desc = unknown service
> > >> > runtime.v1alpha2.RuntimeService"
> > >> > , error: exit status 1
> > >> > [preflight] If you know what you are doing, you can make a check
> > >> non-fatal
> > >> > with `--ignore-preflight-errors=...`
> > >> > To see the stack trace of this error execute with --v=5 or higher
> > >> > [init] Using Kubernetes version: v1.25.0
> > >> > [preflight] Running pre-flight checks
> > >> > [WARNING SystemVerification]: missing optional cgroups:
> blkio
> > >> > error execution phase preflight: [preflight] Some fatal errors
> occurred:
> > >> > [ERROR CRI]: container runtime is not running: output: E0411
> > >> > 11:38:12.079422 13015 remote_runtime.go:948] "Status from runtime
> > >> service
> > >> > failed" err="rpc error: code = Unimplemented desc = unknown service
> > >> > runtime.v1alpha2.RuntimeService"
> > >> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of
> runtime:
> > >> rpc
> > >> > error: code = Unimplemented desc = unknown service
> > >> > runtime.v1alpha2.RuntimeService"
> > >> > , error: exit status 1
> > >> > [preflight] If you know what you are doing, you can make a check
> > >> non-fatal
> > >> > with `--ignore-preflight-errors=...`
> > >> > To see the stack trace of this error execute with --v=5 or higher
> > >> > [init] Using Kubernetes version: v1.25.0
> > >> > [preflight] Running pre-flight checks
> > >> > [WARNING SystemVerification]: missing optional cgroups:
> blkio
> > >> > error execution phase preflight: [preflight] Some fatal errors
> occurred:
> > >> > [ERROR CRI]: container runtime is not running: output: E0411
> > >> > 11:38:12.228669 13043 remote_runtime.go:948] "Status from runtime
> > >> service
> > >> > failed" err="rpc error: code = Unimplemented desc = unknown service
> > >> > runtime.v1alpha2.RuntimeService"
> > >> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of
> runtime:
> > >> rpc
> > >> > error: code = Unimplemented desc = unknown service
> > >> > runtime.v1alpha2.RuntimeService"
> > >> > , error: exit status 1
> > >> > [preflight] If you know what you are doing, you can make a check
> > >> non-fatal
> > >> > with `--ignore-preflight-errors=...`
> > >> > To see the stack trace of this error execute with --v=5 or higher
> > >> > Error: kubeadm init failed!
> > >> >
> > >> >
> > >> > Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <
> rohit.yadav@shapeblue.com> a
> > >> > écrit :
> > >> >
> > >> > > Hi Benoit,
> > >> > >
> > >> > > The CKS feature has been improving over versions, I don't know if
> what
> > >> > > you're trying to achieve is possible with it. Maybe try a
> different
> > >> > version
> > >> > > of the data iso:
> > >> > > http://download.cloudstack.org/cks/
> > >> > >
> > >> > > Alternatively, you can also have a look at the CAPC project:
> > >> > > https://cluster-api-cloudstack.sigs.k8s.io
> > >> > >
> > >> > >
> > >> > > Regards.
> > >> > >
> > >> > >
> > >> > >
> > >> > >
> > >> > > ________________________________
> > >> > > From: benoit lair <ku...@gmail.com>
> > >> > > Sent: Thursday, April 11, 2024 14:16
> > >> > > To: users@cloudstack.apache.org <us...@cloudstack.apache.org>;
> dev <
> > >> > > dev@cloudstack.apache.org>
> > >> > > Subject: Re: ACS 4.16 - Change SystemVM template for CKS
> > >> > >
> > >> > > Any advices ?
> > >> > >
> > >> > > Best regards
> > >> > >
> > >> > > Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com>
> a
> > >> > écrit :
> > >> > >
> > >> > > > I am opened to every alternatives of changing system vm
> templates
> > >> > > > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
> > >> > > >
> > >> > > > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com>
> a
> > >> > > écrit :
> > >> > > >
> > >> > > >> Hello Folks,
> > >> > > >>
> > >> > > >> I am trying to install K8s cluster with community iso 1.28.4
> > >> > > >> I am with a ACS 4.16.0 environment
> > >> > > >> It seems K8S is not working out of the box with v > 1.23.3 due
> to
> > >> > > >> containerd.io version
> > >> > > >>
> > >> > > >> I would like to release a template who will work with K8s >
> 1.23.3
> > >> on
> > >> > > acs
> > >> > > >> 4.16
> > >> > > >> How can i tell CS to take another system vm template, avoiding
> to
> > >> mess
> > >> > > >> normal features with VR and VPC VR keeping the systemvm
> template
> > >> > issued
> > >> > > >> with CS 4.16 ?
> > >> > > >>
> > >> > > >> I am with XCP-NG 8.2.1 in production
> > >> > > >>
> > >> > > >> Thanks for your help or advises
> > >> > > >> Best regards,
> > >> > > >> Benoit
> > >> > > >>
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by Wei ZHOU <us...@gmail.com>.
Hi,
I would suggest you not to spend time on this... 4.16 is EOL
Please upgrade to cloudstack 4.18/4.19, or use old kubernetes versions ..
-Wei
On Fri, Apr 12, 2024 at 2:25 PM benoit lair <ku...@gmail.com> wrote:
>
> i succeded to install systemvm template 4.19 as serving template for
> control and worker nodes with cks community iso 1.28.4
> when cluster saying "starting", i connect to node controller via p2222, saw
> that docker was not installed, but containerd.io yes
>
> I've done the following :
>
> apt install docker-ce
> cp /etc/containerd/config.toml /etc/containerd/config.toml.bck
> containerd config default | tee /etc/containerd/config.toml
> /opt/bin/setup-kube-system
> /opt/bin/deploy-kube-system
>
> on to control node and same after on worker node
> CS UI show kubernetes yaml
> a "kubectl.exe --kubeconfig kube.conf.conf get nodes"
>
> NAME STATUS ROLES AGE VERSION
> k8s-cks-cl19-control-18ed18311c9 Ready control-plane 3h1m v1.28.4
> k8s-cks-cl19-node-18ed1850433 Ready <none> 170m v1.28.4
>
> However CS says the cluster is in alert state
> And dashboad is not working
>
> Any advice ?
>
> when executing this on my laptop :
> kubectl.exe --kubeconfig cl19_k8s_1.28.4.conf proxy
>
> and opening this :
> http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
>
> I have this result :
> {
> "kind": "Status",
> "apiVersion": "v1",
> "metadata": {},
> "status": "Failure",
> "message": "no endpoints available for service \"kubernetes-dashboard\"",
> "reason": "ServiceUnavailable",
> "code": 503
> }
>
> Somebody got this problem with dashboard ?
>
> Did i miss something installing the cluster manually with acs stuff ?
>
> Le jeu. 11 avr. 2024 à 17:40, benoit lair <ku...@gmail.com> a écrit :
>
> > Hi Wei,
> >
> > Thanks for the sharing, i also tried to install the systemvm 4.19
> > I have control node and worker node under template systemvm 4.19 (
> > http://download.cloudstack.org/systemvm/4.19/systemvmtemplate-4.19.0-xen.vhd.bz2
> > )
> > I tried with community cks iso 1.25.0 and 1.28.9
> > On systemvm 4.19 docker was not present by default, just containerd in
> > 1.6.x version
> > I installed docker-ce and docker-ce-cli :
> >
> > apt install -y apt-transport-https ca-certificates curl
> > software-properties-common
> > curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg
> > --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
> > echo "deb [arch=$(dpkg --print-architecture)
> > signed-by=/usr/share/keyrings/docker-archive-keyring.gpg]
> > https://download.docker.com/linux/debian $(lsb_release -cs) stable" |
> > sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
> > remove line about docker in double in /etc/apt/sources.list
> > apt update
> > apt install -y docker-ce
> > apt install -y docker-cli
> > i mounted the 1.28.4 cks community iso in control vm
> >
> > and /opt/bin/setup-kube-system
> > i have these errors :
> >
> > Is it the good way to manually install kube system
> >
> > root@k8s-cks-cl17-control-18ecdc1e5bd:/home/core#
> > /opt/bin/setup-kube-system
> > Installing binaries from /mnt/k8sdisk/
> > 5b1fa8e3e100: Loading layer
> > [==================================================>] 803.8kB/803.8kB
> > 39c831b1aa26: Loading layer
> > [==================================================>] 26.25MB/26.25MB
> > Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
> > 417cb9b79ade: Loading layer
> > [==================================================>] 657.7kB/657.7kB
> > 8d323b160d65: Loading layer
> > [==================================================>] 24.95MB/24.95MB
> > Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
> > 6a4a177e62f3: Loading layer
> > [==================================================>] 115.2kB/115.2kB
> > 398c9baff0ce: Loading layer
> > [==================================================>] 16.07MB/16.07MB
> > Loaded image: registry.k8s.io/coredns/coredns:v1.10.1
> > bd8a70623766: Loading layer
> > [==================================================>] 75.78MB/75.78MB
> > c88361932af5: Loading layer
> > [==================================================>] 508B/508B
> > Loaded image: kubernetesui/dashboard:v2.7.0
> > e023e0e48e6e: Loading layer
> > [==================================================>] 103.7kB/103.7kB
> > 6fbdf253bbc2: Loading layer
> > [==================================================>] 21.2kB/21.2kB
> > 7bea6b893187: Loading layer
> > [==================================================>] 716.5kB/716.5kB
> > ff5700ec5418: Loading layer
> > [==================================================>] 317B/317B
> > d52f02c6501c: Loading layer
> > [==================================================>] 198B/198B
> > e624a5370eca: Loading layer
> > [==================================================>] 113B/113B
> > 1a73b54f556b: Loading layer
> > [==================================================>] 385B/385B
> > d2d7ec0f6756: Loading layer
> > [==================================================>] 355B/355B
> > 4cb10dd2545b: Loading layer
> > [==================================================>] 130.6kB/130.6kB
> > ba9afb2b3e0c: Loading layer
> > [==================================================>] 1.076MB/1.076MB
> > 22bba3da6b0d: Loading layer
> > [==================================================>] 91.29MB/91.29MB
> > a4563151d59b: Loading layer
> > [==================================================>] 9.544MB/9.544MB
> > Loaded image: registry.k8s.io/etcd:3.5.9-0
> > 54ad2ec71039: Loading layer
> > [==================================================>] 103.7kB/103.7kB
> > 6fbdf253bbc2: Loading layer
> > [==================================================>] 21.2kB/21.2kB
> > 7bea6b893187: Loading layer
> > [==================================================>] 716.5kB/716.5kB
> > ff5700ec5418: Loading layer
> > [==================================================>] 317B/317B
> > d52f02c6501c: Loading layer
> > [==================================================>] 198B/198B
> > e624a5370eca: Loading layer
> > [==================================================>] 113B/113B
> > 1a73b54f556b: Loading layer
> > [==================================================>] 385B/385B
> > d2d7ec0f6756: Loading layer
> > [==================================================>] 355B/355B
> > 4cb10dd2545b: Loading layer
> > [==================================================>] 130.6kB/130.6kB
> > 3f3c83f9a2fa: Loading layer
> > [==================================================>] 724.8kB/724.8kB
> > 6a2c4bedb8de: Loading layer
> > [==================================================>] 32.98MB/32.98MB
> > Loaded image: registry.k8s.io/kube-apiserver:v1.28.4
> > e0b89915180d: Loading layer
> > [==================================================>] 31.72MB/31.72MB
> > Loaded image: registry.k8s.io/kube-controller-manager:v1.28.4
> > d2a4b045f3d0: Loading layer
> > [==================================================>] 7.73MB/7.73MB
> > 3905b20a2df2: Loading layer
> > [==================================================>] 16.85MB/16.85MB
> > Loaded image: registry.k8s.io/kube-proxy:v1.28.4
> > f35f4c1ae44f: Loading layer
> > [==================================================>] 17.13MB/17.13MB
> > Loaded image: registry.k8s.io/kube-scheduler:v1.28.4
> > d01384fea991: Loading layer
> > [==================================================>] 19.74MB/19.74MB
> > bcec7eb9e567: Loading layer
> > [==================================================>] 530B/530B
> > Loaded image: kubernetesui/metrics-scraper:v1.0.8
> > e3e5579ddd43: Loading layer
> > [==================================================>] 317.6kB/317.6kB
> > Loaded image: registry.k8s.io/pause:3.9
> > 1b3ee35aacca: Loading layer
> > [==================================================>] 2.796MB/2.796MB
> > 910ce076f504: Loading layer
> > [==================================================>] 7.425MB/7.425MB
> > a8e8b7b8e08a: Loading layer
> > [==================================================>] 8.101MB/8.101MB
> > 084e56a9c24b: Loading layer
> > [==================================================>] 4.376MB/4.376MB
> > 334d70dc85ec: Loading layer
> > [==================================================>] 159B/159B
> > a13197d8bda5: Loading layer
> > [==================================================>] 8.216MB/8.216MB
> > Loaded image: weaveworks/weave-kube:2.8.1
> > 998efb010df6: Loading layer
> > [==================================================>] 728.2kB/728.2kB
> > ed37391def99: Loading layer
> > [==================================================>] 9.28MB/9.28MB
> > 175a472c5f77: Loading layer
> > [==================================================>] 299B/299B
> > a8764e32e9fe: Loading layer
> > [==================================================>] 613B/613B
> > Loaded image: weaveworks/weave-npc:2.8.1
> > net.bridge.bridge-nf-call-iptables = 1
> > [init] Using Kubernetes version: v1.28.4
> > [preflight] Running pre-flight checks
> > error execution phase preflight: [preflight] Some fatal errors occurred:
> > [ERROR CRI]: container runtime is not running: output:
> > time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection:
> > validate CRI v1 runtime API for endpoint
> > \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> > Unimplemented desc = unknown service runtime.v1.RuntimeService"
> > , error: exit status 1
> > [preflight] If you know what you are doing, you can make a check non-fatal
> > with `--ignore-preflight-errors=...`
> > To see the stack trace of this error execute with --v=5 or higher
> > [init] Using Kubernetes version: v1.28.4
> > [preflight] Running pre-flight checks
> > error execution phase preflight: [preflight] Some fatal errors occurred:
> > [ERROR CRI]: container runtime is not running: output:
> > time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection:
> > validate CRI v1 runtime API for endpoint
> > \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> > Unimplemented desc = unknown service runtime.v1.RuntimeService"
> > , error: exit status 1
> > [preflight] If you know what you are doing, you can make a check non-fatal
> > with `--ignore-preflight-errors=...`
> > To see the stack trace of this error execute with --v=5 or higher
> > [init] Using Kubernetes version: v1.28.4
> > [preflight] Running pre-flight checks
> > error execution phase preflight: [preflight] Some fatal errors occurred:
> > [ERROR CRI]: container runtime is not running: output:
> > time="2024-04-11T15:33:27Z" level=fatal msg="validate service connection:
> > validate CRI v1 runtime API for endpoint
> > \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> > Unimplemented desc = unknown service runtime.v1.RuntimeService"
> > , error: exit status 1
> > [preflight] If you know what you are doing, you can make a check non-fatal
> > with `--ignore-preflight-errors=...`
> > To see the stack trace of this error execute with --v=5 or higher
> > Error: kubeadm init failed!
> >
> >
> >
> >
> >
> > Le jeu. 11 avr. 2024 à 17:19, Wei ZHOU <us...@gmail.com> a écrit :
> >
> >> Hi,
> >>
> >> Please refer to
> >> https://github.com/apache/cloudstack/issues/8681#issuecomment-1999083241
> >>
> >> The containerd in 4.16/4.17 sysyemvm template is too old.
> >>
> >> -Wei
> >>
> >>
> >> On Thursday, April 11, 2024, benoit lair <ku...@gmail.com> wrote:
> >>
> >> > Hi Rohit,
> >> >
> >> > I already tested with different iso from cloudstack from CKS repos,
> >> > It is ok with ACS 4.16 with community iso 1.23.3 but it fails from
> >> 1.24.0
> >> > and following others versions
> >> >
> >> > I tried to do an upgrade from 1.23.3 with 1.24.0, it fails,
> >> > I tried to bootstrap a k8s cluster with 1.24.0, also with 1.25 or 1.27.3
> >> > and 1.28.4
> >> >
> >> > Last try, i tested k8s community iso 1.25.0
> >> > in CS UI it saying "Create Kubernetes Cluster k8s-cks-cl16 in
> >> progress"
> >> > and it fails
> >> > If i try to ssh on control node with VR_Pub_IP port 2222
> >> > i have nothing in execution
> >> >
> >> > If i try manual install :
> >> > core@k8s-cks-cl16-control-18eccd0c12c:~$ sudo
> >> /opt/bin/setup-kube-system
> >> > mount: /mnt/k8sdisk: /dev/sr0 already mounted on /mnt/k8sdisk.
> >> > Installing binaries from /mnt/k8sdisk/
> >> > 5b1fa8e3e100: Loading layer
> >> > [==================================================>] 803.8kB/803.8kB
> >> > 39c831b1aa26: Loading layer
> >> > [==================================================>] 26.25MB/26.25MB
> >> > Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
> >> > 417cb9b79ade: Loading layer
> >> > [==================================================>] 657.7kB/657.7kB
> >> > 8d323b160d65: Loading layer
> >> > [==================================================>] 24.95MB/24.95MB
> >> > Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
> >> > 256bc5c338a6: Loading layer
> >> > [==================================================>] 122.2kB/122.2kB
> >> > df1818f16337: Loading layer
> >> > [==================================================>] 14.71MB/14.71MB
> >> > Loaded image: registry.k8s.io/coredns/coredns:v1.9.3
> >> > bd8a70623766: Loading layer
> >> > [==================================================>] 75.78MB/75.78MB
> >> > c88361932af5: Loading layer
> >> > [==================================================>] 508B/508B
> >> > Loaded image: kubernetesui/dashboard:v2.7.0
> >> > 0b031aac6569: Loading layer
> >> > [==================================================>] 804kB/804kB
> >> > cc0597072b83: Loading layer
> >> > [==================================================>] 1.076MB/1.076MB
> >> > 5a8045f731fc: Loading layer
> >> > [==================================================>] 90.58MB/90.58MB
> >> > f5a5ef1a005f: Loading layer
> >> > [==================================================>] 1.188MB/1.188MB
> >> > 5cf3bfe5417d: Loading layer
> >> > [==================================================>] 8.506MB/8.506MB
> >> > Loaded image: registry.k8s.io/etcd:3.5.4-0
> >> > 8d7366c22fd8: Loading layer
> >> > [==================================================>] 804.1kB/804.1kB
> >> > 7d5f5bba7c45: Loading layer
> >> > [==================================================>] 694.9kB/694.9kB
> >> > 8154c9483a6c: Loading layer
> >> > [==================================================>] 32.72MB/32.72MB
> >> > Loaded image: registry.k8s.io/kube-apiserver:v1.25.0
> >> > c5f1a47b47bc: Loading layer
> >> > [==================================================>] 29.76MB/29.76MB
> >> > Loaded image: registry.k8s.io/kube-controller-manager:v1.25.0
> >> > e4d19dd12df2: Loading layer
> >> > [==================================================>] 7.631MB/7.631MB
> >> > ba3637ccd78a: Loading layer
> >> > [==================================================>] 12.63MB/12.63MB
> >> > Loaded image: registry.k8s.io/kube-proxy:v1.25.0
> >> > 026969d859f7: Loading layer
> >> > [==================================================>] 14.29MB/14.29MB
> >> > Loaded image: registry.k8s.io/kube-scheduler:v1.25.0
> >> > d01384fea991: Loading layer
> >> > [==================================================>] 19.74MB/19.74MB
> >> > bcec7eb9e567: Loading layer
> >> > [==================================================>] 530B/530B
> >> > Loaded image: kubernetesui/metrics-scraper:v1.0.8
> >> > 961e93cda9dd: Loading layer
> >> > [==================================================>] 307kB/307kB
> >> > Loaded image: registry.k8s.io/pause:3.8
> >> > 1b3ee35aacca: Loading layer
> >> > [==================================================>] 2.796MB/2.796MB
> >> > 17d89e2a5199: Loading layer
> >> > [==================================================>] 7.425MB/7.425MB
> >> > 148b94859892: Loading layer
> >> > [==================================================>] 8.1MB/8.1MB
> >> > dbea472c1ced: Loading layer
> >> > [==================================================>] 4.376MB/4.376MB
> >> > e81f1a367c17: Loading layer
> >> > [==================================================>] 131B/131B
> >> > 351848f6b295: Loading layer
> >> > [==================================================>] 8.216MB/8.216MB
> >> > Loaded image: weaveworks/weave-kube:latest
> >> > b20f7ff12b24: Loading layer
> >> > [==================================================>] 727.9kB/727.9kB
> >> > 1d49e386162e: Loading layer
> >> > [==================================================>] 9.281MB/9.281MB
> >> > 20145eeac146: Loading layer
> >> > [==================================================>] 297B/297B
> >> > d4cc72748587: Loading layer
> >> > [==================================================>] 600B/600B
> >> > Loaded image: weaveworks/weave-npc:latest
> >> > net.bridge.bridge-nf-call-iptables = 1
> >> > [init] Using Kubernetes version: v1.25.0
> >> > [preflight] Running pre-flight checks
> >> > [WARNING SystemVerification]: missing optional cgroups: blkio
> >> > error execution phase preflight: [preflight] Some fatal errors occurred:
> >> > [ERROR CRI]: container runtime is not running: output: E0411
> >> > 11:38:11.935458 12988 remote_runtime.go:948] "Status from runtime
> >> service
> >> > failed" err="rpc error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > time="2024-04-11T11:38:11Z" level=fatal msg="getting status of runtime:
> >> rpc
> >> > error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > , error: exit status 1
> >> > [preflight] If you know what you are doing, you can make a check
> >> non-fatal
> >> > with `--ignore-preflight-errors=...`
> >> > To see the stack trace of this error execute with --v=5 or higher
> >> > [init] Using Kubernetes version: v1.25.0
> >> > [preflight] Running pre-flight checks
> >> > [WARNING SystemVerification]: missing optional cgroups: blkio
> >> > error execution phase preflight: [preflight] Some fatal errors occurred:
> >> > [ERROR CRI]: container runtime is not running: output: E0411
> >> > 11:38:12.079422 13015 remote_runtime.go:948] "Status from runtime
> >> service
> >> > failed" err="rpc error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime:
> >> rpc
> >> > error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > , error: exit status 1
> >> > [preflight] If you know what you are doing, you can make a check
> >> non-fatal
> >> > with `--ignore-preflight-errors=...`
> >> > To see the stack trace of this error execute with --v=5 or higher
> >> > [init] Using Kubernetes version: v1.25.0
> >> > [preflight] Running pre-flight checks
> >> > [WARNING SystemVerification]: missing optional cgroups: blkio
> >> > error execution phase preflight: [preflight] Some fatal errors occurred:
> >> > [ERROR CRI]: container runtime is not running: output: E0411
> >> > 11:38:12.228669 13043 remote_runtime.go:948] "Status from runtime
> >> service
> >> > failed" err="rpc error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime:
> >> rpc
> >> > error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > , error: exit status 1
> >> > [preflight] If you know what you are doing, you can make a check
> >> non-fatal
> >> > with `--ignore-preflight-errors=...`
> >> > To see the stack trace of this error execute with --v=5 or higher
> >> > Error: kubeadm init failed!
> >> >
> >> >
> >> > Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <ro...@shapeblue.com> a
> >> > écrit :
> >> >
> >> > > Hi Benoit,
> >> > >
> >> > > The CKS feature has been improving over versions, I don't know if what
> >> > > you're trying to achieve is possible with it. Maybe try a different
> >> > version
> >> > > of the data iso:
> >> > > http://download.cloudstack.org/cks/
> >> > >
> >> > > Alternatively, you can also have a look at the CAPC project:
> >> > > https://cluster-api-cloudstack.sigs.k8s.io
> >> > >
> >> > >
> >> > > Regards.
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > ________________________________
> >> > > From: benoit lair <ku...@gmail.com>
> >> > > Sent: Thursday, April 11, 2024 14:16
> >> > > To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <
> >> > > dev@cloudstack.apache.org>
> >> > > Subject: Re: ACS 4.16 - Change SystemVM template for CKS
> >> > >
> >> > > Any advices ?
> >> > >
> >> > > Best regards
> >> > >
> >> > > Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a
> >> > écrit :
> >> > >
> >> > > > I am opened to every alternatives of changing system vm templates
> >> > > > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
> >> > > >
> >> > > > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a
> >> > > écrit :
> >> > > >
> >> > > >> Hello Folks,
> >> > > >>
> >> > > >> I am trying to install K8s cluster with community iso 1.28.4
> >> > > >> I am with a ACS 4.16.0 environment
> >> > > >> It seems K8S is not working out of the box with v > 1.23.3 due to
> >> > > >> containerd.io version
> >> > > >>
> >> > > >> I would like to release a template who will work with K8s > 1.23.3
> >> on
> >> > > acs
> >> > > >> 4.16
> >> > > >> How can i tell CS to take another system vm template, avoiding to
> >> mess
> >> > > >> normal features with VR and VPC VR keeping the systemvm template
> >> > issued
> >> > > >> with CS 4.16 ?
> >> > > >>
> >> > > >> I am with XCP-NG 8.2.1 in production
> >> > > >>
> >> > > >> Thanks for your help or advises
> >> > > >> Best regards,
> >> > > >> Benoit
> >> > > >>
> >> > > >
> >> > >
> >> >
> >>
> >
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by Wei ZHOU <us...@gmail.com>.
Hi,
I would suggest you not to spend time on this... 4.16 is EOL
Please upgrade to cloudstack 4.18/4.19, or use old kubernetes versions ..
-Wei
On Fri, Apr 12, 2024 at 2:25 PM benoit lair <ku...@gmail.com> wrote:
>
> i succeded to install systemvm template 4.19 as serving template for
> control and worker nodes with cks community iso 1.28.4
> when cluster saying "starting", i connect to node controller via p2222, saw
> that docker was not installed, but containerd.io yes
>
> I've done the following :
>
> apt install docker-ce
> cp /etc/containerd/config.toml /etc/containerd/config.toml.bck
> containerd config default | tee /etc/containerd/config.toml
> /opt/bin/setup-kube-system
> /opt/bin/deploy-kube-system
>
> on to control node and same after on worker node
> CS UI show kubernetes yaml
> a "kubectl.exe --kubeconfig kube.conf.conf get nodes"
>
> NAME STATUS ROLES AGE VERSION
> k8s-cks-cl19-control-18ed18311c9 Ready control-plane 3h1m v1.28.4
> k8s-cks-cl19-node-18ed1850433 Ready <none> 170m v1.28.4
>
> However CS says the cluster is in alert state
> And dashboad is not working
>
> Any advice ?
>
> when executing this on my laptop :
> kubectl.exe --kubeconfig cl19_k8s_1.28.4.conf proxy
>
> and opening this :
> http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
>
> I have this result :
> {
> "kind": "Status",
> "apiVersion": "v1",
> "metadata": {},
> "status": "Failure",
> "message": "no endpoints available for service \"kubernetes-dashboard\"",
> "reason": "ServiceUnavailable",
> "code": 503
> }
>
> Somebody got this problem with dashboard ?
>
> Did i miss something installing the cluster manually with acs stuff ?
>
> Le jeu. 11 avr. 2024 à 17:40, benoit lair <ku...@gmail.com> a écrit :
>
> > Hi Wei,
> >
> > Thanks for the sharing, i also tried to install the systemvm 4.19
> > I have control node and worker node under template systemvm 4.19 (
> > http://download.cloudstack.org/systemvm/4.19/systemvmtemplate-4.19.0-xen.vhd.bz2
> > )
> > I tried with community cks iso 1.25.0 and 1.28.9
> > On systemvm 4.19 docker was not present by default, just containerd in
> > 1.6.x version
> > I installed docker-ce and docker-ce-cli :
> >
> > apt install -y apt-transport-https ca-certificates curl
> > software-properties-common
> > curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg
> > --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
> > echo "deb [arch=$(dpkg --print-architecture)
> > signed-by=/usr/share/keyrings/docker-archive-keyring.gpg]
> > https://download.docker.com/linux/debian $(lsb_release -cs) stable" |
> > sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
> > remove line about docker in double in /etc/apt/sources.list
> > apt update
> > apt install -y docker-ce
> > apt install -y docker-cli
> > i mounted the 1.28.4 cks community iso in control vm
> >
> > and /opt/bin/setup-kube-system
> > i have these errors :
> >
> > Is it the good way to manually install kube system
> >
> > root@k8s-cks-cl17-control-18ecdc1e5bd:/home/core#
> > /opt/bin/setup-kube-system
> > Installing binaries from /mnt/k8sdisk/
> > 5b1fa8e3e100: Loading layer
> > [==================================================>] 803.8kB/803.8kB
> > 39c831b1aa26: Loading layer
> > [==================================================>] 26.25MB/26.25MB
> > Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
> > 417cb9b79ade: Loading layer
> > [==================================================>] 657.7kB/657.7kB
> > 8d323b160d65: Loading layer
> > [==================================================>] 24.95MB/24.95MB
> > Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
> > 6a4a177e62f3: Loading layer
> > [==================================================>] 115.2kB/115.2kB
> > 398c9baff0ce: Loading layer
> > [==================================================>] 16.07MB/16.07MB
> > Loaded image: registry.k8s.io/coredns/coredns:v1.10.1
> > bd8a70623766: Loading layer
> > [==================================================>] 75.78MB/75.78MB
> > c88361932af5: Loading layer
> > [==================================================>] 508B/508B
> > Loaded image: kubernetesui/dashboard:v2.7.0
> > e023e0e48e6e: Loading layer
> > [==================================================>] 103.7kB/103.7kB
> > 6fbdf253bbc2: Loading layer
> > [==================================================>] 21.2kB/21.2kB
> > 7bea6b893187: Loading layer
> > [==================================================>] 716.5kB/716.5kB
> > ff5700ec5418: Loading layer
> > [==================================================>] 317B/317B
> > d52f02c6501c: Loading layer
> > [==================================================>] 198B/198B
> > e624a5370eca: Loading layer
> > [==================================================>] 113B/113B
> > 1a73b54f556b: Loading layer
> > [==================================================>] 385B/385B
> > d2d7ec0f6756: Loading layer
> > [==================================================>] 355B/355B
> > 4cb10dd2545b: Loading layer
> > [==================================================>] 130.6kB/130.6kB
> > ba9afb2b3e0c: Loading layer
> > [==================================================>] 1.076MB/1.076MB
> > 22bba3da6b0d: Loading layer
> > [==================================================>] 91.29MB/91.29MB
> > a4563151d59b: Loading layer
> > [==================================================>] 9.544MB/9.544MB
> > Loaded image: registry.k8s.io/etcd:3.5.9-0
> > 54ad2ec71039: Loading layer
> > [==================================================>] 103.7kB/103.7kB
> > 6fbdf253bbc2: Loading layer
> > [==================================================>] 21.2kB/21.2kB
> > 7bea6b893187: Loading layer
> > [==================================================>] 716.5kB/716.5kB
> > ff5700ec5418: Loading layer
> > [==================================================>] 317B/317B
> > d52f02c6501c: Loading layer
> > [==================================================>] 198B/198B
> > e624a5370eca: Loading layer
> > [==================================================>] 113B/113B
> > 1a73b54f556b: Loading layer
> > [==================================================>] 385B/385B
> > d2d7ec0f6756: Loading layer
> > [==================================================>] 355B/355B
> > 4cb10dd2545b: Loading layer
> > [==================================================>] 130.6kB/130.6kB
> > 3f3c83f9a2fa: Loading layer
> > [==================================================>] 724.8kB/724.8kB
> > 6a2c4bedb8de: Loading layer
> > [==================================================>] 32.98MB/32.98MB
> > Loaded image: registry.k8s.io/kube-apiserver:v1.28.4
> > e0b89915180d: Loading layer
> > [==================================================>] 31.72MB/31.72MB
> > Loaded image: registry.k8s.io/kube-controller-manager:v1.28.4
> > d2a4b045f3d0: Loading layer
> > [==================================================>] 7.73MB/7.73MB
> > 3905b20a2df2: Loading layer
> > [==================================================>] 16.85MB/16.85MB
> > Loaded image: registry.k8s.io/kube-proxy:v1.28.4
> > f35f4c1ae44f: Loading layer
> > [==================================================>] 17.13MB/17.13MB
> > Loaded image: registry.k8s.io/kube-scheduler:v1.28.4
> > d01384fea991: Loading layer
> > [==================================================>] 19.74MB/19.74MB
> > bcec7eb9e567: Loading layer
> > [==================================================>] 530B/530B
> > Loaded image: kubernetesui/metrics-scraper:v1.0.8
> > e3e5579ddd43: Loading layer
> > [==================================================>] 317.6kB/317.6kB
> > Loaded image: registry.k8s.io/pause:3.9
> > 1b3ee35aacca: Loading layer
> > [==================================================>] 2.796MB/2.796MB
> > 910ce076f504: Loading layer
> > [==================================================>] 7.425MB/7.425MB
> > a8e8b7b8e08a: Loading layer
> > [==================================================>] 8.101MB/8.101MB
> > 084e56a9c24b: Loading layer
> > [==================================================>] 4.376MB/4.376MB
> > 334d70dc85ec: Loading layer
> > [==================================================>] 159B/159B
> > a13197d8bda5: Loading layer
> > [==================================================>] 8.216MB/8.216MB
> > Loaded image: weaveworks/weave-kube:2.8.1
> > 998efb010df6: Loading layer
> > [==================================================>] 728.2kB/728.2kB
> > ed37391def99: Loading layer
> > [==================================================>] 9.28MB/9.28MB
> > 175a472c5f77: Loading layer
> > [==================================================>] 299B/299B
> > a8764e32e9fe: Loading layer
> > [==================================================>] 613B/613B
> > Loaded image: weaveworks/weave-npc:2.8.1
> > net.bridge.bridge-nf-call-iptables = 1
> > [init] Using Kubernetes version: v1.28.4
> > [preflight] Running pre-flight checks
> > error execution phase preflight: [preflight] Some fatal errors occurred:
> > [ERROR CRI]: container runtime is not running: output:
> > time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection:
> > validate CRI v1 runtime API for endpoint
> > \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> > Unimplemented desc = unknown service runtime.v1.RuntimeService"
> > , error: exit status 1
> > [preflight] If you know what you are doing, you can make a check non-fatal
> > with `--ignore-preflight-errors=...`
> > To see the stack trace of this error execute with --v=5 or higher
> > [init] Using Kubernetes version: v1.28.4
> > [preflight] Running pre-flight checks
> > error execution phase preflight: [preflight] Some fatal errors occurred:
> > [ERROR CRI]: container runtime is not running: output:
> > time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection:
> > validate CRI v1 runtime API for endpoint
> > \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> > Unimplemented desc = unknown service runtime.v1.RuntimeService"
> > , error: exit status 1
> > [preflight] If you know what you are doing, you can make a check non-fatal
> > with `--ignore-preflight-errors=...`
> > To see the stack trace of this error execute with --v=5 or higher
> > [init] Using Kubernetes version: v1.28.4
> > [preflight] Running pre-flight checks
> > error execution phase preflight: [preflight] Some fatal errors occurred:
> > [ERROR CRI]: container runtime is not running: output:
> > time="2024-04-11T15:33:27Z" level=fatal msg="validate service connection:
> > validate CRI v1 runtime API for endpoint
> > \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> > Unimplemented desc = unknown service runtime.v1.RuntimeService"
> > , error: exit status 1
> > [preflight] If you know what you are doing, you can make a check non-fatal
> > with `--ignore-preflight-errors=...`
> > To see the stack trace of this error execute with --v=5 or higher
> > Error: kubeadm init failed!
> >
> >
> >
> >
> >
> > Le jeu. 11 avr. 2024 à 17:19, Wei ZHOU <us...@gmail.com> a écrit :
> >
> >> Hi,
> >>
> >> Please refer to
> >> https://github.com/apache/cloudstack/issues/8681#issuecomment-1999083241
> >>
> >> The containerd in 4.16/4.17 sysyemvm template is too old.
> >>
> >> -Wei
> >>
> >>
> >> On Thursday, April 11, 2024, benoit lair <ku...@gmail.com> wrote:
> >>
> >> > Hi Rohit,
> >> >
> >> > I already tested with different iso from cloudstack from CKS repos,
> >> > It is ok with ACS 4.16 with community iso 1.23.3 but it fails from
> >> 1.24.0
> >> > and following others versions
> >> >
> >> > I tried to do an upgrade from 1.23.3 with 1.24.0, it fails,
> >> > I tried to bootstrap a k8s cluster with 1.24.0, also with 1.25 or 1.27.3
> >> > and 1.28.4
> >> >
> >> > Last try, i tested k8s community iso 1.25.0
> >> > in CS UI it saying "Create Kubernetes Cluster k8s-cks-cl16 in
> >> progress"
> >> > and it fails
> >> > If i try to ssh on control node with VR_Pub_IP port 2222
> >> > i have nothing in execution
> >> >
> >> > If i try manual install :
> >> > core@k8s-cks-cl16-control-18eccd0c12c:~$ sudo
> >> /opt/bin/setup-kube-system
> >> > mount: /mnt/k8sdisk: /dev/sr0 already mounted on /mnt/k8sdisk.
> >> > Installing binaries from /mnt/k8sdisk/
> >> > 5b1fa8e3e100: Loading layer
> >> > [==================================================>] 803.8kB/803.8kB
> >> > 39c831b1aa26: Loading layer
> >> > [==================================================>] 26.25MB/26.25MB
> >> > Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
> >> > 417cb9b79ade: Loading layer
> >> > [==================================================>] 657.7kB/657.7kB
> >> > 8d323b160d65: Loading layer
> >> > [==================================================>] 24.95MB/24.95MB
> >> > Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
> >> > 256bc5c338a6: Loading layer
> >> > [==================================================>] 122.2kB/122.2kB
> >> > df1818f16337: Loading layer
> >> > [==================================================>] 14.71MB/14.71MB
> >> > Loaded image: registry.k8s.io/coredns/coredns:v1.9.3
> >> > bd8a70623766: Loading layer
> >> > [==================================================>] 75.78MB/75.78MB
> >> > c88361932af5: Loading layer
> >> > [==================================================>] 508B/508B
> >> > Loaded image: kubernetesui/dashboard:v2.7.0
> >> > 0b031aac6569: Loading layer
> >> > [==================================================>] 804kB/804kB
> >> > cc0597072b83: Loading layer
> >> > [==================================================>] 1.076MB/1.076MB
> >> > 5a8045f731fc: Loading layer
> >> > [==================================================>] 90.58MB/90.58MB
> >> > f5a5ef1a005f: Loading layer
> >> > [==================================================>] 1.188MB/1.188MB
> >> > 5cf3bfe5417d: Loading layer
> >> > [==================================================>] 8.506MB/8.506MB
> >> > Loaded image: registry.k8s.io/etcd:3.5.4-0
> >> > 8d7366c22fd8: Loading layer
> >> > [==================================================>] 804.1kB/804.1kB
> >> > 7d5f5bba7c45: Loading layer
> >> > [==================================================>] 694.9kB/694.9kB
> >> > 8154c9483a6c: Loading layer
> >> > [==================================================>] 32.72MB/32.72MB
> >> > Loaded image: registry.k8s.io/kube-apiserver:v1.25.0
> >> > c5f1a47b47bc: Loading layer
> >> > [==================================================>] 29.76MB/29.76MB
> >> > Loaded image: registry.k8s.io/kube-controller-manager:v1.25.0
> >> > e4d19dd12df2: Loading layer
> >> > [==================================================>] 7.631MB/7.631MB
> >> > ba3637ccd78a: Loading layer
> >> > [==================================================>] 12.63MB/12.63MB
> >> > Loaded image: registry.k8s.io/kube-proxy:v1.25.0
> >> > 026969d859f7: Loading layer
> >> > [==================================================>] 14.29MB/14.29MB
> >> > Loaded image: registry.k8s.io/kube-scheduler:v1.25.0
> >> > d01384fea991: Loading layer
> >> > [==================================================>] 19.74MB/19.74MB
> >> > bcec7eb9e567: Loading layer
> >> > [==================================================>] 530B/530B
> >> > Loaded image: kubernetesui/metrics-scraper:v1.0.8
> >> > 961e93cda9dd: Loading layer
> >> > [==================================================>] 307kB/307kB
> >> > Loaded image: registry.k8s.io/pause:3.8
> >> > 1b3ee35aacca: Loading layer
> >> > [==================================================>] 2.796MB/2.796MB
> >> > 17d89e2a5199: Loading layer
> >> > [==================================================>] 7.425MB/7.425MB
> >> > 148b94859892: Loading layer
> >> > [==================================================>] 8.1MB/8.1MB
> >> > dbea472c1ced: Loading layer
> >> > [==================================================>] 4.376MB/4.376MB
> >> > e81f1a367c17: Loading layer
> >> > [==================================================>] 131B/131B
> >> > 351848f6b295: Loading layer
> >> > [==================================================>] 8.216MB/8.216MB
> >> > Loaded image: weaveworks/weave-kube:latest
> >> > b20f7ff12b24: Loading layer
> >> > [==================================================>] 727.9kB/727.9kB
> >> > 1d49e386162e: Loading layer
> >> > [==================================================>] 9.281MB/9.281MB
> >> > 20145eeac146: Loading layer
> >> > [==================================================>] 297B/297B
> >> > d4cc72748587: Loading layer
> >> > [==================================================>] 600B/600B
> >> > Loaded image: weaveworks/weave-npc:latest
> >> > net.bridge.bridge-nf-call-iptables = 1
> >> > [init] Using Kubernetes version: v1.25.0
> >> > [preflight] Running pre-flight checks
> >> > [WARNING SystemVerification]: missing optional cgroups: blkio
> >> > error execution phase preflight: [preflight] Some fatal errors occurred:
> >> > [ERROR CRI]: container runtime is not running: output: E0411
> >> > 11:38:11.935458 12988 remote_runtime.go:948] "Status from runtime
> >> service
> >> > failed" err="rpc error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > time="2024-04-11T11:38:11Z" level=fatal msg="getting status of runtime:
> >> rpc
> >> > error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > , error: exit status 1
> >> > [preflight] If you know what you are doing, you can make a check
> >> non-fatal
> >> > with `--ignore-preflight-errors=...`
> >> > To see the stack trace of this error execute with --v=5 or higher
> >> > [init] Using Kubernetes version: v1.25.0
> >> > [preflight] Running pre-flight checks
> >> > [WARNING SystemVerification]: missing optional cgroups: blkio
> >> > error execution phase preflight: [preflight] Some fatal errors occurred:
> >> > [ERROR CRI]: container runtime is not running: output: E0411
> >> > 11:38:12.079422 13015 remote_runtime.go:948] "Status from runtime
> >> service
> >> > failed" err="rpc error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime:
> >> rpc
> >> > error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > , error: exit status 1
> >> > [preflight] If you know what you are doing, you can make a check
> >> non-fatal
> >> > with `--ignore-preflight-errors=...`
> >> > To see the stack trace of this error execute with --v=5 or higher
> >> > [init] Using Kubernetes version: v1.25.0
> >> > [preflight] Running pre-flight checks
> >> > [WARNING SystemVerification]: missing optional cgroups: blkio
> >> > error execution phase preflight: [preflight] Some fatal errors occurred:
> >> > [ERROR CRI]: container runtime is not running: output: E0411
> >> > 11:38:12.228669 13043 remote_runtime.go:948] "Status from runtime
> >> service
> >> > failed" err="rpc error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime:
> >> rpc
> >> > error: code = Unimplemented desc = unknown service
> >> > runtime.v1alpha2.RuntimeService"
> >> > , error: exit status 1
> >> > [preflight] If you know what you are doing, you can make a check
> >> non-fatal
> >> > with `--ignore-preflight-errors=...`
> >> > To see the stack trace of this error execute with --v=5 or higher
> >> > Error: kubeadm init failed!
> >> >
> >> >
> >> > Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <ro...@shapeblue.com> a
> >> > écrit :
> >> >
> >> > > Hi Benoit,
> >> > >
> >> > > The CKS feature has been improving over versions, I don't know if what
> >> > > you're trying to achieve is possible with it. Maybe try a different
> >> > version
> >> > > of the data iso:
> >> > > http://download.cloudstack.org/cks/
> >> > >
> >> > > Alternatively, you can also have a look at the CAPC project:
> >> > > https://cluster-api-cloudstack.sigs.k8s.io
> >> > >
> >> > >
> >> > > Regards.
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > ________________________________
> >> > > From: benoit lair <ku...@gmail.com>
> >> > > Sent: Thursday, April 11, 2024 14:16
> >> > > To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <
> >> > > dev@cloudstack.apache.org>
> >> > > Subject: Re: ACS 4.16 - Change SystemVM template for CKS
> >> > >
> >> > > Any advices ?
> >> > >
> >> > > Best regards
> >> > >
> >> > > Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a
> >> > écrit :
> >> > >
> >> > > > I am opened to every alternatives of changing system vm templates
> >> > > > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
> >> > > >
> >> > > > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a
> >> > > écrit :
> >> > > >
> >> > > >> Hello Folks,
> >> > > >>
> >> > > >> I am trying to install K8s cluster with community iso 1.28.4
> >> > > >> I am with a ACS 4.16.0 environment
> >> > > >> It seems K8S is not working out of the box with v > 1.23.3 due to
> >> > > >> containerd.io version
> >> > > >>
> >> > > >> I would like to release a template who will work with K8s > 1.23.3
> >> on
> >> > > acs
> >> > > >> 4.16
> >> > > >> How can i tell CS to take another system vm template, avoiding to
> >> mess
> >> > > >> normal features with VR and VPC VR keeping the systemvm template
> >> > issued
> >> > > >> with CS 4.16 ?
> >> > > >>
> >> > > >> I am with XCP-NG 8.2.1 in production
> >> > > >>
> >> > > >> Thanks for your help or advises
> >> > > >> Best regards,
> >> > > >> Benoit
> >> > > >>
> >> > > >
> >> > >
> >> >
> >>
> >
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by benoit lair <ku...@gmail.com>.
i succeded to install systemvm template 4.19 as serving template for
control and worker nodes with cks community iso 1.28.4
when cluster saying "starting", i connect to node controller via p2222, saw
that docker was not installed, but containerd.io yes
I've done the following :
apt install docker-ce
cp /etc/containerd/config.toml /etc/containerd/config.toml.bck
containerd config default | tee /etc/containerd/config.toml
/opt/bin/setup-kube-system
/opt/bin/deploy-kube-system
on to control node and same after on worker node
CS UI show kubernetes yaml
a "kubectl.exe --kubeconfig kube.conf.conf get nodes"
NAME STATUS ROLES AGE VERSION
k8s-cks-cl19-control-18ed18311c9 Ready control-plane 3h1m v1.28.4
k8s-cks-cl19-node-18ed1850433 Ready <none> 170m v1.28.4
However CS says the cluster is in alert state
And dashboad is not working
Any advice ?
when executing this on my laptop :
kubectl.exe --kubeconfig cl19_k8s_1.28.4.conf proxy
and opening this :
http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
I have this result :
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "no endpoints available for service \"kubernetes-dashboard\"",
"reason": "ServiceUnavailable",
"code": 503
}
Somebody got this problem with dashboard ?
Did i miss something installing the cluster manually with acs stuff ?
Le jeu. 11 avr. 2024 à 17:40, benoit lair <ku...@gmail.com> a écrit :
> Hi Wei,
>
> Thanks for the sharing, i also tried to install the systemvm 4.19
> I have control node and worker node under template systemvm 4.19 (
> http://download.cloudstack.org/systemvm/4.19/systemvmtemplate-4.19.0-xen.vhd.bz2
> )
> I tried with community cks iso 1.25.0 and 1.28.9
> On systemvm 4.19 docker was not present by default, just containerd in
> 1.6.x version
> I installed docker-ce and docker-ce-cli :
>
> apt install -y apt-transport-https ca-certificates curl
> software-properties-common
> curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg
> --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
> echo "deb [arch=$(dpkg --print-architecture)
> signed-by=/usr/share/keyrings/docker-archive-keyring.gpg]
> https://download.docker.com/linux/debian $(lsb_release -cs) stable" |
> sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
> remove line about docker in double in /etc/apt/sources.list
> apt update
> apt install -y docker-ce
> apt install -y docker-cli
> i mounted the 1.28.4 cks community iso in control vm
>
> and /opt/bin/setup-kube-system
> i have these errors :
>
> Is it the good way to manually install kube system
>
> root@k8s-cks-cl17-control-18ecdc1e5bd:/home/core#
> /opt/bin/setup-kube-system
> Installing binaries from /mnt/k8sdisk/
> 5b1fa8e3e100: Loading layer
> [==================================================>] 803.8kB/803.8kB
> 39c831b1aa26: Loading layer
> [==================================================>] 26.25MB/26.25MB
> Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
> 417cb9b79ade: Loading layer
> [==================================================>] 657.7kB/657.7kB
> 8d323b160d65: Loading layer
> [==================================================>] 24.95MB/24.95MB
> Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
> 6a4a177e62f3: Loading layer
> [==================================================>] 115.2kB/115.2kB
> 398c9baff0ce: Loading layer
> [==================================================>] 16.07MB/16.07MB
> Loaded image: registry.k8s.io/coredns/coredns:v1.10.1
> bd8a70623766: Loading layer
> [==================================================>] 75.78MB/75.78MB
> c88361932af5: Loading layer
> [==================================================>] 508B/508B
> Loaded image: kubernetesui/dashboard:v2.7.0
> e023e0e48e6e: Loading layer
> [==================================================>] 103.7kB/103.7kB
> 6fbdf253bbc2: Loading layer
> [==================================================>] 21.2kB/21.2kB
> 7bea6b893187: Loading layer
> [==================================================>] 716.5kB/716.5kB
> ff5700ec5418: Loading layer
> [==================================================>] 317B/317B
> d52f02c6501c: Loading layer
> [==================================================>] 198B/198B
> e624a5370eca: Loading layer
> [==================================================>] 113B/113B
> 1a73b54f556b: Loading layer
> [==================================================>] 385B/385B
> d2d7ec0f6756: Loading layer
> [==================================================>] 355B/355B
> 4cb10dd2545b: Loading layer
> [==================================================>] 130.6kB/130.6kB
> ba9afb2b3e0c: Loading layer
> [==================================================>] 1.076MB/1.076MB
> 22bba3da6b0d: Loading layer
> [==================================================>] 91.29MB/91.29MB
> a4563151d59b: Loading layer
> [==================================================>] 9.544MB/9.544MB
> Loaded image: registry.k8s.io/etcd:3.5.9-0
> 54ad2ec71039: Loading layer
> [==================================================>] 103.7kB/103.7kB
> 6fbdf253bbc2: Loading layer
> [==================================================>] 21.2kB/21.2kB
> 7bea6b893187: Loading layer
> [==================================================>] 716.5kB/716.5kB
> ff5700ec5418: Loading layer
> [==================================================>] 317B/317B
> d52f02c6501c: Loading layer
> [==================================================>] 198B/198B
> e624a5370eca: Loading layer
> [==================================================>] 113B/113B
> 1a73b54f556b: Loading layer
> [==================================================>] 385B/385B
> d2d7ec0f6756: Loading layer
> [==================================================>] 355B/355B
> 4cb10dd2545b: Loading layer
> [==================================================>] 130.6kB/130.6kB
> 3f3c83f9a2fa: Loading layer
> [==================================================>] 724.8kB/724.8kB
> 6a2c4bedb8de: Loading layer
> [==================================================>] 32.98MB/32.98MB
> Loaded image: registry.k8s.io/kube-apiserver:v1.28.4
> e0b89915180d: Loading layer
> [==================================================>] 31.72MB/31.72MB
> Loaded image: registry.k8s.io/kube-controller-manager:v1.28.4
> d2a4b045f3d0: Loading layer
> [==================================================>] 7.73MB/7.73MB
> 3905b20a2df2: Loading layer
> [==================================================>] 16.85MB/16.85MB
> Loaded image: registry.k8s.io/kube-proxy:v1.28.4
> f35f4c1ae44f: Loading layer
> [==================================================>] 17.13MB/17.13MB
> Loaded image: registry.k8s.io/kube-scheduler:v1.28.4
> d01384fea991: Loading layer
> [==================================================>] 19.74MB/19.74MB
> bcec7eb9e567: Loading layer
> [==================================================>] 530B/530B
> Loaded image: kubernetesui/metrics-scraper:v1.0.8
> e3e5579ddd43: Loading layer
> [==================================================>] 317.6kB/317.6kB
> Loaded image: registry.k8s.io/pause:3.9
> 1b3ee35aacca: Loading layer
> [==================================================>] 2.796MB/2.796MB
> 910ce076f504: Loading layer
> [==================================================>] 7.425MB/7.425MB
> a8e8b7b8e08a: Loading layer
> [==================================================>] 8.101MB/8.101MB
> 084e56a9c24b: Loading layer
> [==================================================>] 4.376MB/4.376MB
> 334d70dc85ec: Loading layer
> [==================================================>] 159B/159B
> a13197d8bda5: Loading layer
> [==================================================>] 8.216MB/8.216MB
> Loaded image: weaveworks/weave-kube:2.8.1
> 998efb010df6: Loading layer
> [==================================================>] 728.2kB/728.2kB
> ed37391def99: Loading layer
> [==================================================>] 9.28MB/9.28MB
> 175a472c5f77: Loading layer
> [==================================================>] 299B/299B
> a8764e32e9fe: Loading layer
> [==================================================>] 613B/613B
> Loaded image: weaveworks/weave-npc:2.8.1
> net.bridge.bridge-nf-call-iptables = 1
> [init] Using Kubernetes version: v1.28.4
> [preflight] Running pre-flight checks
> error execution phase preflight: [preflight] Some fatal errors occurred:
> [ERROR CRI]: container runtime is not running: output:
> time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection:
> validate CRI v1 runtime API for endpoint
> \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> Unimplemented desc = unknown service runtime.v1.RuntimeService"
> , error: exit status 1
> [preflight] If you know what you are doing, you can make a check non-fatal
> with `--ignore-preflight-errors=...`
> To see the stack trace of this error execute with --v=5 or higher
> [init] Using Kubernetes version: v1.28.4
> [preflight] Running pre-flight checks
> error execution phase preflight: [preflight] Some fatal errors occurred:
> [ERROR CRI]: container runtime is not running: output:
> time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection:
> validate CRI v1 runtime API for endpoint
> \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> Unimplemented desc = unknown service runtime.v1.RuntimeService"
> , error: exit status 1
> [preflight] If you know what you are doing, you can make a check non-fatal
> with `--ignore-preflight-errors=...`
> To see the stack trace of this error execute with --v=5 or higher
> [init] Using Kubernetes version: v1.28.4
> [preflight] Running pre-flight checks
> error execution phase preflight: [preflight] Some fatal errors occurred:
> [ERROR CRI]: container runtime is not running: output:
> time="2024-04-11T15:33:27Z" level=fatal msg="validate service connection:
> validate CRI v1 runtime API for endpoint
> \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> Unimplemented desc = unknown service runtime.v1.RuntimeService"
> , error: exit status 1
> [preflight] If you know what you are doing, you can make a check non-fatal
> with `--ignore-preflight-errors=...`
> To see the stack trace of this error execute with --v=5 or higher
> Error: kubeadm init failed!
>
>
>
>
>
> Le jeu. 11 avr. 2024 à 17:19, Wei ZHOU <us...@gmail.com> a écrit :
>
>> Hi,
>>
>> Please refer to
>> https://github.com/apache/cloudstack/issues/8681#issuecomment-1999083241
>>
>> The containerd in 4.16/4.17 sysyemvm template is too old.
>>
>> -Wei
>>
>>
>> On Thursday, April 11, 2024, benoit lair <ku...@gmail.com> wrote:
>>
>> > Hi Rohit,
>> >
>> > I already tested with different iso from cloudstack from CKS repos,
>> > It is ok with ACS 4.16 with community iso 1.23.3 but it fails from
>> 1.24.0
>> > and following others versions
>> >
>> > I tried to do an upgrade from 1.23.3 with 1.24.0, it fails,
>> > I tried to bootstrap a k8s cluster with 1.24.0, also with 1.25 or 1.27.3
>> > and 1.28.4
>> >
>> > Last try, i tested k8s community iso 1.25.0
>> > in CS UI it saying "Create Kubernetes Cluster k8s-cks-cl16 in
>> progress"
>> > and it fails
>> > If i try to ssh on control node with VR_Pub_IP port 2222
>> > i have nothing in execution
>> >
>> > If i try manual install :
>> > core@k8s-cks-cl16-control-18eccd0c12c:~$ sudo
>> /opt/bin/setup-kube-system
>> > mount: /mnt/k8sdisk: /dev/sr0 already mounted on /mnt/k8sdisk.
>> > Installing binaries from /mnt/k8sdisk/
>> > 5b1fa8e3e100: Loading layer
>> > [==================================================>] 803.8kB/803.8kB
>> > 39c831b1aa26: Loading layer
>> > [==================================================>] 26.25MB/26.25MB
>> > Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
>> > 417cb9b79ade: Loading layer
>> > [==================================================>] 657.7kB/657.7kB
>> > 8d323b160d65: Loading layer
>> > [==================================================>] 24.95MB/24.95MB
>> > Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
>> > 256bc5c338a6: Loading layer
>> > [==================================================>] 122.2kB/122.2kB
>> > df1818f16337: Loading layer
>> > [==================================================>] 14.71MB/14.71MB
>> > Loaded image: registry.k8s.io/coredns/coredns:v1.9.3
>> > bd8a70623766: Loading layer
>> > [==================================================>] 75.78MB/75.78MB
>> > c88361932af5: Loading layer
>> > [==================================================>] 508B/508B
>> > Loaded image: kubernetesui/dashboard:v2.7.0
>> > 0b031aac6569: Loading layer
>> > [==================================================>] 804kB/804kB
>> > cc0597072b83: Loading layer
>> > [==================================================>] 1.076MB/1.076MB
>> > 5a8045f731fc: Loading layer
>> > [==================================================>] 90.58MB/90.58MB
>> > f5a5ef1a005f: Loading layer
>> > [==================================================>] 1.188MB/1.188MB
>> > 5cf3bfe5417d: Loading layer
>> > [==================================================>] 8.506MB/8.506MB
>> > Loaded image: registry.k8s.io/etcd:3.5.4-0
>> > 8d7366c22fd8: Loading layer
>> > [==================================================>] 804.1kB/804.1kB
>> > 7d5f5bba7c45: Loading layer
>> > [==================================================>] 694.9kB/694.9kB
>> > 8154c9483a6c: Loading layer
>> > [==================================================>] 32.72MB/32.72MB
>> > Loaded image: registry.k8s.io/kube-apiserver:v1.25.0
>> > c5f1a47b47bc: Loading layer
>> > [==================================================>] 29.76MB/29.76MB
>> > Loaded image: registry.k8s.io/kube-controller-manager:v1.25.0
>> > e4d19dd12df2: Loading layer
>> > [==================================================>] 7.631MB/7.631MB
>> > ba3637ccd78a: Loading layer
>> > [==================================================>] 12.63MB/12.63MB
>> > Loaded image: registry.k8s.io/kube-proxy:v1.25.0
>> > 026969d859f7: Loading layer
>> > [==================================================>] 14.29MB/14.29MB
>> > Loaded image: registry.k8s.io/kube-scheduler:v1.25.0
>> > d01384fea991: Loading layer
>> > [==================================================>] 19.74MB/19.74MB
>> > bcec7eb9e567: Loading layer
>> > [==================================================>] 530B/530B
>> > Loaded image: kubernetesui/metrics-scraper:v1.0.8
>> > 961e93cda9dd: Loading layer
>> > [==================================================>] 307kB/307kB
>> > Loaded image: registry.k8s.io/pause:3.8
>> > 1b3ee35aacca: Loading layer
>> > [==================================================>] 2.796MB/2.796MB
>> > 17d89e2a5199: Loading layer
>> > [==================================================>] 7.425MB/7.425MB
>> > 148b94859892: Loading layer
>> > [==================================================>] 8.1MB/8.1MB
>> > dbea472c1ced: Loading layer
>> > [==================================================>] 4.376MB/4.376MB
>> > e81f1a367c17: Loading layer
>> > [==================================================>] 131B/131B
>> > 351848f6b295: Loading layer
>> > [==================================================>] 8.216MB/8.216MB
>> > Loaded image: weaveworks/weave-kube:latest
>> > b20f7ff12b24: Loading layer
>> > [==================================================>] 727.9kB/727.9kB
>> > 1d49e386162e: Loading layer
>> > [==================================================>] 9.281MB/9.281MB
>> > 20145eeac146: Loading layer
>> > [==================================================>] 297B/297B
>> > d4cc72748587: Loading layer
>> > [==================================================>] 600B/600B
>> > Loaded image: weaveworks/weave-npc:latest
>> > net.bridge.bridge-nf-call-iptables = 1
>> > [init] Using Kubernetes version: v1.25.0
>> > [preflight] Running pre-flight checks
>> > [WARNING SystemVerification]: missing optional cgroups: blkio
>> > error execution phase preflight: [preflight] Some fatal errors occurred:
>> > [ERROR CRI]: container runtime is not running: output: E0411
>> > 11:38:11.935458 12988 remote_runtime.go:948] "Status from runtime
>> service
>> > failed" err="rpc error: code = Unimplemented desc = unknown service
>> > runtime.v1alpha2.RuntimeService"
>> > time="2024-04-11T11:38:11Z" level=fatal msg="getting status of runtime:
>> rpc
>> > error: code = Unimplemented desc = unknown service
>> > runtime.v1alpha2.RuntimeService"
>> > , error: exit status 1
>> > [preflight] If you know what you are doing, you can make a check
>> non-fatal
>> > with `--ignore-preflight-errors=...`
>> > To see the stack trace of this error execute with --v=5 or higher
>> > [init] Using Kubernetes version: v1.25.0
>> > [preflight] Running pre-flight checks
>> > [WARNING SystemVerification]: missing optional cgroups: blkio
>> > error execution phase preflight: [preflight] Some fatal errors occurred:
>> > [ERROR CRI]: container runtime is not running: output: E0411
>> > 11:38:12.079422 13015 remote_runtime.go:948] "Status from runtime
>> service
>> > failed" err="rpc error: code = Unimplemented desc = unknown service
>> > runtime.v1alpha2.RuntimeService"
>> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime:
>> rpc
>> > error: code = Unimplemented desc = unknown service
>> > runtime.v1alpha2.RuntimeService"
>> > , error: exit status 1
>> > [preflight] If you know what you are doing, you can make a check
>> non-fatal
>> > with `--ignore-preflight-errors=...`
>> > To see the stack trace of this error execute with --v=5 or higher
>> > [init] Using Kubernetes version: v1.25.0
>> > [preflight] Running pre-flight checks
>> > [WARNING SystemVerification]: missing optional cgroups: blkio
>> > error execution phase preflight: [preflight] Some fatal errors occurred:
>> > [ERROR CRI]: container runtime is not running: output: E0411
>> > 11:38:12.228669 13043 remote_runtime.go:948] "Status from runtime
>> service
>> > failed" err="rpc error: code = Unimplemented desc = unknown service
>> > runtime.v1alpha2.RuntimeService"
>> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime:
>> rpc
>> > error: code = Unimplemented desc = unknown service
>> > runtime.v1alpha2.RuntimeService"
>> > , error: exit status 1
>> > [preflight] If you know what you are doing, you can make a check
>> non-fatal
>> > with `--ignore-preflight-errors=...`
>> > To see the stack trace of this error execute with --v=5 or higher
>> > Error: kubeadm init failed!
>> >
>> >
>> > Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <ro...@shapeblue.com> a
>> > écrit :
>> >
>> > > Hi Benoit,
>> > >
>> > > The CKS feature has been improving over versions, I don't know if what
>> > > you're trying to achieve is possible with it. Maybe try a different
>> > version
>> > > of the data iso:
>> > > http://download.cloudstack.org/cks/
>> > >
>> > > Alternatively, you can also have a look at the CAPC project:
>> > > https://cluster-api-cloudstack.sigs.k8s.io
>> > >
>> > >
>> > > Regards.
>> > >
>> > >
>> > >
>> > >
>> > > ________________________________
>> > > From: benoit lair <ku...@gmail.com>
>> > > Sent: Thursday, April 11, 2024 14:16
>> > > To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <
>> > > dev@cloudstack.apache.org>
>> > > Subject: Re: ACS 4.16 - Change SystemVM template for CKS
>> > >
>> > > Any advices ?
>> > >
>> > > Best regards
>> > >
>> > > Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a
>> > écrit :
>> > >
>> > > > I am opened to every alternatives of changing system vm templates
>> > > > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
>> > > >
>> > > > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a
>> > > écrit :
>> > > >
>> > > >> Hello Folks,
>> > > >>
>> > > >> I am trying to install K8s cluster with community iso 1.28.4
>> > > >> I am with a ACS 4.16.0 environment
>> > > >> It seems K8S is not working out of the box with v > 1.23.3 due to
>> > > >> containerd.io version
>> > > >>
>> > > >> I would like to release a template who will work with K8s > 1.23.3
>> on
>> > > acs
>> > > >> 4.16
>> > > >> How can i tell CS to take another system vm template, avoiding to
>> mess
>> > > >> normal features with VR and VPC VR keeping the systemvm template
>> > issued
>> > > >> with CS 4.16 ?
>> > > >>
>> > > >> I am with XCP-NG 8.2.1 in production
>> > > >>
>> > > >> Thanks for your help or advises
>> > > >> Best regards,
>> > > >> Benoit
>> > > >>
>> > > >
>> > >
>> >
>>
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by benoit lair <ku...@gmail.com>.
i succeded to install systemvm template 4.19 as serving template for
control and worker nodes with cks community iso 1.28.4
when cluster saying "starting", i connect to node controller via p2222, saw
that docker was not installed, but containerd.io yes
I've done the following :
apt install docker-ce
cp /etc/containerd/config.toml /etc/containerd/config.toml.bck
containerd config default | tee /etc/containerd/config.toml
/opt/bin/setup-kube-system
/opt/bin/deploy-kube-system
on to control node and same after on worker node
CS UI show kubernetes yaml
a "kubectl.exe --kubeconfig kube.conf.conf get nodes"
NAME STATUS ROLES AGE VERSION
k8s-cks-cl19-control-18ed18311c9 Ready control-plane 3h1m v1.28.4
k8s-cks-cl19-node-18ed1850433 Ready <none> 170m v1.28.4
However CS says the cluster is in alert state
And dashboad is not working
Any advice ?
when executing this on my laptop :
kubectl.exe --kubeconfig cl19_k8s_1.28.4.conf proxy
and opening this :
http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
I have this result :
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "no endpoints available for service \"kubernetes-dashboard\"",
"reason": "ServiceUnavailable",
"code": 503
}
Somebody got this problem with dashboard ?
Did i miss something installing the cluster manually with acs stuff ?
Le jeu. 11 avr. 2024 à 17:40, benoit lair <ku...@gmail.com> a écrit :
> Hi Wei,
>
> Thanks for the sharing, i also tried to install the systemvm 4.19
> I have control node and worker node under template systemvm 4.19 (
> http://download.cloudstack.org/systemvm/4.19/systemvmtemplate-4.19.0-xen.vhd.bz2
> )
> I tried with community cks iso 1.25.0 and 1.28.9
> On systemvm 4.19 docker was not present by default, just containerd in
> 1.6.x version
> I installed docker-ce and docker-ce-cli :
>
> apt install -y apt-transport-https ca-certificates curl
> software-properties-common
> curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg
> --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
> echo "deb [arch=$(dpkg --print-architecture)
> signed-by=/usr/share/keyrings/docker-archive-keyring.gpg]
> https://download.docker.com/linux/debian $(lsb_release -cs) stable" |
> sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
> remove line about docker in double in /etc/apt/sources.list
> apt update
> apt install -y docker-ce
> apt install -y docker-cli
> i mounted the 1.28.4 cks community iso in control vm
>
> and /opt/bin/setup-kube-system
> i have these errors :
>
> Is it the good way to manually install kube system
>
> root@k8s-cks-cl17-control-18ecdc1e5bd:/home/core#
> /opt/bin/setup-kube-system
> Installing binaries from /mnt/k8sdisk/
> 5b1fa8e3e100: Loading layer
> [==================================================>] 803.8kB/803.8kB
> 39c831b1aa26: Loading layer
> [==================================================>] 26.25MB/26.25MB
> Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
> 417cb9b79ade: Loading layer
> [==================================================>] 657.7kB/657.7kB
> 8d323b160d65: Loading layer
> [==================================================>] 24.95MB/24.95MB
> Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
> 6a4a177e62f3: Loading layer
> [==================================================>] 115.2kB/115.2kB
> 398c9baff0ce: Loading layer
> [==================================================>] 16.07MB/16.07MB
> Loaded image: registry.k8s.io/coredns/coredns:v1.10.1
> bd8a70623766: Loading layer
> [==================================================>] 75.78MB/75.78MB
> c88361932af5: Loading layer
> [==================================================>] 508B/508B
> Loaded image: kubernetesui/dashboard:v2.7.0
> e023e0e48e6e: Loading layer
> [==================================================>] 103.7kB/103.7kB
> 6fbdf253bbc2: Loading layer
> [==================================================>] 21.2kB/21.2kB
> 7bea6b893187: Loading layer
> [==================================================>] 716.5kB/716.5kB
> ff5700ec5418: Loading layer
> [==================================================>] 317B/317B
> d52f02c6501c: Loading layer
> [==================================================>] 198B/198B
> e624a5370eca: Loading layer
> [==================================================>] 113B/113B
> 1a73b54f556b: Loading layer
> [==================================================>] 385B/385B
> d2d7ec0f6756: Loading layer
> [==================================================>] 355B/355B
> 4cb10dd2545b: Loading layer
> [==================================================>] 130.6kB/130.6kB
> ba9afb2b3e0c: Loading layer
> [==================================================>] 1.076MB/1.076MB
> 22bba3da6b0d: Loading layer
> [==================================================>] 91.29MB/91.29MB
> a4563151d59b: Loading layer
> [==================================================>] 9.544MB/9.544MB
> Loaded image: registry.k8s.io/etcd:3.5.9-0
> 54ad2ec71039: Loading layer
> [==================================================>] 103.7kB/103.7kB
> 6fbdf253bbc2: Loading layer
> [==================================================>] 21.2kB/21.2kB
> 7bea6b893187: Loading layer
> [==================================================>] 716.5kB/716.5kB
> ff5700ec5418: Loading layer
> [==================================================>] 317B/317B
> d52f02c6501c: Loading layer
> [==================================================>] 198B/198B
> e624a5370eca: Loading layer
> [==================================================>] 113B/113B
> 1a73b54f556b: Loading layer
> [==================================================>] 385B/385B
> d2d7ec0f6756: Loading layer
> [==================================================>] 355B/355B
> 4cb10dd2545b: Loading layer
> [==================================================>] 130.6kB/130.6kB
> 3f3c83f9a2fa: Loading layer
> [==================================================>] 724.8kB/724.8kB
> 6a2c4bedb8de: Loading layer
> [==================================================>] 32.98MB/32.98MB
> Loaded image: registry.k8s.io/kube-apiserver:v1.28.4
> e0b89915180d: Loading layer
> [==================================================>] 31.72MB/31.72MB
> Loaded image: registry.k8s.io/kube-controller-manager:v1.28.4
> d2a4b045f3d0: Loading layer
> [==================================================>] 7.73MB/7.73MB
> 3905b20a2df2: Loading layer
> [==================================================>] 16.85MB/16.85MB
> Loaded image: registry.k8s.io/kube-proxy:v1.28.4
> f35f4c1ae44f: Loading layer
> [==================================================>] 17.13MB/17.13MB
> Loaded image: registry.k8s.io/kube-scheduler:v1.28.4
> d01384fea991: Loading layer
> [==================================================>] 19.74MB/19.74MB
> bcec7eb9e567: Loading layer
> [==================================================>] 530B/530B
> Loaded image: kubernetesui/metrics-scraper:v1.0.8
> e3e5579ddd43: Loading layer
> [==================================================>] 317.6kB/317.6kB
> Loaded image: registry.k8s.io/pause:3.9
> 1b3ee35aacca: Loading layer
> [==================================================>] 2.796MB/2.796MB
> 910ce076f504: Loading layer
> [==================================================>] 7.425MB/7.425MB
> a8e8b7b8e08a: Loading layer
> [==================================================>] 8.101MB/8.101MB
> 084e56a9c24b: Loading layer
> [==================================================>] 4.376MB/4.376MB
> 334d70dc85ec: Loading layer
> [==================================================>] 159B/159B
> a13197d8bda5: Loading layer
> [==================================================>] 8.216MB/8.216MB
> Loaded image: weaveworks/weave-kube:2.8.1
> 998efb010df6: Loading layer
> [==================================================>] 728.2kB/728.2kB
> ed37391def99: Loading layer
> [==================================================>] 9.28MB/9.28MB
> 175a472c5f77: Loading layer
> [==================================================>] 299B/299B
> a8764e32e9fe: Loading layer
> [==================================================>] 613B/613B
> Loaded image: weaveworks/weave-npc:2.8.1
> net.bridge.bridge-nf-call-iptables = 1
> [init] Using Kubernetes version: v1.28.4
> [preflight] Running pre-flight checks
> error execution phase preflight: [preflight] Some fatal errors occurred:
> [ERROR CRI]: container runtime is not running: output:
> time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection:
> validate CRI v1 runtime API for endpoint
> \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> Unimplemented desc = unknown service runtime.v1.RuntimeService"
> , error: exit status 1
> [preflight] If you know what you are doing, you can make a check non-fatal
> with `--ignore-preflight-errors=...`
> To see the stack trace of this error execute with --v=5 or higher
> [init] Using Kubernetes version: v1.28.4
> [preflight] Running pre-flight checks
> error execution phase preflight: [preflight] Some fatal errors occurred:
> [ERROR CRI]: container runtime is not running: output:
> time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection:
> validate CRI v1 runtime API for endpoint
> \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> Unimplemented desc = unknown service runtime.v1.RuntimeService"
> , error: exit status 1
> [preflight] If you know what you are doing, you can make a check non-fatal
> with `--ignore-preflight-errors=...`
> To see the stack trace of this error execute with --v=5 or higher
> [init] Using Kubernetes version: v1.28.4
> [preflight] Running pre-flight checks
> error execution phase preflight: [preflight] Some fatal errors occurred:
> [ERROR CRI]: container runtime is not running: output:
> time="2024-04-11T15:33:27Z" level=fatal msg="validate service connection:
> validate CRI v1 runtime API for endpoint
> \"unix:///var/run/containerd/containerd.sock\": rpc error: code =
> Unimplemented desc = unknown service runtime.v1.RuntimeService"
> , error: exit status 1
> [preflight] If you know what you are doing, you can make a check non-fatal
> with `--ignore-preflight-errors=...`
> To see the stack trace of this error execute with --v=5 or higher
> Error: kubeadm init failed!
>
>
>
>
>
> Le jeu. 11 avr. 2024 à 17:19, Wei ZHOU <us...@gmail.com> a écrit :
>
>> Hi,
>>
>> Please refer to
>> https://github.com/apache/cloudstack/issues/8681#issuecomment-1999083241
>>
>> The containerd in 4.16/4.17 sysyemvm template is too old.
>>
>> -Wei
>>
>>
>> On Thursday, April 11, 2024, benoit lair <ku...@gmail.com> wrote:
>>
>> > Hi Rohit,
>> >
>> > I already tested with different iso from cloudstack from CKS repos,
>> > It is ok with ACS 4.16 with community iso 1.23.3 but it fails from
>> 1.24.0
>> > and following others versions
>> >
>> > I tried to do an upgrade from 1.23.3 with 1.24.0, it fails,
>> > I tried to bootstrap a k8s cluster with 1.24.0, also with 1.25 or 1.27.3
>> > and 1.28.4
>> >
>> > Last try, i tested k8s community iso 1.25.0
>> > in CS UI it saying "Create Kubernetes Cluster k8s-cks-cl16 in
>> progress"
>> > and it fails
>> > If i try to ssh on control node with VR_Pub_IP port 2222
>> > i have nothing in execution
>> >
>> > If i try manual install :
>> > core@k8s-cks-cl16-control-18eccd0c12c:~$ sudo
>> /opt/bin/setup-kube-system
>> > mount: /mnt/k8sdisk: /dev/sr0 already mounted on /mnt/k8sdisk.
>> > Installing binaries from /mnt/k8sdisk/
>> > 5b1fa8e3e100: Loading layer
>> > [==================================================>] 803.8kB/803.8kB
>> > 39c831b1aa26: Loading layer
>> > [==================================================>] 26.25MB/26.25MB
>> > Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
>> > 417cb9b79ade: Loading layer
>> > [==================================================>] 657.7kB/657.7kB
>> > 8d323b160d65: Loading layer
>> > [==================================================>] 24.95MB/24.95MB
>> > Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
>> > 256bc5c338a6: Loading layer
>> > [==================================================>] 122.2kB/122.2kB
>> > df1818f16337: Loading layer
>> > [==================================================>] 14.71MB/14.71MB
>> > Loaded image: registry.k8s.io/coredns/coredns:v1.9.3
>> > bd8a70623766: Loading layer
>> > [==================================================>] 75.78MB/75.78MB
>> > c88361932af5: Loading layer
>> > [==================================================>] 508B/508B
>> > Loaded image: kubernetesui/dashboard:v2.7.0
>> > 0b031aac6569: Loading layer
>> > [==================================================>] 804kB/804kB
>> > cc0597072b83: Loading layer
>> > [==================================================>] 1.076MB/1.076MB
>> > 5a8045f731fc: Loading layer
>> > [==================================================>] 90.58MB/90.58MB
>> > f5a5ef1a005f: Loading layer
>> > [==================================================>] 1.188MB/1.188MB
>> > 5cf3bfe5417d: Loading layer
>> > [==================================================>] 8.506MB/8.506MB
>> > Loaded image: registry.k8s.io/etcd:3.5.4-0
>> > 8d7366c22fd8: Loading layer
>> > [==================================================>] 804.1kB/804.1kB
>> > 7d5f5bba7c45: Loading layer
>> > [==================================================>] 694.9kB/694.9kB
>> > 8154c9483a6c: Loading layer
>> > [==================================================>] 32.72MB/32.72MB
>> > Loaded image: registry.k8s.io/kube-apiserver:v1.25.0
>> > c5f1a47b47bc: Loading layer
>> > [==================================================>] 29.76MB/29.76MB
>> > Loaded image: registry.k8s.io/kube-controller-manager:v1.25.0
>> > e4d19dd12df2: Loading layer
>> > [==================================================>] 7.631MB/7.631MB
>> > ba3637ccd78a: Loading layer
>> > [==================================================>] 12.63MB/12.63MB
>> > Loaded image: registry.k8s.io/kube-proxy:v1.25.0
>> > 026969d859f7: Loading layer
>> > [==================================================>] 14.29MB/14.29MB
>> > Loaded image: registry.k8s.io/kube-scheduler:v1.25.0
>> > d01384fea991: Loading layer
>> > [==================================================>] 19.74MB/19.74MB
>> > bcec7eb9e567: Loading layer
>> > [==================================================>] 530B/530B
>> > Loaded image: kubernetesui/metrics-scraper:v1.0.8
>> > 961e93cda9dd: Loading layer
>> > [==================================================>] 307kB/307kB
>> > Loaded image: registry.k8s.io/pause:3.8
>> > 1b3ee35aacca: Loading layer
>> > [==================================================>] 2.796MB/2.796MB
>> > 17d89e2a5199: Loading layer
>> > [==================================================>] 7.425MB/7.425MB
>> > 148b94859892: Loading layer
>> > [==================================================>] 8.1MB/8.1MB
>> > dbea472c1ced: Loading layer
>> > [==================================================>] 4.376MB/4.376MB
>> > e81f1a367c17: Loading layer
>> > [==================================================>] 131B/131B
>> > 351848f6b295: Loading layer
>> > [==================================================>] 8.216MB/8.216MB
>> > Loaded image: weaveworks/weave-kube:latest
>> > b20f7ff12b24: Loading layer
>> > [==================================================>] 727.9kB/727.9kB
>> > 1d49e386162e: Loading layer
>> > [==================================================>] 9.281MB/9.281MB
>> > 20145eeac146: Loading layer
>> > [==================================================>] 297B/297B
>> > d4cc72748587: Loading layer
>> > [==================================================>] 600B/600B
>> > Loaded image: weaveworks/weave-npc:latest
>> > net.bridge.bridge-nf-call-iptables = 1
>> > [init] Using Kubernetes version: v1.25.0
>> > [preflight] Running pre-flight checks
>> > [WARNING SystemVerification]: missing optional cgroups: blkio
>> > error execution phase preflight: [preflight] Some fatal errors occurred:
>> > [ERROR CRI]: container runtime is not running: output: E0411
>> > 11:38:11.935458 12988 remote_runtime.go:948] "Status from runtime
>> service
>> > failed" err="rpc error: code = Unimplemented desc = unknown service
>> > runtime.v1alpha2.RuntimeService"
>> > time="2024-04-11T11:38:11Z" level=fatal msg="getting status of runtime:
>> rpc
>> > error: code = Unimplemented desc = unknown service
>> > runtime.v1alpha2.RuntimeService"
>> > , error: exit status 1
>> > [preflight] If you know what you are doing, you can make a check
>> non-fatal
>> > with `--ignore-preflight-errors=...`
>> > To see the stack trace of this error execute with --v=5 or higher
>> > [init] Using Kubernetes version: v1.25.0
>> > [preflight] Running pre-flight checks
>> > [WARNING SystemVerification]: missing optional cgroups: blkio
>> > error execution phase preflight: [preflight] Some fatal errors occurred:
>> > [ERROR CRI]: container runtime is not running: output: E0411
>> > 11:38:12.079422 13015 remote_runtime.go:948] "Status from runtime
>> service
>> > failed" err="rpc error: code = Unimplemented desc = unknown service
>> > runtime.v1alpha2.RuntimeService"
>> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime:
>> rpc
>> > error: code = Unimplemented desc = unknown service
>> > runtime.v1alpha2.RuntimeService"
>> > , error: exit status 1
>> > [preflight] If you know what you are doing, you can make a check
>> non-fatal
>> > with `--ignore-preflight-errors=...`
>> > To see the stack trace of this error execute with --v=5 or higher
>> > [init] Using Kubernetes version: v1.25.0
>> > [preflight] Running pre-flight checks
>> > [WARNING SystemVerification]: missing optional cgroups: blkio
>> > error execution phase preflight: [preflight] Some fatal errors occurred:
>> > [ERROR CRI]: container runtime is not running: output: E0411
>> > 11:38:12.228669 13043 remote_runtime.go:948] "Status from runtime
>> service
>> > failed" err="rpc error: code = Unimplemented desc = unknown service
>> > runtime.v1alpha2.RuntimeService"
>> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime:
>> rpc
>> > error: code = Unimplemented desc = unknown service
>> > runtime.v1alpha2.RuntimeService"
>> > , error: exit status 1
>> > [preflight] If you know what you are doing, you can make a check
>> non-fatal
>> > with `--ignore-preflight-errors=...`
>> > To see the stack trace of this error execute with --v=5 or higher
>> > Error: kubeadm init failed!
>> >
>> >
>> > Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <ro...@shapeblue.com> a
>> > écrit :
>> >
>> > > Hi Benoit,
>> > >
>> > > The CKS feature has been improving over versions, I don't know if what
>> > > you're trying to achieve is possible with it. Maybe try a different
>> > version
>> > > of the data iso:
>> > > http://download.cloudstack.org/cks/
>> > >
>> > > Alternatively, you can also have a look at the CAPC project:
>> > > https://cluster-api-cloudstack.sigs.k8s.io
>> > >
>> > >
>> > > Regards.
>> > >
>> > >
>> > >
>> > >
>> > > ________________________________
>> > > From: benoit lair <ku...@gmail.com>
>> > > Sent: Thursday, April 11, 2024 14:16
>> > > To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <
>> > > dev@cloudstack.apache.org>
>> > > Subject: Re: ACS 4.16 - Change SystemVM template for CKS
>> > >
>> > > Any advices ?
>> > >
>> > > Best regards
>> > >
>> > > Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a
>> > écrit :
>> > >
>> > > > I am opened to every alternatives of changing system vm templates
>> > > > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
>> > > >
>> > > > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a
>> > > écrit :
>> > > >
>> > > >> Hello Folks,
>> > > >>
>> > > >> I am trying to install K8s cluster with community iso 1.28.4
>> > > >> I am with a ACS 4.16.0 environment
>> > > >> It seems K8S is not working out of the box with v > 1.23.3 due to
>> > > >> containerd.io version
>> > > >>
>> > > >> I would like to release a template who will work with K8s > 1.23.3
>> on
>> > > acs
>> > > >> 4.16
>> > > >> How can i tell CS to take another system vm template, avoiding to
>> mess
>> > > >> normal features with VR and VPC VR keeping the systemvm template
>> > issued
>> > > >> with CS 4.16 ?
>> > > >>
>> > > >> I am with XCP-NG 8.2.1 in production
>> > > >>
>> > > >> Thanks for your help or advises
>> > > >> Best regards,
>> > > >> Benoit
>> > > >>
>> > > >
>> > >
>> >
>>
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by benoit lair <ku...@gmail.com>.
Hi Wei,
Thanks for the sharing, i also tried to install the systemvm 4.19
I have control node and worker node under template systemvm 4.19 (
http://download.cloudstack.org/systemvm/4.19/systemvmtemplate-4.19.0-xen.vhd.bz2
)
I tried with community cks iso 1.25.0 and 1.28.9
On systemvm 4.19 docker was not present by default, just containerd in
1.6.x version
I installed docker-ce and docker-ce-cli :
apt install -y apt-transport-https ca-certificates curl
software-properties-common
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg
--dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture)
signed-by=/usr/share/keyrings/docker-archive-keyring.gpg]
https://download.docker.com/linux/debian $(lsb_release -cs) stable" | sudo
tee /etc/apt/sources.list.d/docker.list > /dev/null
remove line about docker in double in /etc/apt/sources.list
apt update
apt install -y docker-ce
apt install -y docker-cli
i mounted the 1.28.4 cks community iso in control vm
and /opt/bin/setup-kube-system
i have these errors :
Is it the good way to manually install kube system
root@k8s-cks-cl17-control-18ecdc1e5bd:/home/core# /opt/bin/setup-kube-system
Installing binaries from /mnt/k8sdisk/
5b1fa8e3e100: Loading layer
[==================================================>] 803.8kB/803.8kB
39c831b1aa26: Loading layer
[==================================================>] 26.25MB/26.25MB
Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
417cb9b79ade: Loading layer
[==================================================>] 657.7kB/657.7kB
8d323b160d65: Loading layer
[==================================================>] 24.95MB/24.95MB
Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
6a4a177e62f3: Loading layer
[==================================================>] 115.2kB/115.2kB
398c9baff0ce: Loading layer
[==================================================>] 16.07MB/16.07MB
Loaded image: registry.k8s.io/coredns/coredns:v1.10.1
bd8a70623766: Loading layer
[==================================================>] 75.78MB/75.78MB
c88361932af5: Loading layer
[==================================================>] 508B/508B
Loaded image: kubernetesui/dashboard:v2.7.0
e023e0e48e6e: Loading layer
[==================================================>] 103.7kB/103.7kB
6fbdf253bbc2: Loading layer
[==================================================>] 21.2kB/21.2kB
7bea6b893187: Loading layer
[==================================================>] 716.5kB/716.5kB
ff5700ec5418: Loading layer
[==================================================>] 317B/317B
d52f02c6501c: Loading layer
[==================================================>] 198B/198B
e624a5370eca: Loading layer
[==================================================>] 113B/113B
1a73b54f556b: Loading layer
[==================================================>] 385B/385B
d2d7ec0f6756: Loading layer
[==================================================>] 355B/355B
4cb10dd2545b: Loading layer
[==================================================>] 130.6kB/130.6kB
ba9afb2b3e0c: Loading layer
[==================================================>] 1.076MB/1.076MB
22bba3da6b0d: Loading layer
[==================================================>] 91.29MB/91.29MB
a4563151d59b: Loading layer
[==================================================>] 9.544MB/9.544MB
Loaded image: registry.k8s.io/etcd:3.5.9-0
54ad2ec71039: Loading layer
[==================================================>] 103.7kB/103.7kB
6fbdf253bbc2: Loading layer
[==================================================>] 21.2kB/21.2kB
7bea6b893187: Loading layer
[==================================================>] 716.5kB/716.5kB
ff5700ec5418: Loading layer
[==================================================>] 317B/317B
d52f02c6501c: Loading layer
[==================================================>] 198B/198B
e624a5370eca: Loading layer
[==================================================>] 113B/113B
1a73b54f556b: Loading layer
[==================================================>] 385B/385B
d2d7ec0f6756: Loading layer
[==================================================>] 355B/355B
4cb10dd2545b: Loading layer
[==================================================>] 130.6kB/130.6kB
3f3c83f9a2fa: Loading layer
[==================================================>] 724.8kB/724.8kB
6a2c4bedb8de: Loading layer
[==================================================>] 32.98MB/32.98MB
Loaded image: registry.k8s.io/kube-apiserver:v1.28.4
e0b89915180d: Loading layer
[==================================================>] 31.72MB/31.72MB
Loaded image: registry.k8s.io/kube-controller-manager:v1.28.4
d2a4b045f3d0: Loading layer
[==================================================>] 7.73MB/7.73MB
3905b20a2df2: Loading layer
[==================================================>] 16.85MB/16.85MB
Loaded image: registry.k8s.io/kube-proxy:v1.28.4
f35f4c1ae44f: Loading layer
[==================================================>] 17.13MB/17.13MB
Loaded image: registry.k8s.io/kube-scheduler:v1.28.4
d01384fea991: Loading layer
[==================================================>] 19.74MB/19.74MB
bcec7eb9e567: Loading layer
[==================================================>] 530B/530B
Loaded image: kubernetesui/metrics-scraper:v1.0.8
e3e5579ddd43: Loading layer
[==================================================>] 317.6kB/317.6kB
Loaded image: registry.k8s.io/pause:3.9
1b3ee35aacca: Loading layer
[==================================================>] 2.796MB/2.796MB
910ce076f504: Loading layer
[==================================================>] 7.425MB/7.425MB
a8e8b7b8e08a: Loading layer
[==================================================>] 8.101MB/8.101MB
084e56a9c24b: Loading layer
[==================================================>] 4.376MB/4.376MB
334d70dc85ec: Loading layer
[==================================================>] 159B/159B
a13197d8bda5: Loading layer
[==================================================>] 8.216MB/8.216MB
Loaded image: weaveworks/weave-kube:2.8.1
998efb010df6: Loading layer
[==================================================>] 728.2kB/728.2kB
ed37391def99: Loading layer
[==================================================>] 9.28MB/9.28MB
175a472c5f77: Loading layer
[==================================================>] 299B/299B
a8764e32e9fe: Loading layer
[==================================================>] 613B/613B
Loaded image: weaveworks/weave-npc:2.8.1
net.bridge.bridge-nf-call-iptables = 1
[init] Using Kubernetes version: v1.28.4
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output:
time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection:
validate CRI v1 runtime API for endpoint
\"unix:///var/run/containerd/containerd.sock\": rpc error: code =
Unimplemented desc = unknown service runtime.v1.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal
with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
[init] Using Kubernetes version: v1.28.4
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output:
time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection:
validate CRI v1 runtime API for endpoint
\"unix:///var/run/containerd/containerd.sock\": rpc error: code =
Unimplemented desc = unknown service runtime.v1.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal
with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
[init] Using Kubernetes version: v1.28.4
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output:
time="2024-04-11T15:33:27Z" level=fatal msg="validate service connection:
validate CRI v1 runtime API for endpoint
\"unix:///var/run/containerd/containerd.sock\": rpc error: code =
Unimplemented desc = unknown service runtime.v1.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal
with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
Error: kubeadm init failed!
Le jeu. 11 avr. 2024 à 17:19, Wei ZHOU <us...@gmail.com> a écrit :
> Hi,
>
> Please refer to
> https://github.com/apache/cloudstack/issues/8681#issuecomment-1999083241
>
> The containerd in 4.16/4.17 sysyemvm template is too old.
>
> -Wei
>
>
> On Thursday, April 11, 2024, benoit lair <ku...@gmail.com> wrote:
>
> > Hi Rohit,
> >
> > I already tested with different iso from cloudstack from CKS repos,
> > It is ok with ACS 4.16 with community iso 1.23.3 but it fails from 1.24.0
> > and following others versions
> >
> > I tried to do an upgrade from 1.23.3 with 1.24.0, it fails,
> > I tried to bootstrap a k8s cluster with 1.24.0, also with 1.25 or 1.27.3
> > and 1.28.4
> >
> > Last try, i tested k8s community iso 1.25.0
> > in CS UI it saying "Create Kubernetes Cluster k8s-cks-cl16 in progress"
> > and it fails
> > If i try to ssh on control node with VR_Pub_IP port 2222
> > i have nothing in execution
> >
> > If i try manual install :
> > core@k8s-cks-cl16-control-18eccd0c12c:~$ sudo /opt/bin/setup-kube-system
> > mount: /mnt/k8sdisk: /dev/sr0 already mounted on /mnt/k8sdisk.
> > Installing binaries from /mnt/k8sdisk/
> > 5b1fa8e3e100: Loading layer
> > [==================================================>] 803.8kB/803.8kB
> > 39c831b1aa26: Loading layer
> > [==================================================>] 26.25MB/26.25MB
> > Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
> > 417cb9b79ade: Loading layer
> > [==================================================>] 657.7kB/657.7kB
> > 8d323b160d65: Loading layer
> > [==================================================>] 24.95MB/24.95MB
> > Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
> > 256bc5c338a6: Loading layer
> > [==================================================>] 122.2kB/122.2kB
> > df1818f16337: Loading layer
> > [==================================================>] 14.71MB/14.71MB
> > Loaded image: registry.k8s.io/coredns/coredns:v1.9.3
> > bd8a70623766: Loading layer
> > [==================================================>] 75.78MB/75.78MB
> > c88361932af5: Loading layer
> > [==================================================>] 508B/508B
> > Loaded image: kubernetesui/dashboard:v2.7.0
> > 0b031aac6569: Loading layer
> > [==================================================>] 804kB/804kB
> > cc0597072b83: Loading layer
> > [==================================================>] 1.076MB/1.076MB
> > 5a8045f731fc: Loading layer
> > [==================================================>] 90.58MB/90.58MB
> > f5a5ef1a005f: Loading layer
> > [==================================================>] 1.188MB/1.188MB
> > 5cf3bfe5417d: Loading layer
> > [==================================================>] 8.506MB/8.506MB
> > Loaded image: registry.k8s.io/etcd:3.5.4-0
> > 8d7366c22fd8: Loading layer
> > [==================================================>] 804.1kB/804.1kB
> > 7d5f5bba7c45: Loading layer
> > [==================================================>] 694.9kB/694.9kB
> > 8154c9483a6c: Loading layer
> > [==================================================>] 32.72MB/32.72MB
> > Loaded image: registry.k8s.io/kube-apiserver:v1.25.0
> > c5f1a47b47bc: Loading layer
> > [==================================================>] 29.76MB/29.76MB
> > Loaded image: registry.k8s.io/kube-controller-manager:v1.25.0
> > e4d19dd12df2: Loading layer
> > [==================================================>] 7.631MB/7.631MB
> > ba3637ccd78a: Loading layer
> > [==================================================>] 12.63MB/12.63MB
> > Loaded image: registry.k8s.io/kube-proxy:v1.25.0
> > 026969d859f7: Loading layer
> > [==================================================>] 14.29MB/14.29MB
> > Loaded image: registry.k8s.io/kube-scheduler:v1.25.0
> > d01384fea991: Loading layer
> > [==================================================>] 19.74MB/19.74MB
> > bcec7eb9e567: Loading layer
> > [==================================================>] 530B/530B
> > Loaded image: kubernetesui/metrics-scraper:v1.0.8
> > 961e93cda9dd: Loading layer
> > [==================================================>] 307kB/307kB
> > Loaded image: registry.k8s.io/pause:3.8
> > 1b3ee35aacca: Loading layer
> > [==================================================>] 2.796MB/2.796MB
> > 17d89e2a5199: Loading layer
> > [==================================================>] 7.425MB/7.425MB
> > 148b94859892: Loading layer
> > [==================================================>] 8.1MB/8.1MB
> > dbea472c1ced: Loading layer
> > [==================================================>] 4.376MB/4.376MB
> > e81f1a367c17: Loading layer
> > [==================================================>] 131B/131B
> > 351848f6b295: Loading layer
> > [==================================================>] 8.216MB/8.216MB
> > Loaded image: weaveworks/weave-kube:latest
> > b20f7ff12b24: Loading layer
> > [==================================================>] 727.9kB/727.9kB
> > 1d49e386162e: Loading layer
> > [==================================================>] 9.281MB/9.281MB
> > 20145eeac146: Loading layer
> > [==================================================>] 297B/297B
> > d4cc72748587: Loading layer
> > [==================================================>] 600B/600B
> > Loaded image: weaveworks/weave-npc:latest
> > net.bridge.bridge-nf-call-iptables = 1
> > [init] Using Kubernetes version: v1.25.0
> > [preflight] Running pre-flight checks
> > [WARNING SystemVerification]: missing optional cgroups: blkio
> > error execution phase preflight: [preflight] Some fatal errors occurred:
> > [ERROR CRI]: container runtime is not running: output: E0411
> > 11:38:11.935458 12988 remote_runtime.go:948] "Status from runtime
> service
> > failed" err="rpc error: code = Unimplemented desc = unknown service
> > runtime.v1alpha2.RuntimeService"
> > time="2024-04-11T11:38:11Z" level=fatal msg="getting status of runtime:
> rpc
> > error: code = Unimplemented desc = unknown service
> > runtime.v1alpha2.RuntimeService"
> > , error: exit status 1
> > [preflight] If you know what you are doing, you can make a check
> non-fatal
> > with `--ignore-preflight-errors=...`
> > To see the stack trace of this error execute with --v=5 or higher
> > [init] Using Kubernetes version: v1.25.0
> > [preflight] Running pre-flight checks
> > [WARNING SystemVerification]: missing optional cgroups: blkio
> > error execution phase preflight: [preflight] Some fatal errors occurred:
> > [ERROR CRI]: container runtime is not running: output: E0411
> > 11:38:12.079422 13015 remote_runtime.go:948] "Status from runtime
> service
> > failed" err="rpc error: code = Unimplemented desc = unknown service
> > runtime.v1alpha2.RuntimeService"
> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime:
> rpc
> > error: code = Unimplemented desc = unknown service
> > runtime.v1alpha2.RuntimeService"
> > , error: exit status 1
> > [preflight] If you know what you are doing, you can make a check
> non-fatal
> > with `--ignore-preflight-errors=...`
> > To see the stack trace of this error execute with --v=5 or higher
> > [init] Using Kubernetes version: v1.25.0
> > [preflight] Running pre-flight checks
> > [WARNING SystemVerification]: missing optional cgroups: blkio
> > error execution phase preflight: [preflight] Some fatal errors occurred:
> > [ERROR CRI]: container runtime is not running: output: E0411
> > 11:38:12.228669 13043 remote_runtime.go:948] "Status from runtime
> service
> > failed" err="rpc error: code = Unimplemented desc = unknown service
> > runtime.v1alpha2.RuntimeService"
> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime:
> rpc
> > error: code = Unimplemented desc = unknown service
> > runtime.v1alpha2.RuntimeService"
> > , error: exit status 1
> > [preflight] If you know what you are doing, you can make a check
> non-fatal
> > with `--ignore-preflight-errors=...`
> > To see the stack trace of this error execute with --v=5 or higher
> > Error: kubeadm init failed!
> >
> >
> > Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <ro...@shapeblue.com> a
> > écrit :
> >
> > > Hi Benoit,
> > >
> > > The CKS feature has been improving over versions, I don't know if what
> > > you're trying to achieve is possible with it. Maybe try a different
> > version
> > > of the data iso:
> > > http://download.cloudstack.org/cks/
> > >
> > > Alternatively, you can also have a look at the CAPC project:
> > > https://cluster-api-cloudstack.sigs.k8s.io
> > >
> > >
> > > Regards.
> > >
> > >
> > >
> > >
> > > ________________________________
> > > From: benoit lair <ku...@gmail.com>
> > > Sent: Thursday, April 11, 2024 14:16
> > > To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <
> > > dev@cloudstack.apache.org>
> > > Subject: Re: ACS 4.16 - Change SystemVM template for CKS
> > >
> > > Any advices ?
> > >
> > > Best regards
> > >
> > > Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a
> > écrit :
> > >
> > > > I am opened to every alternatives of changing system vm templates
> > > > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
> > > >
> > > > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a
> > > écrit :
> > > >
> > > >> Hello Folks,
> > > >>
> > > >> I am trying to install K8s cluster with community iso 1.28.4
> > > >> I am with a ACS 4.16.0 environment
> > > >> It seems K8S is not working out of the box with v > 1.23.3 due to
> > > >> containerd.io version
> > > >>
> > > >> I would like to release a template who will work with K8s > 1.23.3
> on
> > > acs
> > > >> 4.16
> > > >> How can i tell CS to take another system vm template, avoiding to
> mess
> > > >> normal features with VR and VPC VR keeping the systemvm template
> > issued
> > > >> with CS 4.16 ?
> > > >>
> > > >> I am with XCP-NG 8.2.1 in production
> > > >>
> > > >> Thanks for your help or advises
> > > >> Best regards,
> > > >> Benoit
> > > >>
> > > >
> > >
> >
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by benoit lair <ku...@gmail.com>.
Hi Wei,
Thanks for the sharing, i also tried to install the systemvm 4.19
I have control node and worker node under template systemvm 4.19 (
http://download.cloudstack.org/systemvm/4.19/systemvmtemplate-4.19.0-xen.vhd.bz2
)
I tried with community cks iso 1.25.0 and 1.28.9
On systemvm 4.19 docker was not present by default, just containerd in
1.6.x version
I installed docker-ce and docker-ce-cli :
apt install -y apt-transport-https ca-certificates curl
software-properties-common
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg
--dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture)
signed-by=/usr/share/keyrings/docker-archive-keyring.gpg]
https://download.docker.com/linux/debian $(lsb_release -cs) stable" | sudo
tee /etc/apt/sources.list.d/docker.list > /dev/null
remove line about docker in double in /etc/apt/sources.list
apt update
apt install -y docker-ce
apt install -y docker-cli
i mounted the 1.28.4 cks community iso in control vm
and /opt/bin/setup-kube-system
i have these errors :
Is it the good way to manually install kube system
root@k8s-cks-cl17-control-18ecdc1e5bd:/home/core# /opt/bin/setup-kube-system
Installing binaries from /mnt/k8sdisk/
5b1fa8e3e100: Loading layer
[==================================================>] 803.8kB/803.8kB
39c831b1aa26: Loading layer
[==================================================>] 26.25MB/26.25MB
Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
417cb9b79ade: Loading layer
[==================================================>] 657.7kB/657.7kB
8d323b160d65: Loading layer
[==================================================>] 24.95MB/24.95MB
Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
6a4a177e62f3: Loading layer
[==================================================>] 115.2kB/115.2kB
398c9baff0ce: Loading layer
[==================================================>] 16.07MB/16.07MB
Loaded image: registry.k8s.io/coredns/coredns:v1.10.1
bd8a70623766: Loading layer
[==================================================>] 75.78MB/75.78MB
c88361932af5: Loading layer
[==================================================>] 508B/508B
Loaded image: kubernetesui/dashboard:v2.7.0
e023e0e48e6e: Loading layer
[==================================================>] 103.7kB/103.7kB
6fbdf253bbc2: Loading layer
[==================================================>] 21.2kB/21.2kB
7bea6b893187: Loading layer
[==================================================>] 716.5kB/716.5kB
ff5700ec5418: Loading layer
[==================================================>] 317B/317B
d52f02c6501c: Loading layer
[==================================================>] 198B/198B
e624a5370eca: Loading layer
[==================================================>] 113B/113B
1a73b54f556b: Loading layer
[==================================================>] 385B/385B
d2d7ec0f6756: Loading layer
[==================================================>] 355B/355B
4cb10dd2545b: Loading layer
[==================================================>] 130.6kB/130.6kB
ba9afb2b3e0c: Loading layer
[==================================================>] 1.076MB/1.076MB
22bba3da6b0d: Loading layer
[==================================================>] 91.29MB/91.29MB
a4563151d59b: Loading layer
[==================================================>] 9.544MB/9.544MB
Loaded image: registry.k8s.io/etcd:3.5.9-0
54ad2ec71039: Loading layer
[==================================================>] 103.7kB/103.7kB
6fbdf253bbc2: Loading layer
[==================================================>] 21.2kB/21.2kB
7bea6b893187: Loading layer
[==================================================>] 716.5kB/716.5kB
ff5700ec5418: Loading layer
[==================================================>] 317B/317B
d52f02c6501c: Loading layer
[==================================================>] 198B/198B
e624a5370eca: Loading layer
[==================================================>] 113B/113B
1a73b54f556b: Loading layer
[==================================================>] 385B/385B
d2d7ec0f6756: Loading layer
[==================================================>] 355B/355B
4cb10dd2545b: Loading layer
[==================================================>] 130.6kB/130.6kB
3f3c83f9a2fa: Loading layer
[==================================================>] 724.8kB/724.8kB
6a2c4bedb8de: Loading layer
[==================================================>] 32.98MB/32.98MB
Loaded image: registry.k8s.io/kube-apiserver:v1.28.4
e0b89915180d: Loading layer
[==================================================>] 31.72MB/31.72MB
Loaded image: registry.k8s.io/kube-controller-manager:v1.28.4
d2a4b045f3d0: Loading layer
[==================================================>] 7.73MB/7.73MB
3905b20a2df2: Loading layer
[==================================================>] 16.85MB/16.85MB
Loaded image: registry.k8s.io/kube-proxy:v1.28.4
f35f4c1ae44f: Loading layer
[==================================================>] 17.13MB/17.13MB
Loaded image: registry.k8s.io/kube-scheduler:v1.28.4
d01384fea991: Loading layer
[==================================================>] 19.74MB/19.74MB
bcec7eb9e567: Loading layer
[==================================================>] 530B/530B
Loaded image: kubernetesui/metrics-scraper:v1.0.8
e3e5579ddd43: Loading layer
[==================================================>] 317.6kB/317.6kB
Loaded image: registry.k8s.io/pause:3.9
1b3ee35aacca: Loading layer
[==================================================>] 2.796MB/2.796MB
910ce076f504: Loading layer
[==================================================>] 7.425MB/7.425MB
a8e8b7b8e08a: Loading layer
[==================================================>] 8.101MB/8.101MB
084e56a9c24b: Loading layer
[==================================================>] 4.376MB/4.376MB
334d70dc85ec: Loading layer
[==================================================>] 159B/159B
a13197d8bda5: Loading layer
[==================================================>] 8.216MB/8.216MB
Loaded image: weaveworks/weave-kube:2.8.1
998efb010df6: Loading layer
[==================================================>] 728.2kB/728.2kB
ed37391def99: Loading layer
[==================================================>] 9.28MB/9.28MB
175a472c5f77: Loading layer
[==================================================>] 299B/299B
a8764e32e9fe: Loading layer
[==================================================>] 613B/613B
Loaded image: weaveworks/weave-npc:2.8.1
net.bridge.bridge-nf-call-iptables = 1
[init] Using Kubernetes version: v1.28.4
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output:
time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection:
validate CRI v1 runtime API for endpoint
\"unix:///var/run/containerd/containerd.sock\": rpc error: code =
Unimplemented desc = unknown service runtime.v1.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal
with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
[init] Using Kubernetes version: v1.28.4
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output:
time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection:
validate CRI v1 runtime API for endpoint
\"unix:///var/run/containerd/containerd.sock\": rpc error: code =
Unimplemented desc = unknown service runtime.v1.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal
with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
[init] Using Kubernetes version: v1.28.4
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output:
time="2024-04-11T15:33:27Z" level=fatal msg="validate service connection:
validate CRI v1 runtime API for endpoint
\"unix:///var/run/containerd/containerd.sock\": rpc error: code =
Unimplemented desc = unknown service runtime.v1.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal
with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
Error: kubeadm init failed!
Le jeu. 11 avr. 2024 à 17:19, Wei ZHOU <us...@gmail.com> a écrit :
> Hi,
>
> Please refer to
> https://github.com/apache/cloudstack/issues/8681#issuecomment-1999083241
>
> The containerd in 4.16/4.17 sysyemvm template is too old.
>
> -Wei
>
>
> On Thursday, April 11, 2024, benoit lair <ku...@gmail.com> wrote:
>
> > Hi Rohit,
> >
> > I already tested with different iso from cloudstack from CKS repos,
> > It is ok with ACS 4.16 with community iso 1.23.3 but it fails from 1.24.0
> > and following others versions
> >
> > I tried to do an upgrade from 1.23.3 with 1.24.0, it fails,
> > I tried to bootstrap a k8s cluster with 1.24.0, also with 1.25 or 1.27.3
> > and 1.28.4
> >
> > Last try, i tested k8s community iso 1.25.0
> > in CS UI it saying "Create Kubernetes Cluster k8s-cks-cl16 in progress"
> > and it fails
> > If i try to ssh on control node with VR_Pub_IP port 2222
> > i have nothing in execution
> >
> > If i try manual install :
> > core@k8s-cks-cl16-control-18eccd0c12c:~$ sudo /opt/bin/setup-kube-system
> > mount: /mnt/k8sdisk: /dev/sr0 already mounted on /mnt/k8sdisk.
> > Installing binaries from /mnt/k8sdisk/
> > 5b1fa8e3e100: Loading layer
> > [==================================================>] 803.8kB/803.8kB
> > 39c831b1aa26: Loading layer
> > [==================================================>] 26.25MB/26.25MB
> > Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
> > 417cb9b79ade: Loading layer
> > [==================================================>] 657.7kB/657.7kB
> > 8d323b160d65: Loading layer
> > [==================================================>] 24.95MB/24.95MB
> > Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
> > 256bc5c338a6: Loading layer
> > [==================================================>] 122.2kB/122.2kB
> > df1818f16337: Loading layer
> > [==================================================>] 14.71MB/14.71MB
> > Loaded image: registry.k8s.io/coredns/coredns:v1.9.3
> > bd8a70623766: Loading layer
> > [==================================================>] 75.78MB/75.78MB
> > c88361932af5: Loading layer
> > [==================================================>] 508B/508B
> > Loaded image: kubernetesui/dashboard:v2.7.0
> > 0b031aac6569: Loading layer
> > [==================================================>] 804kB/804kB
> > cc0597072b83: Loading layer
> > [==================================================>] 1.076MB/1.076MB
> > 5a8045f731fc: Loading layer
> > [==================================================>] 90.58MB/90.58MB
> > f5a5ef1a005f: Loading layer
> > [==================================================>] 1.188MB/1.188MB
> > 5cf3bfe5417d: Loading layer
> > [==================================================>] 8.506MB/8.506MB
> > Loaded image: registry.k8s.io/etcd:3.5.4-0
> > 8d7366c22fd8: Loading layer
> > [==================================================>] 804.1kB/804.1kB
> > 7d5f5bba7c45: Loading layer
> > [==================================================>] 694.9kB/694.9kB
> > 8154c9483a6c: Loading layer
> > [==================================================>] 32.72MB/32.72MB
> > Loaded image: registry.k8s.io/kube-apiserver:v1.25.0
> > c5f1a47b47bc: Loading layer
> > [==================================================>] 29.76MB/29.76MB
> > Loaded image: registry.k8s.io/kube-controller-manager:v1.25.0
> > e4d19dd12df2: Loading layer
> > [==================================================>] 7.631MB/7.631MB
> > ba3637ccd78a: Loading layer
> > [==================================================>] 12.63MB/12.63MB
> > Loaded image: registry.k8s.io/kube-proxy:v1.25.0
> > 026969d859f7: Loading layer
> > [==================================================>] 14.29MB/14.29MB
> > Loaded image: registry.k8s.io/kube-scheduler:v1.25.0
> > d01384fea991: Loading layer
> > [==================================================>] 19.74MB/19.74MB
> > bcec7eb9e567: Loading layer
> > [==================================================>] 530B/530B
> > Loaded image: kubernetesui/metrics-scraper:v1.0.8
> > 961e93cda9dd: Loading layer
> > [==================================================>] 307kB/307kB
> > Loaded image: registry.k8s.io/pause:3.8
> > 1b3ee35aacca: Loading layer
> > [==================================================>] 2.796MB/2.796MB
> > 17d89e2a5199: Loading layer
> > [==================================================>] 7.425MB/7.425MB
> > 148b94859892: Loading layer
> > [==================================================>] 8.1MB/8.1MB
> > dbea472c1ced: Loading layer
> > [==================================================>] 4.376MB/4.376MB
> > e81f1a367c17: Loading layer
> > [==================================================>] 131B/131B
> > 351848f6b295: Loading layer
> > [==================================================>] 8.216MB/8.216MB
> > Loaded image: weaveworks/weave-kube:latest
> > b20f7ff12b24: Loading layer
> > [==================================================>] 727.9kB/727.9kB
> > 1d49e386162e: Loading layer
> > [==================================================>] 9.281MB/9.281MB
> > 20145eeac146: Loading layer
> > [==================================================>] 297B/297B
> > d4cc72748587: Loading layer
> > [==================================================>] 600B/600B
> > Loaded image: weaveworks/weave-npc:latest
> > net.bridge.bridge-nf-call-iptables = 1
> > [init] Using Kubernetes version: v1.25.0
> > [preflight] Running pre-flight checks
> > [WARNING SystemVerification]: missing optional cgroups: blkio
> > error execution phase preflight: [preflight] Some fatal errors occurred:
> > [ERROR CRI]: container runtime is not running: output: E0411
> > 11:38:11.935458 12988 remote_runtime.go:948] "Status from runtime
> service
> > failed" err="rpc error: code = Unimplemented desc = unknown service
> > runtime.v1alpha2.RuntimeService"
> > time="2024-04-11T11:38:11Z" level=fatal msg="getting status of runtime:
> rpc
> > error: code = Unimplemented desc = unknown service
> > runtime.v1alpha2.RuntimeService"
> > , error: exit status 1
> > [preflight] If you know what you are doing, you can make a check
> non-fatal
> > with `--ignore-preflight-errors=...`
> > To see the stack trace of this error execute with --v=5 or higher
> > [init] Using Kubernetes version: v1.25.0
> > [preflight] Running pre-flight checks
> > [WARNING SystemVerification]: missing optional cgroups: blkio
> > error execution phase preflight: [preflight] Some fatal errors occurred:
> > [ERROR CRI]: container runtime is not running: output: E0411
> > 11:38:12.079422 13015 remote_runtime.go:948] "Status from runtime
> service
> > failed" err="rpc error: code = Unimplemented desc = unknown service
> > runtime.v1alpha2.RuntimeService"
> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime:
> rpc
> > error: code = Unimplemented desc = unknown service
> > runtime.v1alpha2.RuntimeService"
> > , error: exit status 1
> > [preflight] If you know what you are doing, you can make a check
> non-fatal
> > with `--ignore-preflight-errors=...`
> > To see the stack trace of this error execute with --v=5 or higher
> > [init] Using Kubernetes version: v1.25.0
> > [preflight] Running pre-flight checks
> > [WARNING SystemVerification]: missing optional cgroups: blkio
> > error execution phase preflight: [preflight] Some fatal errors occurred:
> > [ERROR CRI]: container runtime is not running: output: E0411
> > 11:38:12.228669 13043 remote_runtime.go:948] "Status from runtime
> service
> > failed" err="rpc error: code = Unimplemented desc = unknown service
> > runtime.v1alpha2.RuntimeService"
> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime:
> rpc
> > error: code = Unimplemented desc = unknown service
> > runtime.v1alpha2.RuntimeService"
> > , error: exit status 1
> > [preflight] If you know what you are doing, you can make a check
> non-fatal
> > with `--ignore-preflight-errors=...`
> > To see the stack trace of this error execute with --v=5 or higher
> > Error: kubeadm init failed!
> >
> >
> > Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <ro...@shapeblue.com> a
> > écrit :
> >
> > > Hi Benoit,
> > >
> > > The CKS feature has been improving over versions, I don't know if what
> > > you're trying to achieve is possible with it. Maybe try a different
> > version
> > > of the data iso:
> > > http://download.cloudstack.org/cks/
> > >
> > > Alternatively, you can also have a look at the CAPC project:
> > > https://cluster-api-cloudstack.sigs.k8s.io
> > >
> > >
> > > Regards.
> > >
> > >
> > >
> > >
> > > ________________________________
> > > From: benoit lair <ku...@gmail.com>
> > > Sent: Thursday, April 11, 2024 14:16
> > > To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <
> > > dev@cloudstack.apache.org>
> > > Subject: Re: ACS 4.16 - Change SystemVM template for CKS
> > >
> > > Any advices ?
> > >
> > > Best regards
> > >
> > > Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a
> > écrit :
> > >
> > > > I am opened to every alternatives of changing system vm templates
> > > > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
> > > >
> > > > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a
> > > écrit :
> > > >
> > > >> Hello Folks,
> > > >>
> > > >> I am trying to install K8s cluster with community iso 1.28.4
> > > >> I am with a ACS 4.16.0 environment
> > > >> It seems K8S is not working out of the box with v > 1.23.3 due to
> > > >> containerd.io version
> > > >>
> > > >> I would like to release a template who will work with K8s > 1.23.3
> on
> > > acs
> > > >> 4.16
> > > >> How can i tell CS to take another system vm template, avoiding to
> mess
> > > >> normal features with VR and VPC VR keeping the systemvm template
> > issued
> > > >> with CS 4.16 ?
> > > >>
> > > >> I am with XCP-NG 8.2.1 in production
> > > >>
> > > >> Thanks for your help or advises
> > > >> Best regards,
> > > >> Benoit
> > > >>
> > > >
> > >
> >
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by Wei ZHOU <us...@gmail.com>.
Hi,
Please refer to
https://github.com/apache/cloudstack/issues/8681#issuecomment-1999083241
The containerd in 4.16/4.17 sysyemvm template is too old.
-Wei
On Thursday, April 11, 2024, benoit lair <ku...@gmail.com> wrote:
> Hi Rohit,
>
> I already tested with different iso from cloudstack from CKS repos,
> It is ok with ACS 4.16 with community iso 1.23.3 but it fails from 1.24.0
> and following others versions
>
> I tried to do an upgrade from 1.23.3 with 1.24.0, it fails,
> I tried to bootstrap a k8s cluster with 1.24.0, also with 1.25 or 1.27.3
> and 1.28.4
>
> Last try, i tested k8s community iso 1.25.0
> in CS UI it saying "Create Kubernetes Cluster k8s-cks-cl16 in progress"
> and it fails
> If i try to ssh on control node with VR_Pub_IP port 2222
> i have nothing in execution
>
> If i try manual install :
> core@k8s-cks-cl16-control-18eccd0c12c:~$ sudo /opt/bin/setup-kube-system
> mount: /mnt/k8sdisk: /dev/sr0 already mounted on /mnt/k8sdisk.
> Installing binaries from /mnt/k8sdisk/
> 5b1fa8e3e100: Loading layer
> [==================================================>] 803.8kB/803.8kB
> 39c831b1aa26: Loading layer
> [==================================================>] 26.25MB/26.25MB
> Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
> 417cb9b79ade: Loading layer
> [==================================================>] 657.7kB/657.7kB
> 8d323b160d65: Loading layer
> [==================================================>] 24.95MB/24.95MB
> Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
> 256bc5c338a6: Loading layer
> [==================================================>] 122.2kB/122.2kB
> df1818f16337: Loading layer
> [==================================================>] 14.71MB/14.71MB
> Loaded image: registry.k8s.io/coredns/coredns:v1.9.3
> bd8a70623766: Loading layer
> [==================================================>] 75.78MB/75.78MB
> c88361932af5: Loading layer
> [==================================================>] 508B/508B
> Loaded image: kubernetesui/dashboard:v2.7.0
> 0b031aac6569: Loading layer
> [==================================================>] 804kB/804kB
> cc0597072b83: Loading layer
> [==================================================>] 1.076MB/1.076MB
> 5a8045f731fc: Loading layer
> [==================================================>] 90.58MB/90.58MB
> f5a5ef1a005f: Loading layer
> [==================================================>] 1.188MB/1.188MB
> 5cf3bfe5417d: Loading layer
> [==================================================>] 8.506MB/8.506MB
> Loaded image: registry.k8s.io/etcd:3.5.4-0
> 8d7366c22fd8: Loading layer
> [==================================================>] 804.1kB/804.1kB
> 7d5f5bba7c45: Loading layer
> [==================================================>] 694.9kB/694.9kB
> 8154c9483a6c: Loading layer
> [==================================================>] 32.72MB/32.72MB
> Loaded image: registry.k8s.io/kube-apiserver:v1.25.0
> c5f1a47b47bc: Loading layer
> [==================================================>] 29.76MB/29.76MB
> Loaded image: registry.k8s.io/kube-controller-manager:v1.25.0
> e4d19dd12df2: Loading layer
> [==================================================>] 7.631MB/7.631MB
> ba3637ccd78a: Loading layer
> [==================================================>] 12.63MB/12.63MB
> Loaded image: registry.k8s.io/kube-proxy:v1.25.0
> 026969d859f7: Loading layer
> [==================================================>] 14.29MB/14.29MB
> Loaded image: registry.k8s.io/kube-scheduler:v1.25.0
> d01384fea991: Loading layer
> [==================================================>] 19.74MB/19.74MB
> bcec7eb9e567: Loading layer
> [==================================================>] 530B/530B
> Loaded image: kubernetesui/metrics-scraper:v1.0.8
> 961e93cda9dd: Loading layer
> [==================================================>] 307kB/307kB
> Loaded image: registry.k8s.io/pause:3.8
> 1b3ee35aacca: Loading layer
> [==================================================>] 2.796MB/2.796MB
> 17d89e2a5199: Loading layer
> [==================================================>] 7.425MB/7.425MB
> 148b94859892: Loading layer
> [==================================================>] 8.1MB/8.1MB
> dbea472c1ced: Loading layer
> [==================================================>] 4.376MB/4.376MB
> e81f1a367c17: Loading layer
> [==================================================>] 131B/131B
> 351848f6b295: Loading layer
> [==================================================>] 8.216MB/8.216MB
> Loaded image: weaveworks/weave-kube:latest
> b20f7ff12b24: Loading layer
> [==================================================>] 727.9kB/727.9kB
> 1d49e386162e: Loading layer
> [==================================================>] 9.281MB/9.281MB
> 20145eeac146: Loading layer
> [==================================================>] 297B/297B
> d4cc72748587: Loading layer
> [==================================================>] 600B/600B
> Loaded image: weaveworks/weave-npc:latest
> net.bridge.bridge-nf-call-iptables = 1
> [init] Using Kubernetes version: v1.25.0
> [preflight] Running pre-flight checks
> [WARNING SystemVerification]: missing optional cgroups: blkio
> error execution phase preflight: [preflight] Some fatal errors occurred:
> [ERROR CRI]: container runtime is not running: output: E0411
> 11:38:11.935458 12988 remote_runtime.go:948] "Status from runtime service
> failed" err="rpc error: code = Unimplemented desc = unknown service
> runtime.v1alpha2.RuntimeService"
> time="2024-04-11T11:38:11Z" level=fatal msg="getting status of runtime: rpc
> error: code = Unimplemented desc = unknown service
> runtime.v1alpha2.RuntimeService"
> , error: exit status 1
> [preflight] If you know what you are doing, you can make a check non-fatal
> with `--ignore-preflight-errors=...`
> To see the stack trace of this error execute with --v=5 or higher
> [init] Using Kubernetes version: v1.25.0
> [preflight] Running pre-flight checks
> [WARNING SystemVerification]: missing optional cgroups: blkio
> error execution phase preflight: [preflight] Some fatal errors occurred:
> [ERROR CRI]: container runtime is not running: output: E0411
> 11:38:12.079422 13015 remote_runtime.go:948] "Status from runtime service
> failed" err="rpc error: code = Unimplemented desc = unknown service
> runtime.v1alpha2.RuntimeService"
> time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime: rpc
> error: code = Unimplemented desc = unknown service
> runtime.v1alpha2.RuntimeService"
> , error: exit status 1
> [preflight] If you know what you are doing, you can make a check non-fatal
> with `--ignore-preflight-errors=...`
> To see the stack trace of this error execute with --v=5 or higher
> [init] Using Kubernetes version: v1.25.0
> [preflight] Running pre-flight checks
> [WARNING SystemVerification]: missing optional cgroups: blkio
> error execution phase preflight: [preflight] Some fatal errors occurred:
> [ERROR CRI]: container runtime is not running: output: E0411
> 11:38:12.228669 13043 remote_runtime.go:948] "Status from runtime service
> failed" err="rpc error: code = Unimplemented desc = unknown service
> runtime.v1alpha2.RuntimeService"
> time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime: rpc
> error: code = Unimplemented desc = unknown service
> runtime.v1alpha2.RuntimeService"
> , error: exit status 1
> [preflight] If you know what you are doing, you can make a check non-fatal
> with `--ignore-preflight-errors=...`
> To see the stack trace of this error execute with --v=5 or higher
> Error: kubeadm init failed!
>
>
> Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <ro...@shapeblue.com> a
> écrit :
>
> > Hi Benoit,
> >
> > The CKS feature has been improving over versions, I don't know if what
> > you're trying to achieve is possible with it. Maybe try a different
> version
> > of the data iso:
> > http://download.cloudstack.org/cks/
> >
> > Alternatively, you can also have a look at the CAPC project:
> > https://cluster-api-cloudstack.sigs.k8s.io
> >
> >
> > Regards.
> >
> >
> >
> >
> > ________________________________
> > From: benoit lair <ku...@gmail.com>
> > Sent: Thursday, April 11, 2024 14:16
> > To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <
> > dev@cloudstack.apache.org>
> > Subject: Re: ACS 4.16 - Change SystemVM template for CKS
> >
> > Any advices ?
> >
> > Best regards
> >
> > Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a
> écrit :
> >
> > > I am opened to every alternatives of changing system vm templates
> > > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
> > >
> > > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a
> > écrit :
> > >
> > >> Hello Folks,
> > >>
> > >> I am trying to install K8s cluster with community iso 1.28.4
> > >> I am with a ACS 4.16.0 environment
> > >> It seems K8S is not working out of the box with v > 1.23.3 due to
> > >> containerd.io version
> > >>
> > >> I would like to release a template who will work with K8s > 1.23.3 on
> > acs
> > >> 4.16
> > >> How can i tell CS to take another system vm template, avoiding to mess
> > >> normal features with VR and VPC VR keeping the systemvm template
> issued
> > >> with CS 4.16 ?
> > >>
> > >> I am with XCP-NG 8.2.1 in production
> > >>
> > >> Thanks for your help or advises
> > >> Best regards,
> > >> Benoit
> > >>
> > >
> >
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by Wei ZHOU <us...@gmail.com>.
Hi,
Please refer to
https://github.com/apache/cloudstack/issues/8681#issuecomment-1999083241
The containerd in 4.16/4.17 sysyemvm template is too old.
-Wei
On Thursday, April 11, 2024, benoit lair <ku...@gmail.com> wrote:
> Hi Rohit,
>
> I already tested with different iso from cloudstack from CKS repos,
> It is ok with ACS 4.16 with community iso 1.23.3 but it fails from 1.24.0
> and following others versions
>
> I tried to do an upgrade from 1.23.3 with 1.24.0, it fails,
> I tried to bootstrap a k8s cluster with 1.24.0, also with 1.25 or 1.27.3
> and 1.28.4
>
> Last try, i tested k8s community iso 1.25.0
> in CS UI it saying "Create Kubernetes Cluster k8s-cks-cl16 in progress"
> and it fails
> If i try to ssh on control node with VR_Pub_IP port 2222
> i have nothing in execution
>
> If i try manual install :
> core@k8s-cks-cl16-control-18eccd0c12c:~$ sudo /opt/bin/setup-kube-system
> mount: /mnt/k8sdisk: /dev/sr0 already mounted on /mnt/k8sdisk.
> Installing binaries from /mnt/k8sdisk/
> 5b1fa8e3e100: Loading layer
> [==================================================>] 803.8kB/803.8kB
> 39c831b1aa26: Loading layer
> [==================================================>] 26.25MB/26.25MB
> Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
> 417cb9b79ade: Loading layer
> [==================================================>] 657.7kB/657.7kB
> 8d323b160d65: Loading layer
> [==================================================>] 24.95MB/24.95MB
> Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
> 256bc5c338a6: Loading layer
> [==================================================>] 122.2kB/122.2kB
> df1818f16337: Loading layer
> [==================================================>] 14.71MB/14.71MB
> Loaded image: registry.k8s.io/coredns/coredns:v1.9.3
> bd8a70623766: Loading layer
> [==================================================>] 75.78MB/75.78MB
> c88361932af5: Loading layer
> [==================================================>] 508B/508B
> Loaded image: kubernetesui/dashboard:v2.7.0
> 0b031aac6569: Loading layer
> [==================================================>] 804kB/804kB
> cc0597072b83: Loading layer
> [==================================================>] 1.076MB/1.076MB
> 5a8045f731fc: Loading layer
> [==================================================>] 90.58MB/90.58MB
> f5a5ef1a005f: Loading layer
> [==================================================>] 1.188MB/1.188MB
> 5cf3bfe5417d: Loading layer
> [==================================================>] 8.506MB/8.506MB
> Loaded image: registry.k8s.io/etcd:3.5.4-0
> 8d7366c22fd8: Loading layer
> [==================================================>] 804.1kB/804.1kB
> 7d5f5bba7c45: Loading layer
> [==================================================>] 694.9kB/694.9kB
> 8154c9483a6c: Loading layer
> [==================================================>] 32.72MB/32.72MB
> Loaded image: registry.k8s.io/kube-apiserver:v1.25.0
> c5f1a47b47bc: Loading layer
> [==================================================>] 29.76MB/29.76MB
> Loaded image: registry.k8s.io/kube-controller-manager:v1.25.0
> e4d19dd12df2: Loading layer
> [==================================================>] 7.631MB/7.631MB
> ba3637ccd78a: Loading layer
> [==================================================>] 12.63MB/12.63MB
> Loaded image: registry.k8s.io/kube-proxy:v1.25.0
> 026969d859f7: Loading layer
> [==================================================>] 14.29MB/14.29MB
> Loaded image: registry.k8s.io/kube-scheduler:v1.25.0
> d01384fea991: Loading layer
> [==================================================>] 19.74MB/19.74MB
> bcec7eb9e567: Loading layer
> [==================================================>] 530B/530B
> Loaded image: kubernetesui/metrics-scraper:v1.0.8
> 961e93cda9dd: Loading layer
> [==================================================>] 307kB/307kB
> Loaded image: registry.k8s.io/pause:3.8
> 1b3ee35aacca: Loading layer
> [==================================================>] 2.796MB/2.796MB
> 17d89e2a5199: Loading layer
> [==================================================>] 7.425MB/7.425MB
> 148b94859892: Loading layer
> [==================================================>] 8.1MB/8.1MB
> dbea472c1ced: Loading layer
> [==================================================>] 4.376MB/4.376MB
> e81f1a367c17: Loading layer
> [==================================================>] 131B/131B
> 351848f6b295: Loading layer
> [==================================================>] 8.216MB/8.216MB
> Loaded image: weaveworks/weave-kube:latest
> b20f7ff12b24: Loading layer
> [==================================================>] 727.9kB/727.9kB
> 1d49e386162e: Loading layer
> [==================================================>] 9.281MB/9.281MB
> 20145eeac146: Loading layer
> [==================================================>] 297B/297B
> d4cc72748587: Loading layer
> [==================================================>] 600B/600B
> Loaded image: weaveworks/weave-npc:latest
> net.bridge.bridge-nf-call-iptables = 1
> [init] Using Kubernetes version: v1.25.0
> [preflight] Running pre-flight checks
> [WARNING SystemVerification]: missing optional cgroups: blkio
> error execution phase preflight: [preflight] Some fatal errors occurred:
> [ERROR CRI]: container runtime is not running: output: E0411
> 11:38:11.935458 12988 remote_runtime.go:948] "Status from runtime service
> failed" err="rpc error: code = Unimplemented desc = unknown service
> runtime.v1alpha2.RuntimeService"
> time="2024-04-11T11:38:11Z" level=fatal msg="getting status of runtime: rpc
> error: code = Unimplemented desc = unknown service
> runtime.v1alpha2.RuntimeService"
> , error: exit status 1
> [preflight] If you know what you are doing, you can make a check non-fatal
> with `--ignore-preflight-errors=...`
> To see the stack trace of this error execute with --v=5 or higher
> [init] Using Kubernetes version: v1.25.0
> [preflight] Running pre-flight checks
> [WARNING SystemVerification]: missing optional cgroups: blkio
> error execution phase preflight: [preflight] Some fatal errors occurred:
> [ERROR CRI]: container runtime is not running: output: E0411
> 11:38:12.079422 13015 remote_runtime.go:948] "Status from runtime service
> failed" err="rpc error: code = Unimplemented desc = unknown service
> runtime.v1alpha2.RuntimeService"
> time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime: rpc
> error: code = Unimplemented desc = unknown service
> runtime.v1alpha2.RuntimeService"
> , error: exit status 1
> [preflight] If you know what you are doing, you can make a check non-fatal
> with `--ignore-preflight-errors=...`
> To see the stack trace of this error execute with --v=5 or higher
> [init] Using Kubernetes version: v1.25.0
> [preflight] Running pre-flight checks
> [WARNING SystemVerification]: missing optional cgroups: blkio
> error execution phase preflight: [preflight] Some fatal errors occurred:
> [ERROR CRI]: container runtime is not running: output: E0411
> 11:38:12.228669 13043 remote_runtime.go:948] "Status from runtime service
> failed" err="rpc error: code = Unimplemented desc = unknown service
> runtime.v1alpha2.RuntimeService"
> time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime: rpc
> error: code = Unimplemented desc = unknown service
> runtime.v1alpha2.RuntimeService"
> , error: exit status 1
> [preflight] If you know what you are doing, you can make a check non-fatal
> with `--ignore-preflight-errors=...`
> To see the stack trace of this error execute with --v=5 or higher
> Error: kubeadm init failed!
>
>
> Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <ro...@shapeblue.com> a
> écrit :
>
> > Hi Benoit,
> >
> > The CKS feature has been improving over versions, I don't know if what
> > you're trying to achieve is possible with it. Maybe try a different
> version
> > of the data iso:
> > http://download.cloudstack.org/cks/
> >
> > Alternatively, you can also have a look at the CAPC project:
> > https://cluster-api-cloudstack.sigs.k8s.io
> >
> >
> > Regards.
> >
> >
> >
> >
> > ________________________________
> > From: benoit lair <ku...@gmail.com>
> > Sent: Thursday, April 11, 2024 14:16
> > To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <
> > dev@cloudstack.apache.org>
> > Subject: Re: ACS 4.16 - Change SystemVM template for CKS
> >
> > Any advices ?
> >
> > Best regards
> >
> > Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a
> écrit :
> >
> > > I am opened to every alternatives of changing system vm templates
> > > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
> > >
> > > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a
> > écrit :
> > >
> > >> Hello Folks,
> > >>
> > >> I am trying to install K8s cluster with community iso 1.28.4
> > >> I am with a ACS 4.16.0 environment
> > >> It seems K8S is not working out of the box with v > 1.23.3 due to
> > >> containerd.io version
> > >>
> > >> I would like to release a template who will work with K8s > 1.23.3 on
> > acs
> > >> 4.16
> > >> How can i tell CS to take another system vm template, avoiding to mess
> > >> normal features with VR and VPC VR keeping the systemvm template
> issued
> > >> with CS 4.16 ?
> > >>
> > >> I am with XCP-NG 8.2.1 in production
> > >>
> > >> Thanks for your help or advises
> > >> Best regards,
> > >> Benoit
> > >>
> > >
> >
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by benoit lair <ku...@gmail.com>.
Hi Rohit,
I already tested with different iso from cloudstack from CKS repos,
It is ok with ACS 4.16 with community iso 1.23.3 but it fails from 1.24.0
and following others versions
I tried to do an upgrade from 1.23.3 with 1.24.0, it fails,
I tried to bootstrap a k8s cluster with 1.24.0, also with 1.25 or 1.27.3
and 1.28.4
Last try, i tested k8s community iso 1.25.0
in CS UI it saying "Create Kubernetes Cluster k8s-cks-cl16 in progress"
and it fails
If i try to ssh on control node with VR_Pub_IP port 2222
i have nothing in execution
If i try manual install :
core@k8s-cks-cl16-control-18eccd0c12c:~$ sudo /opt/bin/setup-kube-system
mount: /mnt/k8sdisk: /dev/sr0 already mounted on /mnt/k8sdisk.
Installing binaries from /mnt/k8sdisk/
5b1fa8e3e100: Loading layer
[==================================================>] 803.8kB/803.8kB
39c831b1aa26: Loading layer
[==================================================>] 26.25MB/26.25MB
Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
417cb9b79ade: Loading layer
[==================================================>] 657.7kB/657.7kB
8d323b160d65: Loading layer
[==================================================>] 24.95MB/24.95MB
Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
256bc5c338a6: Loading layer
[==================================================>] 122.2kB/122.2kB
df1818f16337: Loading layer
[==================================================>] 14.71MB/14.71MB
Loaded image: registry.k8s.io/coredns/coredns:v1.9.3
bd8a70623766: Loading layer
[==================================================>] 75.78MB/75.78MB
c88361932af5: Loading layer
[==================================================>] 508B/508B
Loaded image: kubernetesui/dashboard:v2.7.0
0b031aac6569: Loading layer
[==================================================>] 804kB/804kB
cc0597072b83: Loading layer
[==================================================>] 1.076MB/1.076MB
5a8045f731fc: Loading layer
[==================================================>] 90.58MB/90.58MB
f5a5ef1a005f: Loading layer
[==================================================>] 1.188MB/1.188MB
5cf3bfe5417d: Loading layer
[==================================================>] 8.506MB/8.506MB
Loaded image: registry.k8s.io/etcd:3.5.4-0
8d7366c22fd8: Loading layer
[==================================================>] 804.1kB/804.1kB
7d5f5bba7c45: Loading layer
[==================================================>] 694.9kB/694.9kB
8154c9483a6c: Loading layer
[==================================================>] 32.72MB/32.72MB
Loaded image: registry.k8s.io/kube-apiserver:v1.25.0
c5f1a47b47bc: Loading layer
[==================================================>] 29.76MB/29.76MB
Loaded image: registry.k8s.io/kube-controller-manager:v1.25.0
e4d19dd12df2: Loading layer
[==================================================>] 7.631MB/7.631MB
ba3637ccd78a: Loading layer
[==================================================>] 12.63MB/12.63MB
Loaded image: registry.k8s.io/kube-proxy:v1.25.0
026969d859f7: Loading layer
[==================================================>] 14.29MB/14.29MB
Loaded image: registry.k8s.io/kube-scheduler:v1.25.0
d01384fea991: Loading layer
[==================================================>] 19.74MB/19.74MB
bcec7eb9e567: Loading layer
[==================================================>] 530B/530B
Loaded image: kubernetesui/metrics-scraper:v1.0.8
961e93cda9dd: Loading layer
[==================================================>] 307kB/307kB
Loaded image: registry.k8s.io/pause:3.8
1b3ee35aacca: Loading layer
[==================================================>] 2.796MB/2.796MB
17d89e2a5199: Loading layer
[==================================================>] 7.425MB/7.425MB
148b94859892: Loading layer
[==================================================>] 8.1MB/8.1MB
dbea472c1ced: Loading layer
[==================================================>] 4.376MB/4.376MB
e81f1a367c17: Loading layer
[==================================================>] 131B/131B
351848f6b295: Loading layer
[==================================================>] 8.216MB/8.216MB
Loaded image: weaveworks/weave-kube:latest
b20f7ff12b24: Loading layer
[==================================================>] 727.9kB/727.9kB
1d49e386162e: Loading layer
[==================================================>] 9.281MB/9.281MB
20145eeac146: Loading layer
[==================================================>] 297B/297B
d4cc72748587: Loading layer
[==================================================>] 600B/600B
Loaded image: weaveworks/weave-npc:latest
net.bridge.bridge-nf-call-iptables = 1
[init] Using Kubernetes version: v1.25.0
[preflight] Running pre-flight checks
[WARNING SystemVerification]: missing optional cgroups: blkio
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output: E0411
11:38:11.935458 12988 remote_runtime.go:948] "Status from runtime service
failed" err="rpc error: code = Unimplemented desc = unknown service
runtime.v1alpha2.RuntimeService"
time="2024-04-11T11:38:11Z" level=fatal msg="getting status of runtime: rpc
error: code = Unimplemented desc = unknown service
runtime.v1alpha2.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal
with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
[init] Using Kubernetes version: v1.25.0
[preflight] Running pre-flight checks
[WARNING SystemVerification]: missing optional cgroups: blkio
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output: E0411
11:38:12.079422 13015 remote_runtime.go:948] "Status from runtime service
failed" err="rpc error: code = Unimplemented desc = unknown service
runtime.v1alpha2.RuntimeService"
time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime: rpc
error: code = Unimplemented desc = unknown service
runtime.v1alpha2.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal
with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
[init] Using Kubernetes version: v1.25.0
[preflight] Running pre-flight checks
[WARNING SystemVerification]: missing optional cgroups: blkio
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output: E0411
11:38:12.228669 13043 remote_runtime.go:948] "Status from runtime service
failed" err="rpc error: code = Unimplemented desc = unknown service
runtime.v1alpha2.RuntimeService"
time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime: rpc
error: code = Unimplemented desc = unknown service
runtime.v1alpha2.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal
with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
Error: kubeadm init failed!
Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <ro...@shapeblue.com> a
écrit :
> Hi Benoit,
>
> The CKS feature has been improving over versions, I don't know if what
> you're trying to achieve is possible with it. Maybe try a different version
> of the data iso:
> http://download.cloudstack.org/cks/
>
> Alternatively, you can also have a look at the CAPC project:
> https://cluster-api-cloudstack.sigs.k8s.io
>
>
> Regards.
>
>
>
>
> ________________________________
> From: benoit lair <ku...@gmail.com>
> Sent: Thursday, April 11, 2024 14:16
> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <
> dev@cloudstack.apache.org>
> Subject: Re: ACS 4.16 - Change SystemVM template for CKS
>
> Any advices ?
>
> Best regards
>
> Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a écrit :
>
> > I am opened to every alternatives of changing system vm templates
> > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
> >
> > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a
> écrit :
> >
> >> Hello Folks,
> >>
> >> I am trying to install K8s cluster with community iso 1.28.4
> >> I am with a ACS 4.16.0 environment
> >> It seems K8S is not working out of the box with v > 1.23.3 due to
> >> containerd.io version
> >>
> >> I would like to release a template who will work with K8s > 1.23.3 on
> acs
> >> 4.16
> >> How can i tell CS to take another system vm template, avoiding to mess
> >> normal features with VR and VPC VR keeping the systemvm template issued
> >> with CS 4.16 ?
> >>
> >> I am with XCP-NG 8.2.1 in production
> >>
> >> Thanks for your help or advises
> >> Best regards,
> >> Benoit
> >>
> >
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by benoit lair <ku...@gmail.com>.
Hi Rohit,
I already tested with different iso from cloudstack from CKS repos,
It is ok with ACS 4.16 with community iso 1.23.3 but it fails from 1.24.0
and following others versions
I tried to do an upgrade from 1.23.3 with 1.24.0, it fails,
I tried to bootstrap a k8s cluster with 1.24.0, also with 1.25 or 1.27.3
and 1.28.4
Last try, i tested k8s community iso 1.25.0
in CS UI it saying "Create Kubernetes Cluster k8s-cks-cl16 in progress"
and it fails
If i try to ssh on control node with VR_Pub_IP port 2222
i have nothing in execution
If i try manual install :
core@k8s-cks-cl16-control-18eccd0c12c:~$ sudo /opt/bin/setup-kube-system
mount: /mnt/k8sdisk: /dev/sr0 already mounted on /mnt/k8sdisk.
Installing binaries from /mnt/k8sdisk/
5b1fa8e3e100: Loading layer
[==================================================>] 803.8kB/803.8kB
39c831b1aa26: Loading layer
[==================================================>] 26.25MB/26.25MB
Loaded image: apache/cloudstack-kubernetes-autoscaler:latest
417cb9b79ade: Loading layer
[==================================================>] 657.7kB/657.7kB
8d323b160d65: Loading layer
[==================================================>] 24.95MB/24.95MB
Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0
256bc5c338a6: Loading layer
[==================================================>] 122.2kB/122.2kB
df1818f16337: Loading layer
[==================================================>] 14.71MB/14.71MB
Loaded image: registry.k8s.io/coredns/coredns:v1.9.3
bd8a70623766: Loading layer
[==================================================>] 75.78MB/75.78MB
c88361932af5: Loading layer
[==================================================>] 508B/508B
Loaded image: kubernetesui/dashboard:v2.7.0
0b031aac6569: Loading layer
[==================================================>] 804kB/804kB
cc0597072b83: Loading layer
[==================================================>] 1.076MB/1.076MB
5a8045f731fc: Loading layer
[==================================================>] 90.58MB/90.58MB
f5a5ef1a005f: Loading layer
[==================================================>] 1.188MB/1.188MB
5cf3bfe5417d: Loading layer
[==================================================>] 8.506MB/8.506MB
Loaded image: registry.k8s.io/etcd:3.5.4-0
8d7366c22fd8: Loading layer
[==================================================>] 804.1kB/804.1kB
7d5f5bba7c45: Loading layer
[==================================================>] 694.9kB/694.9kB
8154c9483a6c: Loading layer
[==================================================>] 32.72MB/32.72MB
Loaded image: registry.k8s.io/kube-apiserver:v1.25.0
c5f1a47b47bc: Loading layer
[==================================================>] 29.76MB/29.76MB
Loaded image: registry.k8s.io/kube-controller-manager:v1.25.0
e4d19dd12df2: Loading layer
[==================================================>] 7.631MB/7.631MB
ba3637ccd78a: Loading layer
[==================================================>] 12.63MB/12.63MB
Loaded image: registry.k8s.io/kube-proxy:v1.25.0
026969d859f7: Loading layer
[==================================================>] 14.29MB/14.29MB
Loaded image: registry.k8s.io/kube-scheduler:v1.25.0
d01384fea991: Loading layer
[==================================================>] 19.74MB/19.74MB
bcec7eb9e567: Loading layer
[==================================================>] 530B/530B
Loaded image: kubernetesui/metrics-scraper:v1.0.8
961e93cda9dd: Loading layer
[==================================================>] 307kB/307kB
Loaded image: registry.k8s.io/pause:3.8
1b3ee35aacca: Loading layer
[==================================================>] 2.796MB/2.796MB
17d89e2a5199: Loading layer
[==================================================>] 7.425MB/7.425MB
148b94859892: Loading layer
[==================================================>] 8.1MB/8.1MB
dbea472c1ced: Loading layer
[==================================================>] 4.376MB/4.376MB
e81f1a367c17: Loading layer
[==================================================>] 131B/131B
351848f6b295: Loading layer
[==================================================>] 8.216MB/8.216MB
Loaded image: weaveworks/weave-kube:latest
b20f7ff12b24: Loading layer
[==================================================>] 727.9kB/727.9kB
1d49e386162e: Loading layer
[==================================================>] 9.281MB/9.281MB
20145eeac146: Loading layer
[==================================================>] 297B/297B
d4cc72748587: Loading layer
[==================================================>] 600B/600B
Loaded image: weaveworks/weave-npc:latest
net.bridge.bridge-nf-call-iptables = 1
[init] Using Kubernetes version: v1.25.0
[preflight] Running pre-flight checks
[WARNING SystemVerification]: missing optional cgroups: blkio
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output: E0411
11:38:11.935458 12988 remote_runtime.go:948] "Status from runtime service
failed" err="rpc error: code = Unimplemented desc = unknown service
runtime.v1alpha2.RuntimeService"
time="2024-04-11T11:38:11Z" level=fatal msg="getting status of runtime: rpc
error: code = Unimplemented desc = unknown service
runtime.v1alpha2.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal
with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
[init] Using Kubernetes version: v1.25.0
[preflight] Running pre-flight checks
[WARNING SystemVerification]: missing optional cgroups: blkio
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output: E0411
11:38:12.079422 13015 remote_runtime.go:948] "Status from runtime service
failed" err="rpc error: code = Unimplemented desc = unknown service
runtime.v1alpha2.RuntimeService"
time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime: rpc
error: code = Unimplemented desc = unknown service
runtime.v1alpha2.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal
with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
[init] Using Kubernetes version: v1.25.0
[preflight] Running pre-flight checks
[WARNING SystemVerification]: missing optional cgroups: blkio
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output: E0411
11:38:12.228669 13043 remote_runtime.go:948] "Status from runtime service
failed" err="rpc error: code = Unimplemented desc = unknown service
runtime.v1alpha2.RuntimeService"
time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime: rpc
error: code = Unimplemented desc = unknown service
runtime.v1alpha2.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal
with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
Error: kubeadm init failed!
Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <ro...@shapeblue.com> a
écrit :
> Hi Benoit,
>
> The CKS feature has been improving over versions, I don't know if what
> you're trying to achieve is possible with it. Maybe try a different version
> of the data iso:
> http://download.cloudstack.org/cks/
>
> Alternatively, you can also have a look at the CAPC project:
> https://cluster-api-cloudstack.sigs.k8s.io
>
>
> Regards.
>
>
>
>
> ________________________________
> From: benoit lair <ku...@gmail.com>
> Sent: Thursday, April 11, 2024 14:16
> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <
> dev@cloudstack.apache.org>
> Subject: Re: ACS 4.16 - Change SystemVM template for CKS
>
> Any advices ?
>
> Best regards
>
> Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a écrit :
>
> > I am opened to every alternatives of changing system vm templates
> > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
> >
> > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a
> écrit :
> >
> >> Hello Folks,
> >>
> >> I am trying to install K8s cluster with community iso 1.28.4
> >> I am with a ACS 4.16.0 environment
> >> It seems K8S is not working out of the box with v > 1.23.3 due to
> >> containerd.io version
> >>
> >> I would like to release a template who will work with K8s > 1.23.3 on
> acs
> >> 4.16
> >> How can i tell CS to take another system vm template, avoiding to mess
> >> normal features with VR and VPC VR keeping the systemvm template issued
> >> with CS 4.16 ?
> >>
> >> I am with XCP-NG 8.2.1 in production
> >>
> >> Thanks for your help or advises
> >> Best regards,
> >> Benoit
> >>
> >
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by benoit lair <ku...@gmail.com>.
Rohit,
It is possible to install CAPC on a CS 4.16 ?
Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <ro...@shapeblue.com> a
écrit :
> Hi Benoit,
>
> The CKS feature has been improving over versions, I don't know if what
> you're trying to achieve is possible with it. Maybe try a different version
> of the data iso:
> http://download.cloudstack.org/cks/
>
> Alternatively, you can also have a look at the CAPC project:
> https://cluster-api-cloudstack.sigs.k8s.io
>
>
> Regards.
>
>
>
>
> ________________________________
> From: benoit lair <ku...@gmail.com>
> Sent: Thursday, April 11, 2024 14:16
> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <
> dev@cloudstack.apache.org>
> Subject: Re: ACS 4.16 - Change SystemVM template for CKS
>
> Any advices ?
>
> Best regards
>
> Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a écrit :
>
> > I am opened to every alternatives of changing system vm templates
> > I just need to run K8s clyusters 1.28 with my CS 4.16 :)
> >
> > Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a
> écrit :
> >
> >> Hello Folks,
> >>
> >> I am trying to install K8s cluster with community iso 1.28.4
> >> I am with a ACS 4.16.0 environment
> >> It seems K8S is not working out of the box with v > 1.23.3 due to
> >> containerd.io version
> >>
> >> I would like to release a template who will work with K8s > 1.23.3 on
> acs
> >> 4.16
> >> How can i tell CS to take another system vm template, avoiding to mess
> >> normal features with VR and VPC VR keeping the systemvm template issued
> >> with CS 4.16 ?
> >>
> >> I am with XCP-NG 8.2.1 in production
> >>
> >> Thanks for your help or advises
> >> Best regards,
> >> Benoit
> >>
> >
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi Benoit,
The CKS feature has been improving over versions, I don't know if what you're trying to achieve is possible with it. Maybe try a different version of the data iso:
http://download.cloudstack.org/cks/
Alternatively, you can also have a look at the CAPC project: https://cluster-api-cloudstack.sigs.k8s.io
Regards.
________________________________
From: benoit lair <ku...@gmail.com>
Sent: Thursday, April 11, 2024 14:16
To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <de...@cloudstack.apache.org>
Subject: Re: ACS 4.16 - Change SystemVM template for CKS
Any advices ?
Best regards
Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a écrit :
> I am opened to every alternatives of changing system vm templates
> I just need to run K8s clyusters 1.28 with my CS 4.16 :)
>
> Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a écrit :
>
>> Hello Folks,
>>
>> I am trying to install K8s cluster with community iso 1.28.4
>> I am with a ACS 4.16.0 environment
>> It seems K8S is not working out of the box with v > 1.23.3 due to
>> containerd.io version
>>
>> I would like to release a template who will work with K8s > 1.23.3 on acs
>> 4.16
>> How can i tell CS to take another system vm template, avoiding to mess
>> normal features with VR and VPC VR keeping the systemvm template issued
>> with CS 4.16 ?
>>
>> I am with XCP-NG 8.2.1 in production
>>
>> Thanks for your help or advises
>> Best regards,
>> Benoit
>>
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi Benoit,
The CKS feature has been improving over versions, I don't know if what you're trying to achieve is possible with it. Maybe try a different version of the data iso:
http://download.cloudstack.org/cks/
Alternatively, you can also have a look at the CAPC project: https://cluster-api-cloudstack.sigs.k8s.io
Regards.
________________________________
From: benoit lair <ku...@gmail.com>
Sent: Thursday, April 11, 2024 14:16
To: users@cloudstack.apache.org <us...@cloudstack.apache.org>; dev <de...@cloudstack.apache.org>
Subject: Re: ACS 4.16 - Change SystemVM template for CKS
Any advices ?
Best regards
Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a écrit :
> I am opened to every alternatives of changing system vm templates
> I just need to run K8s clyusters 1.28 with my CS 4.16 :)
>
> Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a écrit :
>
>> Hello Folks,
>>
>> I am trying to install K8s cluster with community iso 1.28.4
>> I am with a ACS 4.16.0 environment
>> It seems K8S is not working out of the box with v > 1.23.3 due to
>> containerd.io version
>>
>> I would like to release a template who will work with K8s > 1.23.3 on acs
>> 4.16
>> How can i tell CS to take another system vm template, avoiding to mess
>> normal features with VR and VPC VR keeping the systemvm template issued
>> with CS 4.16 ?
>>
>> I am with XCP-NG 8.2.1 in production
>>
>> Thanks for your help or advises
>> Best regards,
>> Benoit
>>
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by benoit lair <ku...@gmail.com>.
Any advices ?
Best regards
Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a écrit :
> I am opened to every alternatives of changing system vm templates
> I just need to run K8s clyusters 1.28 with my CS 4.16 :)
>
> Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a écrit :
>
>> Hello Folks,
>>
>> I am trying to install K8s cluster with community iso 1.28.4
>> I am with a ACS 4.16.0 environment
>> It seems K8S is not working out of the box with v > 1.23.3 due to
>> containerd.io version
>>
>> I would like to release a template who will work with K8s > 1.23.3 on acs
>> 4.16
>> How can i tell CS to take another system vm template, avoiding to mess
>> normal features with VR and VPC VR keeping the systemvm template issued
>> with CS 4.16 ?
>>
>> I am with XCP-NG 8.2.1 in production
>>
>> Thanks for your help or advises
>> Best regards,
>> Benoit
>>
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by benoit lair <ku...@gmail.com>.
Any advices ?
Best regards
Le lun. 8 avr. 2024 à 16:53, benoit lair <ku...@gmail.com> a écrit :
> I am opened to every alternatives of changing system vm templates
> I just need to run K8s clyusters 1.28 with my CS 4.16 :)
>
> Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a écrit :
>
>> Hello Folks,
>>
>> I am trying to install K8s cluster with community iso 1.28.4
>> I am with a ACS 4.16.0 environment
>> It seems K8S is not working out of the box with v > 1.23.3 due to
>> containerd.io version
>>
>> I would like to release a template who will work with K8s > 1.23.3 on acs
>> 4.16
>> How can i tell CS to take another system vm template, avoiding to mess
>> normal features with VR and VPC VR keeping the systemvm template issued
>> with CS 4.16 ?
>>
>> I am with XCP-NG 8.2.1 in production
>>
>> Thanks for your help or advises
>> Best regards,
>> Benoit
>>
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by benoit lair <ku...@gmail.com>.
I am opened to every alternatives of changing system vm templates
I just need to run K8s clyusters 1.28 with my CS 4.16 :)
Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a écrit :
> Hello Folks,
>
> I am trying to install K8s cluster with community iso 1.28.4
> I am with a ACS 4.16.0 environment
> It seems K8S is not working out of the box with v > 1.23.3 due to
> containerd.io version
>
> I would like to release a template who will work with K8s > 1.23.3 on acs
> 4.16
> How can i tell CS to take another system vm template, avoiding to mess
> normal features with VR and VPC VR keeping the systemvm template issued
> with CS 4.16 ?
>
> I am with XCP-NG 8.2.1 in production
>
> Thanks for your help or advises
> Best regards,
> Benoit
>
Re: ACS 4.16 - Change SystemVM template for CKS
Posted by benoit lair <ku...@gmail.com>.
I am opened to every alternatives of changing system vm templates
I just need to run K8s clyusters 1.28 with my CS 4.16 :)
Le lun. 8 avr. 2024 à 16:52, benoit lair <ku...@gmail.com> a écrit :
> Hello Folks,
>
> I am trying to install K8s cluster with community iso 1.28.4
> I am with a ACS 4.16.0 environment
> It seems K8S is not working out of the box with v > 1.23.3 due to
> containerd.io version
>
> I would like to release a template who will work with K8s > 1.23.3 on acs
> 4.16
> How can i tell CS to take another system vm template, avoiding to mess
> normal features with VR and VPC VR keeping the systemvm template issued
> with CS 4.16 ?
>
> I am with XCP-NG 8.2.1 in production
>
> Thanks for your help or advises
> Best regards,
> Benoit
>