Kubernetes stuck in the starting state

[Sanjay Kumar <ss...@gmail.com>]

Hello All

Kubernetes cluster is stucked in the 'Starting' state, even though VM's,
LB, Firewall Rules are been created properly.
Have any one faced such issue?

[image: image.png]

Any help would be really appreciated. Thank you!

Regards,
Sanjay Kumar

Re: Kubernetes stuck in the starting state

[Sanjay Kumar <ss...@gmail.com>]

Hi Wei,

We did re-register the k8 and it's started to work.


On Fri, Mar 17, 2023 at 6:34 PM Wei ZHOU <us...@gmail.com> wrote:

> Hi Sanjay,
>
> Good to know it. Do you use a different the systemvm template ?
>
> -Wei
>
>
> On Friday, 17 March 2023, Sanjay Kumar <ss...@gmail.com> wrote:
>
> > Hi Wei,
> >
> > Thank you so much for your help!
> >
> > We have successfully deploy the k8 cluster after re-added the template.
> >
> >
> > On Tue, Mar 14, 2023 at 8:46 PM Wei ZHOU <us...@gmail.com> wrote:
> >
> > > You should run these commands in the control node which has name "<cks
> > > name>-control-xxx"
> > >
> > > -Wei
> > >
> > >
> > >
> > > On Tue, 14 Mar 2023 at 15:01, Sanjay Kumar <ss...@gmail.com>
> > wrote:
> > >
> > > > Hi Wei,
> > > >
> > > > Thanks for quick dig the issue. is there any way to get fix the
> > dashboard
> > > > of cluster is showing in alerts state?
> > > >
> > > > cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get
> nodes
> > > > The connection to the server localhost:8080 was refused - did you
> > specify
> > > > the right host or port?
> > > > cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get pods
> > -A
> > > > The connection to the server localhost:8080 was refused - did you
> > specify
> > > > the right host or port?
> > > >
> > > >
> > > > On Tue, Mar 14, 2023 at 7:21 PM Wei ZHOU <us...@gmail.com>
> > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > You can do some checks in the control node. for example
> > > > >
> > > > > sudo /opt/bin/kubectl get nodes
> > > > > sudo /opt/bin/kubectl get pods -A
> > > > > sudo /opt/bin/kubectl describe pod kubernetes-dashboard-xxxx-yyyy
> -n
> > > > > kubernetes-dashboard
> > > > >
> > > > > I have no problem with creating cks clusters.
> > > > >
> > > > > -Wei
> > > > >
> > > > > On Tue, 14 Mar 2023 at 14:44, Sanjay Kumar <ss...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > Hi Wei,
> > > > > >
> > > > > > I am able to do ssh but cluster is showing alerts state.
> > > > > >
> > > > > > root@CS01:~# ssh -p 2222 -i
> > > /var/lib/cloudstack/management/.ssh/id_rsa
> > > > > > cloud@103.192.198.23
> > > > > > Linux k8s-test-cluster-control-186debc31f4 5.10.0-19-amd64 #1 SMP
> > > > Debian
> > > > > > 5.10.149-2 (2022-10-21) x86_64
> > > > > >
> > > > > > The programs included with the Debian GNU/Linux system are free
> > > > software;
> > > > > > the exact distribution terms for each program are described in
> the
> > > > > > individual files in /usr/share/doc/*/copyright.
> > > > > >
> > > > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > > > permitted by applicable law.
> > > > > > Last login: Tue Mar 14 13:40:18 2023 from 103.73.190.91
> > > > > >
> > > > > > The programs included with the Debian GNU/Linux system are free
> > > > software;
> > > > > > the exact distribution terms for each program are described in
> the
> > > > > > individual files in /usr/share/doc/*/copyright.
> > > > > >
> > > > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > > > permitted by applicable law.
> > > > > > cloud@k8s-test-cluster-control-186debc31f4:~$ exit
> > > > > > logout
> > > > > > Connection to 103.192.198.23 closed.
> > > > > > root@ACS01:~# ssh -p 2223 -i
> > > > /var/lib/cloudstack/management/.ssh/id_rsa
> > > > > > cloud@103.192.198.23
> > > > > > Linux k8s-test-cluster-node-186debc6be7 5.10.0-19-amd64 #1 SMP
> > Debian
> > > > > > 5.10.149-2 (2022-10-21) x86_64
> > > > > >
> > > > > > The programs included with the Debian GNU/Linux system are free
> > > > software;
> > > > > > the exact distribution terms for each program are described in
> the
> > > > > > individual files in /usr/share/doc/*/copyright.
> > > > > >
> > > > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > > > permitted by applicable law.
> > > > > > Last login: Tue Mar 14 13:39:56 2023 from 103.73.190.91
> > > > > >
> > > > > > The programs included with the Debian GNU/Linux system are free
> > > > software;
> > > > > > the exact distribution terms for each program are described in
> the
> > > > > > individual files in /usr/share/doc/*/copyright.
> > > > > >
> > > > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > > > permitted by applicable law.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tue, Mar 14, 2023 at 6:58 PM Wei ZHOU <us...@gmail.com>
> > > > wrote:
> > > > > >
> > > > > >> Hi,
> > > > > >>
> > > > > >>
> > > > > >> You can ssh into the cks control node from the cloudstack
> > management
> > > > > >> server.
> > > > > >>
> > > > > >> ssh -p 2222 -i <path of id_rsa> cloud@<public ip of cks
> cluster>
> > > > > >>
> > > > > >> Here is an example:
> > > > > >> # ssh -p 2222 -i /var/lib/cloudstack/management/.ssh/id_rsa
> > > > > >> cloud@10.0.52.195
> > > > > >>
> > > > > >> The path of id_rsa is different on the ubuntu/rhel management
> > > servers.
> > > > > >>
> > > > > >> You can also ssh into other cks nodes with different ports
> > starting
> > > > with
> > > > > >> 2223.
> > > > > >>
> > > > > >> -Wei
> > > > > >> -Wei
> > > > > >>
> > > > > >>
> > > > > >> On Tue, 14 Mar 2023 at 13:14, Sanjay Kumar <
> sslinuxpath@gmail.com
> > >
> > > > > wrote:
> > > > > >>
> > > > > >> > Hi Wei,
> > > > > >> >
> > > > > >> > Could you please help me out on this? How could we access the
> > > > console
> > > > > >> node
> > > > > >> > or any link that could be help for us.
> > > > > >> >
> > > > > >> >
> > > > > >> > On Tue, Mar 14, 2023 at 2:53 PM Wei ZHOU <
> ustcweizhou@gmail.com
> > >
> > > > > wrote:
> > > > > >> >
> > > > > >> > > It looks like the K8s API server is up, but the dashboard is
> > > not.
> > > > > >> > >
> > > > > >> > > Can you log into the console node (from the cloudstack
> > > management
> > > > > >> server)
> > > > > >> > > and check the status?
> > > > > >> > >
> > > > > >> > > -Wei
> > > > > >> > >
> > > > > >> > > On Tue, 14 Mar 2023 at 09:24, Sanjay Kumar <
> > > sslinuxpath@gmail.com
> > > > >
> > > > > >> > wrote:
> > > > > >> > >
> > > > > >> > > > Hi Wei,
> > > > > >> > > >
> > > > > >> > > > Thank you so much for your prompt response!
> > > > > >> > > >
> > > > > >> > > > Now the running kubernetes cluster is showing in alerts
> > state
> > > > > after
> > > > > >> 1
> > > > > >> > > > hours.
> > > > > >> > > >
> > > > > >> > > > cloudstack version: 4.17.2
> > > > > >> > > > kubernetes version : Kubernetes 1.23.3
> > > > > >> > > > network details (services, rules, etc) : Network name
> > > > > >> > > > (k8s-test-cluster-network)
> > > > > >> > > > vm details (template, offering, etc)
> > > > > >> > k8s-test-cluster-control-186debc31f4
> > > > > >> > > > , SystemVM Template (KVM) , Compute offering
> (Basic-1-Linux)
> > > > > >> > > >
> > > > > >> > > > Please access the logs from below link. It was long size
> > > that's
> > > > > why
> > > > > >> we
> > > > > >> > > > uploaded in drive.
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > > https://drive.google.com/drive/folders/1TC_hw_h_S_
> > TwqnT8fhoB4S3f2WHVlncx?usp=sharing
> > > > > >> > > >
> > > > > >> > > > [image: image.png]
> > > > > >> > > >
> > > > > >> > > > With Regards,
> > > > > >> > > > Sanjay Kumar
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > > On Tue, Mar 14, 2023 at 1:18 PM Wei ZHOU <
> > > ustcweizhou@gmail.com
> > > > >
> > > > > >> > wrote:
> > > > > >> > > >
> > > > > >> > > >> Hi Sanjay,
> > > > > >> > > >>
> > > > > >> > > >> To be clear, can you share the details like below ?
> > > > > >> > > >>
> > > > > >> > > >> cloudstack version
> > > > > >> > > >> kubernetes version
> > > > > >> > > >> network details (services, rules, etc)
> > > > > >> > > >> vm details (template, offering, etc)
> > > > > >> > > >> management logs
> > > > > >> > > >> agent logs
> > > > > >> > > >> anything else you can share.
> > > > > >> > > >>
> > > > > >> > > >>
> > > > > >> > > >> -Wei
> > > > > >> > > >> On Tue, 14 Mar 2023 at 08:26, Sanjay Kumar <
> > > > > sslinuxpath@gmail.com>
> > > > > >> > > wrote:
> > > > > >> > > >>
> > > > > >> > > >> > Hi Wei,
> > > > > >> > > >> >
> > > > > >> > > >> > Thanks for the quick response!
> > > > > >> > > >> >
> > > > > >> > > >> > We have checked the event as well and VM's, LB,
> Firewall
> > > > Rules
> > > > > >> have
> > > > > >> > > been
> > > > > >> > > >> > created properly but the cluster still showing starting
> > > > state.
> > > > > we
> > > > > >> > are
> > > > > >> > > >> > attaching the screenshot for the same.
> > > > > >> > > >> >
> > > > > >> > > >> >
> > > > > >> > > >> > On Tue, Mar 14, 2023 at 12:48 PM Wei ZHOU <
> > > > > ustcweizhou@gmail.com
> > > > > >> >
> > > > > >> > > >> wrote:
> > > > > >> > > >> >
> > > > > >> > > >> >> HI,
> > > > > >> > > >> >>
> > > > > >> > > >> >> Can you give more details  ?
> > > > > >> > > >> >>
> > > > > >> > > >> >> -Wei
> > > > > >> > > >> >>
> > > > > >> > > >> >> On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <
> > > > > >> sslinuxpath@gmail.com>
> > > > > >> > > >> wrote:
> > > > > >> > > >> >>
> > > > > >> > > >> >> > Hello All
> > > > > >> > > >> >> >
> > > > > >> > > >> >> > Kubernetes cluster is stucked in the 'Starting'
> state,
> > > > even
> > > > > >> > though
> > > > > >> > > >> VM's,
> > > > > >> > > >> >> > LB, Firewall Rules are been created properly.
> > > > > >> > > >> >> > Have any one faced such issue?
> > > > > >> > > >> >> >
> > > > > >> > > >> >> > [image: image.png]
> > > > > >> > > >> >> >
> > > > > >> > > >> >> > Any help would be really appreciated. Thank you!
> > > > > >> > > >> >> >
> > > > > >> > > >> >> > Regards,
> > > > > >> > > >> >> > Sanjay Kumar
> > > > > >> > > >> >> >
> > > > > >> > > >> >> >
> > > > > >> > > >> >> >
> > > > > >> > > >> >> >
> > > > > >> > > >> >> >
> > > > > >> > > >> >> >
> > > > > >> > > >> >>
> > > > > >> > > >> >
> > > > > >> > > >>
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Kubernetes stuck in the starting state

[Wei ZHOU <us...@gmail.com>]

Hi Sanjay,

Good to know it. Do you use a different the systemvm template ?

-Wei


On Friday, 17 March 2023, Sanjay Kumar <ss...@gmail.com> wrote:

> Hi Wei,
>
> Thank you so much for your help!
>
> We have successfully deploy the k8 cluster after re-added the template.
>
>
> On Tue, Mar 14, 2023 at 8:46 PM Wei ZHOU <us...@gmail.com> wrote:
>
> > You should run these commands in the control node which has name "<cks
> > name>-control-xxx"
> >
> > -Wei
> >
> >
> >
> > On Tue, 14 Mar 2023 at 15:01, Sanjay Kumar <ss...@gmail.com>
> wrote:
> >
> > > Hi Wei,
> > >
> > > Thanks for quick dig the issue. is there any way to get fix the
> dashboard
> > > of cluster is showing in alerts state?
> > >
> > > cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get nodes
> > > The connection to the server localhost:8080 was refused - did you
> specify
> > > the right host or port?
> > > cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get pods
> -A
> > > The connection to the server localhost:8080 was refused - did you
> specify
> > > the right host or port?
> > >
> > >
> > > On Tue, Mar 14, 2023 at 7:21 PM Wei ZHOU <us...@gmail.com>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > You can do some checks in the control node. for example
> > > >
> > > > sudo /opt/bin/kubectl get nodes
> > > > sudo /opt/bin/kubectl get pods -A
> > > > sudo /opt/bin/kubectl describe pod kubernetes-dashboard-xxxx-yyyy -n
> > > > kubernetes-dashboard
> > > >
> > > > I have no problem with creating cks clusters.
> > > >
> > > > -Wei
> > > >
> > > > On Tue, 14 Mar 2023 at 14:44, Sanjay Kumar <ss...@gmail.com>
> > > wrote:
> > > >
> > > > > Hi Wei,
> > > > >
> > > > > I am able to do ssh but cluster is showing alerts state.
> > > > >
> > > > > root@CS01:~# ssh -p 2222 -i
> > /var/lib/cloudstack/management/.ssh/id_rsa
> > > > > cloud@103.192.198.23
> > > > > Linux k8s-test-cluster-control-186debc31f4 5.10.0-19-amd64 #1 SMP
> > > Debian
> > > > > 5.10.149-2 (2022-10-21) x86_64
> > > > >
> > > > > The programs included with the Debian GNU/Linux system are free
> > > software;
> > > > > the exact distribution terms for each program are described in the
> > > > > individual files in /usr/share/doc/*/copyright.
> > > > >
> > > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > > permitted by applicable law.
> > > > > Last login: Tue Mar 14 13:40:18 2023 from 103.73.190.91
> > > > >
> > > > > The programs included with the Debian GNU/Linux system are free
> > > software;
> > > > > the exact distribution terms for each program are described in the
> > > > > individual files in /usr/share/doc/*/copyright.
> > > > >
> > > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > > permitted by applicable law.
> > > > > cloud@k8s-test-cluster-control-186debc31f4:~$ exit
> > > > > logout
> > > > > Connection to 103.192.198.23 closed.
> > > > > root@ACS01:~# ssh -p 2223 -i
> > > /var/lib/cloudstack/management/.ssh/id_rsa
> > > > > cloud@103.192.198.23
> > > > > Linux k8s-test-cluster-node-186debc6be7 5.10.0-19-amd64 #1 SMP
> Debian
> > > > > 5.10.149-2 (2022-10-21) x86_64
> > > > >
> > > > > The programs included with the Debian GNU/Linux system are free
> > > software;
> > > > > the exact distribution terms for each program are described in the
> > > > > individual files in /usr/share/doc/*/copyright.
> > > > >
> > > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > > permitted by applicable law.
> > > > > Last login: Tue Mar 14 13:39:56 2023 from 103.73.190.91
> > > > >
> > > > > The programs included with the Debian GNU/Linux system are free
> > > software;
> > > > > the exact distribution terms for each program are described in the
> > > > > individual files in /usr/share/doc/*/copyright.
> > > > >
> > > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > > permitted by applicable law.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Mar 14, 2023 at 6:58 PM Wei ZHOU <us...@gmail.com>
> > > wrote:
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >>
> > > > >> You can ssh into the cks control node from the cloudstack
> management
> > > > >> server.
> > > > >>
> > > > >> ssh -p 2222 -i <path of id_rsa> cloud@<public ip of cks cluster>
> > > > >>
> > > > >> Here is an example:
> > > > >> # ssh -p 2222 -i /var/lib/cloudstack/management/.ssh/id_rsa
> > > > >> cloud@10.0.52.195
> > > > >>
> > > > >> The path of id_rsa is different on the ubuntu/rhel management
> > servers.
> > > > >>
> > > > >> You can also ssh into other cks nodes with different ports
> starting
> > > with
> > > > >> 2223.
> > > > >>
> > > > >> -Wei
> > > > >> -Wei
> > > > >>
> > > > >>
> > > > >> On Tue, 14 Mar 2023 at 13:14, Sanjay Kumar <sslinuxpath@gmail.com
> >
> > > > wrote:
> > > > >>
> > > > >> > Hi Wei,
> > > > >> >
> > > > >> > Could you please help me out on this? How could we access the
> > > console
> > > > >> node
> > > > >> > or any link that could be help for us.
> > > > >> >
> > > > >> >
> > > > >> > On Tue, Mar 14, 2023 at 2:53 PM Wei ZHOU <ustcweizhou@gmail.com
> >
> > > > wrote:
> > > > >> >
> > > > >> > > It looks like the K8s API server is up, but the dashboard is
> > not.
> > > > >> > >
> > > > >> > > Can you log into the console node (from the cloudstack
> > management
> > > > >> server)
> > > > >> > > and check the status?
> > > > >> > >
> > > > >> > > -Wei
> > > > >> > >
> > > > >> > > On Tue, 14 Mar 2023 at 09:24, Sanjay Kumar <
> > sslinuxpath@gmail.com
> > > >
> > > > >> > wrote:
> > > > >> > >
> > > > >> > > > Hi Wei,
> > > > >> > > >
> > > > >> > > > Thank you so much for your prompt response!
> > > > >> > > >
> > > > >> > > > Now the running kubernetes cluster is showing in alerts
> state
> > > > after
> > > > >> 1
> > > > >> > > > hours.
> > > > >> > > >
> > > > >> > > > cloudstack version: 4.17.2
> > > > >> > > > kubernetes version : Kubernetes 1.23.3
> > > > >> > > > network details (services, rules, etc) : Network name
> > > > >> > > > (k8s-test-cluster-network)
> > > > >> > > > vm details (template, offering, etc)
> > > > >> > k8s-test-cluster-control-186debc31f4
> > > > >> > > > , SystemVM Template (KVM) , Compute offering (Basic-1-Linux)
> > > > >> > > >
> > > > >> > > > Please access the logs from below link. It was long size
> > that's
> > > > why
> > > > >> we
> > > > >> > > > uploaded in drive.
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> > https://drive.google.com/drive/folders/1TC_hw_h_S_
> TwqnT8fhoB4S3f2WHVlncx?usp=sharing
> > > > >> > > >
> > > > >> > > > [image: image.png]
> > > > >> > > >
> > > > >> > > > With Regards,
> > > > >> > > > Sanjay Kumar
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > On Tue, Mar 14, 2023 at 1:18 PM Wei ZHOU <
> > ustcweizhou@gmail.com
> > > >
> > > > >> > wrote:
> > > > >> > > >
> > > > >> > > >> Hi Sanjay,
> > > > >> > > >>
> > > > >> > > >> To be clear, can you share the details like below ?
> > > > >> > > >>
> > > > >> > > >> cloudstack version
> > > > >> > > >> kubernetes version
> > > > >> > > >> network details (services, rules, etc)
> > > > >> > > >> vm details (template, offering, etc)
> > > > >> > > >> management logs
> > > > >> > > >> agent logs
> > > > >> > > >> anything else you can share.
> > > > >> > > >>
> > > > >> > > >>
> > > > >> > > >> -Wei
> > > > >> > > >> On Tue, 14 Mar 2023 at 08:26, Sanjay Kumar <
> > > > sslinuxpath@gmail.com>
> > > > >> > > wrote:
> > > > >> > > >>
> > > > >> > > >> > Hi Wei,
> > > > >> > > >> >
> > > > >> > > >> > Thanks for the quick response!
> > > > >> > > >> >
> > > > >> > > >> > We have checked the event as well and VM's, LB, Firewall
> > > Rules
> > > > >> have
> > > > >> > > been
> > > > >> > > >> > created properly but the cluster still showing starting
> > > state.
> > > > we
> > > > >> > are
> > > > >> > > >> > attaching the screenshot for the same.
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> > On Tue, Mar 14, 2023 at 12:48 PM Wei ZHOU <
> > > > ustcweizhou@gmail.com
> > > > >> >
> > > > >> > > >> wrote:
> > > > >> > > >> >
> > > > >> > > >> >> HI,
> > > > >> > > >> >>
> > > > >> > > >> >> Can you give more details  ?
> > > > >> > > >> >>
> > > > >> > > >> >> -Wei
> > > > >> > > >> >>
> > > > >> > > >> >> On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <
> > > > >> sslinuxpath@gmail.com>
> > > > >> > > >> wrote:
> > > > >> > > >> >>
> > > > >> > > >> >> > Hello All
> > > > >> > > >> >> >
> > > > >> > > >> >> > Kubernetes cluster is stucked in the 'Starting' state,
> > > even
> > > > >> > though
> > > > >> > > >> VM's,
> > > > >> > > >> >> > LB, Firewall Rules are been created properly.
> > > > >> > > >> >> > Have any one faced such issue?
> > > > >> > > >> >> >
> > > > >> > > >> >> > [image: image.png]
> > > > >> > > >> >> >
> > > > >> > > >> >> > Any help would be really appreciated. Thank you!
> > > > >> > > >> >> >
> > > > >> > > >> >> > Regards,
> > > > >> > > >> >> > Sanjay Kumar
> > > > >> > > >> >> >
> > > > >> > > >> >> >
> > > > >> > > >> >> >
> > > > >> > > >> >> >
> > > > >> > > >> >> >
> > > > >> > > >> >> >
> > > > >> > > >> >>
> > > > >> > > >> >
> > > > >> > > >>
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > >
> > >
> >
>

Re: Kubernetes stuck in the starting state

[Sanjay Kumar <ss...@gmail.com>]

Hi Wei,

Thank you so much for your help!

We have successfully deploy the k8 cluster after re-added the template.


On Tue, Mar 14, 2023 at 8:46 PM Wei ZHOU <us...@gmail.com> wrote:

> You should run these commands in the control node which has name "<cks
> name>-control-xxx"
>
> -Wei
>
>
>
> On Tue, 14 Mar 2023 at 15:01, Sanjay Kumar <ss...@gmail.com> wrote:
>
> > Hi Wei,
> >
> > Thanks for quick dig the issue. is there any way to get fix the dashboard
> > of cluster is showing in alerts state?
> >
> > cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get nodes
> > The connection to the server localhost:8080 was refused - did you specify
> > the right host or port?
> > cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get pods -A
> > The connection to the server localhost:8080 was refused - did you specify
> > the right host or port?
> >
> >
> > On Tue, Mar 14, 2023 at 7:21 PM Wei ZHOU <us...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > You can do some checks in the control node. for example
> > >
> > > sudo /opt/bin/kubectl get nodes
> > > sudo /opt/bin/kubectl get pods -A
> > > sudo /opt/bin/kubectl describe pod kubernetes-dashboard-xxxx-yyyy -n
> > > kubernetes-dashboard
> > >
> > > I have no problem with creating cks clusters.
> > >
> > > -Wei
> > >
> > > On Tue, 14 Mar 2023 at 14:44, Sanjay Kumar <ss...@gmail.com>
> > wrote:
> > >
> > > > Hi Wei,
> > > >
> > > > I am able to do ssh but cluster is showing alerts state.
> > > >
> > > > root@CS01:~# ssh -p 2222 -i
> /var/lib/cloudstack/management/.ssh/id_rsa
> > > > cloud@103.192.198.23
> > > > Linux k8s-test-cluster-control-186debc31f4 5.10.0-19-amd64 #1 SMP
> > Debian
> > > > 5.10.149-2 (2022-10-21) x86_64
> > > >
> > > > The programs included with the Debian GNU/Linux system are free
> > software;
> > > > the exact distribution terms for each program are described in the
> > > > individual files in /usr/share/doc/*/copyright.
> > > >
> > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > permitted by applicable law.
> > > > Last login: Tue Mar 14 13:40:18 2023 from 103.73.190.91
> > > >
> > > > The programs included with the Debian GNU/Linux system are free
> > software;
> > > > the exact distribution terms for each program are described in the
> > > > individual files in /usr/share/doc/*/copyright.
> > > >
> > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > permitted by applicable law.
> > > > cloud@k8s-test-cluster-control-186debc31f4:~$ exit
> > > > logout
> > > > Connection to 103.192.198.23 closed.
> > > > root@ACS01:~# ssh -p 2223 -i
> > /var/lib/cloudstack/management/.ssh/id_rsa
> > > > cloud@103.192.198.23
> > > > Linux k8s-test-cluster-node-186debc6be7 5.10.0-19-amd64 #1 SMP Debian
> > > > 5.10.149-2 (2022-10-21) x86_64
> > > >
> > > > The programs included with the Debian GNU/Linux system are free
> > software;
> > > > the exact distribution terms for each program are described in the
> > > > individual files in /usr/share/doc/*/copyright.
> > > >
> > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > permitted by applicable law.
> > > > Last login: Tue Mar 14 13:39:56 2023 from 103.73.190.91
> > > >
> > > > The programs included with the Debian GNU/Linux system are free
> > software;
> > > > the exact distribution terms for each program are described in the
> > > > individual files in /usr/share/doc/*/copyright.
> > > >
> > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > permitted by applicable law.
> > > >
> > > >
> > > >
> > > >
> > > > On Tue, Mar 14, 2023 at 6:58 PM Wei ZHOU <us...@gmail.com>
> > wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >>
> > > >> You can ssh into the cks control node from the cloudstack management
> > > >> server.
> > > >>
> > > >> ssh -p 2222 -i <path of id_rsa> cloud@<public ip of cks cluster>
> > > >>
> > > >> Here is an example:
> > > >> # ssh -p 2222 -i /var/lib/cloudstack/management/.ssh/id_rsa
> > > >> cloud@10.0.52.195
> > > >>
> > > >> The path of id_rsa is different on the ubuntu/rhel management
> servers.
> > > >>
> > > >> You can also ssh into other cks nodes with different ports starting
> > with
> > > >> 2223.
> > > >>
> > > >> -Wei
> > > >> -Wei
> > > >>
> > > >>
> > > >> On Tue, 14 Mar 2023 at 13:14, Sanjay Kumar <ss...@gmail.com>
> > > wrote:
> > > >>
> > > >> > Hi Wei,
> > > >> >
> > > >> > Could you please help me out on this? How could we access the
> > console
> > > >> node
> > > >> > or any link that could be help for us.
> > > >> >
> > > >> >
> > > >> > On Tue, Mar 14, 2023 at 2:53 PM Wei ZHOU <us...@gmail.com>
> > > wrote:
> > > >> >
> > > >> > > It looks like the K8s API server is up, but the dashboard is
> not.
> > > >> > >
> > > >> > > Can you log into the console node (from the cloudstack
> management
> > > >> server)
> > > >> > > and check the status?
> > > >> > >
> > > >> > > -Wei
> > > >> > >
> > > >> > > On Tue, 14 Mar 2023 at 09:24, Sanjay Kumar <
> sslinuxpath@gmail.com
> > >
> > > >> > wrote:
> > > >> > >
> > > >> > > > Hi Wei,
> > > >> > > >
> > > >> > > > Thank you so much for your prompt response!
> > > >> > > >
> > > >> > > > Now the running kubernetes cluster is showing in alerts state
> > > after
> > > >> 1
> > > >> > > > hours.
> > > >> > > >
> > > >> > > > cloudstack version: 4.17.2
> > > >> > > > kubernetes version : Kubernetes 1.23.3
> > > >> > > > network details (services, rules, etc) : Network name
> > > >> > > > (k8s-test-cluster-network)
> > > >> > > > vm details (template, offering, etc)
> > > >> > k8s-test-cluster-control-186debc31f4
> > > >> > > > , SystemVM Template (KVM) , Compute offering (Basic-1-Linux)
> > > >> > > >
> > > >> > > > Please access the logs from below link. It was long size
> that's
> > > why
> > > >> we
> > > >> > > > uploaded in drive.
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://drive.google.com/drive/folders/1TC_hw_h_S_TwqnT8fhoB4S3f2WHVlncx?usp=sharing
> > > >> > > >
> > > >> > > > [image: image.png]
> > > >> > > >
> > > >> > > > With Regards,
> > > >> > > > Sanjay Kumar
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > > On Tue, Mar 14, 2023 at 1:18 PM Wei ZHOU <
> ustcweizhou@gmail.com
> > >
> > > >> > wrote:
> > > >> > > >
> > > >> > > >> Hi Sanjay,
> > > >> > > >>
> > > >> > > >> To be clear, can you share the details like below ?
> > > >> > > >>
> > > >> > > >> cloudstack version
> > > >> > > >> kubernetes version
> > > >> > > >> network details (services, rules, etc)
> > > >> > > >> vm details (template, offering, etc)
> > > >> > > >> management logs
> > > >> > > >> agent logs
> > > >> > > >> anything else you can share.
> > > >> > > >>
> > > >> > > >>
> > > >> > > >> -Wei
> > > >> > > >> On Tue, 14 Mar 2023 at 08:26, Sanjay Kumar <
> > > sslinuxpath@gmail.com>
> > > >> > > wrote:
> > > >> > > >>
> > > >> > > >> > Hi Wei,
> > > >> > > >> >
> > > >> > > >> > Thanks for the quick response!
> > > >> > > >> >
> > > >> > > >> > We have checked the event as well and VM's, LB, Firewall
> > Rules
> > > >> have
> > > >> > > been
> > > >> > > >> > created properly but the cluster still showing starting
> > state.
> > > we
> > > >> > are
> > > >> > > >> > attaching the screenshot for the same.
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> > On Tue, Mar 14, 2023 at 12:48 PM Wei ZHOU <
> > > ustcweizhou@gmail.com
> > > >> >
> > > >> > > >> wrote:
> > > >> > > >> >
> > > >> > > >> >> HI,
> > > >> > > >> >>
> > > >> > > >> >> Can you give more details  ?
> > > >> > > >> >>
> > > >> > > >> >> -Wei
> > > >> > > >> >>
> > > >> > > >> >> On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <
> > > >> sslinuxpath@gmail.com>
> > > >> > > >> wrote:
> > > >> > > >> >>
> > > >> > > >> >> > Hello All
> > > >> > > >> >> >
> > > >> > > >> >> > Kubernetes cluster is stucked in the 'Starting' state,
> > even
> > > >> > though
> > > >> > > >> VM's,
> > > >> > > >> >> > LB, Firewall Rules are been created properly.
> > > >> > > >> >> > Have any one faced such issue?
> > > >> > > >> >> >
> > > >> > > >> >> > [image: image.png]
> > > >> > > >> >> >
> > > >> > > >> >> > Any help would be really appreciated. Thank you!
> > > >> > > >> >> >
> > > >> > > >> >> > Regards,
> > > >> > > >> >> > Sanjay Kumar
> > > >> > > >> >> >
> > > >> > > >> >> >
> > > >> > > >> >> >
> > > >> > > >> >> >
> > > >> > > >> >> >
> > > >> > > >> >> >
> > > >> > > >> >>
> > > >> > > >> >
> > > >> > > >>
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > >
> >
>

Re: Kubernetes stuck in the starting state

[Wei ZHOU <us...@gmail.com>]

Hi,

You can use sudo. "cloud" user has sudo priviledge.

-Wei
On Friday, 17 March 2023, 김재종 <cb...@naver.com> wrote:

> Hi  Wei ZHOU
> My name is jaejong.
> Yes source nat IP is VR ip.
> My source nat IP is 10.0.35.12.
>
> i can login to my control and node as follows:
> ssh -p 2223 -v -i /var/cloudstack/management/.ssh/id_rsa cloud@10.0.35.12
>
> Now how can i  get root role?
> attempt by root not allowed due to Permission denied.
> I'd like to edit /etc/sysctl.conf to add vm.max_map_count=262144.
> opensearch require this option.
>
> thank you!!
> by jaejong
>
> -----Original Message-----
> From: "Wei ZHOU"<us...@gmail.com>
> To: <us...@cloudstack.apache.org>;
> Cc:
> Sent: 2023-03-17 (금) 19:06:30 (GMT+09:00)
> Subject: Re: Kubernetes stuck in the starting state
>
> Hi,
>
> What's the output of the command ? You can add "-v" to the command to get
> more information.
>
> Each cks cluster has a public IP which is normally the source nat IP of the
> isolated network.
> It is expected that the Public IP is Allocated.
>
> -Wei
>
>
> On Fri, 17 Mar 2023 at 10:58, 김재종 <cb...@naver.com> wrote:
>
> > ssh -p 2222 -i <path of id_rsa> cloud@<public ip of cks cluster>
> >
> > Is there public ip of cks cluster? then is id of VR?
> > I also have the problem that can not acess to cks nodes.
> >
> > My isolated network state is Implemented. but The public ip state is
> > Allocated
> > Indicating the network configuration is in allocated but not setup.
> > VR with this public ip is Running state.
> > Why is the same ip state defferent?
> >
> >
> >
> >
> >
> >
> > -----Original Message-----
> > From: "Wei ZHOU"<us...@gmail.com>
> > To: <us...@cloudstack.apache.org>;
> > Cc:
> > Sent: 2023-03-15 (수) 00:16:03 (GMT+09:00)
> > Subject: Re: Kubernetes stuck in the starting state
> >
> > You should run these commands in the control node which has name "<cks
> > name>-control-xxx"
> >
> > -Wei
> >
> >
> >
> > On Tue, 14 Mar 2023 at 15:01, Sanjay Kumar <ss...@gmail.com>
> wrote:
> >
> > > Hi Wei,
> > >
> > > Thanks for quick dig the issue. is there any way to get fix the
> dashboard
> > > of cluster is showing in alerts state?
> > >
> > > cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get nodes
> > > The connection to the server localhost:8080 was refused - did you
> specify
> > > the right host or port?
> > > cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get pods
> -A
> > > The connection to the server localhost:8080 was refused - did you
> specify
> > > the right host or port?
> > >
> > >
> > > On Tue, Mar 14, 2023 at 7:21 PM Wei ZHOU <us...@gmail.com>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > You can do some checks in the control node. for example
> > > >
> > > > sudo /opt/bin/kubectl get nodes
> > > > sudo /opt/bin/kubectl get pods -A
> > > > sudo /opt/bin/kubectl describe pod kubernetes-dashboard-xxxx-yyyy -n
> > > > kubernetes-dashboard
> > > >
> > > > I have no problem with creating cks clusters.
> > > >
> > > > -Wei
> > > >
> > > > On Tue, 14 Mar 2023 at 14:44, Sanjay Kumar <ss...@gmail.com>
> > > wrote:
> > > >
> > > > > Hi Wei,
> > > > >
> > > > > I am able to do ssh but cluster is showing alerts state.
> > > > >
> > > > > root@CS01:~# ssh -p 2222 -i
> > /var/lib/cloudstack/management/.ssh/id_rsa
> > > > > cloud@103.192.198.23
> > > > > Linux k8s-test-cluster-control-186debc31f4 5.10.0-19-amd64 #1 SMP
> > > Debian
> > > > > 5.10.149-2 (2022-10-21) x86_64
> > > > >
> > > > > The programs included with the Debian GNU/Linux system are free
> > > software;
> > > > > the exact distribution terms for each program are described in the
> > > > > individual files in /usr/share/doc/*/copyright.
> > > > >
> > > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > > permitted by applicable law.
> > > > > Last login: Tue Mar 14 13:40:18 2023 from 103.73.190.91
> > > > >
> > > > > The programs included with the Debian GNU/Linux system are free
> > > software;
> > > > > the exact distribution terms for each program are described in the
> > > > > individual files in /usr/share/doc/*/copyright.
> > > > >
> > > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > > permitted by applicable law.
> > > > > cloud@k8s-test-cluster-control-186debc31f4:~$ exit
> > > > > logout
> > > > > Connection to 103.192.198.23 closed.
> > > > > root@ACS01:~# ssh -p 2223 -i
> > > /var/lib/cloudstack/management/.ssh/id_rsa
> > > > > cloud@103.192.198.23
> > > > > Linux k8s-test-cluster-node-186debc6be7 5.10.0-19-amd64 #1 SMP
> Debian
> > > > > 5.10.149-2 (2022-10-21) x86_64
> > > > >
> > > > > The programs included with the Debian GNU/Linux system are free
> > > software;
> > > > > the exact distribution terms for each program are described in the
> > > > > individual files in /usr/share/doc/*/copyright.
> > > > >
> > > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > > permitted by applicable law.
> > > > > Last login: Tue Mar 14 13:39:56 2023 from 103.73.190.91
> > > > >
> > > > > The programs included with the Debian GNU/Linux system are free
> > > software;
> > > > > the exact distribution terms for each program are described in the
> > > > > individual files in /usr/share/doc/*/copyright.
> > > > >
> > > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > > permitted by applicable law.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Mar 14, 2023 at 6:58 PM Wei ZHOU <us...@gmail.com>
> > > wrote:
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >>
> > > > >> You can ssh into the cks control node from the cloudstack
> management
> > > > >> server.
> > > > >>
> > > > >> ssh -p 2222 -i <path of id_rsa> cloud@<public ip of cks cluster>
> > > > >>
> > > > >> Here is an example:
> > > > >> # ssh -p 2222 -i /var/lib/cloudstack/management/.ssh/id_rsa
> > > > >> cloud@10.0.52.195
> > > > >>
> > > > >> The path of id_rsa is different on the ubuntu/rhel management
> > servers.
> > > > >>
> > > > >> You can also ssh into other cks nodes with different ports
> starting
> > > with
> > > > >> 2223.
> > > > >>
> > > > >> -Wei
> > > > >> -Wei
> > > > >>
> > > > >>
> > > > >> On Tue, 14 Mar 2023 at 13:14, Sanjay Kumar <sslinuxpath@gmail.com
> >
> > > > wrote:
> > > > >>
> > > > >> > Hi Wei,
> > > > >> >
> > > > >> > Could you please help me out on this? How could we access the
> > > console
> > > > >> node
> > > > >> > or any link that could be help for us.
> > > > >> >
> > > > >> >
> > > > >> > On Tue, Mar 14, 2023 at 2:53 PM Wei ZHOU <ustcweizhou@gmail.com
> >
> > > > wrote:
> > > > >> >
> > > > >> > > It looks like the K8s API server is up, but the dashboard is
> > not.
> > > > >> > >
> > > > >> > > Can you log into the console node (from the cloudstack
> > management
> > > > >> server)
> > > > >> > > and check the status?
> > > > >> > >
> > > > >> > > -Wei
> > > > >> > >
> > > > >> > > On Tue, 14 Mar 2023 at 09:24, Sanjay Kumar <
> > sslinuxpath@gmail.com
> > > >
> > > > >> > wrote:
> > > > >> > >
> > > > >> > > > Hi Wei,
> > > > >> > > >
> > > > >> > > > Thank you so much for your prompt response!
> > > > >> > > >
> > > > >> > > > Now the running kubernetes cluster is showing in alerts
> state
> > > > after
> > > > >> 1
> > > > >> > > > hours.
> > > > >> > > >
> > > > >> > > > cloudstack version: 4.17.2
> > > > >> > > > kubernetes version : Kubernetes 1.23.3
> > > > >> > > > network details (services, rules, etc) : Network name
> > > > >> > > > (k8s-test-cluster-network)
> > > > >> > > > vm details (template, offering, etc)
> > > > >> > k8s-test-cluster-control-186debc31f4
> > > > >> > > > , SystemVM Template (KVM) , Compute offering (Basic-1-Linux)
> > > > >> > > >
> > > > >> > > > Please access the logs from below link. It was long size
> > that's
> > > > why
> > > > >> we
> > > > >> > > > uploaded in drive.
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> > https://drive.google.com/drive/folders/1TC_hw_h_S_
> TwqnT8fhoB4S3f2WHVlncx?usp=sharing
> > > > >> > > >
> > > > >> > > > [image: image.png]
> > > > >> > > >
> > > > >> > > > With Regards,
> > > > >> > > > Sanjay Kumar
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > On Tue, Mar 14, 2023 at 1:18 PM Wei ZHOU <
> > ustcweizhou@gmail.com
> > > >
> > > > >> > wrote:
> > > > >> > > >
> > > > >> > > >> Hi Sanjay,
> > > > >> > > >>
> > > > >> > > >> To be clear, can you share the details like below ?
> > > > >> > > >>
> > > > >> > > >> cloudstack version
> > > > >> > > >> kubernetes version
> > > > >> > > >> network details (services, rules, etc)
> > > > >> > > >> vm details (template, offering, etc)
> > > > >> > > >> management logs
> > > > >> > > >> agent logs
> > > > >> > > >> anything else you can share.
> > > > >> > > >>
> > > > >> > > >>
> > > > >> > > >> -Wei
> > > > >> > > >> On Tue, 14 Mar 2023 at 08:26, Sanjay Kumar <
> > > > sslinuxpath@gmail.com>
> > > > >> > > wrote:
> > > > >> > > >>
> > > > >> > > >> > Hi Wei,
> > > > >> > > >> >
> > > > >> > > >> > Thanks for the quick response!
> > > > >> > > >> >
> > > > >> > > >> > We have checked the event as well and VM's, LB, Firewall
> > > Rules
> > > > >> have
> > > > >> > > been
> > > > >> > > >> > created properly but the cluster still showing starting
> > > state.
> > > > we
> > > > >> > are
> > > > >> > > >> > attaching the screenshot for the same.
> > > > >> > > >> >
> > > > >> > > >> >
> > > > >> > > >> > On Tue, Mar 14, 2023 at 12:48 PM Wei ZHOU <
> > > > ustcweizhou@gmail.com
> > > > >> >
> > > > >> > > >> wrote:
> > > > >> > > >> >
> > > > >> > > >> >> HI,
> > > > >> > > >> >>
> > > > >> > > >> >> Can you give more details  ?
> > > > >> > > >> >>
> > > > >> > > >> >> -Wei
> > > > >> > > >> >>
> > > > >> > > >> >> On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <
> > > > >> sslinuxpath@gmail.com>
> > > > >> > > >> wrote:
> > > > >> > > >> >>
> > > > >> > > >> >> > Hello All
> > > > >> > > >> >> >
> > > > >> > > >> >> > Kubernetes cluster is stucked in the 'Starting' state,
> > > even
> > > > >> > though
> > > > >> > > >> VM's,
> > > > >> > > >> >> > LB, Firewall Rules are been created properly.
> > > > >> > > >> >> > Have any one faced such issue?
> > > > >> > > >> >> >
> > > > >> > > >> >> > [image: image.png]
> > > > >> > > >> >> >
> > > > >> > > >> >> > Any help would be really appreciated. Thank you!
> > > > >> > > >> >> >
> > > > >> > > >> >> > Regards,
> > > > >> > > >> >> > Sanjay Kumar
> > > > >> > > >> >> >
> > > > >> > > >> >> >
> > > > >> > > >> >> >
> > > > >> > > >> >> >
> > > > >> > > >> >> >
> > > > >> > > >> >> >
> > > > >> > > >> >>
> > > > >> > > >> >
> > > > >> > > >>
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > >
> > >
> >
> >
>
>

Re: Kubernetes stuck in the starting state

[김재종 <cb...@naver.com>]

Hi  Wei ZHOU
My name is jaejong.
Yes source nat IP is VR ip.
My source nat IP is 10.0.35.12.

i can login to my control and node as follows:
ssh -p 2223 -v -i /var/cloudstack/management/.ssh/id_rsa cloud@10.0.35.12 

Now how can i  get root role?
attempt by root not allowed due to Permission denied.
I'd like to edit /etc/sysctl.conf to add vm.max_map_count=262144.
opensearch require this option.

thank you!! 
by jaejong

-----Original Message-----
From: "Wei ZHOU"<us...@gmail.com>
To: <us...@cloudstack.apache.org>;
Cc:
Sent: 2023-03-17 (금) 19:06:30 (GMT+09:00)
Subject: Re: Kubernetes stuck in the starting state

Hi,

What's the output of the command ? You can add "-v" to the command to get
more information.

Each cks cluster has a public IP which is normally the source nat IP of the
isolated network.
It is expected that the Public IP is Allocated.

-Wei


On Fri, 17 Mar 2023 at 10:58, 김재종 <cb...@naver.com> wrote:

> ssh -p 2222 -i <path of id_rsa> cloud@<public ip of cks cluster>
>
> Is there public ip of cks cluster? then is id of VR?
> I also have the problem that can not acess to cks nodes.
>
> My isolated network state is Implemented. but The public ip state is
> Allocated
> Indicating the network configuration is in allocated but not setup.
> VR with this public ip is Running state.
> Why is the same ip state defferent?
>
>
>
>
>
>
> -----Original Message-----
> From: "Wei ZHOU"<us...@gmail.com>
> To: <us...@cloudstack.apache.org>;
> Cc:
> Sent: 2023-03-15 (수) 00:16:03 (GMT+09:00)
> Subject: Re: Kubernetes stuck in the starting state
>
> You should run these commands in the control node which has name "<cks
> name>-control-xxx"
>
> -Wei
>
>
>
> On Tue, 14 Mar 2023 at 15:01, Sanjay Kumar <ss...@gmail.com> wrote:
>
> > Hi Wei,
> >
> > Thanks for quick dig the issue. is there any way to get fix the dashboard
> > of cluster is showing in alerts state?
> >
> > cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get nodes
> > The connection to the server localhost:8080 was refused - did you specify
> > the right host or port?
> > cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get pods -A
> > The connection to the server localhost:8080 was refused - did you specify
> > the right host or port?
> >
> >
> > On Tue, Mar 14, 2023 at 7:21 PM Wei ZHOU <us...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > You can do some checks in the control node. for example
> > >
> > > sudo /opt/bin/kubectl get nodes
> > > sudo /opt/bin/kubectl get pods -A
> > > sudo /opt/bin/kubectl describe pod kubernetes-dashboard-xxxx-yyyy -n
> > > kubernetes-dashboard
> > >
> > > I have no problem with creating cks clusters.
> > >
> > > -Wei
> > >
> > > On Tue, 14 Mar 2023 at 14:44, Sanjay Kumar <ss...@gmail.com>
> > wrote:
> > >
> > > > Hi Wei,
> > > >
> > > > I am able to do ssh but cluster is showing alerts state.
> > > >
> > > > root@CS01:~# ssh -p 2222 -i
> /var/lib/cloudstack/management/.ssh/id_rsa
> > > > cloud@103.192.198.23
> > > > Linux k8s-test-cluster-control-186debc31f4 5.10.0-19-amd64 #1 SMP
> > Debian
> > > > 5.10.149-2 (2022-10-21) x86_64
> > > >
> > > > The programs included with the Debian GNU/Linux system are free
> > software;
> > > > the exact distribution terms for each program are described in the
> > > > individual files in /usr/share/doc/*/copyright.
> > > >
> > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > permitted by applicable law.
> > > > Last login: Tue Mar 14 13:40:18 2023 from 103.73.190.91
> > > >
> > > > The programs included with the Debian GNU/Linux system are free
> > software;
> > > > the exact distribution terms for each program are described in the
> > > > individual files in /usr/share/doc/*/copyright.
> > > >
> > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > permitted by applicable law.
> > > > cloud@k8s-test-cluster-control-186debc31f4:~$ exit
> > > > logout
> > > > Connection to 103.192.198.23 closed.
> > > > root@ACS01:~# ssh -p 2223 -i
> > /var/lib/cloudstack/management/.ssh/id_rsa
> > > > cloud@103.192.198.23
> > > > Linux k8s-test-cluster-node-186debc6be7 5.10.0-19-amd64 #1 SMP Debian
> > > > 5.10.149-2 (2022-10-21) x86_64
> > > >
> > > > The programs included with the Debian GNU/Linux system are free
> > software;
> > > > the exact distribution terms for each program are described in the
> > > > individual files in /usr/share/doc/*/copyright.
> > > >
> > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > permitted by applicable law.
> > > > Last login: Tue Mar 14 13:39:56 2023 from 103.73.190.91
> > > >
> > > > The programs included with the Debian GNU/Linux system are free
> > software;
> > > > the exact distribution terms for each program are described in the
> > > > individual files in /usr/share/doc/*/copyright.
> > > >
> > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > permitted by applicable law.
> > > >
> > > >
> > > >
> > > >
> > > > On Tue, Mar 14, 2023 at 6:58 PM Wei ZHOU <us...@gmail.com>
> > wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >>
> > > >> You can ssh into the cks control node from the cloudstack management
> > > >> server.
> > > >>
> > > >> ssh -p 2222 -i <path of id_rsa> cloud@<public ip of cks cluster>
> > > >>
> > > >> Here is an example:
> > > >> # ssh -p 2222 -i /var/lib/cloudstack/management/.ssh/id_rsa
> > > >> cloud@10.0.52.195
> > > >>
> > > >> The path of id_rsa is different on the ubuntu/rhel management
> servers.
> > > >>
> > > >> You can also ssh into other cks nodes with different ports starting
> > with
> > > >> 2223.
> > > >>
> > > >> -Wei
> > > >> -Wei
> > > >>
> > > >>
> > > >> On Tue, 14 Mar 2023 at 13:14, Sanjay Kumar <ss...@gmail.com>
> > > wrote:
> > > >>
> > > >> > Hi Wei,
> > > >> >
> > > >> > Could you please help me out on this? How could we access the
> > console
> > > >> node
> > > >> > or any link that could be help for us.
> > > >> >
> > > >> >
> > > >> > On Tue, Mar 14, 2023 at 2:53 PM Wei ZHOU <us...@gmail.com>
> > > wrote:
> > > >> >
> > > >> > > It looks like the K8s API server is up, but the dashboard is
> not.
> > > >> > >
> > > >> > > Can you log into the console node (from the cloudstack
> management
> > > >> server)
> > > >> > > and check the status?
> > > >> > >
> > > >> > > -Wei
> > > >> > >
> > > >> > > On Tue, 14 Mar 2023 at 09:24, Sanjay Kumar <
> sslinuxpath@gmail.com
> > >
> > > >> > wrote:
> > > >> > >
> > > >> > > > Hi Wei,
> > > >> > > >
> > > >> > > > Thank you so much for your prompt response!
> > > >> > > >
> > > >> > > > Now the running kubernetes cluster is showing in alerts state
> > > after
> > > >> 1
> > > >> > > > hours.
> > > >> > > >
> > > >> > > > cloudstack version: 4.17.2
> > > >> > > > kubernetes version : Kubernetes 1.23.3
> > > >> > > > network details (services, rules, etc) : Network name
> > > >> > > > (k8s-test-cluster-network)
> > > >> > > > vm details (template, offering, etc)
> > > >> > k8s-test-cluster-control-186debc31f4
> > > >> > > > , SystemVM Template (KVM) , Compute offering (Basic-1-Linux)
> > > >> > > >
> > > >> > > > Please access the logs from below link. It was long size
> that's
> > > why
> > > >> we
> > > >> > > > uploaded in drive.
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://drive.google.com/drive/folders/1TC_hw_h_S_TwqnT8fhoB4S3f2WHVlncx?usp=sharing
> > > >> > > >
> > > >> > > > [image: image.png]
> > > >> > > >
> > > >> > > > With Regards,
> > > >> > > > Sanjay Kumar
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > > On Tue, Mar 14, 2023 at 1:18 PM Wei ZHOU <
> ustcweizhou@gmail.com
> > >
> > > >> > wrote:
> > > >> > > >
> > > >> > > >> Hi Sanjay,
> > > >> > > >>
> > > >> > > >> To be clear, can you share the details like below ?
> > > >> > > >>
> > > >> > > >> cloudstack version
> > > >> > > >> kubernetes version
> > > >> > > >> network details (services, rules, etc)
> > > >> > > >> vm details (template, offering, etc)
> > > >> > > >> management logs
> > > >> > > >> agent logs
> > > >> > > >> anything else you can share.
> > > >> > > >>
> > > >> > > >>
> > > >> > > >> -Wei
> > > >> > > >> On Tue, 14 Mar 2023 at 08:26, Sanjay Kumar <
> > > sslinuxpath@gmail.com>
> > > >> > > wrote:
> > > >> > > >>
> > > >> > > >> > Hi Wei,
> > > >> > > >> >
> > > >> > > >> > Thanks for the quick response!
> > > >> > > >> >
> > > >> > > >> > We have checked the event as well and VM's, LB, Firewall
> > Rules
> > > >> have
> > > >> > > been
> > > >> > > >> > created properly but the cluster still showing starting
> > state.
> > > we
> > > >> > are
> > > >> > > >> > attaching the screenshot for the same.
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> > On Tue, Mar 14, 2023 at 12:48 PM Wei ZHOU <
> > > ustcweizhou@gmail.com
> > > >> >
> > > >> > > >> wrote:
> > > >> > > >> >
> > > >> > > >> >> HI,
> > > >> > > >> >>
> > > >> > > >> >> Can you give more details  ?
> > > >> > > >> >>
> > > >> > > >> >> -Wei
> > > >> > > >> >>
> > > >> > > >> >> On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <
> > > >> sslinuxpath@gmail.com>
> > > >> > > >> wrote:
> > > >> > > >> >>
> > > >> > > >> >> > Hello All
> > > >> > > >> >> >
> > > >> > > >> >> > Kubernetes cluster is stucked in the 'Starting' state,
> > even
> > > >> > though
> > > >> > > >> VM's,
> > > >> > > >> >> > LB, Firewall Rules are been created properly.
> > > >> > > >> >> > Have any one faced such issue?
> > > >> > > >> >> >
> > > >> > > >> >> > [image: image.png]
> > > >> > > >> >> >
> > > >> > > >> >> > Any help would be really appreciated. Thank you!
> > > >> > > >> >> >
> > > >> > > >> >> > Regards,
> > > >> > > >> >> > Sanjay Kumar
> > > >> > > >> >> >
> > > >> > > >> >> >
> > > >> > > >> >> >
> > > >> > > >> >> >
> > > >> > > >> >> >
> > > >> > > >> >> >
> > > >> > > >> >>
> > > >> > > >> >
> > > >> > > >>
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > >
> >
>
>

Re: Kubernetes stuck in the starting state

[Wei ZHOU <us...@gmail.com>]

Hi,

What's the output of the command ? You can add "-v" to the command to get
more information.

Each cks cluster has a public IP which is normally the source nat IP of the
isolated network.
It is expected that the Public IP is Allocated.

-Wei


On Fri, 17 Mar 2023 at 10:58, 김재종 <cb...@naver.com> wrote:

> ssh -p 2222 -i <path of id_rsa> cloud@<public ip of cks cluster>
>
> Is there public ip of cks cluster? then is id of VR?
> I also have the problem that can not acess to cks nodes.
>
> My isolated network state is Implemented. but The public ip state is
> Allocated
> Indicating the network configuration is in allocated but not setup.
> VR with this public ip is Running state.
> Why is the same ip state defferent?
>
>
>
>
>
>
> -----Original Message-----
> From: "Wei ZHOU"<us...@gmail.com>
> To: <us...@cloudstack.apache.org>;
> Cc:
> Sent: 2023-03-15 (수) 00:16:03 (GMT+09:00)
> Subject: Re: Kubernetes stuck in the starting state
>
> You should run these commands in the control node which has name "<cks
> name>-control-xxx"
>
> -Wei
>
>
>
> On Tue, 14 Mar 2023 at 15:01, Sanjay Kumar <ss...@gmail.com> wrote:
>
> > Hi Wei,
> >
> > Thanks for quick dig the issue. is there any way to get fix the dashboard
> > of cluster is showing in alerts state?
> >
> > cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get nodes
> > The connection to the server localhost:8080 was refused - did you specify
> > the right host or port?
> > cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get pods -A
> > The connection to the server localhost:8080 was refused - did you specify
> > the right host or port?
> >
> >
> > On Tue, Mar 14, 2023 at 7:21 PM Wei ZHOU <us...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > You can do some checks in the control node. for example
> > >
> > > sudo /opt/bin/kubectl get nodes
> > > sudo /opt/bin/kubectl get pods -A
> > > sudo /opt/bin/kubectl describe pod kubernetes-dashboard-xxxx-yyyy -n
> > > kubernetes-dashboard
> > >
> > > I have no problem with creating cks clusters.
> > >
> > > -Wei
> > >
> > > On Tue, 14 Mar 2023 at 14:44, Sanjay Kumar <ss...@gmail.com>
> > wrote:
> > >
> > > > Hi Wei,
> > > >
> > > > I am able to do ssh but cluster is showing alerts state.
> > > >
> > > > root@CS01:~# ssh -p 2222 -i
> /var/lib/cloudstack/management/.ssh/id_rsa
> > > > cloud@103.192.198.23
> > > > Linux k8s-test-cluster-control-186debc31f4 5.10.0-19-amd64 #1 SMP
> > Debian
> > > > 5.10.149-2 (2022-10-21) x86_64
> > > >
> > > > The programs included with the Debian GNU/Linux system are free
> > software;
> > > > the exact distribution terms for each program are described in the
> > > > individual files in /usr/share/doc/*/copyright.
> > > >
> > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > permitted by applicable law.
> > > > Last login: Tue Mar 14 13:40:18 2023 from 103.73.190.91
> > > >
> > > > The programs included with the Debian GNU/Linux system are free
> > software;
> > > > the exact distribution terms for each program are described in the
> > > > individual files in /usr/share/doc/*/copyright.
> > > >
> > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > permitted by applicable law.
> > > > cloud@k8s-test-cluster-control-186debc31f4:~$ exit
> > > > logout
> > > > Connection to 103.192.198.23 closed.
> > > > root@ACS01:~# ssh -p 2223 -i
> > /var/lib/cloudstack/management/.ssh/id_rsa
> > > > cloud@103.192.198.23
> > > > Linux k8s-test-cluster-node-186debc6be7 5.10.0-19-amd64 #1 SMP Debian
> > > > 5.10.149-2 (2022-10-21) x86_64
> > > >
> > > > The programs included with the Debian GNU/Linux system are free
> > software;
> > > > the exact distribution terms for each program are described in the
> > > > individual files in /usr/share/doc/*/copyright.
> > > >
> > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > permitted by applicable law.
> > > > Last login: Tue Mar 14 13:39:56 2023 from 103.73.190.91
> > > >
> > > > The programs included with the Debian GNU/Linux system are free
> > software;
> > > > the exact distribution terms for each program are described in the
> > > > individual files in /usr/share/doc/*/copyright.
> > > >
> > > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > > permitted by applicable law.
> > > >
> > > >
> > > >
> > > >
> > > > On Tue, Mar 14, 2023 at 6:58 PM Wei ZHOU <us...@gmail.com>
> > wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >>
> > > >> You can ssh into the cks control node from the cloudstack management
> > > >> server.
> > > >>
> > > >> ssh -p 2222 -i <path of id_rsa> cloud@<public ip of cks cluster>
> > > >>
> > > >> Here is an example:
> > > >> # ssh -p 2222 -i /var/lib/cloudstack/management/.ssh/id_rsa
> > > >> cloud@10.0.52.195
> > > >>
> > > >> The path of id_rsa is different on the ubuntu/rhel management
> servers.
> > > >>
> > > >> You can also ssh into other cks nodes with different ports starting
> > with
> > > >> 2223.
> > > >>
> > > >> -Wei
> > > >> -Wei
> > > >>
> > > >>
> > > >> On Tue, 14 Mar 2023 at 13:14, Sanjay Kumar <ss...@gmail.com>
> > > wrote:
> > > >>
> > > >> > Hi Wei,
> > > >> >
> > > >> > Could you please help me out on this? How could we access the
> > console
> > > >> node
> > > >> > or any link that could be help for us.
> > > >> >
> > > >> >
> > > >> > On Tue, Mar 14, 2023 at 2:53 PM Wei ZHOU <us...@gmail.com>
> > > wrote:
> > > >> >
> > > >> > > It looks like the K8s API server is up, but the dashboard is
> not.
> > > >> > >
> > > >> > > Can you log into the console node (from the cloudstack
> management
> > > >> server)
> > > >> > > and check the status?
> > > >> > >
> > > >> > > -Wei
> > > >> > >
> > > >> > > On Tue, 14 Mar 2023 at 09:24, Sanjay Kumar <
> sslinuxpath@gmail.com
> > >
> > > >> > wrote:
> > > >> > >
> > > >> > > > Hi Wei,
> > > >> > > >
> > > >> > > > Thank you so much for your prompt response!
> > > >> > > >
> > > >> > > > Now the running kubernetes cluster is showing in alerts state
> > > after
> > > >> 1
> > > >> > > > hours.
> > > >> > > >
> > > >> > > > cloudstack version: 4.17.2
> > > >> > > > kubernetes version : Kubernetes 1.23.3
> > > >> > > > network details (services, rules, etc) : Network name
> > > >> > > > (k8s-test-cluster-network)
> > > >> > > > vm details (template, offering, etc)
> > > >> > k8s-test-cluster-control-186debc31f4
> > > >> > > > , SystemVM Template (KVM) , Compute offering (Basic-1-Linux)
> > > >> > > >
> > > >> > > > Please access the logs from below link. It was long size
> that's
> > > why
> > > >> we
> > > >> > > > uploaded in drive.
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://drive.google.com/drive/folders/1TC_hw_h_S_TwqnT8fhoB4S3f2WHVlncx?usp=sharing
> > > >> > > >
> > > >> > > > [image: image.png]
> > > >> > > >
> > > >> > > > With Regards,
> > > >> > > > Sanjay Kumar
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > > On Tue, Mar 14, 2023 at 1:18 PM Wei ZHOU <
> ustcweizhou@gmail.com
> > >
> > > >> > wrote:
> > > >> > > >
> > > >> > > >> Hi Sanjay,
> > > >> > > >>
> > > >> > > >> To be clear, can you share the details like below ?
> > > >> > > >>
> > > >> > > >> cloudstack version
> > > >> > > >> kubernetes version
> > > >> > > >> network details (services, rules, etc)
> > > >> > > >> vm details (template, offering, etc)
> > > >> > > >> management logs
> > > >> > > >> agent logs
> > > >> > > >> anything else you can share.
> > > >> > > >>
> > > >> > > >>
> > > >> > > >> -Wei
> > > >> > > >> On Tue, 14 Mar 2023 at 08:26, Sanjay Kumar <
> > > sslinuxpath@gmail.com>
> > > >> > > wrote:
> > > >> > > >>
> > > >> > > >> > Hi Wei,
> > > >> > > >> >
> > > >> > > >> > Thanks for the quick response!
> > > >> > > >> >
> > > >> > > >> > We have checked the event as well and VM's, LB, Firewall
> > Rules
> > > >> have
> > > >> > > been
> > > >> > > >> > created properly but the cluster still showing starting
> > state.
> > > we
> > > >> > are
> > > >> > > >> > attaching the screenshot for the same.
> > > >> > > >> >
> > > >> > > >> >
> > > >> > > >> > On Tue, Mar 14, 2023 at 12:48 PM Wei ZHOU <
> > > ustcweizhou@gmail.com
> > > >> >
> > > >> > > >> wrote:
> > > >> > > >> >
> > > >> > > >> >> HI,
> > > >> > > >> >>
> > > >> > > >> >> Can you give more details  ?
> > > >> > > >> >>
> > > >> > > >> >> -Wei
> > > >> > > >> >>
> > > >> > > >> >> On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <
> > > >> sslinuxpath@gmail.com>
> > > >> > > >> wrote:
> > > >> > > >> >>
> > > >> > > >> >> > Hello All
> > > >> > > >> >> >
> > > >> > > >> >> > Kubernetes cluster is stucked in the 'Starting' state,
> > even
> > > >> > though
> > > >> > > >> VM's,
> > > >> > > >> >> > LB, Firewall Rules are been created properly.
> > > >> > > >> >> > Have any one faced such issue?
> > > >> > > >> >> >
> > > >> > > >> >> > [image: image.png]
> > > >> > > >> >> >
> > > >> > > >> >> > Any help would be really appreciated. Thank you!
> > > >> > > >> >> >
> > > >> > > >> >> > Regards,
> > > >> > > >> >> > Sanjay Kumar
> > > >> > > >> >> >
> > > >> > > >> >> >
> > > >> > > >> >> >
> > > >> > > >> >> >
> > > >> > > >> >> >
> > > >> > > >> >> >
> > > >> > > >> >>
> > > >> > > >> >
> > > >> > > >>
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > >
> >
>
>

Re: Kubernetes stuck in the starting state

[김재종 <cb...@naver.com>]

ssh -p 2222 -i <path of id_rsa> cloud@<public ip of cks cluster>

Is there public ip of cks cluster? then is id of VR?
I also have the problem that can not acess to cks nodes.

My isolated network state is Implemented. but The public ip state is Allocated
Indicating the network configuration is in allocated but not setup.
VR with this public ip is Running state.
Why is the same ip state defferent? 




 

-----Original Message-----
From: "Wei ZHOU"<us...@gmail.com>
To: <us...@cloudstack.apache.org>;
Cc:
Sent: 2023-03-15 (수) 00:16:03 (GMT+09:00)
Subject: Re: Kubernetes stuck in the starting state

You should run these commands in the control node which has name "<cks
name>-control-xxx"

-Wei



On Tue, 14 Mar 2023 at 15:01, Sanjay Kumar <ss...@gmail.com> wrote:

> Hi Wei,
>
> Thanks for quick dig the issue. is there any way to get fix the dashboard
> of cluster is showing in alerts state?
>
> cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get nodes
> The connection to the server localhost:8080 was refused - did you specify
> the right host or port?
> cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get pods -A
> The connection to the server localhost:8080 was refused - did you specify
> the right host or port?
>
>
> On Tue, Mar 14, 2023 at 7:21 PM Wei ZHOU <us...@gmail.com> wrote:
>
> > Hi,
> >
> > You can do some checks in the control node. for example
> >
> > sudo /opt/bin/kubectl get nodes
> > sudo /opt/bin/kubectl get pods -A
> > sudo /opt/bin/kubectl describe pod kubernetes-dashboard-xxxx-yyyy -n
> > kubernetes-dashboard
> >
> > I have no problem with creating cks clusters.
> >
> > -Wei
> >
> > On Tue, 14 Mar 2023 at 14:44, Sanjay Kumar <ss...@gmail.com>
> wrote:
> >
> > > Hi Wei,
> > >
> > > I am able to do ssh but cluster is showing alerts state.
> > >
> > > root@CS01:~# ssh -p 2222 -i /var/lib/cloudstack/management/.ssh/id_rsa
> > > cloud@103.192.198.23
> > > Linux k8s-test-cluster-control-186debc31f4 5.10.0-19-amd64 #1 SMP
> Debian
> > > 5.10.149-2 (2022-10-21) x86_64
> > >
> > > The programs included with the Debian GNU/Linux system are free
> software;
> > > the exact distribution terms for each program are described in the
> > > individual files in /usr/share/doc/*/copyright.
> > >
> > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > permitted by applicable law.
> > > Last login: Tue Mar 14 13:40:18 2023 from 103.73.190.91
> > >
> > > The programs included with the Debian GNU/Linux system are free
> software;
> > > the exact distribution terms for each program are described in the
> > > individual files in /usr/share/doc/*/copyright.
> > >
> > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > permitted by applicable law.
> > > cloud@k8s-test-cluster-control-186debc31f4:~$ exit
> > > logout
> > > Connection to 103.192.198.23 closed.
> > > root@ACS01:~# ssh -p 2223 -i
> /var/lib/cloudstack/management/.ssh/id_rsa
> > > cloud@103.192.198.23
> > > Linux k8s-test-cluster-node-186debc6be7 5.10.0-19-amd64 #1 SMP Debian
> > > 5.10.149-2 (2022-10-21) x86_64
> > >
> > > The programs included with the Debian GNU/Linux system are free
> software;
> > > the exact distribution terms for each program are described in the
> > > individual files in /usr/share/doc/*/copyright.
> > >
> > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > permitted by applicable law.
> > > Last login: Tue Mar 14 13:39:56 2023 from 103.73.190.91
> > >
> > > The programs included with the Debian GNU/Linux system are free
> software;
> > > the exact distribution terms for each program are described in the
> > > individual files in /usr/share/doc/*/copyright.
> > >
> > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > permitted by applicable law.
> > >
> > >
> > >
> > >
> > > On Tue, Mar 14, 2023 at 6:58 PM Wei ZHOU <us...@gmail.com>
> wrote:
> > >
> > >> Hi,
> > >>
> > >>
> > >> You can ssh into the cks control node from the cloudstack management
> > >> server.
> > >>
> > >> ssh -p 2222 -i <path of id_rsa> cloud@<public ip of cks cluster>
> > >>
> > >> Here is an example:
> > >> # ssh -p 2222 -i /var/lib/cloudstack/management/.ssh/id_rsa
> > >> cloud@10.0.52.195
> > >>
> > >> The path of id_rsa is different on the ubuntu/rhel management servers.
> > >>
> > >> You can also ssh into other cks nodes with different ports starting
> with
> > >> 2223.
> > >>
> > >> -Wei
> > >> -Wei
> > >>
> > >>
> > >> On Tue, 14 Mar 2023 at 13:14, Sanjay Kumar <ss...@gmail.com>
> > wrote:
> > >>
> > >> > Hi Wei,
> > >> >
> > >> > Could you please help me out on this? How could we access the
> console
> > >> node
> > >> > or any link that could be help for us.
> > >> >
> > >> >
> > >> > On Tue, Mar 14, 2023 at 2:53 PM Wei ZHOU <us...@gmail.com>
> > wrote:
> > >> >
> > >> > > It looks like the K8s API server is up, but the dashboard is not.
> > >> > >
> > >> > > Can you log into the console node (from the cloudstack management
> > >> server)
> > >> > > and check the status?
> > >> > >
> > >> > > -Wei
> > >> > >
> > >> > > On Tue, 14 Mar 2023 at 09:24, Sanjay Kumar <sslinuxpath@gmail.com
> >
> > >> > wrote:
> > >> > >
> > >> > > > Hi Wei,
> > >> > > >
> > >> > > > Thank you so much for your prompt response!
> > >> > > >
> > >> > > > Now the running kubernetes cluster is showing in alerts state
> > after
> > >> 1
> > >> > > > hours.
> > >> > > >
> > >> > > > cloudstack version: 4.17.2
> > >> > > > kubernetes version : Kubernetes 1.23.3
> > >> > > > network details (services, rules, etc) : Network name
> > >> > > > (k8s-test-cluster-network)
> > >> > > > vm details (template, offering, etc)
> > >> > k8s-test-cluster-control-186debc31f4
> > >> > > > , SystemVM Template (KVM) , Compute offering (Basic-1-Linux)
> > >> > > >
> > >> > > > Please access the logs from below link. It was long size that's
> > why
> > >> we
> > >> > > > uploaded in drive.
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://drive.google.com/drive/folders/1TC_hw_h_S_TwqnT8fhoB4S3f2WHVlncx?usp=sharing
> > >> > > >
> > >> > > > [image: image.png]
> > >> > > >
> > >> > > > With Regards,
> > >> > > > Sanjay Kumar
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Tue, Mar 14, 2023 at 1:18 PM Wei ZHOU <ustcweizhou@gmail.com
> >
> > >> > wrote:
> > >> > > >
> > >> > > >> Hi Sanjay,
> > >> > > >>
> > >> > > >> To be clear, can you share the details like below ?
> > >> > > >>
> > >> > > >> cloudstack version
> > >> > > >> kubernetes version
> > >> > > >> network details (services, rules, etc)
> > >> > > >> vm details (template, offering, etc)
> > >> > > >> management logs
> > >> > > >> agent logs
> > >> > > >> anything else you can share.
> > >> > > >>
> > >> > > >>
> > >> > > >> -Wei
> > >> > > >> On Tue, 14 Mar 2023 at 08:26, Sanjay Kumar <
> > sslinuxpath@gmail.com>
> > >> > > wrote:
> > >> > > >>
> > >> > > >> > Hi Wei,
> > >> > > >> >
> > >> > > >> > Thanks for the quick response!
> > >> > > >> >
> > >> > > >> > We have checked the event as well and VM's, LB, Firewall
> Rules
> > >> have
> > >> > > been
> > >> > > >> > created properly but the cluster still showing starting
> state.
> > we
> > >> > are
> > >> > > >> > attaching the screenshot for the same.
> > >> > > >> >
> > >> > > >> >
> > >> > > >> > On Tue, Mar 14, 2023 at 12:48 PM Wei ZHOU <
> > ustcweizhou@gmail.com
> > >> >
> > >> > > >> wrote:
> > >> > > >> >
> > >> > > >> >> HI,
> > >> > > >> >>
> > >> > > >> >> Can you give more details  ?
> > >> > > >> >>
> > >> > > >> >> -Wei
> > >> > > >> >>
> > >> > > >> >> On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <
> > >> sslinuxpath@gmail.com>
> > >> > > >> wrote:
> > >> > > >> >>
> > >> > > >> >> > Hello All
> > >> > > >> >> >
> > >> > > >> >> > Kubernetes cluster is stucked in the 'Starting' state,
> even
> > >> > though
> > >> > > >> VM's,
> > >> > > >> >> > LB, Firewall Rules are been created properly.
> > >> > > >> >> > Have any one faced such issue?
> > >> > > >> >> >
> > >> > > >> >> > [image: image.png]
> > >> > > >> >> >
> > >> > > >> >> > Any help would be really appreciated. Thank you!
> > >> > > >> >> >
> > >> > > >> >> > Regards,
> > >> > > >> >> > Sanjay Kumar
> > >> > > >> >> >
> > >> > > >> >> >
> > >> > > >> >> >
> > >> > > >> >> >
> > >> > > >> >> >
> > >> > > >> >> >
> > >> > > >> >>
> > >> > > >> >
> > >> > > >>
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>

Re: Kubernetes stuck in the starting state

[Wei ZHOU <us...@gmail.com>]

You should run these commands in the control node which has name "<cks
name>-control-xxx"

-Wei



On Tue, 14 Mar 2023 at 15:01, Sanjay Kumar <ss...@gmail.com> wrote:

> Hi Wei,
>
> Thanks for quick dig the issue. is there any way to get fix the dashboard
> of cluster is showing in alerts state?
>
> cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get nodes
> The connection to the server localhost:8080 was refused - did you specify
> the right host or port?
> cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get pods -A
> The connection to the server localhost:8080 was refused - did you specify
> the right host or port?
>
>
> On Tue, Mar 14, 2023 at 7:21 PM Wei ZHOU <us...@gmail.com> wrote:
>
> > Hi,
> >
> > You can do some checks in the control node. for example
> >
> > sudo /opt/bin/kubectl get nodes
> > sudo /opt/bin/kubectl get pods -A
> > sudo /opt/bin/kubectl describe pod kubernetes-dashboard-xxxx-yyyy -n
> > kubernetes-dashboard
> >
> > I have no problem with creating cks clusters.
> >
> > -Wei
> >
> > On Tue, 14 Mar 2023 at 14:44, Sanjay Kumar <ss...@gmail.com>
> wrote:
> >
> > > Hi Wei,
> > >
> > > I am able to do ssh but cluster is showing alerts state.
> > >
> > > root@CS01:~# ssh -p 2222 -i /var/lib/cloudstack/management/.ssh/id_rsa
> > > cloud@103.192.198.23
> > > Linux k8s-test-cluster-control-186debc31f4 5.10.0-19-amd64 #1 SMP
> Debian
> > > 5.10.149-2 (2022-10-21) x86_64
> > >
> > > The programs included with the Debian GNU/Linux system are free
> software;
> > > the exact distribution terms for each program are described in the
> > > individual files in /usr/share/doc/*/copyright.
> > >
> > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > permitted by applicable law.
> > > Last login: Tue Mar 14 13:40:18 2023 from 103.73.190.91
> > >
> > > The programs included with the Debian GNU/Linux system are free
> software;
> > > the exact distribution terms for each program are described in the
> > > individual files in /usr/share/doc/*/copyright.
> > >
> > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > permitted by applicable law.
> > > cloud@k8s-test-cluster-control-186debc31f4:~$ exit
> > > logout
> > > Connection to 103.192.198.23 closed.
> > > root@ACS01:~# ssh -p 2223 -i
> /var/lib/cloudstack/management/.ssh/id_rsa
> > > cloud@103.192.198.23
> > > Linux k8s-test-cluster-node-186debc6be7 5.10.0-19-amd64 #1 SMP Debian
> > > 5.10.149-2 (2022-10-21) x86_64
> > >
> > > The programs included with the Debian GNU/Linux system are free
> software;
> > > the exact distribution terms for each program are described in the
> > > individual files in /usr/share/doc/*/copyright.
> > >
> > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > permitted by applicable law.
> > > Last login: Tue Mar 14 13:39:56 2023 from 103.73.190.91
> > >
> > > The programs included with the Debian GNU/Linux system are free
> software;
> > > the exact distribution terms for each program are described in the
> > > individual files in /usr/share/doc/*/copyright.
> > >
> > > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > > permitted by applicable law.
> > >
> > >
> > >
> > >
> > > On Tue, Mar 14, 2023 at 6:58 PM Wei ZHOU <us...@gmail.com>
> wrote:
> > >
> > >> Hi,
> > >>
> > >>
> > >> You can ssh into the cks control node from the cloudstack management
> > >> server.
> > >>
> > >> ssh -p 2222 -i <path of id_rsa> cloud@<public ip of cks cluster>
> > >>
> > >> Here is an example:
> > >> # ssh -p 2222 -i /var/lib/cloudstack/management/.ssh/id_rsa
> > >> cloud@10.0.52.195
> > >>
> > >> The path of id_rsa is different on the ubuntu/rhel management servers.
> > >>
> > >> You can also ssh into other cks nodes with different ports starting
> with
> > >> 2223.
> > >>
> > >> -Wei
> > >> -Wei
> > >>
> > >>
> > >> On Tue, 14 Mar 2023 at 13:14, Sanjay Kumar <ss...@gmail.com>
> > wrote:
> > >>
> > >> > Hi Wei,
> > >> >
> > >> > Could you please help me out on this? How could we access the
> console
> > >> node
> > >> > or any link that could be help for us.
> > >> >
> > >> >
> > >> > On Tue, Mar 14, 2023 at 2:53 PM Wei ZHOU <us...@gmail.com>
> > wrote:
> > >> >
> > >> > > It looks like the K8s API server is up, but the dashboard is not.
> > >> > >
> > >> > > Can you log into the console node (from the cloudstack management
> > >> server)
> > >> > > and check the status?
> > >> > >
> > >> > > -Wei
> > >> > >
> > >> > > On Tue, 14 Mar 2023 at 09:24, Sanjay Kumar <sslinuxpath@gmail.com
> >
> > >> > wrote:
> > >> > >
> > >> > > > Hi Wei,
> > >> > > >
> > >> > > > Thank you so much for your prompt response!
> > >> > > >
> > >> > > > Now the running kubernetes cluster is showing in alerts state
> > after
> > >> 1
> > >> > > > hours.
> > >> > > >
> > >> > > > cloudstack version: 4.17.2
> > >> > > > kubernetes version : Kubernetes 1.23.3
> > >> > > > network details (services, rules, etc) : Network name
> > >> > > > (k8s-test-cluster-network)
> > >> > > > vm details (template, offering, etc)
> > >> > k8s-test-cluster-control-186debc31f4
> > >> > > > , SystemVM Template (KVM) , Compute offering (Basic-1-Linux)
> > >> > > >
> > >> > > > Please access the logs from below link. It was long size that's
> > why
> > >> we
> > >> > > > uploaded in drive.
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://drive.google.com/drive/folders/1TC_hw_h_S_TwqnT8fhoB4S3f2WHVlncx?usp=sharing
> > >> > > >
> > >> > > > [image: image.png]
> > >> > > >
> > >> > > > With Regards,
> > >> > > > Sanjay Kumar
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Tue, Mar 14, 2023 at 1:18 PM Wei ZHOU <ustcweizhou@gmail.com
> >
> > >> > wrote:
> > >> > > >
> > >> > > >> Hi Sanjay,
> > >> > > >>
> > >> > > >> To be clear, can you share the details like below ?
> > >> > > >>
> > >> > > >> cloudstack version
> > >> > > >> kubernetes version
> > >> > > >> network details (services, rules, etc)
> > >> > > >> vm details (template, offering, etc)
> > >> > > >> management logs
> > >> > > >> agent logs
> > >> > > >> anything else you can share.
> > >> > > >>
> > >> > > >>
> > >> > > >> -Wei
> > >> > > >> On Tue, 14 Mar 2023 at 08:26, Sanjay Kumar <
> > sslinuxpath@gmail.com>
> > >> > > wrote:
> > >> > > >>
> > >> > > >> > Hi Wei,
> > >> > > >> >
> > >> > > >> > Thanks for the quick response!
> > >> > > >> >
> > >> > > >> > We have checked the event as well and VM's, LB, Firewall
> Rules
> > >> have
> > >> > > been
> > >> > > >> > created properly but the cluster still showing starting
> state.
> > we
> > >> > are
> > >> > > >> > attaching the screenshot for the same.
> > >> > > >> >
> > >> > > >> >
> > >> > > >> > On Tue, Mar 14, 2023 at 12:48 PM Wei ZHOU <
> > ustcweizhou@gmail.com
> > >> >
> > >> > > >> wrote:
> > >> > > >> >
> > >> > > >> >> HI,
> > >> > > >> >>
> > >> > > >> >> Can you give more details  ?
> > >> > > >> >>
> > >> > > >> >> -Wei
> > >> > > >> >>
> > >> > > >> >> On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <
> > >> sslinuxpath@gmail.com>
> > >> > > >> wrote:
> > >> > > >> >>
> > >> > > >> >> > Hello All
> > >> > > >> >> >
> > >> > > >> >> > Kubernetes cluster is stucked in the 'Starting' state,
> even
> > >> > though
> > >> > > >> VM's,
> > >> > > >> >> > LB, Firewall Rules are been created properly.
> > >> > > >> >> > Have any one faced such issue?
> > >> > > >> >> >
> > >> > > >> >> > [image: image.png]
> > >> > > >> >> >
> > >> > > >> >> > Any help would be really appreciated. Thank you!
> > >> > > >> >> >
> > >> > > >> >> > Regards,
> > >> > > >> >> > Sanjay Kumar
> > >> > > >> >> >
> > >> > > >> >> >
> > >> > > >> >> >
> > >> > > >> >> >
> > >> > > >> >> >
> > >> > > >> >> >
> > >> > > >> >>
> > >> > > >> >
> > >> > > >>
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>

Re: Kubernetes stuck in the starting state

[Sanjay Kumar <ss...@gmail.com>]

Hi Wei,

Thanks for quick dig the issue. is there any way to get fix the dashboard
of cluster is showing in alerts state?

cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get nodes
The connection to the server localhost:8080 was refused - did you specify
the right host or port?
cloud@k8s-test-cluster-node-186debc6be7:~$ /opt/bin/kubectl get pods -A
The connection to the server localhost:8080 was refused - did you specify
the right host or port?


On Tue, Mar 14, 2023 at 7:21 PM Wei ZHOU <us...@gmail.com> wrote:

> Hi,
>
> You can do some checks in the control node. for example
>
> sudo /opt/bin/kubectl get nodes
> sudo /opt/bin/kubectl get pods -A
> sudo /opt/bin/kubectl describe pod kubernetes-dashboard-xxxx-yyyy -n
> kubernetes-dashboard
>
> I have no problem with creating cks clusters.
>
> -Wei
>
> On Tue, 14 Mar 2023 at 14:44, Sanjay Kumar <ss...@gmail.com> wrote:
>
> > Hi Wei,
> >
> > I am able to do ssh but cluster is showing alerts state.
> >
> > root@CS01:~# ssh -p 2222 -i /var/lib/cloudstack/management/.ssh/id_rsa
> > cloud@103.192.198.23
> > Linux k8s-test-cluster-control-186debc31f4 5.10.0-19-amd64 #1 SMP Debian
> > 5.10.149-2 (2022-10-21) x86_64
> >
> > The programs included with the Debian GNU/Linux system are free software;
> > the exact distribution terms for each program are described in the
> > individual files in /usr/share/doc/*/copyright.
> >
> > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > permitted by applicable law.
> > Last login: Tue Mar 14 13:40:18 2023 from 103.73.190.91
> >
> > The programs included with the Debian GNU/Linux system are free software;
> > the exact distribution terms for each program are described in the
> > individual files in /usr/share/doc/*/copyright.
> >
> > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > permitted by applicable law.
> > cloud@k8s-test-cluster-control-186debc31f4:~$ exit
> > logout
> > Connection to 103.192.198.23 closed.
> > root@ACS01:~# ssh -p 2223 -i /var/lib/cloudstack/management/.ssh/id_rsa
> > cloud@103.192.198.23
> > Linux k8s-test-cluster-node-186debc6be7 5.10.0-19-amd64 #1 SMP Debian
> > 5.10.149-2 (2022-10-21) x86_64
> >
> > The programs included with the Debian GNU/Linux system are free software;
> > the exact distribution terms for each program are described in the
> > individual files in /usr/share/doc/*/copyright.
> >
> > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > permitted by applicable law.
> > Last login: Tue Mar 14 13:39:56 2023 from 103.73.190.91
> >
> > The programs included with the Debian GNU/Linux system are free software;
> > the exact distribution terms for each program are described in the
> > individual files in /usr/share/doc/*/copyright.
> >
> > Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> > permitted by applicable law.
> >
> >
> >
> >
> > On Tue, Mar 14, 2023 at 6:58 PM Wei ZHOU <us...@gmail.com> wrote:
> >
> >> Hi,
> >>
> >>
> >> You can ssh into the cks control node from the cloudstack management
> >> server.
> >>
> >> ssh -p 2222 -i <path of id_rsa> cloud@<public ip of cks cluster>
> >>
> >> Here is an example:
> >> # ssh -p 2222 -i /var/lib/cloudstack/management/.ssh/id_rsa
> >> cloud@10.0.52.195
> >>
> >> The path of id_rsa is different on the ubuntu/rhel management servers.
> >>
> >> You can also ssh into other cks nodes with different ports starting with
> >> 2223.
> >>
> >> -Wei
> >> -Wei
> >>
> >>
> >> On Tue, 14 Mar 2023 at 13:14, Sanjay Kumar <ss...@gmail.com>
> wrote:
> >>
> >> > Hi Wei,
> >> >
> >> > Could you please help me out on this? How could we access the console
> >> node
> >> > or any link that could be help for us.
> >> >
> >> >
> >> > On Tue, Mar 14, 2023 at 2:53 PM Wei ZHOU <us...@gmail.com>
> wrote:
> >> >
> >> > > It looks like the K8s API server is up, but the dashboard is not.
> >> > >
> >> > > Can you log into the console node (from the cloudstack management
> >> server)
> >> > > and check the status?
> >> > >
> >> > > -Wei
> >> > >
> >> > > On Tue, 14 Mar 2023 at 09:24, Sanjay Kumar <ss...@gmail.com>
> >> > wrote:
> >> > >
> >> > > > Hi Wei,
> >> > > >
> >> > > > Thank you so much for your prompt response!
> >> > > >
> >> > > > Now the running kubernetes cluster is showing in alerts state
> after
> >> 1
> >> > > > hours.
> >> > > >
> >> > > > cloudstack version: 4.17.2
> >> > > > kubernetes version : Kubernetes 1.23.3
> >> > > > network details (services, rules, etc) : Network name
> >> > > > (k8s-test-cluster-network)
> >> > > > vm details (template, offering, etc)
> >> > k8s-test-cluster-control-186debc31f4
> >> > > > , SystemVM Template (KVM) , Compute offering (Basic-1-Linux)
> >> > > >
> >> > > > Please access the logs from below link. It was long size that's
> why
> >> we
> >> > > > uploaded in drive.
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> https://drive.google.com/drive/folders/1TC_hw_h_S_TwqnT8fhoB4S3f2WHVlncx?usp=sharing
> >> > > >
> >> > > > [image: image.png]
> >> > > >
> >> > > > With Regards,
> >> > > > Sanjay Kumar
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Tue, Mar 14, 2023 at 1:18 PM Wei ZHOU <us...@gmail.com>
> >> > wrote:
> >> > > >
> >> > > >> Hi Sanjay,
> >> > > >>
> >> > > >> To be clear, can you share the details like below ?
> >> > > >>
> >> > > >> cloudstack version
> >> > > >> kubernetes version
> >> > > >> network details (services, rules, etc)
> >> > > >> vm details (template, offering, etc)
> >> > > >> management logs
> >> > > >> agent logs
> >> > > >> anything else you can share.
> >> > > >>
> >> > > >>
> >> > > >> -Wei
> >> > > >> On Tue, 14 Mar 2023 at 08:26, Sanjay Kumar <
> sslinuxpath@gmail.com>
> >> > > wrote:
> >> > > >>
> >> > > >> > Hi Wei,
> >> > > >> >
> >> > > >> > Thanks for the quick response!
> >> > > >> >
> >> > > >> > We have checked the event as well and VM's, LB, Firewall Rules
> >> have
> >> > > been
> >> > > >> > created properly but the cluster still showing starting state.
> we
> >> > are
> >> > > >> > attaching the screenshot for the same.
> >> > > >> >
> >> > > >> >
> >> > > >> > On Tue, Mar 14, 2023 at 12:48 PM Wei ZHOU <
> ustcweizhou@gmail.com
> >> >
> >> > > >> wrote:
> >> > > >> >
> >> > > >> >> HI,
> >> > > >> >>
> >> > > >> >> Can you give more details  ?
> >> > > >> >>
> >> > > >> >> -Wei
> >> > > >> >>
> >> > > >> >> On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <
> >> sslinuxpath@gmail.com>
> >> > > >> wrote:
> >> > > >> >>
> >> > > >> >> > Hello All
> >> > > >> >> >
> >> > > >> >> > Kubernetes cluster is stucked in the 'Starting' state, even
> >> > though
> >> > > >> VM's,
> >> > > >> >> > LB, Firewall Rules are been created properly.
> >> > > >> >> > Have any one faced such issue?
> >> > > >> >> >
> >> > > >> >> > [image: image.png]
> >> > > >> >> >
> >> > > >> >> > Any help would be really appreciated. Thank you!
> >> > > >> >> >
> >> > > >> >> > Regards,
> >> > > >> >> > Sanjay Kumar
> >> > > >> >> >
> >> > > >> >> >
> >> > > >> >> >
> >> > > >> >> >
> >> > > >> >> >
> >> > > >> >> >
> >> > > >> >>
> >> > > >> >
> >> > > >>
> >> > > >
> >> > >
> >> >
> >>
> >
>

Re: Kubernetes stuck in the starting state

[Wei ZHOU <us...@gmail.com>]

Hi,

You can do some checks in the control node. for example

sudo /opt/bin/kubectl get nodes
sudo /opt/bin/kubectl get pods -A
sudo /opt/bin/kubectl describe pod kubernetes-dashboard-xxxx-yyyy -n
kubernetes-dashboard

I have no problem with creating cks clusters.

-Wei

On Tue, 14 Mar 2023 at 14:44, Sanjay Kumar <ss...@gmail.com> wrote:

> Hi Wei,
>
> I am able to do ssh but cluster is showing alerts state.
>
> root@CS01:~# ssh -p 2222 -i /var/lib/cloudstack/management/.ssh/id_rsa
> cloud@103.192.198.23
> Linux k8s-test-cluster-control-186debc31f4 5.10.0-19-amd64 #1 SMP Debian
> 5.10.149-2 (2022-10-21) x86_64
>
> The programs included with the Debian GNU/Linux system are free software;
> the exact distribution terms for each program are described in the
> individual files in /usr/share/doc/*/copyright.
>
> Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> permitted by applicable law.
> Last login: Tue Mar 14 13:40:18 2023 from 103.73.190.91
>
> The programs included with the Debian GNU/Linux system are free software;
> the exact distribution terms for each program are described in the
> individual files in /usr/share/doc/*/copyright.
>
> Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> permitted by applicable law.
> cloud@k8s-test-cluster-control-186debc31f4:~$ exit
> logout
> Connection to 103.192.198.23 closed.
> root@ACS01:~# ssh -p 2223 -i /var/lib/cloudstack/management/.ssh/id_rsa
> cloud@103.192.198.23
> Linux k8s-test-cluster-node-186debc6be7 5.10.0-19-amd64 #1 SMP Debian
> 5.10.149-2 (2022-10-21) x86_64
>
> The programs included with the Debian GNU/Linux system are free software;
> the exact distribution terms for each program are described in the
> individual files in /usr/share/doc/*/copyright.
>
> Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> permitted by applicable law.
> Last login: Tue Mar 14 13:39:56 2023 from 103.73.190.91
>
> The programs included with the Debian GNU/Linux system are free software;
> the exact distribution terms for each program are described in the
> individual files in /usr/share/doc/*/copyright.
>
> Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> permitted by applicable law.
>
>
>
>
> On Tue, Mar 14, 2023 at 6:58 PM Wei ZHOU <us...@gmail.com> wrote:
>
>> Hi,
>>
>>
>> You can ssh into the cks control node from the cloudstack management
>> server.
>>
>> ssh -p 2222 -i <path of id_rsa> cloud@<public ip of cks cluster>
>>
>> Here is an example:
>> # ssh -p 2222 -i /var/lib/cloudstack/management/.ssh/id_rsa
>> cloud@10.0.52.195
>>
>> The path of id_rsa is different on the ubuntu/rhel management servers.
>>
>> You can also ssh into other cks nodes with different ports starting with
>> 2223.
>>
>> -Wei
>> -Wei
>>
>>
>> On Tue, 14 Mar 2023 at 13:14, Sanjay Kumar <ss...@gmail.com> wrote:
>>
>> > Hi Wei,
>> >
>> > Could you please help me out on this? How could we access the console
>> node
>> > or any link that could be help for us.
>> >
>> >
>> > On Tue, Mar 14, 2023 at 2:53 PM Wei ZHOU <us...@gmail.com> wrote:
>> >
>> > > It looks like the K8s API server is up, but the dashboard is not.
>> > >
>> > > Can you log into the console node (from the cloudstack management
>> server)
>> > > and check the status?
>> > >
>> > > -Wei
>> > >
>> > > On Tue, 14 Mar 2023 at 09:24, Sanjay Kumar <ss...@gmail.com>
>> > wrote:
>> > >
>> > > > Hi Wei,
>> > > >
>> > > > Thank you so much for your prompt response!
>> > > >
>> > > > Now the running kubernetes cluster is showing in alerts state after
>> 1
>> > > > hours.
>> > > >
>> > > > cloudstack version: 4.17.2
>> > > > kubernetes version : Kubernetes 1.23.3
>> > > > network details (services, rules, etc) : Network name
>> > > > (k8s-test-cluster-network)
>> > > > vm details (template, offering, etc)
>> > k8s-test-cluster-control-186debc31f4
>> > > > , SystemVM Template (KVM) , Compute offering (Basic-1-Linux)
>> > > >
>> > > > Please access the logs from below link. It was long size that's why
>> we
>> > > > uploaded in drive.
>> > > >
>> > > >
>> > > >
>> > >
>> >
>> https://drive.google.com/drive/folders/1TC_hw_h_S_TwqnT8fhoB4S3f2WHVlncx?usp=sharing
>> > > >
>> > > > [image: image.png]
>> > > >
>> > > > With Regards,
>> > > > Sanjay Kumar
>> > > >
>> > > >
>> > > >
>> > > > On Tue, Mar 14, 2023 at 1:18 PM Wei ZHOU <us...@gmail.com>
>> > wrote:
>> > > >
>> > > >> Hi Sanjay,
>> > > >>
>> > > >> To be clear, can you share the details like below ?
>> > > >>
>> > > >> cloudstack version
>> > > >> kubernetes version
>> > > >> network details (services, rules, etc)
>> > > >> vm details (template, offering, etc)
>> > > >> management logs
>> > > >> agent logs
>> > > >> anything else you can share.
>> > > >>
>> > > >>
>> > > >> -Wei
>> > > >> On Tue, 14 Mar 2023 at 08:26, Sanjay Kumar <ss...@gmail.com>
>> > > wrote:
>> > > >>
>> > > >> > Hi Wei,
>> > > >> >
>> > > >> > Thanks for the quick response!
>> > > >> >
>> > > >> > We have checked the event as well and VM's, LB, Firewall Rules
>> have
>> > > been
>> > > >> > created properly but the cluster still showing starting state. we
>> > are
>> > > >> > attaching the screenshot for the same.
>> > > >> >
>> > > >> >
>> > > >> > On Tue, Mar 14, 2023 at 12:48 PM Wei ZHOU <ustcweizhou@gmail.com
>> >
>> > > >> wrote:
>> > > >> >
>> > > >> >> HI,
>> > > >> >>
>> > > >> >> Can you give more details  ?
>> > > >> >>
>> > > >> >> -Wei
>> > > >> >>
>> > > >> >> On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <
>> sslinuxpath@gmail.com>
>> > > >> wrote:
>> > > >> >>
>> > > >> >> > Hello All
>> > > >> >> >
>> > > >> >> > Kubernetes cluster is stucked in the 'Starting' state, even
>> > though
>> > > >> VM's,
>> > > >> >> > LB, Firewall Rules are been created properly.
>> > > >> >> > Have any one faced such issue?
>> > > >> >> >
>> > > >> >> > [image: image.png]
>> > > >> >> >
>> > > >> >> > Any help would be really appreciated. Thank you!
>> > > >> >> >
>> > > >> >> > Regards,
>> > > >> >> > Sanjay Kumar
>> > > >> >> >
>> > > >> >> >
>> > > >> >> >
>> > > >> >> >
>> > > >> >> >
>> > > >> >> >
>> > > >> >>
>> > > >> >
>> > > >>
>> > > >
>> > >
>> >
>>
>

Re: Kubernetes stuck in the starting state

[Sanjay Kumar <ss...@gmail.com>]

Hi Wei,

I am able to do ssh but cluster is showing alerts state.

root@CS01:~# ssh -p 2222 -i /var/lib/cloudstack/management/.ssh/id_rsa
cloud@103.192.198.23
Linux k8s-test-cluster-control-186debc31f4 5.10.0-19-amd64 #1 SMP Debian
5.10.149-2 (2022-10-21) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Tue Mar 14 13:40:18 2023 from 103.73.190.91

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
cloud@k8s-test-cluster-control-186debc31f4:~$ exit
logout
Connection to 103.192.198.23 closed.
root@ACS01:~# ssh -p 2223 -i /var/lib/cloudstack/management/.ssh/id_rsa
cloud@103.192.198.23
Linux k8s-test-cluster-node-186debc6be7 5.10.0-19-amd64 #1 SMP Debian
5.10.149-2 (2022-10-21) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Tue Mar 14 13:39:56 2023 from 103.73.190.91

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.




On Tue, Mar 14, 2023 at 6:58 PM Wei ZHOU <us...@gmail.com> wrote:

> Hi,
>
>
> You can ssh into the cks control node from the cloudstack management
> server.
>
> ssh -p 2222 -i <path of id_rsa> cloud@<public ip of cks cluster>
>
> Here is an example:
> # ssh -p 2222 -i /var/lib/cloudstack/management/.ssh/id_rsa
> cloud@10.0.52.195
>
> The path of id_rsa is different on the ubuntu/rhel management servers.
>
> You can also ssh into other cks nodes with different ports starting with
> 2223.
>
> -Wei
> -Wei
>
>
> On Tue, 14 Mar 2023 at 13:14, Sanjay Kumar <ss...@gmail.com> wrote:
>
> > Hi Wei,
> >
> > Could you please help me out on this? How could we access the console
> node
> > or any link that could be help for us.
> >
> >
> > On Tue, Mar 14, 2023 at 2:53 PM Wei ZHOU <us...@gmail.com> wrote:
> >
> > > It looks like the K8s API server is up, but the dashboard is not.
> > >
> > > Can you log into the console node (from the cloudstack management
> server)
> > > and check the status?
> > >
> > > -Wei
> > >
> > > On Tue, 14 Mar 2023 at 09:24, Sanjay Kumar <ss...@gmail.com>
> > wrote:
> > >
> > > > Hi Wei,
> > > >
> > > > Thank you so much for your prompt response!
> > > >
> > > > Now the running kubernetes cluster is showing in alerts state after 1
> > > > hours.
> > > >
> > > > cloudstack version: 4.17.2
> > > > kubernetes version : Kubernetes 1.23.3
> > > > network details (services, rules, etc) : Network name
> > > > (k8s-test-cluster-network)
> > > > vm details (template, offering, etc)
> > k8s-test-cluster-control-186debc31f4
> > > > , SystemVM Template (KVM) , Compute offering (Basic-1-Linux)
> > > >
> > > > Please access the logs from below link. It was long size that's why
> we
> > > > uploaded in drive.
> > > >
> > > >
> > > >
> > >
> >
> https://drive.google.com/drive/folders/1TC_hw_h_S_TwqnT8fhoB4S3f2WHVlncx?usp=sharing
> > > >
> > > > [image: image.png]
> > > >
> > > > With Regards,
> > > > Sanjay Kumar
> > > >
> > > >
> > > >
> > > > On Tue, Mar 14, 2023 at 1:18 PM Wei ZHOU <us...@gmail.com>
> > wrote:
> > > >
> > > >> Hi Sanjay,
> > > >>
> > > >> To be clear, can you share the details like below ?
> > > >>
> > > >> cloudstack version
> > > >> kubernetes version
> > > >> network details (services, rules, etc)
> > > >> vm details (template, offering, etc)
> > > >> management logs
> > > >> agent logs
> > > >> anything else you can share.
> > > >>
> > > >>
> > > >> -Wei
> > > >> On Tue, 14 Mar 2023 at 08:26, Sanjay Kumar <ss...@gmail.com>
> > > wrote:
> > > >>
> > > >> > Hi Wei,
> > > >> >
> > > >> > Thanks for the quick response!
> > > >> >
> > > >> > We have checked the event as well and VM's, LB, Firewall Rules
> have
> > > been
> > > >> > created properly but the cluster still showing starting state. we
> > are
> > > >> > attaching the screenshot for the same.
> > > >> >
> > > >> >
> > > >> > On Tue, Mar 14, 2023 at 12:48 PM Wei ZHOU <us...@gmail.com>
> > > >> wrote:
> > > >> >
> > > >> >> HI,
> > > >> >>
> > > >> >> Can you give more details  ?
> > > >> >>
> > > >> >> -Wei
> > > >> >>
> > > >> >> On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <
> sslinuxpath@gmail.com>
> > > >> wrote:
> > > >> >>
> > > >> >> > Hello All
> > > >> >> >
> > > >> >> > Kubernetes cluster is stucked in the 'Starting' state, even
> > though
> > > >> VM's,
> > > >> >> > LB, Firewall Rules are been created properly.
> > > >> >> > Have any one faced such issue?
> > > >> >> >
> > > >> >> > [image: image.png]
> > > >> >> >
> > > >> >> > Any help would be really appreciated. Thank you!
> > > >> >> >
> > > >> >> > Regards,
> > > >> >> > Sanjay Kumar
> > > >> >> >
> > > >> >> >
> > > >> >> >
> > > >> >> >
> > > >> >> >
> > > >> >> >
> > > >> >>
> > > >> >
> > > >>
> > > >
> > >
> >
>

Re: Kubernetes stuck in the starting state

[Wei ZHOU <us...@gmail.com>]

Hi,


You can ssh into the cks control node from the cloudstack management server.

ssh -p 2222 -i <path of id_rsa> cloud@<public ip of cks cluster>

Here is an example:
# ssh -p 2222 -i /var/lib/cloudstack/management/.ssh/id_rsa
cloud@10.0.52.195

The path of id_rsa is different on the ubuntu/rhel management servers.

You can also ssh into other cks nodes with different ports starting with
2223.

-Wei
-Wei


On Tue, 14 Mar 2023 at 13:14, Sanjay Kumar <ss...@gmail.com> wrote:

> Hi Wei,
>
> Could you please help me out on this? How could we access the console node
> or any link that could be help for us.
>
>
> On Tue, Mar 14, 2023 at 2:53 PM Wei ZHOU <us...@gmail.com> wrote:
>
> > It looks like the K8s API server is up, but the dashboard is not.
> >
> > Can you log into the console node (from the cloudstack management server)
> > and check the status?
> >
> > -Wei
> >
> > On Tue, 14 Mar 2023 at 09:24, Sanjay Kumar <ss...@gmail.com>
> wrote:
> >
> > > Hi Wei,
> > >
> > > Thank you so much for your prompt response!
> > >
> > > Now the running kubernetes cluster is showing in alerts state after 1
> > > hours.
> > >
> > > cloudstack version: 4.17.2
> > > kubernetes version : Kubernetes 1.23.3
> > > network details (services, rules, etc) : Network name
> > > (k8s-test-cluster-network)
> > > vm details (template, offering, etc)
> k8s-test-cluster-control-186debc31f4
> > > , SystemVM Template (KVM) , Compute offering (Basic-1-Linux)
> > >
> > > Please access the logs from below link. It was long size that's why we
> > > uploaded in drive.
> > >
> > >
> > >
> >
> https://drive.google.com/drive/folders/1TC_hw_h_S_TwqnT8fhoB4S3f2WHVlncx?usp=sharing
> > >
> > > [image: image.png]
> > >
> > > With Regards,
> > > Sanjay Kumar
> > >
> > >
> > >
> > > On Tue, Mar 14, 2023 at 1:18 PM Wei ZHOU <us...@gmail.com>
> wrote:
> > >
> > >> Hi Sanjay,
> > >>
> > >> To be clear, can you share the details like below ?
> > >>
> > >> cloudstack version
> > >> kubernetes version
> > >> network details (services, rules, etc)
> > >> vm details (template, offering, etc)
> > >> management logs
> > >> agent logs
> > >> anything else you can share.
> > >>
> > >>
> > >> -Wei
> > >> On Tue, 14 Mar 2023 at 08:26, Sanjay Kumar <ss...@gmail.com>
> > wrote:
> > >>
> > >> > Hi Wei,
> > >> >
> > >> > Thanks for the quick response!
> > >> >
> > >> > We have checked the event as well and VM's, LB, Firewall Rules have
> > been
> > >> > created properly but the cluster still showing starting state. we
> are
> > >> > attaching the screenshot for the same.
> > >> >
> > >> >
> > >> > On Tue, Mar 14, 2023 at 12:48 PM Wei ZHOU <us...@gmail.com>
> > >> wrote:
> > >> >
> > >> >> HI,
> > >> >>
> > >> >> Can you give more details  ?
> > >> >>
> > >> >> -Wei
> > >> >>
> > >> >> On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <ss...@gmail.com>
> > >> wrote:
> > >> >>
> > >> >> > Hello All
> > >> >> >
> > >> >> > Kubernetes cluster is stucked in the 'Starting' state, even
> though
> > >> VM's,
> > >> >> > LB, Firewall Rules are been created properly.
> > >> >> > Have any one faced such issue?
> > >> >> >
> > >> >> > [image: image.png]
> > >> >> >
> > >> >> > Any help would be really appreciated. Thank you!
> > >> >> >
> > >> >> > Regards,
> > >> >> > Sanjay Kumar
> > >> >> >
> > >> >> >
> > >> >> >
> > >> >> >
> > >> >> >
> > >> >> >
> > >> >>
> > >> >
> > >>
> > >
> >
>

Re: Kubernetes stuck in the starting state

[Sanjay Kumar <ss...@gmail.com>]

Hi Wei,

Could you please help me out on this? How could we access the console node
or any link that could be help for us.


On Tue, Mar 14, 2023 at 2:53 PM Wei ZHOU <us...@gmail.com> wrote:

> It looks like the K8s API server is up, but the dashboard is not.
>
> Can you log into the console node (from the cloudstack management server)
> and check the status?
>
> -Wei
>
> On Tue, 14 Mar 2023 at 09:24, Sanjay Kumar <ss...@gmail.com> wrote:
>
> > Hi Wei,
> >
> > Thank you so much for your prompt response!
> >
> > Now the running kubernetes cluster is showing in alerts state after 1
> > hours.
> >
> > cloudstack version: 4.17.2
> > kubernetes version : Kubernetes 1.23.3
> > network details (services, rules, etc) : Network name
> > (k8s-test-cluster-network)
> > vm details (template, offering, etc) k8s-test-cluster-control-186debc31f4
> > , SystemVM Template (KVM) , Compute offering (Basic-1-Linux)
> >
> > Please access the logs from below link. It was long size that's why we
> > uploaded in drive.
> >
> >
> >
> https://drive.google.com/drive/folders/1TC_hw_h_S_TwqnT8fhoB4S3f2WHVlncx?usp=sharing
> >
> > [image: image.png]
> >
> > With Regards,
> > Sanjay Kumar
> >
> >
> >
> > On Tue, Mar 14, 2023 at 1:18 PM Wei ZHOU <us...@gmail.com> wrote:
> >
> >> Hi Sanjay,
> >>
> >> To be clear, can you share the details like below ?
> >>
> >> cloudstack version
> >> kubernetes version
> >> network details (services, rules, etc)
> >> vm details (template, offering, etc)
> >> management logs
> >> agent logs
> >> anything else you can share.
> >>
> >>
> >> -Wei
> >> On Tue, 14 Mar 2023 at 08:26, Sanjay Kumar <ss...@gmail.com>
> wrote:
> >>
> >> > Hi Wei,
> >> >
> >> > Thanks for the quick response!
> >> >
> >> > We have checked the event as well and VM's, LB, Firewall Rules have
> been
> >> > created properly but the cluster still showing starting state. we are
> >> > attaching the screenshot for the same.
> >> >
> >> >
> >> > On Tue, Mar 14, 2023 at 12:48 PM Wei ZHOU <us...@gmail.com>
> >> wrote:
> >> >
> >> >> HI,
> >> >>
> >> >> Can you give more details  ?
> >> >>
> >> >> -Wei
> >> >>
> >> >> On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <ss...@gmail.com>
> >> wrote:
> >> >>
> >> >> > Hello All
> >> >> >
> >> >> > Kubernetes cluster is stucked in the 'Starting' state, even though
> >> VM's,
> >> >> > LB, Firewall Rules are been created properly.
> >> >> > Have any one faced such issue?
> >> >> >
> >> >> > [image: image.png]
> >> >> >
> >> >> > Any help would be really appreciated. Thank you!
> >> >> >
> >> >> > Regards,
> >> >> > Sanjay Kumar
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >>
> >> >
> >>
> >
>

Re: Kubernetes stuck in the starting state

[Wei ZHOU <us...@gmail.com>]

It looks like the K8s API server is up, but the dashboard is not.

Can you log into the console node (from the cloudstack management server)
and check the status?

-Wei

On Tue, 14 Mar 2023 at 09:24, Sanjay Kumar <ss...@gmail.com> wrote:

> Hi Wei,
>
> Thank you so much for your prompt response!
>
> Now the running kubernetes cluster is showing in alerts state after 1
> hours.
>
> cloudstack version: 4.17.2
> kubernetes version : Kubernetes 1.23.3
> network details (services, rules, etc) : Network name
> (k8s-test-cluster-network)
> vm details (template, offering, etc) k8s-test-cluster-control-186debc31f4
> , SystemVM Template (KVM) , Compute offering (Basic-1-Linux)
>
> Please access the logs from below link. It was long size that's why we
> uploaded in drive.
>
>
> https://drive.google.com/drive/folders/1TC_hw_h_S_TwqnT8fhoB4S3f2WHVlncx?usp=sharing
>
> [image: image.png]
>
> With Regards,
> Sanjay Kumar
>
>
>
> On Tue, Mar 14, 2023 at 1:18 PM Wei ZHOU <us...@gmail.com> wrote:
>
>> Hi Sanjay,
>>
>> To be clear, can you share the details like below ?
>>
>> cloudstack version
>> kubernetes version
>> network details (services, rules, etc)
>> vm details (template, offering, etc)
>> management logs
>> agent logs
>> anything else you can share.
>>
>>
>> -Wei
>> On Tue, 14 Mar 2023 at 08:26, Sanjay Kumar <ss...@gmail.com> wrote:
>>
>> > Hi Wei,
>> >
>> > Thanks for the quick response!
>> >
>> > We have checked the event as well and VM's, LB, Firewall Rules have been
>> > created properly but the cluster still showing starting state. we are
>> > attaching the screenshot for the same.
>> >
>> >
>> > On Tue, Mar 14, 2023 at 12:48 PM Wei ZHOU <us...@gmail.com>
>> wrote:
>> >
>> >> HI,
>> >>
>> >> Can you give more details  ?
>> >>
>> >> -Wei
>> >>
>> >> On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <ss...@gmail.com>
>> wrote:
>> >>
>> >> > Hello All
>> >> >
>> >> > Kubernetes cluster is stucked in the 'Starting' state, even though
>> VM's,
>> >> > LB, Firewall Rules are been created properly.
>> >> > Have any one faced such issue?
>> >> >
>> >> > [image: image.png]
>> >> >
>> >> > Any help would be really appreciated. Thank you!
>> >> >
>> >> > Regards,
>> >> > Sanjay Kumar
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >>
>> >
>>
>

Re: Kubernetes stuck in the starting state

[Sanjay Kumar <ss...@gmail.com>]

Hi Wei,

Thank you so much for your prompt response!

Now the running kubernetes cluster is showing in alerts state after 1
hours.

cloudstack version: 4.17.2
kubernetes version : Kubernetes 1.23.3
network details (services, rules, etc) : Network name
(k8s-test-cluster-network)
vm details (template, offering, etc) k8s-test-cluster-control-186debc31f4 ,
SystemVM Template (KVM) , Compute offering (Basic-1-Linux)

Please access the logs from below link. It was long size that's why we
uploaded in drive.

https://drive.google.com/drive/folders/1TC_hw_h_S_TwqnT8fhoB4S3f2WHVlncx?usp=sharing

[image: image.png]

With Regards,
Sanjay Kumar



On Tue, Mar 14, 2023 at 1:18 PM Wei ZHOU <us...@gmail.com> wrote:

> Hi Sanjay,
>
> To be clear, can you share the details like below ?
>
> cloudstack version
> kubernetes version
> network details (services, rules, etc)
> vm details (template, offering, etc)
> management logs
> agent logs
> anything else you can share.
>
>
> -Wei
> On Tue, 14 Mar 2023 at 08:26, Sanjay Kumar <ss...@gmail.com> wrote:
>
> > Hi Wei,
> >
> > Thanks for the quick response!
> >
> > We have checked the event as well and VM's, LB, Firewall Rules have been
> > created properly but the cluster still showing starting state. we are
> > attaching the screenshot for the same.
> >
> >
> > On Tue, Mar 14, 2023 at 12:48 PM Wei ZHOU <us...@gmail.com> wrote:
> >
> >> HI,
> >>
> >> Can you give more details  ?
> >>
> >> -Wei
> >>
> >> On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <ss...@gmail.com>
> wrote:
> >>
> >> > Hello All
> >> >
> >> > Kubernetes cluster is stucked in the 'Starting' state, even though
> VM's,
> >> > LB, Firewall Rules are been created properly.
> >> > Have any one faced such issue?
> >> >
> >> > [image: image.png]
> >> >
> >> > Any help would be really appreciated. Thank you!
> >> >
> >> > Regards,
> >> > Sanjay Kumar
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >>
> >
>

Re: Kubernetes stuck in the starting state

[Wei ZHOU <us...@gmail.com>]

Hi Sanjay,

To be clear, can you share the details like below ?

cloudstack version
kubernetes version
network details (services, rules, etc)
vm details (template, offering, etc)
management logs
agent logs
anything else you can share.


-Wei
On Tue, 14 Mar 2023 at 08:26, Sanjay Kumar <ss...@gmail.com> wrote:

> Hi Wei,
>
> Thanks for the quick response!
>
> We have checked the event as well and VM's, LB, Firewall Rules have been
> created properly but the cluster still showing starting state. we are
> attaching the screenshot for the same.
>
>
> On Tue, Mar 14, 2023 at 12:48 PM Wei ZHOU <us...@gmail.com> wrote:
>
>> HI,
>>
>> Can you give more details  ?
>>
>> -Wei
>>
>> On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <ss...@gmail.com> wrote:
>>
>> > Hello All
>> >
>> > Kubernetes cluster is stucked in the 'Starting' state, even though VM's,
>> > LB, Firewall Rules are been created properly.
>> > Have any one faced such issue?
>> >
>> > [image: image.png]
>> >
>> > Any help would be really appreciated. Thank you!
>> >
>> > Regards,
>> > Sanjay Kumar
>> >
>> >
>> >
>> >
>> >
>> >
>>
>

Re: Kubernetes stuck in the starting state

[Sanjay Kumar <ss...@gmail.com>]

Hi Wei,

Thanks for the quick response!

We have checked the event as well and VM's, LB, Firewall Rules have been
created properly but the cluster still showing starting state. we are
attaching the screenshot for the same.


On Tue, Mar 14, 2023 at 12:48 PM Wei ZHOU <us...@gmail.com> wrote:

> HI,
>
> Can you give more details  ?
>
> -Wei
>
> On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <ss...@gmail.com> wrote:
>
> > Hello All
> >
> > Kubernetes cluster is stucked in the 'Starting' state, even though VM's,
> > LB, Firewall Rules are been created properly.
> > Have any one faced such issue?
> >
> > [image: image.png]
> >
> > Any help would be really appreciated. Thank you!
> >
> > Regards,
> > Sanjay Kumar
> >
> >
> >
> >
> >
> >
>

Re: Kubernetes stuck in the starting state

[Wei ZHOU <us...@gmail.com>]

HI,

Can you give more details  ?

-Wei

On Tue, 14 Mar 2023 at 07:20, Sanjay Kumar <ss...@gmail.com> wrote:

> Hello All
>
> Kubernetes cluster is stucked in the 'Starting' state, even though VM's,
> LB, Firewall Rules are been created properly.
> Have any one faced such issue?
>
> [image: image.png]
>
> Any help would be really appreciated. Thank you!
>
> Regards,
> Sanjay Kumar
>
>
>
>
>
>