You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "DaanHoogland (via GitHub)" <gi...@apache.org> on 2024/05/24 09:06:57 UTC
[I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
DaanHoogland opened a new issue, #9121:
URL: https://github.com/apache/cloudstack/issues/9121
<!--
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release and main branch are affected too.
Always add information AFTER of these HTML comments, but no need to delete the comments.
-->
##### ISSUE TYPE
<!-- Pick one below and delete the rest -->
* Bug Report
##### COMPONENT NAME
<!--
Categorize the issue, e.g. API, VR, VPN, UI, etc.
-->
~~~
CKS
~~~
##### CLOUDSTACK VERSION
<!--
New line separated list of affected versions, commit ID for issues on main branch.
-->
~~~
4.19.0
~~~
##### CONFIGURATION
<!--
Information about the configuration if relevant, e.g. basic network, advanced networking, etc. N/A otherwise
-->
simple installation with CKS enabled
##### OS / ENVIRONMENT
<!--
Information about the environment if relevant, N/A otherwise
-->
4.19 with any hypervisor/network model
##### SUMMARY
<!-- Explain the problem/feature briefly -->
when starting a CKS cluster the control node does not enable ssh and thus the cluster never comes up.
##### STEPS TO REPRODUCE
<!--
For bugs, show exactly how to reproduce the problem, using a minimal test-case. Use Screenshots if accurate.
For new features, show how the feature would be used.
-->
<!-- Paste example playbooks or commands between quotes below -->
~~~
enable cks
install a cks image
deploy a cluster
~~~
<!-- You can also paste gist.github.com links for larger files -->
##### EXPECTED RESULTS
<!-- What did you expect to happen when running the steps above? -->
~~~
cluster comes up (control node accessable with ssh)
~~~
##### ACTUAL RESULTS
<!-- What actually happened? -->
<!-- Paste verbatim command output between quotes below -->
~~~
no ssh deamon active on the control node and hence deploy times out.
~~~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "JoaoJandre (via GitHub)" <gi...@apache.org>.
JoaoJandre commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2353779122
> if this issue happens with vmware 8.0u1/u2/u3, it should have been addressed by #9625
>
> cc @DaanHoogland @rohityadavcloud @sureshanaparti @JoaoJandre
@weizhouapache I do not have a VMware 8 env to test this.
Could someone validate if the issue persists after #9625?
cc @DaanHoogland @rohityadavcloud @sureshanaparti
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "luganofer (via GitHub)" <gi...@apache.org>.
luganofer commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2129695893
@weizhouapache
In the tests I did recreating the problem, the connectivity between the public ip of CKS cluster and the managers is enabled and the problem occurs anyway. Also, if one of the nodes is accessed via console and the ssh service is started manually, from the managers I can establish the ssh connection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2149802989
> it was 80u1 , @weizhouapache
- 80u1 (8.0.1.0) is not working. See #7572. Do not run 4.18/4.19 test with it. However, 4.20 seems to be working with 80u1.
- we use 8.0b (8.0.0.2) in Trillian tests with vmware-80. It has been run many times. The test results look good.
- the reporter uses 8.0c (8.0.0.3, if the version is correct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2130417410
> @weizhouapache At least in my lab environment it happens in every deployment I try and with several different k8s versions (1.28.4, 1.27.8, 1.27.3, 1.26.6)
@luganofer
can you also the hypervisor type and version, the link of cks iso ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2149194217
> > cks tried by me is 1.27.8, but user reported trying several versions. Host os is vmware, and I will verify others and update the description. I am first checking 4.18.1 (and possibly before) to see when it was introduced.
>
> Sorry, I forgot to feedback; xcpng and kvm seem to work, just vmware is broken.
which vmware version did you test? @DaanHoogland
It seems to be working in Trillian tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2354679689
Tested on both 8.0u2 and 8.0u3 both clusters are marked as running, so I think it safe to assume this is solved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2149362008
What test is verifying this @weizhouapache ? (as I recall it was 70u3, but I'll check)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "luganofer (via GitHub)" <gi...@apache.org>.
luganofer commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2148757049
Hi @Pearl1594, no error logs en console VM.
Only the following error is observed in the logs of the managers:
ERROR [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-84:ctx-216a9879 job-150679 ctx-7f68a95d) (logid:e493dc8f) Failed to setup Kubernetes cluster : maradona in usable state as unable to access control node VMs of the cluster
From my perspective, the problem is related to nodes that do not initialise correctly (cloud-init ?). They receive ip by dhcp from the VR, but do not change the hostname and fundamentally do not start the ssh service so the deployed nodes cannot be reached by the acs managers (via ssh) and the correct deployment of the k8s cluster is not completed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2149369357
it was 80u1 , @weizhouapache
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2129840038
> the problem occurs anyway. Also, if one of the nodes is accessed via console and the ssh service is started manually, from the managers I can establish the ssh connection.
ok @luganofer
did it happen every time ? or just once ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2148902353
> Hi @Pearl1594, no error logs en console VM.
> Only the following error is observed in managers logs:
>
> ERROR [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-84:ctx-216a9879 job-150679 ctx-7f68a95d) (logid:e493dc8f) Failed to setup Kubernetes cluster : maradona in usable state as unable to access control node VMs of the cluster
>
> From my perspective, the problem is related to nodes that do not initialise correctly (cloud-init ?). They receive ip by dhcp from the VR, but do not change the hostname and fundamentally do not start the ssh service so the deployed nodes cannot be reached by the acs managers (via ssh) and the correct deployment of the k8s cluster is not completed.
>
@luganofer
If you are able to log into the vm (is the password still "password"?) and restart ssh, can you check the cloud-init logs? /var/log/cloud-init-*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "Pearl1594 (via GitHub)" <gi...@apache.org>.
Pearl1594 commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2146084754
@luganofer As the nodes / VMs come up, do you see any error logs in the VM console?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "luganofer (via GitHub)" <gi...@apache.org>.
luganofer commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2129980343
@weizhouapache At least in my lab environment it happens in every deployment I try and with several different k8s versions (1.28.4, 1.27.8, 1.27.3, 1.26.6)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2129025216
@DaanHoogland
can you share the hypervisor type and version ? cks iso link ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland closed issue #9121: CKS: cluster does not fully deploy due to sshd not started on control node
URL: https://github.com/apache/cloudstack/issues/9121
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2341500750
if this issue happens with vmware 8.0u1/u2/u3, it should have been addressed by #9625
cc @DaanHoogland @rohityadavcloud @sureshanaparti @JoaoJandre
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2304444474
@DaanHoogland
there is a known issue that systemvm/cks node is stuck at Starting on vmware 80u1
https://github.com/apache/cloudstack/issues/7572
@DaanHoogland will you move this to 4.20.0.0 milestone and test it later ?
@sureshanaparti is working on vmware 80u1/u2/u3 support in 4.20.0.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2129031199
cks tried by me is 1.27.8, but user reported trying several versions. Host os is vmware, and I will verify others and update the description. I am first checking 4.18.1 (and possibly before) to see when it was introduced.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "weizhouapache (via GitHub)" <gi...@apache.org>.
weizhouapache commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2129036205
and network type, etc
the public ip of the CKS cluster should be accessible from cloudstack mgmt server
in some setup, if the mgmt server (in private network) cannot access the cks nodes (via public IP) and get the status of cks cluster, the cluster might end in Error state
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "luganofer (via GitHub)" <gi...@apache.org>.
luganofer commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2130480099
@weizhouapache
I am using VMware vSphere 8.0c and all the ISOs were downloaded from the following link: https://download.cloudstack.org/cks/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] CKS: cluster does not fully deploy due to sshd not started on control node [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on issue #9121:
URL: https://github.com/apache/cloudstack/issues/9121#issuecomment-2149106902
> cks tried by me is 1.27.8, but user reported trying several versions. Host os is vmware, and I will verify others and update the description. I am first checking 4.18.1 (and possibly before) to see when it was introduced.
Sorry, I forgot to feedback; xcpng and kvm seem to work, just vmware is broken.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org