You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2022/03/11 06:07:09 UTC

[GitHub] [cloudstack] sidre9k opened a new issue #6094: Cloudstack Kuberentes getting stuck in Starting State

sidre9k opened a new issue #6094:
URL: https://github.com/apache/cloudstack/issues/6094


   ##### ISSUE TYPE
    * Bug Report
   
   ##### CLOUDSTACK VERSION
   4.16
   
   ##### CONFIGURATION
   Advanced networking
   
   ##### OS / ENVIRONMENT
   CentOS 7.9
   
   ##### SUMMARY
   Kubernetes Cluster creation gets stuck in Starting state indefinitely
   
   ##### STEPS TO REPRODUCE
   This issue happens intermittently while creating new kuberentes cluster in cloudstack.
   
   ##### EXPECTED RESULTS
   Kubernetes Cluster creation should transition to Running state from Starting state
   
   ##### ACTUAL RESULTS
   Kubernetes Cluster creation gets stuck in "Starting" state indefinitely
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] sidre9k commented on issue #6094: Cloudstack Kuberentes getting stuck in Starting State

Posted by GitBox <gi...@apache.org>.

sidre9k commented on issue #6094:
URL: https://github.com/apache/cloudstack/issues/6094#issuecomment-1072207402


   @sureshanaparti Attaching MGMT servers logs herewith.
   [k8s-sid.log](https://github.com/apache/cloudstack/files/8303033/k8s-sid.log)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] sureshanaparti commented on issue #6094: Cloudstack Kuberentes getting stuck in Starting State

Posted by GitBox <gi...@apache.org>.

sureshanaparti commented on issue #6094:
URL: https://github.com/apache/cloudstack/issues/6094#issuecomment-1066704210


   Hi @sidre9k Can you share the management server logs during this issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] sureshanaparti commented on issue #6094: Cloudstack Kuberentes getting stuck in Starting State

Posted by GitBox <gi...@apache.org>.

sureshanaparti commented on issue #6094:
URL: https://github.com/apache/cloudstack/issues/6094#issuecomment-1066708681


   @sidre9k Is it the same issue as https://github.com/apache/cloudstack/issues/5999 ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] sidre9k commented on issue #6094: Cloudstack Kuberentes getting stuck in Starting State

Posted by GitBox <gi...@apache.org>.

sidre9k commented on issue #6094:
URL: https://github.com/apache/cloudstack/issues/6094#issuecomment-1066719099


   > [sureshanaparti](/sureshanaparti)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] sidre9k commented on issue #6094: Cloudstack Kuberentes getting stuck in Starting State

Posted by GitBox <gi...@apache.org>.

sidre9k commented on issue #6094:
URL: https://github.com/apache/cloudstack/issues/6094#issuecomment-1072206469


   @sureshanaparti Attaching mgmt servers logs herewith.
   `2022-03-10 17:48:58,360 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:18,371 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:38,382 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:58,393 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:18,404 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:38,415 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:58,426 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:18,437 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:38,448 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:58,459 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:18,470 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:38,481 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:58,492 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:18,502 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:38,513 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:58,524 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:18,529 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:38,540 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:58,550 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:03,869 INFO  [c.c.a.ApiServer] (qtp1603198149-26090:ctx-50dbb34c ctx-bfeb2abd) (logid:8094e808) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:14,316 INFO  [c.c.a.ApiServer] (qtp1603198149-26090:ctx-9b7fafc0 ctx-79479e6f) (logid:885c6916) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:17,359 INFO  [c.c.a.ApiServer] (qtp1603198149-26087:ctx-efd30324 ctx-5916a61e) (logid:974dfb72) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:18,561 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:38,571 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:58,582 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:18,584 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:38,596 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:58,606 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:18,617 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:38,628 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:58,636 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:18,647 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:38,658 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:40,237 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-acdc0c28) (logid:ae83f9b3) Seq 3-724516590053290113: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 17:58:58,669 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:18,680 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:38,691 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:58,702 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:18,712 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:38,723 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:58,725 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:18,736 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:38,747 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:58,758 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:18,768 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:38,779 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:58,789 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:18,800 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:38,811 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:58,822 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:18,833 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:38,844 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:58,855 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:18,866 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:38,876 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:58,889 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:18,900 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:38,911 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:58,922 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:18,932 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:38,944 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:58,955 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:18,967 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:38,946 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-5639dca3) (logid:fc60836a) Seq 3-724516590053290160: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:08:38,978 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:58,980 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:18,989 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:39,000 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:59,009 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:19,020 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:39,030 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:59,041 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:19,052 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:39,063 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:59,074 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:19,085 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:39,096 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:59,107 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:19,117 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:39,128 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:59,129 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:19,140 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:39,146 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:59,157 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:19,162 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:39,172 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:59,182 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:19,193 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:39,204 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:59,214 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:19,220 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:39,225 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:59,236 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:19,247 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:38,950 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-2d313558) (logid:4622835f) Seq 3-724516590053290208: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:18:39,257 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:59,268 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:19,278 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:39,289 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:59,290 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:19,301 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:39,312 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:59,323 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:19,334 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:39,345 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:59,356 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:19,367 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:39,378 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:59,389 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:19,399 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:39,410 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:59,421 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:19,430 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:39,441 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:59,452 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:19,464 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:39,475 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:59,485 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:19,497 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:39,508 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:59,513 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:19,524 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:39,535 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:59,546 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:19,558 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:39,569 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:40,086 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-5e7c7e3b) (logid:370d4c7c) Seq 3-724516590053290257: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:28:59,580 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:19,590 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:39,601 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:59,611 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:30:09,612 ERROR [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Failed to setup Kubernetes cluster : k8s-sid in usable state as unable to access control node VMs of the cluster
   2022-03-10 18:30:15,088 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-control-17f73b3832e in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:21,849 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-control-17f73b3a94f in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:23,218 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b3cc83 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:28,736 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b3eb89 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:30,903 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b40a72 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:30,903 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-101:ctx-aed659b4 job-1811) (logid:0a27d378) Complete async job-1811, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Failed to setup Kubernetes cluster : k8s-sid in usable state as unable to access control node VMs of the cluster"}
   `
   
   
   
   Find channel
   
   FAVORITES
   Copy Link
   [Implementations](https://nxtchat.nxtgen.com/emerging/channels/implementations)
   Copy Link
   [OpenNebula](https://nxtchat.nxtgen.com/emerging/channels/opennebula)
   Copy Link
   [InfraOps](https://nxtchat.nxtgen.com/emerging/channels/infraops)
   Copy Link
   [Monitoring / Observability](https://nxtchat.nxtgen.com/emerging/channels/monitoring--observability)
   Copy Link
   [ET-System Engineering](https://nxtchat.nxtgen.com/emerging/channels/et-system-engineering)
   Copy Link
   [CEPH Storage](https://nxtchat.nxtgen.com/emerging/channels/ceph)
   Copy Link
   [EDGE](https://nxtchat.nxtgen.com/emerging/channels/edge)
   Copy Link
   [Networking](https://nxtchat.nxtgen.com/emerging/channels/networkingcni)
   Copy Link
   [HPC](https://nxtchat.nxtgen.com/emerging/channels/hpc)
   
   CHANNELS
   Copy Link
   [Emerging Technologies](https://nxtchat.nxtgen.com/emerging/channels/town-square)
   Copy Link
   [Off-Topic](https://nxtchat.nxtgen.com/emerging/channels/off-topic)
   Copy Link
   [Uptime Kuma](https://nxtchat.nxtgen.com/emerging/channels/uptime-kuma)
   
   DIRECT MESSAGES
   
   Copy Link
   [jayanth](https://nxtchat.nxtgen.com/emerging/messages/@jayanth)
   Copy Link
   [ejaz](https://nxtchat.nxtgen.com/emerging/messages/@ejaz)
   Copy Link
   [rajesh_dangi](https://nxtchat.nxtgen.com/emerging/messages/@rajesh_dangi)
   Copy Link
   [siddhit (you)](https://nxtchat.nxtgen.com/emerging/messages/@siddhit)
   
   Invite Members
   
   Online
   
   
   
   Add a channel description
   At 1:15 PM Friday, March 18, jayanth wrote, , 1 attachment
   February 23
   	
   siddhit
   [4:28 PM](https://nxtchat.nxtgen.com/emerging/pl/jitze868aty6fmpate3ueswxih)
   
   February 24
   	
   jayanth
   [8:59 AM](https://nxtchat.nxtgen.com/emerging/pl/c4qh9i4attn3f89tffasyz4fra)
   https://docs.cloudstack.apache.org/en/latest/conceptsandterminology/concepts.html#massively-scalable-infrastructure-management
   
   	
   jayanth
   [9:44 AM](https://nxtchat.nxtgen.com/emerging/pl/ywjsxqfsz3bitdi6kgey3iy5ch)
   https://docs.cloudstack.apache.org/en/latest/conceptsandterminology/choosing_deployment_architecture.html#multi-site-deployment
   
   EDGE?
   
   	
   jayanth
   [10:33 AM](https://nxtchat.nxtgen.com/emerging/pl/9t11aqh8tbg8idc3fyexdw663r)
   
   	
   jayanth
   [11:51 AM](https://nxtchat.nxtgen.com/emerging/pl/j9psbgf7m3dbpq61proq1a7r7o)
   rgw trust forwarded https
   rgw_crypt_require_ssl
   	
   jayanth
   [12:36 PM](https://nxtchat.nxtgen.com/emerging/pl/x8sorbqekprcdg3ot38zma1yah)
   https://docs.ceph.com/en/latest/radosgw/encryption/#automatic-encryption-for-testing-only
   
   	
   siddhit
   [5:26 PM](https://nxtchat.nxtgen.com/emerging/pl/8qu9ywh6t3rmpr37kdrhcmqp8y)
   GPU: https://lab.piszki.pl/cloudstack-kvm-and-running-vm-with-vgpu/
   
   Piszki Lab | EN
   [CloudStack – KVM and running VM with vGPU](https://lab.piszki.pl/cloudstack-kvm-and-running-vm-with-vgpu/)
   GPU cards (such as Nvidia V100) have recently gained popularity in many companies and other places (such as universities). Such a card can be used in many ways, for CUDA calculations but also for v… 
   
   February 27
   	
   jayanth
   [10:18 AM](https://nxtchat.nxtgen.com/emerging/pl/c3n6dx1zofdqdn7bqmf3khqpje)
   Policy based routing is present.
   
   
   	
   siddhit
   [12:38 PM](https://nxtchat.nxtgen.com/emerging/pl/61rzkcd6z7r598ozacr4ywjn3r)
   Is this on cloudstack appliance?
   
   Or opennebula?
   
   	
   jayanth
   [1:51 PM](https://nxtchat.nxtgen.com/emerging/pl/uaxigzr4fibficwoonhtda59tc)
   Cloudstack. 
   
   	
   siddhit
   [5:30 PM](https://nxtchat.nxtgen.com/emerging/pl/potsjx1zjtd7zgx5kdkdi8tsaw)
   Okay
   
   February 28
   	
   jayanth
   [5:01 PM](https://nxtchat.nxtgen.com/emerging/pl/eipp1appy7frtgt6mugs1nqywo)
   https://www.virtuozzo.com/resources/
   
   Virtuozzo
   [Resources](https://www.virtuozzo.com/resources/)
   Here you will find all the useful resources and other collateral such as whitepapers, video, case studies and much more.
   
   	
   jayanth
   [5:22 PM](https://nxtchat.nxtgen.com/emerging/pl/yp5htfsu77858gm583ytdz8cgr)
   https://www.virtuozzo.com/wp-content/uploads/2021/04/vip-use-cases-sb-en-ltr.pdf
   
   	
   jayanth
   [5:41 PM](https://nxtchat.nxtgen.com/emerging/pl/53t9i5pu6tg4zbu7otwuy8djee)
   https://github.com/apache/cloudstack/pull/5909
   
   GitHub
   [Quota custom tariffs by GutoVeronezi · Pull Request #5909 · apache/cloudstack](https://github.com/apache/cloudstack/pull/5909)
   Description This PR solves issue #5891. The main focus of this PR is to provide a flexible manner for operators to charge and bill their cloud resources. Issue #5891 has more details and flowcharts... 
   
   March 01
   	
   jayanth
   [12:01 PM](https://nxtchat.nxtgen.com/emerging/pl/cqz5b53pd3robm3aq8w458kmew)
   
   ..
   
   March 02
   	
   [jayanth]()
   [9:35 AM](https://nxtchat.nxtgen.com/emerging/pl/cm35r69ibtddpm9aa54r4y8fhh)
   https://www.nginx.com/c/microservices-march-2022-kubernetes-networking/?utm_medium=email&utm_source=nginxdb&utm_campaign=ww-nx_pgkub_ok&utm_content=ev&mkt_tok=NjUzLVNNQy03ODMAAAGC5i-vOEyuqUIpQmiGekNmtmJz1zXsCwfV_zzvEYot_F1E6NyeXls_CytWWsqXd9tA5449TjxIAGrzbcgUo_ap2Rc5s9ha-_8LevzpzT7JWFRI4PcBNQ
   
   NGINX
   [Microservices March 2022: Kubernetes Networking - NGINX](https://www.nginx.com/c/microservices-march-2022-kubernetes-networking/?utm_medium=email&utm_source=nginxdb&utm_campaign=ww-nx_pgkub_ok&utm_content=ev&mkt_tok=NjUzLVNNQy03ODMAAAGC5i-vOEyuqUIpQmiGekNmtmJz1zXsCwfV_zzvEYot_F1E6NyeXls_CytWWsqXd9tA5449TjxIAGrzbcgUo_ap2Rc5s9ha-_8LevzpzT7JWFRI4PcBNQ)
   Microservices March 2022 is a free program that takes your Kubernetes networking skills from Zero to Hero. You’ll learn about Ingress controllers, service meshes, API gateways, and more! Register today. 
   
   	
   jayanth
   [7:57 PM](https://nxtchat.nxtgen.com/emerging/pl/wqk7gokyajnafm8o6frujk8uzy)
   Responsibilities:
   
   Deploy, manage, support and monitor production-grade cloud solutions.
   Evaluate, recommend, productionize Open Source solutions and systems for strengthening existing and new cloud stacks.
   Perform Proof of Concept (PoC) builds and extensive functional evaluation of Open Source systems.
   Interact with the customers and users to deploy and support cloud requirements, meeting deadlines and committed SLAs.
   Provide detailed Incident Reports with RCA.
   Troubleshoot problems encountered during installation, configuration and maintenance. Maintain the hardware and software assets inventory.
   Supporting activities of the Engineer are to provide robust cloud platform by ensuring effective administration, secure network protocols, access constraints and processes in place.
   Maintain comprehensive documentation of daily tasks and projects and tickets.
   Maintain and contribute to in-house Knowledge Base to streamline redundant issues / resolutions process.
   Create and maintain Standard Operating Procedures and run-books for the deployed solutions.
   Coordinate with in-house development teams in an orderly manner to meet the business requirements.
   
   Skills required:
   
   Primarily proficient in GNU/Linux, and expertise on various Linux distributions.
   Strong hands-on knowledge on System Administration, Networking, Storage and Cloud Security.
   Expertise on the components of the cloud i.e., Compute, Networking, Storage and Security.
   Deep and hands-on understanding of the protocols DNS, DHCP, SSH, TCP, UDP, S3 etc.,
   Sound knowledge on Virtualization and Open Source Hypervisors.
   Knowledge on Reverse proxies and Load Balancers is an added advantage.
   Open Source software enthusiast.
   Patient and calm attitude to confront day to day problems encountered.
   Flexibility to work under critical conditions, dexterity in providing efficient cloud, storage and networking support.
   
   	
   jayanth
   [10:46 PM](https://nxtchat.nxtgen.com/emerging/pl/epcurjdcutfhjem614cryp954a)
   Siddhit, have sent the wrong one!
   
   Job Title:
   
   Associate - System Engineer
   
   Duties and Responsibilities:
   
   Evaluate, deploy, manage and support production-grade cloud solutions.
   Design, implementation, configuration, and administration of Linux and Windows Operating Systems to support various workloads.
   Monitoring and maintaining the deployed infrastructure.
   Ensuring the availability of deployed systems.
   Maintain the infrastructure inventory and sanity.
   Evaluate new Open Source Software solutions.
   Deploy Virtual and Physical servers.
   Perform Proof of Concept (PoC) builds and extensive functional evaluation of Open Source systems.
   Upgrade Applications and Tools, install patches, troubleshoot and resolve application problems, performance tuning and capacity planning.
   Interact with the customers and users to deploy and support cloud requirements, meeting deadlines and committed SLAs.
   Maintain and contribute to in-house Knowledge Base to streamline redundant issues / resolutions process.
   Create and maintain Standard Operating Procedures and run-books for the deployed solutions.
   Coordinate with in-house teams in an orderly manner to meet the business requirements.
   Patient and calm attitude to confront day to day problems encountered.
   Flexibility to work under critical conditions, dexterity in providing efficient cloud, storage and networking support.
   Keen interest contributing to and building communities in open source is a plus.
   
   Preferred qualification:
   
   Completed a university or technical school education focusing on IT, computer science, or engineering (or comparable professional experience).
   Hands-on knowledge in GNU/Linux, and expertise on renowned Linux distributions.
   Proficiency with common tools of the trade for automation.
   Fundamental networking knowledge.
   Ability to perform System Administration tasks.
   Fluency in verbal and written English.
   Fundamental knowledge on the components of the cloud i.e., Compute, Networking, Storage and Security.
   Understanding of the concepts of virtualization and cloud computing.
   Knowledge of Kernel-based Virtual Machine (KVM) is an added advantage.
   Knowledge of Kubernetes is an added advantage.
   Open Source enthusiast.
   
   About NxtGen:
   
   NxtGen provides data center and cloud services from our own high density data center (HDDC) facilities or can deploy centrally managed on-premise data centers (OPDC™). Our enterprise cloud services provide private clouds hosted within OPDCs and hosted private clouds or hybrid cloud infrastructure hosted at their HDDCs. NxtGen’s business-class infrastructure helps companies simplify IT infrastructure, reduce running costs, and enable business growth by creating additional capacity from existing infrastructure and providing the right public or private infrastructure to meet dynamic business demands.
   
   About the team:
   
   We are team of young engineers who are passionate about solving infrastructure challenges and provide robust cloud solutions to our end customers. Our key focus area is to embrace and adapt various open-source technologies, glue them together and build effective solutions around it. Become part of our growing team if this opportunity excites you! 
   
   Show more
   March 03
   	
   siddhit
   [7:37 AM](https://nxtchat.nxtgen.com/emerging/pl/esk8psu893nj3mygtwtgf4c73o)
   This is good
   
   	
   jayanth
   [9:31 AM](https://nxtchat.nxtgen.com/emerging/pl/5y619xbokbn13jwgquhka48ogr)
   Okay, I'm posting then.
   
   	
   jayanth
   [10:45 AM](https://nxtchat.nxtgen.com/emerging/pl/ycnxt49cxidh3x36j6actusxay)
   Have updated the SpeedCloud credentials on the sheet which has been shared to you.
   
   March 04
   	
   jayanth
   [11:18 AM](https://nxtchat.nxtgen.com/emerging/pl/qaeqgbzqmpy95exxkhjti6in7w)
   https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
   
   [HTTP headers - HTTP | MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers)
   HTTP headers let the client and the server pass additional information with an HTTP request or response. An HTTP header consists of its case-insensitive name followed by a colon (:), then by its value. Whitespace before the value is ignored. 
   
   March 07
   	
   jayanth
   [11:33 AM](https://nxtchat.nxtgen.com/emerging/pl/8fp9uszbmjgcffakkmd5cbu9mo)
   Anuj asked me to redeploy the VM.
   
   	
   siddhit
   [11:33 AM](https://nxtchat.nxtgen.com/emerging/pl/4i6qr6xxntdpi8pbzpk8w1dboo)
   Ok
   
   	
   jayanth
   [11:33 AM](https://nxtchat.nxtgen.com/emerging/pl/jx1o9ztnuj8uifjmqp31fgrbir)
   Doing that.
   
   	
   jayanth
   [4:25 PM](https://nxtchat.nxtgen.com/emerging/pl/ku5dcgcffinyxxix9mdn3dzbka)
   Anuj VM Creation done.
   
   March 08
   	
   jayanth
   [12:04 PM](https://nxtchat.nxtgen.com/emerging/pl/dzh9gu944ffitjxq9d7dzgwg8y)
   https://us04web.zoom.us/j/78293871969?pwd=CoEAvBx4MLDhGGsSBE8Hp_8H8SomT0.1 
   
   Zoom Video
   [Join our Cloud HD Video Meeting](https://us04web.zoom.us/j/78293871969?pwd=CoEAvBx4MLDhGGsSBE8Hp_8H8SomT0.1)
   Zoom is the leader in modern enterprise video communications, with an easy, reliable cloud platform for video and audio conferencing, chat, and webinars across mobile, desktop, and room systems. Zoom Rooms is the original software-based conference room solution used around the world in board, con...
   March 09
   	
   jayanth
   [12:33 PM](https://nxtchat.nxtgen.com/emerging/pl/s6m7gzb6jj81pb6jqxnnx3tfro)
   https://www.haproxy.com/blog/load-balancing-affinity-persistence-sticky-sessions-what-you-need-to-know/
   
   Very nice explanation.
   
   HAProxy Technologies
   [Load Balancing, Affinity, Persistence, Sticky Sessions: What You Need to Know - HAProxy Technologies](https://www.haproxy.com/blog/load-balancing-affinity-persistence-sticky-sessions-what-you-need-to-know/)
   This blog post shows why and how to use a load balancer, the differences between Affinity, Persistence and Sticky Sessions and how to enable them in HAProxy 
   
   	
   jayanth
   [2:10 PM](https://nxtchat.nxtgen.com/emerging/pl/wtin4nwoobyufj7cnxm7syzbha)[](https://nxtchat.nxtgen.com/emerging/messages/@jayanth#)[](https://nxtchat.nxtgen.com/api/v4/files/namgby3mdjrixyitbybraag16w?download=1)
   Sasi_DevOps_Fresher.pdf
   PDF162KB
   Thursday
   	
   siddhit
   [1:09 PM](https://nxtchat.nxtgen.com/emerging/pl/4zxnfsa7ejrb8nqtxif181s1ya)
   Lets dig deeper w.r.t yestd degraded cephfs logs and see if any pattern or event which could have caused the outage
   
   
   	
   siddhit
   [1:55 PM](https://nxtchat.nxtgen.com/emerging/pl/8oach3ge4tbyj8uj87n97hjodo)
   
   Pls Check IOPS and throughupt for 4MB and 512 KB blocksize for both RBD and NFS
   
   	
   siddhit
   [1:55 PM](https://nxtchat.nxtgen.com/emerging/pl/a849wy4a4byq58cuo8a7p9cuhc)
   
   i guess we did for 64 KB right?
   
   	
   jayanth
   [1:56 PM](https://nxtchat.nxtgen.com/emerging/pl/yyooeok8iid4dntesudzynizth)
   Correct.
   
   	
   jayanth
   [1:56 PM](https://nxtchat.nxtgen.com/emerging/pl/ba55nuqohtfi3p3zch3pdd8pqw)
   
   Sure.
   
   	
   jayanth
   1:58 PM
   
   IOPS at 4K, throughput at 64K.
   
   We'll have all tests done.
   
   	
   [siddhit]()
   2:52 PM
   SiteOPsAHD_BSNLIDC SiteOPsAHD_BSNLIDC@nxtgen.com
   
   	
   [siddhit]()
   [3:30 PM](https://nxtchat.nxtgen.com/emerging/pl/3fi9pd7h8jrpikudcxd3wdw77r)
   https://www.sebastien-han.fr/blog/2012/07/06/nfs-over-rbd/
   
   Today
   New Messages
   	
   jayanth
   [9:48 AM](https://nxtchat.nxtgen.com/emerging/pl/pb76bhi633b8zdgio6kkkgq8hc)
   
   AHD firewall itself is down/experiencing issues. 
   
   [9:48 AM](https://nxtchat.nxtgen.com/emerging/pl/wc9upw9w938ffypybztr63g3ya)
   14.192.19.27
   
   
   	
   jayanth
   [1:23 PM](https://nxtchat.nxtgen.com/emerging/pl/e99mpke7n3dhbmnz8y5hhhyjea)
   
   Any updates from Anuj - Riversilica? 
   
   	
   siddhit
   [1:56 PM](https://nxtchat.nxtgen.com/emerging/pl/cut8d4uxsig5zktn3tjdmzfokw)
   Nope.Informed Rajesh to expedite.Seems they are tied in some project.
   
   	
   jayanth
   [1:57 PM](https://nxtchat.nxtgen.com/emerging/pl/9mo178tgpbyubeg88tnqka6aao)
   Oh okay.
   
   	
   siddhit
   [4:18 PM](https://nxtchat.nxtgen.com/emerging/pl/1zg9wui13pbxpexr8h4ikg356r)
   is our pritunl setup still operational?
   
   	
   jayanth
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/u9drmh1gwpf3bf1i8y6ugor7ky)
   Ah, Pritunl? 
   
   barebones openvpn setup is still operational. No Pritunl controlled setups exist.
   
   	
   siddhit
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/8pi3jouwhpgmdrz4mp5ti977zo)
   ok
   
   	
   jayanth
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/swjgjd9upfdtmyak7piquxmtwc)
   Which location are you looking for?
   
   	
   siddhit
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/wq11zpciipbs3xxbgggffkx95o)
   none specific
   
   	
   jayanth
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/nitdgtj793rz8e9z7474rke7uy)
   Okay.
   
   	
   siddhit
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/enubdjzwi7yhue3qhkwu1it69o)
   just wanted to refer dashboard
   
   	
   jayanth
   [4:20 PM](https://nxtchat.nxtgen.com/emerging/pl/toujgdnfjfr33e3h38yt9nerqr)
   https://speedcloudstage.nxtgen.com/#vms-tab/1579
   
   check this out then.
   
   It's turned off currently.
   
   	
   siddhit
   [4:23 PM](https://nxtchat.nxtgen.com/emerging/pl/uxw578mrrprkpbzstjh66uxoew)
   
   can you add staging creds also in master sheet?
   
   	
   jayanth
   [4:23 PM](https://nxtchat.nxtgen.com/emerging/pl/ykhgwxqb4frj9yf377ae6serpw)
   Sure.
   
   	
   siddhit
   [4:39 PM](https://nxtchat.nxtgen.com/emerging/pl/3cezcr11bjb9up1d8mjxyufauy)
   Any new profiles for interview?
   
   	
   jayanth
   [4:40 PM](https://nxtchat.nxtgen.com/emerging/pl/9p7g1qf3ttr9jj7eoagqm9razh)
   2 new applicants. To be interviewed very soon.
   
   	
   siddhit
   [4:40 PM](https://nxtchat.nxtgen.com/emerging/pl/xrp9wxxun7ya9et9z71n3moaac)
   ok
   
   keep thread posted
   
   	
   jayanth
   [4:40 PM](https://nxtchat.nxtgen.com/emerging/pl/yqrm7nghijn1bp3s1gsdtb6c1a)
   Sure,
   
   	
   jayanth
   [4:49 PM](https://nxtchat.nxtgen.com/emerging/pl/gmuu3sgqxtf9tn66imbb695ejr)
   No interviews have been done from your end right for Associate profiles right, just confirming. 
   
   	
   siddhit
   [4:59 PM](https://nxtchat.nxtgen.com/emerging/pl/rrj6fkcy5jnn9fopipep6mayka)
   not after you initiated profiling 
   
   	
   jayanth
   [4:59 PM](https://nxtchat.nxtgen.com/emerging/pl/s1kemtckft8k5qdcpenoq5c8nh)
   Okay.
   
   	
   siddhit
   [4:59 PM](https://nxtchat.nxtgen.com/emerging/pl/1cdi7x1or7drmdayro185iomuo)
   serveral before them..will collate and share list for past reviewed candidates
   
   	
   jayanth
   [4:59 PM](https://nxtchat.nxtgen.com/emerging/pl/u6mf6gaucj8qbktj6zhao67apr)
   Okay.
   
   	
   jayanth
   [5:05 PM](https://nxtchat.nxtgen.com/emerging/pl/basmfcbkkfr8jk9h8b6sz5kq9r)
   
   Done.
   
   Today
   	
   jayanth
   [1:15 PM](https://nxtchat.nxtgen.com/emerging/pl/wwahu1uz3ib7ijfpmrayp5i4zr)[](https://nxtchat.nxtgen.com/emerging/messages/@jayanth#)[](https://nxtchat.nxtgen.com/api/v4/files/w8csjfze7jnkbppw7mxsertrdh?download=1)
   k8s-sid.log
   LOG36KB
   Write to jayanth
   
   No file chosen
   
   [Help](https://nxtchat.nxtgen.com/help/messaging?locale=en)[](https://nxtchat.nxtgen.com/api/v4/files/w8csjfze7jnkbppw7mxsertrdh?download=1)
   k8s-sid.log
   JayanthShared In ~Jayanth
   
   2022-03-10 17:48:58,360 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:18,371 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:38,382 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:58,393 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:18,404 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:38,415 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:58,426 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:18,437 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:38,448 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:58,459 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:18,470 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:38,481 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:58,492 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:18,502 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:38,513 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:58,524 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:18,529 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:38,540 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:58,550 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:03,869 INFO  [c.c.a.ApiServer] (qtp1603198149-26090:ctx-50dbb34c ctx-bfeb2abd) (logid:8094e808) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:14,316 INFO  [c.c.a.ApiServer] (qtp1603198149-26090:ctx-9b7fafc0 ctx-79479e6f) (logid:885c6916) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:17,359 INFO  [c.c.a.ApiServer] (qtp1603198149-26087:ctx-efd30324 ctx-5916a61e) (logid:974dfb72) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:18,561 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:38,571 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:58,582 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:18,584 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:38,596 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:58,606 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:18,617 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:38,628 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:58,636 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:18,647 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:38,658 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:40,237 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-acdc0c28) (logid:ae83f9b3) Seq 3-724516590053290113: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 17:58:58,669 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:18,680 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:38,691 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:58,702 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:18,712 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:38,723 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:58,725 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:18,736 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:38,747 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:58,758 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:18,768 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:38,779 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:58,789 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:18,800 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:38,811 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:58,822 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:18,833 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:38,844 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:58,855 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:18,866 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:38,876 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:58,889 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:18,900 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:38,911 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:58,922 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:18,932 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:38,944 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:58,955 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:18,967 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:38,946 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-5639dca3) (logid:fc60836a) Seq 3-724516590053290160: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:08:38,978 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:58,980 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:18,989 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:39,000 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:59,009 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:19,020 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:39,030 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:59,041 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:19,052 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:39,063 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:59,074 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:19,085 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:39,096 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:59,107 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:19,117 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:39,128 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:59,129 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:19,140 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:39,146 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:59,157 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:19,162 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:39,172 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:59,182 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:19,193 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:39,204 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:59,214 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:19,220 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:39,225 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:59,236 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:19,247 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:38,950 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-2d313558) (logid:4622835f) Seq 3-724516590053290208: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:18:39,257 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:59,268 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:19,278 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:39,289 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:59,290 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:19,301 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:39,312 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:59,323 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:19,334 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:39,345 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:59,356 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:19,367 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:39,378 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:59,389 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:19,399 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:39,410 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:59,421 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:19,430 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:39,441 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:59,452 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:19,464 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:39,475 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:59,485 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:19,497 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:39,508 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:59,513 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:19,524 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:39,535 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:59,546 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:19,558 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:39,569 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:40,086 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-5e7c7e3b) (logid:370d4c7c) Seq 3-724516590053290257: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:28:59,580 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:19,590 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:39,601 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:59,611 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:30:09,612 ERROR [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Failed to setup Kubernetes cluster : k8s-sid in usable state as unable to access control node VMs of the cluster
   2022-03-10 18:30:15,088 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-control-17f73b3832e in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:21,849 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-control-17f73b3a94f in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:23,218 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b3cc83 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:28,736 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b3eb89 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:30,903 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b40a72 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:30,903 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-101:ctx-aed659b4 job-1811) (logid:0a27d378) Complete async job-1811, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Failed to setup Kubernetes cluster : k8s-sid in usable state as unable to access control node VMs of the cluster"}`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] sidre9k removed a comment on issue #6094: Cloudstack Kuberentes getting stuck in Starting State

Posted by GitBox <gi...@apache.org>.

sidre9k removed a comment on issue #6094:
URL: https://github.com/apache/cloudstack/issues/6094#issuecomment-1072206469


   @sureshanaparti Attaching mgmt servers logs herewith.
   `2022-03-10 17:48:58,360 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:18,371 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:38,382 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:58,393 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:18,404 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:38,415 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:58,426 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:18,437 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:38,448 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:58,459 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:18,470 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:38,481 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:58,492 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:18,502 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:38,513 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:58,524 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:18,529 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:38,540 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:58,550 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:03,869 INFO  [c.c.a.ApiServer] (qtp1603198149-26090:ctx-50dbb34c ctx-bfeb2abd) (logid:8094e808) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:14,316 INFO  [c.c.a.ApiServer] (qtp1603198149-26090:ctx-9b7fafc0 ctx-79479e6f) (logid:885c6916) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:17,359 INFO  [c.c.a.ApiServer] (qtp1603198149-26087:ctx-efd30324 ctx-5916a61e) (logid:974dfb72) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:18,561 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:38,571 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:58,582 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:18,584 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:38,596 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:58,606 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:18,617 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:38,628 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:58,636 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:18,647 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:38,658 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:40,237 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-acdc0c28) (logid:ae83f9b3) Seq 3-724516590053290113: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 17:58:58,669 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:18,680 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:38,691 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:58,702 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:18,712 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:38,723 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:58,725 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:18,736 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:38,747 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:58,758 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:18,768 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:38,779 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:58,789 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:18,800 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:38,811 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:58,822 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:18,833 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:38,844 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:58,855 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:18,866 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:38,876 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:58,889 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:18,900 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:38,911 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:58,922 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:18,932 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:38,944 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:58,955 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:18,967 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:38,946 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-5639dca3) (logid:fc60836a) Seq 3-724516590053290160: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:08:38,978 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:58,980 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:18,989 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:39,000 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:59,009 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:19,020 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:39,030 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:59,041 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:19,052 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:39,063 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:59,074 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:19,085 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:39,096 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:59,107 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:19,117 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:39,128 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:59,129 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:19,140 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:39,146 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:59,157 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:19,162 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:39,172 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:59,182 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:19,193 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:39,204 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:59,214 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:19,220 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:39,225 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:59,236 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:19,247 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:38,950 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-2d313558) (logid:4622835f) Seq 3-724516590053290208: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:18:39,257 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:59,268 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:19,278 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:39,289 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:59,290 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:19,301 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:39,312 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:59,323 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:19,334 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:39,345 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:59,356 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:19,367 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:39,378 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:59,389 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:19,399 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:39,410 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:59,421 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:19,430 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:39,441 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:59,452 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:19,464 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:39,475 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:59,485 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:19,497 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:39,508 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:59,513 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:19,524 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:39,535 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:59,546 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:19,558 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:39,569 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:40,086 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-5e7c7e3b) (logid:370d4c7c) Seq 3-724516590053290257: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:28:59,580 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:19,590 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:39,601 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:59,611 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:30:09,612 ERROR [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Failed to setup Kubernetes cluster : k8s-sid in usable state as unable to access control node VMs of the cluster
   2022-03-10 18:30:15,088 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-control-17f73b3832e in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:21,849 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-control-17f73b3a94f in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:23,218 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b3cc83 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:28,736 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b3eb89 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:30,903 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b40a72 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:30,903 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-101:ctx-aed659b4 job-1811) (logid:0a27d378) Complete async job-1811, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Failed to setup Kubernetes cluster : k8s-sid in usable state as unable to access control node VMs of the cluster"}
   `
   
   
   
   Find channel
   
   FAVORITES
   Copy Link
   [Implementations](https://nxtchat.nxtgen.com/emerging/channels/implementations)
   Copy Link
   [OpenNebula](https://nxtchat.nxtgen.com/emerging/channels/opennebula)
   Copy Link
   [InfraOps](https://nxtchat.nxtgen.com/emerging/channels/infraops)
   Copy Link
   [Monitoring / Observability](https://nxtchat.nxtgen.com/emerging/channels/monitoring--observability)
   Copy Link
   [ET-System Engineering](https://nxtchat.nxtgen.com/emerging/channels/et-system-engineering)
   Copy Link
   [CEPH Storage](https://nxtchat.nxtgen.com/emerging/channels/ceph)
   Copy Link
   [EDGE](https://nxtchat.nxtgen.com/emerging/channels/edge)
   Copy Link
   [Networking](https://nxtchat.nxtgen.com/emerging/channels/networkingcni)
   Copy Link
   [HPC](https://nxtchat.nxtgen.com/emerging/channels/hpc)
   
   CHANNELS
   Copy Link
   [Emerging Technologies](https://nxtchat.nxtgen.com/emerging/channels/town-square)
   Copy Link
   [Off-Topic](https://nxtchat.nxtgen.com/emerging/channels/off-topic)
   Copy Link
   [Uptime Kuma](https://nxtchat.nxtgen.com/emerging/channels/uptime-kuma)
   
   DIRECT MESSAGES
   
   Copy Link
   [jayanth](https://nxtchat.nxtgen.com/emerging/messages/@jayanth)
   Copy Link
   [ejaz](https://nxtchat.nxtgen.com/emerging/messages/@ejaz)
   Copy Link
   [rajesh_dangi](https://nxtchat.nxtgen.com/emerging/messages/@rajesh_dangi)
   Copy Link
   [siddhit (you)](https://nxtchat.nxtgen.com/emerging/messages/@siddhit)
   
   Invite Members
   
   Online
   
   
   
   Add a channel description
   At 1:15 PM Friday, March 18, jayanth wrote, , 1 attachment
   February 23
   	
   siddhit
   [4:28 PM](https://nxtchat.nxtgen.com/emerging/pl/jitze868aty6fmpate3ueswxih)
   
   February 24
   	
   jayanth
   [8:59 AM](https://nxtchat.nxtgen.com/emerging/pl/c4qh9i4attn3f89tffasyz4fra)
   https://docs.cloudstack.apache.org/en/latest/conceptsandterminology/concepts.html#massively-scalable-infrastructure-management
   
   	
   jayanth
   [9:44 AM](https://nxtchat.nxtgen.com/emerging/pl/ywjsxqfsz3bitdi6kgey3iy5ch)
   https://docs.cloudstack.apache.org/en/latest/conceptsandterminology/choosing_deployment_architecture.html#multi-site-deployment
   
   EDGE?
   
   	
   jayanth
   [10:33 AM](https://nxtchat.nxtgen.com/emerging/pl/9t11aqh8tbg8idc3fyexdw663r)
   
   	
   jayanth
   [11:51 AM](https://nxtchat.nxtgen.com/emerging/pl/j9psbgf7m3dbpq61proq1a7r7o)
   rgw trust forwarded https
   rgw_crypt_require_ssl
   	
   jayanth
   [12:36 PM](https://nxtchat.nxtgen.com/emerging/pl/x8sorbqekprcdg3ot38zma1yah)
   https://docs.ceph.com/en/latest/radosgw/encryption/#automatic-encryption-for-testing-only
   
   	
   siddhit
   [5:26 PM](https://nxtchat.nxtgen.com/emerging/pl/8qu9ywh6t3rmpr37kdrhcmqp8y)
   GPU: https://lab.piszki.pl/cloudstack-kvm-and-running-vm-with-vgpu/
   
   Piszki Lab | EN
   [CloudStack – KVM and running VM with vGPU](https://lab.piszki.pl/cloudstack-kvm-and-running-vm-with-vgpu/)
   GPU cards (such as Nvidia V100) have recently gained popularity in many companies and other places (such as universities). Such a card can be used in many ways, for CUDA calculations but also for v… 
   
   February 27
   	
   jayanth
   [10:18 AM](https://nxtchat.nxtgen.com/emerging/pl/c3n6dx1zofdqdn7bqmf3khqpje)
   Policy based routing is present.
   
   
   	
   siddhit
   [12:38 PM](https://nxtchat.nxtgen.com/emerging/pl/61rzkcd6z7r598ozacr4ywjn3r)
   Is this on cloudstack appliance?
   
   Or opennebula?
   
   	
   jayanth
   [1:51 PM](https://nxtchat.nxtgen.com/emerging/pl/uaxigzr4fibficwoonhtda59tc)
   Cloudstack. 
   
   	
   siddhit
   [5:30 PM](https://nxtchat.nxtgen.com/emerging/pl/potsjx1zjtd7zgx5kdkdi8tsaw)
   Okay
   
   February 28
   	
   jayanth
   [5:01 PM](https://nxtchat.nxtgen.com/emerging/pl/eipp1appy7frtgt6mugs1nqywo)
   https://www.virtuozzo.com/resources/
   
   Virtuozzo
   [Resources](https://www.virtuozzo.com/resources/)
   Here you will find all the useful resources and other collateral such as whitepapers, video, case studies and much more.
   
   	
   jayanth
   [5:22 PM](https://nxtchat.nxtgen.com/emerging/pl/yp5htfsu77858gm583ytdz8cgr)
   https://www.virtuozzo.com/wp-content/uploads/2021/04/vip-use-cases-sb-en-ltr.pdf
   
   	
   jayanth
   [5:41 PM](https://nxtchat.nxtgen.com/emerging/pl/53t9i5pu6tg4zbu7otwuy8djee)
   https://github.com/apache/cloudstack/pull/5909
   
   GitHub
   [Quota custom tariffs by GutoVeronezi · Pull Request #5909 · apache/cloudstack](https://github.com/apache/cloudstack/pull/5909)
   Description This PR solves issue #5891. The main focus of this PR is to provide a flexible manner for operators to charge and bill their cloud resources. Issue #5891 has more details and flowcharts... 
   
   March 01
   	
   jayanth
   [12:01 PM](https://nxtchat.nxtgen.com/emerging/pl/cqz5b53pd3robm3aq8w458kmew)
   
   ..
   
   March 02
   	
   [jayanth]()
   [9:35 AM](https://nxtchat.nxtgen.com/emerging/pl/cm35r69ibtddpm9aa54r4y8fhh)
   https://www.nginx.com/c/microservices-march-2022-kubernetes-networking/?utm_medium=email&utm_source=nginxdb&utm_campaign=ww-nx_pgkub_ok&utm_content=ev&mkt_tok=NjUzLVNNQy03ODMAAAGC5i-vOEyuqUIpQmiGekNmtmJz1zXsCwfV_zzvEYot_F1E6NyeXls_CytWWsqXd9tA5449TjxIAGrzbcgUo_ap2Rc5s9ha-_8LevzpzT7JWFRI4PcBNQ
   
   NGINX
   [Microservices March 2022: Kubernetes Networking - NGINX](https://www.nginx.com/c/microservices-march-2022-kubernetes-networking/?utm_medium=email&utm_source=nginxdb&utm_campaign=ww-nx_pgkub_ok&utm_content=ev&mkt_tok=NjUzLVNNQy03ODMAAAGC5i-vOEyuqUIpQmiGekNmtmJz1zXsCwfV_zzvEYot_F1E6NyeXls_CytWWsqXd9tA5449TjxIAGrzbcgUo_ap2Rc5s9ha-_8LevzpzT7JWFRI4PcBNQ)
   Microservices March 2022 is a free program that takes your Kubernetes networking skills from Zero to Hero. You’ll learn about Ingress controllers, service meshes, API gateways, and more! Register today. 
   
   	
   jayanth
   [7:57 PM](https://nxtchat.nxtgen.com/emerging/pl/wqk7gokyajnafm8o6frujk8uzy)
   Responsibilities:
   
   Deploy, manage, support and monitor production-grade cloud solutions.
   Evaluate, recommend, productionize Open Source solutions and systems for strengthening existing and new cloud stacks.
   Perform Proof of Concept (PoC) builds and extensive functional evaluation of Open Source systems.
   Interact with the customers and users to deploy and support cloud requirements, meeting deadlines and committed SLAs.
   Provide detailed Incident Reports with RCA.
   Troubleshoot problems encountered during installation, configuration and maintenance. Maintain the hardware and software assets inventory.
   Supporting activities of the Engineer are to provide robust cloud platform by ensuring effective administration, secure network protocols, access constraints and processes in place.
   Maintain comprehensive documentation of daily tasks and projects and tickets.
   Maintain and contribute to in-house Knowledge Base to streamline redundant issues / resolutions process.
   Create and maintain Standard Operating Procedures and run-books for the deployed solutions.
   Coordinate with in-house development teams in an orderly manner to meet the business requirements.
   
   Skills required:
   
   Primarily proficient in GNU/Linux, and expertise on various Linux distributions.
   Strong hands-on knowledge on System Administration, Networking, Storage and Cloud Security.
   Expertise on the components of the cloud i.e., Compute, Networking, Storage and Security.
   Deep and hands-on understanding of the protocols DNS, DHCP, SSH, TCP, UDP, S3 etc.,
   Sound knowledge on Virtualization and Open Source Hypervisors.
   Knowledge on Reverse proxies and Load Balancers is an added advantage.
   Open Source software enthusiast.
   Patient and calm attitude to confront day to day problems encountered.
   Flexibility to work under critical conditions, dexterity in providing efficient cloud, storage and networking support.
   
   	
   jayanth
   [10:46 PM](https://nxtchat.nxtgen.com/emerging/pl/epcurjdcutfhjem614cryp954a)
   Siddhit, have sent the wrong one!
   
   Job Title:
   
   Associate - System Engineer
   
   Duties and Responsibilities:
   
   Evaluate, deploy, manage and support production-grade cloud solutions.
   Design, implementation, configuration, and administration of Linux and Windows Operating Systems to support various workloads.
   Monitoring and maintaining the deployed infrastructure.
   Ensuring the availability of deployed systems.
   Maintain the infrastructure inventory and sanity.
   Evaluate new Open Source Software solutions.
   Deploy Virtual and Physical servers.
   Perform Proof of Concept (PoC) builds and extensive functional evaluation of Open Source systems.
   Upgrade Applications and Tools, install patches, troubleshoot and resolve application problems, performance tuning and capacity planning.
   Interact with the customers and users to deploy and support cloud requirements, meeting deadlines and committed SLAs.
   Maintain and contribute to in-house Knowledge Base to streamline redundant issues / resolutions process.
   Create and maintain Standard Operating Procedures and run-books for the deployed solutions.
   Coordinate with in-house teams in an orderly manner to meet the business requirements.
   Patient and calm attitude to confront day to day problems encountered.
   Flexibility to work under critical conditions, dexterity in providing efficient cloud, storage and networking support.
   Keen interest contributing to and building communities in open source is a plus.
   
   Preferred qualification:
   
   Completed a university or technical school education focusing on IT, computer science, or engineering (or comparable professional experience).
   Hands-on knowledge in GNU/Linux, and expertise on renowned Linux distributions.
   Proficiency with common tools of the trade for automation.
   Fundamental networking knowledge.
   Ability to perform System Administration tasks.
   Fluency in verbal and written English.
   Fundamental knowledge on the components of the cloud i.e., Compute, Networking, Storage and Security.
   Understanding of the concepts of virtualization and cloud computing.
   Knowledge of Kernel-based Virtual Machine (KVM) is an added advantage.
   Knowledge of Kubernetes is an added advantage.
   Open Source enthusiast.
   
   About NxtGen:
   
   NxtGen provides data center and cloud services from our own high density data center (HDDC) facilities or can deploy centrally managed on-premise data centers (OPDC™). Our enterprise cloud services provide private clouds hosted within OPDCs and hosted private clouds or hybrid cloud infrastructure hosted at their HDDCs. NxtGen’s business-class infrastructure helps companies simplify IT infrastructure, reduce running costs, and enable business growth by creating additional capacity from existing infrastructure and providing the right public or private infrastructure to meet dynamic business demands.
   
   About the team:
   
   We are team of young engineers who are passionate about solving infrastructure challenges and provide robust cloud solutions to our end customers. Our key focus area is to embrace and adapt various open-source technologies, glue them together and build effective solutions around it. Become part of our growing team if this opportunity excites you! 
   
   Show more
   March 03
   	
   siddhit
   [7:37 AM](https://nxtchat.nxtgen.com/emerging/pl/esk8psu893nj3mygtwtgf4c73o)
   This is good
   
   	
   jayanth
   [9:31 AM](https://nxtchat.nxtgen.com/emerging/pl/5y619xbokbn13jwgquhka48ogr)
   Okay, I'm posting then.
   
   	
   jayanth
   [10:45 AM](https://nxtchat.nxtgen.com/emerging/pl/ycnxt49cxidh3x36j6actusxay)
   Have updated the SpeedCloud credentials on the sheet which has been shared to you.
   
   March 04
   	
   jayanth
   [11:18 AM](https://nxtchat.nxtgen.com/emerging/pl/qaeqgbzqmpy95exxkhjti6in7w)
   https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
   
   [HTTP headers - HTTP | MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers)
   HTTP headers let the client and the server pass additional information with an HTTP request or response. An HTTP header consists of its case-insensitive name followed by a colon (:), then by its value. Whitespace before the value is ignored. 
   
   March 07
   	
   jayanth
   [11:33 AM](https://nxtchat.nxtgen.com/emerging/pl/8fp9uszbmjgcffakkmd5cbu9mo)
   Anuj asked me to redeploy the VM.
   
   	
   siddhit
   [11:33 AM](https://nxtchat.nxtgen.com/emerging/pl/4i6qr6xxntdpi8pbzpk8w1dboo)
   Ok
   
   	
   jayanth
   [11:33 AM](https://nxtchat.nxtgen.com/emerging/pl/jx1o9ztnuj8uifjmqp31fgrbir)
   Doing that.
   
   	
   jayanth
   [4:25 PM](https://nxtchat.nxtgen.com/emerging/pl/ku5dcgcffinyxxix9mdn3dzbka)
   Anuj VM Creation done.
   
   March 08
   	
   jayanth
   [12:04 PM](https://nxtchat.nxtgen.com/emerging/pl/dzh9gu944ffitjxq9d7dzgwg8y)
   https://us04web.zoom.us/j/78293871969?pwd=CoEAvBx4MLDhGGsSBE8Hp_8H8SomT0.1 
   
   Zoom Video
   [Join our Cloud HD Video Meeting](https://us04web.zoom.us/j/78293871969?pwd=CoEAvBx4MLDhGGsSBE8Hp_8H8SomT0.1)
   Zoom is the leader in modern enterprise video communications, with an easy, reliable cloud platform for video and audio conferencing, chat, and webinars across mobile, desktop, and room systems. Zoom Rooms is the original software-based conference room solution used around the world in board, con...
   March 09
   	
   jayanth
   [12:33 PM](https://nxtchat.nxtgen.com/emerging/pl/s6m7gzb6jj81pb6jqxnnx3tfro)
   https://www.haproxy.com/blog/load-balancing-affinity-persistence-sticky-sessions-what-you-need-to-know/
   
   Very nice explanation.
   
   HAProxy Technologies
   [Load Balancing, Affinity, Persistence, Sticky Sessions: What You Need to Know - HAProxy Technologies](https://www.haproxy.com/blog/load-balancing-affinity-persistence-sticky-sessions-what-you-need-to-know/)
   This blog post shows why and how to use a load balancer, the differences between Affinity, Persistence and Sticky Sessions and how to enable them in HAProxy 
   
   	
   jayanth
   [2:10 PM](https://nxtchat.nxtgen.com/emerging/pl/wtin4nwoobyufj7cnxm7syzbha)[](https://nxtchat.nxtgen.com/emerging/messages/@jayanth#)[](https://nxtchat.nxtgen.com/api/v4/files/namgby3mdjrixyitbybraag16w?download=1)
   Sasi_DevOps_Fresher.pdf
   PDF162KB
   Thursday
   	
   siddhit
   [1:09 PM](https://nxtchat.nxtgen.com/emerging/pl/4zxnfsa7ejrb8nqtxif181s1ya)
   Lets dig deeper w.r.t yestd degraded cephfs logs and see if any pattern or event which could have caused the outage
   
   
   	
   siddhit
   [1:55 PM](https://nxtchat.nxtgen.com/emerging/pl/8oach3ge4tbyj8uj87n97hjodo)
   
   Pls Check IOPS and throughupt for 4MB and 512 KB blocksize for both RBD and NFS
   
   	
   siddhit
   [1:55 PM](https://nxtchat.nxtgen.com/emerging/pl/a849wy4a4byq58cuo8a7p9cuhc)
   
   i guess we did for 64 KB right?
   
   	
   jayanth
   [1:56 PM](https://nxtchat.nxtgen.com/emerging/pl/yyooeok8iid4dntesudzynizth)
   Correct.
   
   	
   jayanth
   [1:56 PM](https://nxtchat.nxtgen.com/emerging/pl/ba55nuqohtfi3p3zch3pdd8pqw)
   
   Sure.
   
   	
   jayanth
   1:58 PM
   
   IOPS at 4K, throughput at 64K.
   
   We'll have all tests done.
   
   	
   [siddhit]()
   2:52 PM
   SiteOPsAHD_BSNLIDC SiteOPsAHD_BSNLIDC@nxtgen.com
   
   	
   [siddhit]()
   [3:30 PM](https://nxtchat.nxtgen.com/emerging/pl/3fi9pd7h8jrpikudcxd3wdw77r)
   https://www.sebastien-han.fr/blog/2012/07/06/nfs-over-rbd/
   
   Today
   New Messages
   	
   jayanth
   [9:48 AM](https://nxtchat.nxtgen.com/emerging/pl/pb76bhi633b8zdgio6kkkgq8hc)
   
   AHD firewall itself is down/experiencing issues. 
   
   [9:48 AM](https://nxtchat.nxtgen.com/emerging/pl/wc9upw9w938ffypybztr63g3ya)
   14.192.19.27
   
   
   	
   jayanth
   [1:23 PM](https://nxtchat.nxtgen.com/emerging/pl/e99mpke7n3dhbmnz8y5hhhyjea)
   
   Any updates from Anuj - Riversilica? 
   
   	
   siddhit
   [1:56 PM](https://nxtchat.nxtgen.com/emerging/pl/cut8d4uxsig5zktn3tjdmzfokw)
   Nope.Informed Rajesh to expedite.Seems they are tied in some project.
   
   	
   jayanth
   [1:57 PM](https://nxtchat.nxtgen.com/emerging/pl/9mo178tgpbyubeg88tnqka6aao)
   Oh okay.
   
   	
   siddhit
   [4:18 PM](https://nxtchat.nxtgen.com/emerging/pl/1zg9wui13pbxpexr8h4ikg356r)
   is our pritunl setup still operational?
   
   	
   jayanth
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/u9drmh1gwpf3bf1i8y6ugor7ky)
   Ah, Pritunl? 
   
   barebones openvpn setup is still operational. No Pritunl controlled setups exist.
   
   	
   siddhit
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/8pi3jouwhpgmdrz4mp5ti977zo)
   ok
   
   	
   jayanth
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/swjgjd9upfdtmyak7piquxmtwc)
   Which location are you looking for?
   
   	
   siddhit
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/wq11zpciipbs3xxbgggffkx95o)
   none specific
   
   	
   jayanth
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/nitdgtj793rz8e9z7474rke7uy)
   Okay.
   
   	
   siddhit
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/enubdjzwi7yhue3qhkwu1it69o)
   just wanted to refer dashboard
   
   	
   jayanth
   [4:20 PM](https://nxtchat.nxtgen.com/emerging/pl/toujgdnfjfr33e3h38yt9nerqr)
   https://speedcloudstage.nxtgen.com/#vms-tab/1579
   
   check this out then.
   
   It's turned off currently.
   
   	
   siddhit
   [4:23 PM](https://nxtchat.nxtgen.com/emerging/pl/uxw578mrrprkpbzstjh66uxoew)
   
   can you add staging creds also in master sheet?
   
   	
   jayanth
   [4:23 PM](https://nxtchat.nxtgen.com/emerging/pl/ykhgwxqb4frj9yf377ae6serpw)
   Sure.
   
   	
   siddhit
   [4:39 PM](https://nxtchat.nxtgen.com/emerging/pl/3cezcr11bjb9up1d8mjxyufauy)
   Any new profiles for interview?
   
   	
   jayanth
   [4:40 PM](https://nxtchat.nxtgen.com/emerging/pl/9p7g1qf3ttr9jj7eoagqm9razh)
   2 new applicants. To be interviewed very soon.
   
   	
   siddhit
   [4:40 PM](https://nxtchat.nxtgen.com/emerging/pl/xrp9wxxun7ya9et9z71n3moaac)
   ok
   
   keep thread posted
   
   	
   jayanth
   [4:40 PM](https://nxtchat.nxtgen.com/emerging/pl/yqrm7nghijn1bp3s1gsdtb6c1a)
   Sure,
   
   	
   jayanth
   [4:49 PM](https://nxtchat.nxtgen.com/emerging/pl/gmuu3sgqxtf9tn66imbb695ejr)
   No interviews have been done from your end right for Associate profiles right, just confirming. 
   
   	
   siddhit
   [4:59 PM](https://nxtchat.nxtgen.com/emerging/pl/rrj6fkcy5jnn9fopipep6mayka)
   not after you initiated profiling 
   
   	
   jayanth
   [4:59 PM](https://nxtchat.nxtgen.com/emerging/pl/s1kemtckft8k5qdcpenoq5c8nh)
   Okay.
   
   	
   siddhit
   [4:59 PM](https://nxtchat.nxtgen.com/emerging/pl/1cdi7x1or7drmdayro185iomuo)
   serveral before them..will collate and share list for past reviewed candidates
   
   	
   jayanth
   [4:59 PM](https://nxtchat.nxtgen.com/emerging/pl/u6mf6gaucj8qbktj6zhao67apr)
   Okay.
   
   	
   jayanth
   [5:05 PM](https://nxtchat.nxtgen.com/emerging/pl/basmfcbkkfr8jk9h8b6sz5kq9r)
   
   Done.
   
   Today
   	
   jayanth
   [1:15 PM](https://nxtchat.nxtgen.com/emerging/pl/wwahu1uz3ib7ijfpmrayp5i4zr)[](https://nxtchat.nxtgen.com/emerging/messages/@jayanth#)[](https://nxtchat.nxtgen.com/api/v4/files/w8csjfze7jnkbppw7mxsertrdh?download=1)
   k8s-sid.log
   LOG36KB
   Write to jayanth
   
   No file chosen
   
   [Help](https://nxtchat.nxtgen.com/help/messaging?locale=en)[](https://nxtchat.nxtgen.com/api/v4/files/w8csjfze7jnkbppw7mxsertrdh?download=1)
   k8s-sid.log
   JayanthShared In ~Jayanth
   
   2022-03-10 17:48:58,360 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:18,371 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:38,382 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:58,393 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:18,404 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:38,415 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:58,426 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:18,437 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:38,448 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:58,459 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:18,470 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:38,481 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:58,492 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:18,502 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:38,513 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:58,524 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:18,529 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:38,540 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:58,550 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:03,869 INFO  [c.c.a.ApiServer] (qtp1603198149-26090:ctx-50dbb34c ctx-bfeb2abd) (logid:8094e808) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:14,316 INFO  [c.c.a.ApiServer] (qtp1603198149-26090:ctx-9b7fafc0 ctx-79479e6f) (logid:885c6916) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:17,359 INFO  [c.c.a.ApiServer] (qtp1603198149-26087:ctx-efd30324 ctx-5916a61e) (logid:974dfb72) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:18,561 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:38,571 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:58,582 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:18,584 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:38,596 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:58,606 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:18,617 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:38,628 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:58,636 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:18,647 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:38,658 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:40,237 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-acdc0c28) (logid:ae83f9b3) Seq 3-724516590053290113: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 17:58:58,669 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:18,680 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:38,691 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:58,702 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:18,712 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:38,723 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:58,725 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:18,736 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:38,747 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:58,758 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:18,768 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:38,779 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:58,789 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:18,800 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:38,811 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:58,822 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:18,833 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:38,844 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:58,855 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:18,866 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:38,876 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:58,889 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:18,900 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:38,911 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:58,922 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:18,932 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:38,944 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:58,955 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:18,967 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:38,946 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-5639dca3) (logid:fc60836a) Seq 3-724516590053290160: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:08:38,978 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:58,980 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:18,989 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:39,000 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:59,009 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:19,020 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:39,030 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:59,041 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:19,052 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:39,063 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:59,074 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:19,085 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:39,096 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:59,107 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:19,117 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:39,128 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:59,129 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:19,140 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:39,146 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:59,157 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:19,162 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:39,172 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:59,182 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:19,193 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:39,204 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:59,214 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:19,220 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:39,225 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:59,236 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:19,247 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:38,950 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-2d313558) (logid:4622835f) Seq 3-724516590053290208: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:18:39,257 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:59,268 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:19,278 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:39,289 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:59,290 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:19,301 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:39,312 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:59,323 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:19,334 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:39,345 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:59,356 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:19,367 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:39,378 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:59,389 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:19,399 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:39,410 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:59,421 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:19,430 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:39,441 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:59,452 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:19,464 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:39,475 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:59,485 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:19,497 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:39,508 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:59,513 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:19,524 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:39,535 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:59,546 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:19,558 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:39,569 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:40,086 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-5e7c7e3b) (logid:370d4c7c) Seq 3-724516590053290257: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:28:59,580 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:19,590 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:39,601 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:59,611 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:30:09,612 ERROR [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Failed to setup Kubernetes cluster : k8s-sid in usable state as unable to access control node VMs of the cluster
   2022-03-10 18:30:15,088 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-control-17f73b3832e in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:21,849 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-control-17f73b3a94f in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:23,218 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b3cc83 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:28,736 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b3eb89 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:30,903 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b40a72 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:30,903 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-101:ctx-aed659b4 job-1811) (logid:0a27d378) Complete async job-1811, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Failed to setup Kubernetes cluster : k8s-sid in usable state as unable to access control node VMs of the cluster"}`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] sidre9k commented on issue #6094: Cloudstack Kuberentes getting stuck in Starting State

Posted by GitBox <gi...@apache.org>.

sidre9k commented on issue #6094:
URL: https://github.com/apache/cloudstack/issues/6094#issuecomment-1072206469






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] sidre9k closed issue #6094: Cloudstack Kuberentes getting stuck in Starting State

Posted by GitBox <gi...@apache.org>.

sidre9k closed issue #6094:
URL: https://github.com/apache/cloudstack/issues/6094


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] sidre9k commented on issue #6094: Cloudstack Kuberentes getting stuck in Starting State

Posted by GitBox <gi...@apache.org>.

sidre9k commented on issue #6094:
URL: https://github.com/apache/cloudstack/issues/6094#issuecomment-1066720347


   > @sidre9k Is it the same issue as #5999 ?
   
   Yes Suresh.Its same issue as described in #5999. I will share management server logs at earliest.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] sidre9k removed a comment on issue #6094: Cloudstack Kuberentes getting stuck in Starting State

Posted by GitBox <gi...@apache.org>.

sidre9k removed a comment on issue #6094:
URL: https://github.com/apache/cloudstack/issues/6094#issuecomment-1066719099


   > [sureshanaparti](/sureshanaparti)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] sidre9k removed a comment on issue #6094: Cloudstack Kuberentes getting stuck in Starting State

Posted by GitBox <gi...@apache.org>.

sidre9k removed a comment on issue #6094:
URL: https://github.com/apache/cloudstack/issues/6094#issuecomment-1072206469


   @sureshanaparti Attaching mgmt servers logs herewith.
   `2022-03-10 17:48:58,360 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:18,371 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:38,382 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:58,393 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:18,404 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:38,415 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:58,426 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:18,437 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:38,448 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:58,459 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:18,470 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:38,481 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:58,492 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:18,502 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:38,513 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:58,524 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:18,529 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:38,540 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:58,550 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:03,869 INFO  [c.c.a.ApiServer] (qtp1603198149-26090:ctx-50dbb34c ctx-bfeb2abd) (logid:8094e808) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:14,316 INFO  [c.c.a.ApiServer] (qtp1603198149-26090:ctx-9b7fafc0 ctx-79479e6f) (logid:885c6916) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:17,359 INFO  [c.c.a.ApiServer] (qtp1603198149-26087:ctx-efd30324 ctx-5916a61e) (logid:974dfb72) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:18,561 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:38,571 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:58,582 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:18,584 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:38,596 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:58,606 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:18,617 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:38,628 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:58,636 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:18,647 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:38,658 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:40,237 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-acdc0c28) (logid:ae83f9b3) Seq 3-724516590053290113: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 17:58:58,669 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:18,680 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:38,691 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:58,702 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:18,712 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:38,723 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:58,725 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:18,736 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:38,747 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:58,758 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:18,768 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:38,779 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:58,789 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:18,800 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:38,811 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:58,822 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:18,833 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:38,844 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:58,855 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:18,866 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:38,876 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:58,889 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:18,900 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:38,911 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:58,922 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:18,932 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:38,944 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:58,955 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:18,967 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:38,946 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-5639dca3) (logid:fc60836a) Seq 3-724516590053290160: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:08:38,978 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:58,980 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:18,989 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:39,000 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:59,009 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:19,020 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:39,030 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:59,041 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:19,052 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:39,063 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:59,074 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:19,085 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:39,096 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:59,107 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:19,117 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:39,128 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:59,129 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:19,140 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:39,146 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:59,157 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:19,162 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:39,172 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:59,182 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:19,193 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:39,204 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:59,214 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:19,220 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:39,225 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:59,236 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:19,247 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:38,950 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-2d313558) (logid:4622835f) Seq 3-724516590053290208: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:18:39,257 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:59,268 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:19,278 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:39,289 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:59,290 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:19,301 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:39,312 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:59,323 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:19,334 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:39,345 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:59,356 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:19,367 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:39,378 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:59,389 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:19,399 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:39,410 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:59,421 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:19,430 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:39,441 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:59,452 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:19,464 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:39,475 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:59,485 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:19,497 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:39,508 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:59,513 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:19,524 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:39,535 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:59,546 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:19,558 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:39,569 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:40,086 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-5e7c7e3b) (logid:370d4c7c) Seq 3-724516590053290257: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:28:59,580 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:19,590 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:39,601 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:59,611 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:30:09,612 ERROR [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Failed to setup Kubernetes cluster : k8s-sid in usable state as unable to access control node VMs of the cluster
   2022-03-10 18:30:15,088 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-control-17f73b3832e in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:21,849 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-control-17f73b3a94f in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:23,218 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b3cc83 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:28,736 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b3eb89 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:30,903 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b40a72 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:30,903 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-101:ctx-aed659b4 job-1811) (logid:0a27d378) Complete async job-1811, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Failed to setup Kubernetes cluster : k8s-sid in usable state as unable to access control node VMs of the cluster"}
   `
   
   
   
   Find channel
   
   FAVORITES
   Copy Link
   [Implementations](https://nxtchat.nxtgen.com/emerging/channels/implementations)
   Copy Link
   [OpenNebula](https://nxtchat.nxtgen.com/emerging/channels/opennebula)
   Copy Link
   [InfraOps](https://nxtchat.nxtgen.com/emerging/channels/infraops)
   Copy Link
   [Monitoring / Observability](https://nxtchat.nxtgen.com/emerging/channels/monitoring--observability)
   Copy Link
   [ET-System Engineering](https://nxtchat.nxtgen.com/emerging/channels/et-system-engineering)
   Copy Link
   [CEPH Storage](https://nxtchat.nxtgen.com/emerging/channels/ceph)
   Copy Link
   [EDGE](https://nxtchat.nxtgen.com/emerging/channels/edge)
   Copy Link
   [Networking](https://nxtchat.nxtgen.com/emerging/channels/networkingcni)
   Copy Link
   [HPC](https://nxtchat.nxtgen.com/emerging/channels/hpc)
   
   CHANNELS
   Copy Link
   [Emerging Technologies](https://nxtchat.nxtgen.com/emerging/channels/town-square)
   Copy Link
   [Off-Topic](https://nxtchat.nxtgen.com/emerging/channels/off-topic)
   Copy Link
   [Uptime Kuma](https://nxtchat.nxtgen.com/emerging/channels/uptime-kuma)
   
   DIRECT MESSAGES
   
   Copy Link
   [jayanth](https://nxtchat.nxtgen.com/emerging/messages/@jayanth)
   Copy Link
   [ejaz](https://nxtchat.nxtgen.com/emerging/messages/@ejaz)
   Copy Link
   [rajesh_dangi](https://nxtchat.nxtgen.com/emerging/messages/@rajesh_dangi)
   Copy Link
   [siddhit (you)](https://nxtchat.nxtgen.com/emerging/messages/@siddhit)
   
   Invite Members
   
   Online
   
   
   
   Add a channel description
   At 1:15 PM Friday, March 18, jayanth wrote, , 1 attachment
   February 23
   	
   siddhit
   [4:28 PM](https://nxtchat.nxtgen.com/emerging/pl/jitze868aty6fmpate3ueswxih)
   
   February 24
   	
   jayanth
   [8:59 AM](https://nxtchat.nxtgen.com/emerging/pl/c4qh9i4attn3f89tffasyz4fra)
   https://docs.cloudstack.apache.org/en/latest/conceptsandterminology/concepts.html#massively-scalable-infrastructure-management
   
   	
   jayanth
   [9:44 AM](https://nxtchat.nxtgen.com/emerging/pl/ywjsxqfsz3bitdi6kgey3iy5ch)
   https://docs.cloudstack.apache.org/en/latest/conceptsandterminology/choosing_deployment_architecture.html#multi-site-deployment
   
   EDGE?
   
   	
   jayanth
   [10:33 AM](https://nxtchat.nxtgen.com/emerging/pl/9t11aqh8tbg8idc3fyexdw663r)
   
   	
   jayanth
   [11:51 AM](https://nxtchat.nxtgen.com/emerging/pl/j9psbgf7m3dbpq61proq1a7r7o)
   rgw trust forwarded https
   rgw_crypt_require_ssl
   	
   jayanth
   [12:36 PM](https://nxtchat.nxtgen.com/emerging/pl/x8sorbqekprcdg3ot38zma1yah)
   https://docs.ceph.com/en/latest/radosgw/encryption/#automatic-encryption-for-testing-only
   
   	
   siddhit
   [5:26 PM](https://nxtchat.nxtgen.com/emerging/pl/8qu9ywh6t3rmpr37kdrhcmqp8y)
   GPU: https://lab.piszki.pl/cloudstack-kvm-and-running-vm-with-vgpu/
   
   Piszki Lab | EN
   [CloudStack – KVM and running VM with vGPU](https://lab.piszki.pl/cloudstack-kvm-and-running-vm-with-vgpu/)
   GPU cards (such as Nvidia V100) have recently gained popularity in many companies and other places (such as universities). Such a card can be used in many ways, for CUDA calculations but also for v… 
   
   February 27
   	
   jayanth
   [10:18 AM](https://nxtchat.nxtgen.com/emerging/pl/c3n6dx1zofdqdn7bqmf3khqpje)
   Policy based routing is present.
   
   
   	
   siddhit
   [12:38 PM](https://nxtchat.nxtgen.com/emerging/pl/61rzkcd6z7r598ozacr4ywjn3r)
   Is this on cloudstack appliance?
   
   Or opennebula?
   
   	
   jayanth
   [1:51 PM](https://nxtchat.nxtgen.com/emerging/pl/uaxigzr4fibficwoonhtda59tc)
   Cloudstack. 
   
   	
   siddhit
   [5:30 PM](https://nxtchat.nxtgen.com/emerging/pl/potsjx1zjtd7zgx5kdkdi8tsaw)
   Okay
   
   February 28
   	
   jayanth
   [5:01 PM](https://nxtchat.nxtgen.com/emerging/pl/eipp1appy7frtgt6mugs1nqywo)
   https://www.virtuozzo.com/resources/
   
   Virtuozzo
   [Resources](https://www.virtuozzo.com/resources/)
   Here you will find all the useful resources and other collateral such as whitepapers, video, case studies and much more.
   
   	
   jayanth
   [5:22 PM](https://nxtchat.nxtgen.com/emerging/pl/yp5htfsu77858gm583ytdz8cgr)
   https://www.virtuozzo.com/wp-content/uploads/2021/04/vip-use-cases-sb-en-ltr.pdf
   
   	
   jayanth
   [5:41 PM](https://nxtchat.nxtgen.com/emerging/pl/53t9i5pu6tg4zbu7otwuy8djee)
   https://github.com/apache/cloudstack/pull/5909
   
   GitHub
   [Quota custom tariffs by GutoVeronezi · Pull Request #5909 · apache/cloudstack](https://github.com/apache/cloudstack/pull/5909)
   Description This PR solves issue #5891. The main focus of this PR is to provide a flexible manner for operators to charge and bill their cloud resources. Issue #5891 has more details and flowcharts... 
   
   March 01
   	
   jayanth
   [12:01 PM](https://nxtchat.nxtgen.com/emerging/pl/cqz5b53pd3robm3aq8w458kmew)
   
   ..
   
   March 02
   	
   [jayanth]()
   [9:35 AM](https://nxtchat.nxtgen.com/emerging/pl/cm35r69ibtddpm9aa54r4y8fhh)
   https://www.nginx.com/c/microservices-march-2022-kubernetes-networking/?utm_medium=email&utm_source=nginxdb&utm_campaign=ww-nx_pgkub_ok&utm_content=ev&mkt_tok=NjUzLVNNQy03ODMAAAGC5i-vOEyuqUIpQmiGekNmtmJz1zXsCwfV_zzvEYot_F1E6NyeXls_CytWWsqXd9tA5449TjxIAGrzbcgUo_ap2Rc5s9ha-_8LevzpzT7JWFRI4PcBNQ
   
   NGINX
   [Microservices March 2022: Kubernetes Networking - NGINX](https://www.nginx.com/c/microservices-march-2022-kubernetes-networking/?utm_medium=email&utm_source=nginxdb&utm_campaign=ww-nx_pgkub_ok&utm_content=ev&mkt_tok=NjUzLVNNQy03ODMAAAGC5i-vOEyuqUIpQmiGekNmtmJz1zXsCwfV_zzvEYot_F1E6NyeXls_CytWWsqXd9tA5449TjxIAGrzbcgUo_ap2Rc5s9ha-_8LevzpzT7JWFRI4PcBNQ)
   Microservices March 2022 is a free program that takes your Kubernetes networking skills from Zero to Hero. You’ll learn about Ingress controllers, service meshes, API gateways, and more! Register today. 
   
   	
   jayanth
   [7:57 PM](https://nxtchat.nxtgen.com/emerging/pl/wqk7gokyajnafm8o6frujk8uzy)
   Responsibilities:
   
   Deploy, manage, support and monitor production-grade cloud solutions.
   Evaluate, recommend, productionize Open Source solutions and systems for strengthening existing and new cloud stacks.
   Perform Proof of Concept (PoC) builds and extensive functional evaluation of Open Source systems.
   Interact with the customers and users to deploy and support cloud requirements, meeting deadlines and committed SLAs.
   Provide detailed Incident Reports with RCA.
   Troubleshoot problems encountered during installation, configuration and maintenance. Maintain the hardware and software assets inventory.
   Supporting activities of the Engineer are to provide robust cloud platform by ensuring effective administration, secure network protocols, access constraints and processes in place.
   Maintain comprehensive documentation of daily tasks and projects and tickets.
   Maintain and contribute to in-house Knowledge Base to streamline redundant issues / resolutions process.
   Create and maintain Standard Operating Procedures and run-books for the deployed solutions.
   Coordinate with in-house development teams in an orderly manner to meet the business requirements.
   
   Skills required:
   
   Primarily proficient in GNU/Linux, and expertise on various Linux distributions.
   Strong hands-on knowledge on System Administration, Networking, Storage and Cloud Security.
   Expertise on the components of the cloud i.e., Compute, Networking, Storage and Security.
   Deep and hands-on understanding of the protocols DNS, DHCP, SSH, TCP, UDP, S3 etc.,
   Sound knowledge on Virtualization and Open Source Hypervisors.
   Knowledge on Reverse proxies and Load Balancers is an added advantage.
   Open Source software enthusiast.
   Patient and calm attitude to confront day to day problems encountered.
   Flexibility to work under critical conditions, dexterity in providing efficient cloud, storage and networking support.
   
   	
   jayanth
   [10:46 PM](https://nxtchat.nxtgen.com/emerging/pl/epcurjdcutfhjem614cryp954a)
   Siddhit, have sent the wrong one!
   
   Job Title:
   
   Associate - System Engineer
   
   Duties and Responsibilities:
   
   Evaluate, deploy, manage and support production-grade cloud solutions.
   Design, implementation, configuration, and administration of Linux and Windows Operating Systems to support various workloads.
   Monitoring and maintaining the deployed infrastructure.
   Ensuring the availability of deployed systems.
   Maintain the infrastructure inventory and sanity.
   Evaluate new Open Source Software solutions.
   Deploy Virtual and Physical servers.
   Perform Proof of Concept (PoC) builds and extensive functional evaluation of Open Source systems.
   Upgrade Applications and Tools, install patches, troubleshoot and resolve application problems, performance tuning and capacity planning.
   Interact with the customers and users to deploy and support cloud requirements, meeting deadlines and committed SLAs.
   Maintain and contribute to in-house Knowledge Base to streamline redundant issues / resolutions process.
   Create and maintain Standard Operating Procedures and run-books for the deployed solutions.
   Coordinate with in-house teams in an orderly manner to meet the business requirements.
   Patient and calm attitude to confront day to day problems encountered.
   Flexibility to work under critical conditions, dexterity in providing efficient cloud, storage and networking support.
   Keen interest contributing to and building communities in open source is a plus.
   
   Preferred qualification:
   
   Completed a university or technical school education focusing on IT, computer science, or engineering (or comparable professional experience).
   Hands-on knowledge in GNU/Linux, and expertise on renowned Linux distributions.
   Proficiency with common tools of the trade for automation.
   Fundamental networking knowledge.
   Ability to perform System Administration tasks.
   Fluency in verbal and written English.
   Fundamental knowledge on the components of the cloud i.e., Compute, Networking, Storage and Security.
   Understanding of the concepts of virtualization and cloud computing.
   Knowledge of Kernel-based Virtual Machine (KVM) is an added advantage.
   Knowledge of Kubernetes is an added advantage.
   Open Source enthusiast.
   
   About NxtGen:
   
   NxtGen provides data center and cloud services from our own high density data center (HDDC) facilities or can deploy centrally managed on-premise data centers (OPDC™). Our enterprise cloud services provide private clouds hosted within OPDCs and hosted private clouds or hybrid cloud infrastructure hosted at their HDDCs. NxtGen’s business-class infrastructure helps companies simplify IT infrastructure, reduce running costs, and enable business growth by creating additional capacity from existing infrastructure and providing the right public or private infrastructure to meet dynamic business demands.
   
   About the team:
   
   We are team of young engineers who are passionate about solving infrastructure challenges and provide robust cloud solutions to our end customers. Our key focus area is to embrace and adapt various open-source technologies, glue them together and build effective solutions around it. Become part of our growing team if this opportunity excites you! 
   
   Show more
   March 03
   	
   siddhit
   [7:37 AM](https://nxtchat.nxtgen.com/emerging/pl/esk8psu893nj3mygtwtgf4c73o)
   This is good
   
   	
   jayanth
   [9:31 AM](https://nxtchat.nxtgen.com/emerging/pl/5y619xbokbn13jwgquhka48ogr)
   Okay, I'm posting then.
   
   	
   jayanth
   [10:45 AM](https://nxtchat.nxtgen.com/emerging/pl/ycnxt49cxidh3x36j6actusxay)
   Have updated the SpeedCloud credentials on the sheet which has been shared to you.
   
   March 04
   	
   jayanth
   [11:18 AM](https://nxtchat.nxtgen.com/emerging/pl/qaeqgbzqmpy95exxkhjti6in7w)
   https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
   
   [HTTP headers - HTTP | MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers)
   HTTP headers let the client and the server pass additional information with an HTTP request or response. An HTTP header consists of its case-insensitive name followed by a colon (:), then by its value. Whitespace before the value is ignored. 
   
   March 07
   	
   jayanth
   [11:33 AM](https://nxtchat.nxtgen.com/emerging/pl/8fp9uszbmjgcffakkmd5cbu9mo)
   Anuj asked me to redeploy the VM.
   
   	
   siddhit
   [11:33 AM](https://nxtchat.nxtgen.com/emerging/pl/4i6qr6xxntdpi8pbzpk8w1dboo)
   Ok
   
   	
   jayanth
   [11:33 AM](https://nxtchat.nxtgen.com/emerging/pl/jx1o9ztnuj8uifjmqp31fgrbir)
   Doing that.
   
   	
   jayanth
   [4:25 PM](https://nxtchat.nxtgen.com/emerging/pl/ku5dcgcffinyxxix9mdn3dzbka)
   Anuj VM Creation done.
   
   March 08
   	
   jayanth
   [12:04 PM](https://nxtchat.nxtgen.com/emerging/pl/dzh9gu944ffitjxq9d7dzgwg8y)
   https://us04web.zoom.us/j/78293871969?pwd=CoEAvBx4MLDhGGsSBE8Hp_8H8SomT0.1 
   
   Zoom Video
   [Join our Cloud HD Video Meeting](https://us04web.zoom.us/j/78293871969?pwd=CoEAvBx4MLDhGGsSBE8Hp_8H8SomT0.1)
   Zoom is the leader in modern enterprise video communications, with an easy, reliable cloud platform for video and audio conferencing, chat, and webinars across mobile, desktop, and room systems. Zoom Rooms is the original software-based conference room solution used around the world in board, con...
   March 09
   	
   jayanth
   [12:33 PM](https://nxtchat.nxtgen.com/emerging/pl/s6m7gzb6jj81pb6jqxnnx3tfro)
   https://www.haproxy.com/blog/load-balancing-affinity-persistence-sticky-sessions-what-you-need-to-know/
   
   Very nice explanation.
   
   HAProxy Technologies
   [Load Balancing, Affinity, Persistence, Sticky Sessions: What You Need to Know - HAProxy Technologies](https://www.haproxy.com/blog/load-balancing-affinity-persistence-sticky-sessions-what-you-need-to-know/)
   This blog post shows why and how to use a load balancer, the differences between Affinity, Persistence and Sticky Sessions and how to enable them in HAProxy 
   
   	
   jayanth
   [2:10 PM](https://nxtchat.nxtgen.com/emerging/pl/wtin4nwoobyufj7cnxm7syzbha)[](https://nxtchat.nxtgen.com/emerging/messages/@jayanth#)[](https://nxtchat.nxtgen.com/api/v4/files/namgby3mdjrixyitbybraag16w?download=1)
   Sasi_DevOps_Fresher.pdf
   PDF162KB
   Thursday
   	
   siddhit
   [1:09 PM](https://nxtchat.nxtgen.com/emerging/pl/4zxnfsa7ejrb8nqtxif181s1ya)
   Lets dig deeper w.r.t yestd degraded cephfs logs and see if any pattern or event which could have caused the outage
   
   
   	
   siddhit
   [1:55 PM](https://nxtchat.nxtgen.com/emerging/pl/8oach3ge4tbyj8uj87n97hjodo)
   
   Pls Check IOPS and throughupt for 4MB and 512 KB blocksize for both RBD and NFS
   
   	
   siddhit
   [1:55 PM](https://nxtchat.nxtgen.com/emerging/pl/a849wy4a4byq58cuo8a7p9cuhc)
   
   i guess we did for 64 KB right?
   
   	
   jayanth
   [1:56 PM](https://nxtchat.nxtgen.com/emerging/pl/yyooeok8iid4dntesudzynizth)
   Correct.
   
   	
   jayanth
   [1:56 PM](https://nxtchat.nxtgen.com/emerging/pl/ba55nuqohtfi3p3zch3pdd8pqw)
   
   Sure.
   
   	
   jayanth
   1:58 PM
   
   IOPS at 4K, throughput at 64K.
   
   We'll have all tests done.
   
   	
   [siddhit]()
   2:52 PM
   SiteOPsAHD_BSNLIDC SiteOPsAHD_BSNLIDC@nxtgen.com
   
   	
   [siddhit]()
   [3:30 PM](https://nxtchat.nxtgen.com/emerging/pl/3fi9pd7h8jrpikudcxd3wdw77r)
   https://www.sebastien-han.fr/blog/2012/07/06/nfs-over-rbd/
   
   Today
   New Messages
   	
   jayanth
   [9:48 AM](https://nxtchat.nxtgen.com/emerging/pl/pb76bhi633b8zdgio6kkkgq8hc)
   
   AHD firewall itself is down/experiencing issues. 
   
   [9:48 AM](https://nxtchat.nxtgen.com/emerging/pl/wc9upw9w938ffypybztr63g3ya)
   14.192.19.27
   
   
   	
   jayanth
   [1:23 PM](https://nxtchat.nxtgen.com/emerging/pl/e99mpke7n3dhbmnz8y5hhhyjea)
   
   Any updates from Anuj - Riversilica? 
   
   	
   siddhit
   [1:56 PM](https://nxtchat.nxtgen.com/emerging/pl/cut8d4uxsig5zktn3tjdmzfokw)
   Nope.Informed Rajesh to expedite.Seems they are tied in some project.
   
   	
   jayanth
   [1:57 PM](https://nxtchat.nxtgen.com/emerging/pl/9mo178tgpbyubeg88tnqka6aao)
   Oh okay.
   
   	
   siddhit
   [4:18 PM](https://nxtchat.nxtgen.com/emerging/pl/1zg9wui13pbxpexr8h4ikg356r)
   is our pritunl setup still operational?
   
   	
   jayanth
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/u9drmh1gwpf3bf1i8y6ugor7ky)
   Ah, Pritunl? 
   
   barebones openvpn setup is still operational. No Pritunl controlled setups exist.
   
   	
   siddhit
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/8pi3jouwhpgmdrz4mp5ti977zo)
   ok
   
   	
   jayanth
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/swjgjd9upfdtmyak7piquxmtwc)
   Which location are you looking for?
   
   	
   siddhit
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/wq11zpciipbs3xxbgggffkx95o)
   none specific
   
   	
   jayanth
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/nitdgtj793rz8e9z7474rke7uy)
   Okay.
   
   	
   siddhit
   [4:19 PM](https://nxtchat.nxtgen.com/emerging/pl/enubdjzwi7yhue3qhkwu1it69o)
   just wanted to refer dashboard
   
   	
   jayanth
   [4:20 PM](https://nxtchat.nxtgen.com/emerging/pl/toujgdnfjfr33e3h38yt9nerqr)
   https://speedcloudstage.nxtgen.com/#vms-tab/1579
   
   check this out then.
   
   It's turned off currently.
   
   	
   siddhit
   [4:23 PM](https://nxtchat.nxtgen.com/emerging/pl/uxw578mrrprkpbzstjh66uxoew)
   
   can you add staging creds also in master sheet?
   
   	
   jayanth
   [4:23 PM](https://nxtchat.nxtgen.com/emerging/pl/ykhgwxqb4frj9yf377ae6serpw)
   Sure.
   
   	
   siddhit
   [4:39 PM](https://nxtchat.nxtgen.com/emerging/pl/3cezcr11bjb9up1d8mjxyufauy)
   Any new profiles for interview?
   
   	
   jayanth
   [4:40 PM](https://nxtchat.nxtgen.com/emerging/pl/9p7g1qf3ttr9jj7eoagqm9razh)
   2 new applicants. To be interviewed very soon.
   
   	
   siddhit
   [4:40 PM](https://nxtchat.nxtgen.com/emerging/pl/xrp9wxxun7ya9et9z71n3moaac)
   ok
   
   keep thread posted
   
   	
   jayanth
   [4:40 PM](https://nxtchat.nxtgen.com/emerging/pl/yqrm7nghijn1bp3s1gsdtb6c1a)
   Sure,
   
   	
   jayanth
   [4:49 PM](https://nxtchat.nxtgen.com/emerging/pl/gmuu3sgqxtf9tn66imbb695ejr)
   No interviews have been done from your end right for Associate profiles right, just confirming. 
   
   	
   siddhit
   [4:59 PM](https://nxtchat.nxtgen.com/emerging/pl/rrj6fkcy5jnn9fopipep6mayka)
   not after you initiated profiling 
   
   	
   jayanth
   [4:59 PM](https://nxtchat.nxtgen.com/emerging/pl/s1kemtckft8k5qdcpenoq5c8nh)
   Okay.
   
   	
   siddhit
   [4:59 PM](https://nxtchat.nxtgen.com/emerging/pl/1cdi7x1or7drmdayro185iomuo)
   serveral before them..will collate and share list for past reviewed candidates
   
   	
   jayanth
   [4:59 PM](https://nxtchat.nxtgen.com/emerging/pl/u6mf6gaucj8qbktj6zhao67apr)
   Okay.
   
   	
   jayanth
   [5:05 PM](https://nxtchat.nxtgen.com/emerging/pl/basmfcbkkfr8jk9h8b6sz5kq9r)
   
   Done.
   
   Today
   	
   jayanth
   [1:15 PM](https://nxtchat.nxtgen.com/emerging/pl/wwahu1uz3ib7ijfpmrayp5i4zr)[](https://nxtchat.nxtgen.com/emerging/messages/@jayanth#)[](https://nxtchat.nxtgen.com/api/v4/files/w8csjfze7jnkbppw7mxsertrdh?download=1)
   k8s-sid.log
   LOG36KB
   Write to jayanth
   
   No file chosen
   
   [Help](https://nxtchat.nxtgen.com/help/messaging?locale=en)[](https://nxtchat.nxtgen.com/api/v4/files/w8csjfze7jnkbppw7mxsertrdh?download=1)
   k8s-sid.log
   JayanthShared In ~Jayanth
   
   2022-03-10 17:48:58,360 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:18,371 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:38,382 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:49:58,393 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:18,404 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:38,415 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:50:58,426 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:18,437 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:38,448 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:51:58,459 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:18,470 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:38,481 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:52:58,492 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:18,502 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:38,513 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:53:58,524 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:18,529 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:38,540 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:54:58,550 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:03,869 INFO  [c.c.a.ApiServer] (qtp1603198149-26090:ctx-50dbb34c ctx-bfeb2abd) (logid:8094e808) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:14,316 INFO  [c.c.a.ApiServer] (qtp1603198149-26090:ctx-9b7fafc0 ctx-79479e6f) (logid:885c6916) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:17,359 INFO  [c.c.a.ApiServer] (qtp1603198149-26087:ctx-efd30324 ctx-5916a61e) (logid:974dfb72) Setup is in progress for Kubernetes cluster : k8s-sid, config not available at this moment
   2022-03-10 17:55:18,561 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:38,571 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:55:58,582 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:18,584 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:38,596 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:56:58,606 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:18,617 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:38,628 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:57:58,636 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:18,647 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:38,658 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:58:40,237 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-acdc0c28) (logid:ae83f9b3) Seq 3-724516590053290113: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 17:58:58,669 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:18,680 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:38,691 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 17:59:58,702 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:18,712 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:38,723 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:00:58,725 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:18,736 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:38,747 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:01:58,758 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:18,768 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:38,779 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:02:58,789 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:18,800 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:38,811 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:03:58,822 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:18,833 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:38,844 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:04:58,855 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:18,866 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:38,876 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:05:58,889 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:18,900 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:38,911 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:06:58,922 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:18,932 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:38,944 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:07:58,955 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:18,967 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:38,946 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-5639dca3) (logid:fc60836a) Seq 3-724516590053290160: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:08:38,978 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:08:58,980 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:18,989 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:39,000 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:09:59,009 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:19,020 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:39,030 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:10:59,041 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:19,052 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:39,063 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:11:59,074 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:19,085 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:39,096 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:12:59,107 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:19,117 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:39,128 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:13:59,129 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:19,140 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:39,146 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:14:59,157 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:19,162 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:39,172 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:15:59,182 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:19,193 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:39,204 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:16:59,214 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:19,220 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:39,225 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:17:59,236 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:19,247 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:38,950 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-2d313558) (logid:4622835f) Seq 3-724516590053290208: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:18:39,257 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:18:59,268 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:19,278 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:39,289 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:19:59,290 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:19,301 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:39,312 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:20:59,323 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:19,334 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:39,345 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:21:59,356 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:19,367 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:39,378 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:22:59,389 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:19,399 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:39,410 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:23:59,421 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:19,430 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:39,441 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:24:59,452 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:19,464 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:39,475 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:25:59,485 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:19,497 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:39,508 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:26:59,513 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:19,524 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:39,535 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:27:59,546 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:19,558 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:39,569 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:28:40,086 DEBUG [c.c.a.t.Request] (RouterStatusMonitor-1:ctx-5e7c7e3b) (logid:370d4c7c) Seq 3-724516590053290257: Sending  { Cmd , MgmtId: 2486404317571, via: 3(mum-hv-02.speedcloud.com), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"healthChecksConfig":{"gateways":"gatewaysIps=169.254.0.1 103.230.37.153 ","portForwarding":"sourceIp=103.230.37.155,sourcePortStart=2222,sourcePortEnd=2222,destIp=172.26.21.128,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2223,sourcePortEnd=2223,destIp=172.26.21.201,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2224,sourcePortEnd=2224,destIp=172.26.21.29,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2225,sourcePortEnd=2225,destIp=172.26.21.65,destPortStart=22,destPortEnd=22;sourceIp=103.230.37.155,sourcePortStart=2226,sourcePortEnd=2226,destIp=172.26.21.219,destPortStart=22,destPortEnd=22;","systemThresholds":"minDiskNeeded
 =100.0,maxCpuUsage=100.0,maxMemoryUsage=100.0;","haproxyData":"maxconn=4096,sourcePortStart=6443,sourcePortEnd=6443,sourceIp=103.230.37.155,destPortStart=6443,destPortEnd=6443,algorithm=roundrobin,protocol=tcp,stickiness=None,keepAliveEnabled=false,vmIps=172.26.21.128 172.26.21.201;","virtualMachines":"vmName=k8s-sid-control-17f73b3832e,macAddress=02:00:51:8b:00:02,ip=172.26.21.128,dhcp=true,dns=true;vmName=k8s-sid-control-17f73b3a94f,macAddress=02:00:69:e8:00:03,ip=172.26.21.201,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3cc83,macAddress=02:00:2c:d0:00:04,ip=172.26.21.29,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b3eb89,macAddress=02:00:30:b3:00:05,ip=172.26.21.65,dhcp=true,dns=true;vmName=k8s-sid-node-17f73b40a72,macAddress=02:00:29:0b:00:06,ip=172.26.21.219,dhcp=true,dns=true;","routerVersion":"templateVersion=Cloudstack Release 4.16.0 Wed 22 Sep 2021 09:32:21 AM UTC,scriptsVersion=a68d1eec046236c8fa05fce9b60d2a98
   2022-03-10 18:28:59,580 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:19,590 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:39,601 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:29:59,611 INFO  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Waiting for Kubernetes cluster : k8s-sid control node VMs to be accessible
   2022-03-10 18:30:09,612 ERROR [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Failed to setup Kubernetes cluster : k8s-sid in usable state as unable to access control node VMs of the cluster
   2022-03-10 18:30:15,088 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-control-17f73b3832e in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:21,849 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-control-17f73b3a94f in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:23,218 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b3cc83 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:28,736 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b3eb89 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:30,903 INFO  [c.c.k.c.a.KubernetesClusterActionWorker] (API-Job-Executor-101:ctx-aed659b4 job-1811 ctx-c5dd498d) (logid:0a27d378) Detached Kubernetes binaries from VM : k8s-sid-node-17f73b40a72 in the Kubernetes cluster : k8s-sid
   2022-03-10 18:30:30,903 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-101:ctx-aed659b4 job-1811) (logid:0a27d378) Complete async job-1811, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Failed to setup Kubernetes cluster : k8s-sid in usable state as unable to access control node VMs of the cluster"}`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org