You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by da...@outlook.be,
da...@outlook.be on 2019/06/03 11:14:23 UTC
Unable to ping/ssh my VMs after a stop/start
Hello,
I'm a new cloudstack user. I've started with a fresh install on 2 ubuntu 18.04 boxes : 1 for the management server , 1 for a KVM node.
I'm able to create ubuntu VMs to which I can connect via ssh from the KVM node.
But when I stop/start the VMs, I can no longer ping or ssh the VMs.
================================
The sequence of operations are :
================================
0) Check bridge on KVM node before VM creation
----------------------------------------------
xxx@node001:~$ brctl show cloudbr0
bridge name bridge id STP enabled interfaces
cloudbr0 8000.ce1467c23351 no eno1
vnet0
vnet3
vnet4
vnet6
vnet7
1) Create a VM instance from the web UI
---------------------------------------
=> IP 192.168.0.61
=> internal name i-2-21-VM
xxx@node001:~$ brctl show cloudbr0
bridge name bridge id STP enabled interfaces
cloudbr0 8000.ce1467c23351 no eno1
vnet0
vnet3
vnet4
vnet6
vnet7
vnet8
xxx@node001:~$ ifconfig vnet8
vnet8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::fc00:13ff:fe00:16 prefixlen 64 scopeid 0x20<link>
ether fe:00:13:00:00:16 txqueuelen 1000 (Ethernet)
RX packets 22 bytes 2540 (2.5 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 37 bytes 5398 (5.3 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
=> arp entry : 192.168.0.61 ether 1e:00:13:00:00:16 C cloudbr0
2) From the KVM node, ssh or ping 192.168.0.61 works perfectly
--------------------------------------------------------------
xxx@node001:~$ ping 192.168.0.61
PING 192.168.0.61 (192.168.0.61) 56(84) bytes of data.
64 bytes from 192.168.0.61: icmp_seq=1 ttl=64 time=0.631 ms
64 bytes from 192.168.0.61: icmp_seq=2 ttl=64 time=0.209 ms
64 bytes from 192.168.0.61: icmp_seq=3 ttl=64 time=0.226 ms
^C
--- 192.168.0.61 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2056ms
rtt min/avg/max/mdev = 0.209/0.355/0.631/0.195 ms
3) Stop the VM instance from the UI
-----------------------------------
xxx@node001:~$ brctl show cloudbr0
bridge name bridge id STP enabled interfaces
cloudbr0 8000.ce1467c23351 no eno1
vnet0
vnet3
vnet4
vnet6
vnet7
xxx@node001:~$ ping 192.168.0.61
PING 192.168.0.61 (192.168.0.61) 56(84) bytes of data.
^C
--- 192.168.0.61 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2044ms
=> arp entry : 192.168.0.61 (incomplete) cloudbr0
4) Start the VM instance from the UI
------------------------------------
xxx@node001:~$ brctl show cloudbr0
bridge name bridge id STP enabled interfaces
cloudbr0 8000.ce1467c23351 no eno1
vnet0
vnet3
vnet4
vnet6
vnet7
vnet8
xxx@node001:~$ ifconfig vnet8
vnet8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::fc00:13ff:fe00:16 prefixlen 64 scopeid 0x20<link>
ether fe:00:13:00:00:16 txqueuelen 1000 (Ethernet)
RX packets 89 bytes 4330 (4.3 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 25 bytes 3518 (3.5 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
5) ISSUE : From the KVM node, ssh 192.168.0.61 no longer work
-------------------------------------------------------------
xxx@node001:~$ ping 192.168.0.61
PING 192.168.0.61 (192.168.0.61) 56(84) bytes of data.
From 192.168.0.28 icmp_seq=1 Destination Host Unreachable
From 192.168.0.28 icmp_seq=2 Destination Host Unreachable
From 192.168.0.28 icmp_seq=3 Destination Host Unreachable
^C
=> arp entry : 192.168.0.61 (incomplete) cloudbr0
======================
Management server LOGS
======================
VM Creation
-----------
2019-06-03 12:52:21,201 DEBUG [c.c.v.VirtualMachineManagerImpl] (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocating entries for VM: VM[User|i-2-21-VM]
2019-06-03 12:52:21,201 DEBUG [c.c.v.VirtualMachineManagerImpl] (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocating nics for VM[User|i-2-21-VM]
2019-06-03 12:52:21,201 DEBUG [o.a.c.e.o.NetworkOrchestrator] (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocating nic for vm VM[User|i-2-21-VM] in network Ntwk[204|Guest|6] with requested profile NicProfile[0-0-null-null-null
2019-06-03 12:52:21,206 DEBUG [c.c.v.VirtualMachineManagerImpl] (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocating disks for VM[User|i-2-21-VM]
2019-06-03 12:52:21,215 DEBUG [c.c.v.VirtualMachineManagerImpl] (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocation completed for VM: VM[User|i-2-21-VM]
2019-06-03 12:52:21,215 DEBUG [c.c.v.UserVmManagerImpl] (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Successfully allocated DB entry for VM[User|i-2-21-VM]
2019-06-03 12:52:23,090 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-71:ctx-57028b14 job-188/job-189 ctx-8ad63f37) (logid:56adfd38) Successfully transitioned to start state for VM[User|i-2-21-VM] reservation id = 24b9ed17-834d-4e0a-a6c7-b6521f7813b2
2019-06-03 12:52:23,178 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-71:ctx-57028b14 job-188/job-189 ctx-8ad63f37) (logid:56adfd38) Deployment found - P0=VM[User|i-2-21-VM], P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(21|ROOT-->Pool(2))]
8174: Sending { Cmd , MgmtId: 75006884826222, via: 1(node001), Ver: v1, Flags: 100111, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"ef1cda00-68e7-4958-b35a-3df95ba8c029","uuid":"ef1cda00-68e7-4958-b35a-3df95ba8c029","id":207,"format":"QCOW2","accountId":2,"hvm":true,"displayText":"ubuntu 16.04.6 server","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8","isManaged":false}},"name":"b3527b5d-1d1e-4b9d-bc03-308f023b655e","size":5368709120,"hypervisorType":"KVM","bootable":false,"uniqueName":"b3527b5d-1d1e-4b9d-bc03-308f023b655e","directDownload":false}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8","isManaged":false}},"name":"ROOT-21","size":5368709120,"volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"hypervisorType":"KVM"}},"executeInSequence":true,"options":{},"options2":{},"wait":0}}] }
2019-06-03 12:52:26,648 DEBUG [c.c.a.t.Request] (Work-Job-Executor-71:ctx-57028b14 job-188/job-189 ctx-8ad63f37) (logid:56adfd38) Seq 1-7235595751324128175: Sending { Cmd , MgmtId: 75006884826222, via: 1(node001), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StartCommand":{"vm":{"id":21,"name":"i-2-21-VM","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Ubuntu 16.04 (64-bit)","platformEmulator":"Ubuntu 16.04","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"COaaT8974fN86DWrTYVQrA","params":{"keyboard":"us","deployvm":"true","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"4fb4b946-5d2b-4b66-a29a-cf218886615a","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8","isManaged":false}},"name":"ROOT-21","size":5368709120,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","type":"ROOT","_details":{"storageHost":"192.168.0.34","managed":"false","storagePort":"2049","volumeSize":"5368709120"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false,"bootable":false,"directDownload":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false"},"uuid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","ip":"192.168.0.61","netmask":"255.255.255.0","gateway":"192.168.0.1","mac":"1e:00:13:00:00:16","dns1":"1.1.1.1","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true}],"guestOsDetails":{}},"hostIp":"192.168.0.28","executeInSequence":false,"wait":0}}] }
2019-06-03 12:52:27,507 DEBUG [c.c.a.t.Request] (AgentManager-Handler-8:null) (logid:) Seq 1-7235595751324128175: Processing: { Ans: , MgmtId: 75006884826222, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":21,"name":"i-2-21-VM","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Ubuntu 16.04 (64-bit)","platformEmulator":"Ubuntu 16.04","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"COaaT8974fN86DWrTYVQrA","vncAddr":"192.168.0.28","params":{"keyboard":"us","deployvm":"true","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"4fb4b946-5d2b-4b66-a29a-cf218886615a","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8","isManaged":false}},"name":"ROOT-21","size":5368709120,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","type":"ROOT","_details":{"storageHost":"192.168.0.34","managed":"false","storagePort":"2049","volumeSize":"5368709120"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false,"bootable":false,"directDownload":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false"},"uuid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","ip":"192.168.0.61","netmask":"255.255.255.0","gateway":"192.168.0.1","mac":"1e:00:13:00:00:16","dns1":"1.1.1.1","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true}],"guestOsDetails":{}},"result":true,"wait":0}}] }
2019-06-03 12:52:27,885 DEBUG [c.c.n.s.SecurityGroupManagerImpl] (SecGrp-Worker-25:ctx-1fab9c8b) (logid:89e0d0db) SecurityGroupManager v2: sending ruleset update for vm i-2-21-VM:ingress num rules=0:egress num rules=0 num cidrs=0 sig=d41d8cd98f00b204e9800998ecf8427e
2019-06-03 12:52:27,886 DEBUG [c.c.a.t.Request] (SecGrp-Worker-25:ctx-1fab9c8b) (logid:89e0d0db) Seq 1-7235595751324128176: Sending { Cmd , MgmtId: 75006884826222, via: 1(node001), Ver: v1, Flags: 100111, [{"com.cloud.agent.api.SecurityGroupRulesCmd":{"guestIp":"192.168.0.61","vmName":"i-2-21-VM","guestMac":"1e:00:13:00:00:16","signature":"d41d8cd98f00b204e9800998ecf8427e","seqNum":1,"vmId":21,"msId":75006884826222,"ingressRuleSet":[],"egressRuleSet":[],"wait":0}}] }
2019-06-03 12:52:28,081 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-71:ctx-57028b14 job-188/job-189 ctx-8ad63f37) (logid:56adfd38) Start completed for VM VM[User|i-2-21-VM]
2019-06-03 12:52:28,311 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-51:ctx-505e4d01 job-188 ctx-e3665401) (logid:56adfd38) Complete async job-188, jobStatus: SUCCEEDED, resultCode: 0, result: org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{"id":"4fb4b946-5d2b-4b66-a29a-cf218886615a","name":"vmu001","displayname":"vmu001","account":"admin","userid":"0d19d291-80c4-11e9-88d7-4437e6b9cc6e","username":"admin","domainid":"e20d260a-80c0-11e9-88d7-4437e6b9cc6e","domain":"ROOT","created":"2019-06-03T12:52:21+0200","state":"Running","haenable":false,"zoneid":"0d0c2a76-42e7-4ac7-9037-3712448fb0b0","zonename":"Zone1","hostid":"192c59bd-aa9a-47fa-9e9c-63df100d4449","hostname":"node001","templateid":"ef1cda00-68e7-4958-b35a-3df95ba8c029","templatename":"ubuntu 16.04.6 server","templatedisplaytext":"ubuntu 16.04.6 server","passwordenabled":false,"serviceofferingid":"c1f3654e-e7c7-4db5-b064-948eb2bbe517","serviceofferingname":"Small Instance","cpunumber":1,"cpuspeed":500,"memory":512,"guestosid":"ed88d9c7-80c3-11e9-88d7-4437e6b9cc6e","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[{"id":"0d4ecb8f-80c4-11e9-88d7-4437e6b9cc6e","name":"default","description":"Default Security Group","account":"admin","ingressrule":[],"egressrule":[],"tags":[],"virtualmachineids":[]}],"nic":[{"id":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","networkid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","networkname":"defaultGuestNetwork","netmask":"255.255.255.0","gateway":"192.168.0.1","ipaddress":"192.168.0.61","broadcasturi":"vlan://untagged","traffictype":"Guest","type":"Shared","isdefault":true,"macaddress":"1e:00:13:00:00:16","secondaryip":[],"extradhcpoption":[]}],"hypervisor":"KVM","instancename":"i-2-21-VM","details":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"affinitygroup":[],"displayvm":true,"isdynamicallyscalable":false,"ostypeid":256,"tags":[],"jobid":"56adfd38-67d5-4472-b241-300b8baad7f3","jobstatus":0}
VM Stop
-------
2019-06-03 12:57:23,179 DEBUG [c.c.a.t.Request] (AgentManager-Handler-8:null) (logid:) Seq 1-7235595751324128197: Processing: { Ans: , MgmtId: 75006884826222, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.GetVmDiskStatsAnswer":{"hostName":"node001","vmDiskStatsMap":{"i-2-21-VM":[{"vmName":"i-2-21-VM","path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","ioRead":4423,"ioWrite":140,"bytesWrite":1795072,"bytesRead":144043008}]},"result":true,"details":"","wait":0}}] }
2019-06-03 12:57:23,375 DEBUG [c.c.a.t.Request] (AgentManager-Handler-14:null) (logid:) Seq 1-7235595751324128198: Processing: { Ans: , MgmtId: 75006884826222, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.GetVmNetworkStatsAnswer":{"hostName":"node001","vmNetworkStatsMap":{"i-2-21-VM":[{"vmName":"i-2-21-VM","macAddress":"1e:00:13:00:00:16","bytesSent":3134,"bytesReceived":7366}]},"result":true,"details":"","wait":0}}] }
2019-06-03 12:57:23,458 DEBUG [c.c.a.t.Request] (Work-Job-Executor-72:ctx-ec8a2106 job-190/job-191 ctx-72c73494) (logid:92805b41) Seq 1-7235595751324128199: Sending { Cmd , MgmtId: 75006884826222, via: 1(node001), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"forceStop":false,"volumesToDisconnect":[],"vmName":"i-2-21-VM","executeInSequence":false,"wait":0}}] }
2019-06-03 12:59:26,120 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-72:ctx-ec8a2106 job-190/job-191 ctx-72c73494) (logid:92805b41) VM[User|i-2-21-VM] is stopped on the host. Proceeding to release resource held.
2019-06-03 12:59:26,261 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-72:ctx-ec8a2106 job-190/job-191 ctx-72c73494) (logid:92805b41) Successfully released network resources for the vm VM[User|i-2-21-VM]
2019-06-03 12:59:26,261 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-72:ctx-ec8a2106 job-190/job-191 ctx-72c73494) (logid:92805b41) Successfully released storage resources for the vm VM[User|i-2-21-VM]
2019-06-03 12:59:27,015 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-52:ctx-546ec715 job-190 ctx-13580c5f) (logid:92805b41) Complete async job-190, jobStatus: SUCCEEDED, resultCode: 0, result: org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{"id":"4fb4b946-5d2b-4b66-a29a-cf218886615a","name":"vmu001","displayname":"vmu001","account":"admin","userid":"0d19d291-80c4-11e9-88d7-4437e6b9cc6e","username":"admin","domainid":"e20d260a-80c0-11e9-88d7-4437e6b9cc6e","domain":"ROOT","created":"2019-06-03T12:52:21+0200","state":"Stopped","haenable":false,"zoneid":"0d0c2a76-42e7-4ac7-9037-3712448fb0b0","zonename":"Zone1","templateid":"ef1cda00-68e7-4958-b35a-3df95ba8c029","templatename":"ubuntu 16.04.6 server","templatedisplaytext":"ubuntu 16.04.6 server","passwordenabled":false,"serviceofferingid":"c1f3654e-e7c7-4db5-b064-948eb2bbe517","serviceofferingname":"Small Instance","cpunumber":1,"cpuspeed":500,"memory":512,"cpuused":"1.99%","networkkbsread":3,"networkkbswrite":1,"diskkbsread":0,"diskkbswrite":864,"memorykbs":524288,"memoryintfreekbs":279324,"memorytargetkbs":524288,"diskioread":0,"diskiowrite":99,"guestosid":"ed88d9c7-80c3-11e9-88d7-4437e6b9cc6e","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[{"id":"0d4ecb8f-80c4-11e9-88d7-4437e6b9cc6e","name":"default","description":"Default Security Group","account":"admin","ingressrule":[],"egressrule":[],"tags":[],"virtualmachineids":[]}],"nic":[{"id":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","networkid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","networkname":"defaultGuestNetwork","netmask":"255.255.255.0","gateway":"192.168.0.1","ipaddress":"192.168.0.61","broadcasturi":"vlan://untagged","traffictype":"Guest","type":"Shared","isdefault":true,"macaddress":"1e:00:13:00:00:16","secondaryip":[],"extradhcpoption":[]}],"hypervisor":"KVM","instancename":"i-2-21-VM","details":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"affinitygroup":[],"displayvm":true,"isdynamicallyscalable":false,"ostypeid":256,"tags":[],"jobid":"92805b41-9391-4211-8eb4-3b7d06f27030","jobstatus":0}
VM Start
--------
2019-06-03 13:02:35,531 DEBUG [c.c.a.t.Request] (Work-Job-Executor-73:ctx-625e7d4f job-192/job-193 ctx-8e8437af) (logid:b1f99bee) Seq 1-7235595751324128220: Sending { Cmd , MgmtId: 75006884826222, via: 1(node001), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StartCommand":{"vm":{"id":21,"name":"i-2-21-VM","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Ubuntu 16.04 (64-bit)","platformEmulator":"Ubuntu 16.04","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"COaaT8974fN86DWrTYVQrA","params":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"4fb4b946-5d2b-4b66-a29a-cf218886615a","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8","isManaged":false}},"name":"ROOT-21","size":5368709120,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","type":"ROOT","_details":{"storageHost":"192.168.0.34","managed":"false","storagePort":"2049","volumeSize":"5368709120"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false,"bootable":false,"directDownload":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false"},"uuid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","ip":"192.168.0.61","netmask":"255.255.255.0","gateway":"192.168.0.1","mac":"1e:00:13:00:00:16","dns1":"1.1.1.1","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true}],"guestOsDetails":{}},"hostIp":"192.168.0.28","executeInSequence":false,"wait":0}}] }
2019-06-03 13:02:36,231 DEBUG [c.c.a.t.Request] (AgentManager-Handler-6:null) (logid:) Seq 1-7235595751324128220: Processing: { Ans: , MgmtId: 75006884826222, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":21,"name":"i-2-21-VM","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Ubuntu 16.04 (64-bit)","platformEmulator":"Ubuntu 16.04","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"COaaT8974fN86DWrTYVQrA","vncAddr":"192.168.0.28","params":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"4fb4b946-5d2b-4b66-a29a-cf218886615a","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8","isManaged":false}},"name":"ROOT-21","size":5368709120,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","type":"ROOT","_details":{"storageHost":"192.168.0.34","managed":"false","storagePort":"2049","volumeSize":"5368709120"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false,"bootable":false,"directDownload":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false"},"uuid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","ip":"192.168.0.61","netmask":"255.255.255.0","gateway":"192.168.0.1","mac":"1e:00:13:00:00:16","dns1":"1.1.1.1","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true}],"guestOsDetails":{}},"result":true,"wait":0}}] }
2019-06-03 13:02:36,648 DEBUG [c.c.n.s.SecurityGroupManagerImpl] (SecGrp-Worker-34:ctx-1a1c6a04) (logid:4f0cd9ce) SecurityGroupManager v2: sending ruleset update for vm i-2-21-VM:ingress num rules=0:egress num rules=0 num cidrs=0 sig=d41d8cd98f00b204e9800998ecf8427e
2019-06-03 13:02:36,648 DEBUG [c.c.a.t.Request] (SecGrp-Worker-34:ctx-1a1c6a04) (logid:4f0cd9ce) Seq 1-7235595751324128221: Sending { Cmd , MgmtId: 75006884826222, via: 1(node001), Ver: v1, Flags: 100111, [{"com.cloud.agent.api.SecurityGroupRulesCmd":{"guestIp":"192.168.0.61","vmName":"i-2-21-VM","guestMac":"1e:00:13:00:00:16","signature":"d41d8cd98f00b204e9800998ecf8427e","seqNum":2,"vmId":21,"msId":75006884826222,"ingressRuleSet":[],"egressRuleSet":[],"wait":0}}] }
2019-06-03 13:02:36,778 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-73:ctx-625e7d4f job-192/job-193 ctx-8e8437af) (logid:b1f99bee) Start completed for VM VM[User|i-2-21-VM]
2019-06-03 13:02:37,012 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-53:ctx-9ad730ea job-192 ctx-b612185b) (logid:b1f99bee) Complete async job-192, jobStatus: SUCCEEDED, resultCode: 0, result: org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{"id":"4fb4b946-5d2b-4b66-a29a-cf218886615a","name":"vmu001","displayname":"vmu001","account":"admin","userid":"0d19d291-80c4-11e9-88d7-4437e6b9cc6e","username":"admin","domainid":"e20d260a-80c0-11e9-88d7-4437e6b9cc6e","domain":"ROOT","created":"2019-06-03T12:52:21+0200","state":"Running","haenable":false,"zoneid":"0d0c2a76-42e7-4ac7-9037-3712448fb0b0","zonename":"Zone1","hostid":"192c59bd-aa9a-47fa-9e9c-63df100d4449","hostname":"node001","templateid":"ef1cda00-68e7-4958-b35a-3df95ba8c029","templatename":"ubuntu 16.04.6 server","templatedisplaytext":"ubuntu 16.04.6 server","passwordenabled":false,"serviceofferingid":"c1f3654e-e7c7-4db5-b064-948eb2bbe517","serviceofferingname":"Small Instance","cpunumber":1,"cpuspeed":500,"memory":512,"cpuused":"1.99%","networkkbsread":3,"networkkbswrite":1,"diskkbsread":0,"diskkbswrite":864,"memorykbs":524288,"memoryintfreekbs":279324,"memorytargetkbs":524288,"diskioread":0,"diskiowrite":99,"guestosid":"ed88d9c7-80c3-11e9-88d7-4437e6b9cc6e","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[{"id":"0d4ecb8f-80c4-11e9-88d7-4437e6b9cc6e","name":"default","description":"Default Security Group","account":"admin","ingressrule":[],"egressrule":[],"tags":[],"virtualmachineids":[]}],"nic":[{"id":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","networkid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","networkname":"defaultGuestNetwork","netmask":"255.255.255.0","gateway":"192.168.0.1","ipaddress":"192.168.0.61","broadcasturi":"vlan://untagged","traffictype":"Guest","type":"Shared","isdefault":true,"macaddress":"1e:00:13:00:00:16","secondaryip":[],"extradhcpoption":[]}],"hypervisor":"KVM","instancename":"i-2-21-VM","details":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"affinitygroup":[],"displayvm":true,"isdynamicallyscalable":false,"ostypeid":256,"tags":[],"jobid":"b1f99bee-ce28-4a06-8929-8549806d60ba","jobstatus":0}
root@mgmtserver:~#
Thanks in advance for your help
Regards
Daniel
RE: Unable to ping/ssh my VMs after a stop/start
Posted by daniel bellido <da...@outlook.be>.
Hello Ivan,
Adding this on the hypervisor (to stop network bridge traffic from traversing IPtables / ARPtables) fixed the issue :
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
Thanks
________________________________
De : daniel bellido <da...@outlook.be>
Envoyé : lundi 3 juin 2019 21:24
À : users@cloudstack.apache.org
Objet : RE: Unable to ping/ssh my VMs after a stop/start
Hello Ivan,
Thanks a lot for your reply.
Indeed the next step will be to be able to ping from elsewhere than the hypervisor but some additional network configs are required and I didn't look at it yet.
So for the moment I'm just trying to ping/ssh my VMs from the hypervisor node where the bridge cloudbr0 is located.
After the stop/start, I also noticed that I was able to open a console from the UI to the VM. From there a ifconfig returns me 192.168.0.49 as IP on NIC ens3 but I can't ping that IP from the supervisor.
By the way iptables -L on the hypervisor returns this :
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT udp -- anywhere anywhere udp dpt:domain
ACCEPT tcp -- anywhere anywhere tcp dpt:domain
ACCEPT udp -- anywhere anywhere udp dpt:bootps
ACCEPT tcp -- anywhere anywhere tcp dpt:bootps
Chain FORWARD (policy ACCEPT)
target prot opt source destination
BF-cloudbr0 all -- anywhere anywhere PHYSDEV match --physdev-is-bridged
BF-cloudbr0 all -- anywhere anywhere PHYSDEV match --physdev-is-bridged
DROP all -- anywhere anywhere
DROP all -- anywhere anywhere
ACCEPT all -- anywhere 192.168.122.0/24 ctstate RELATED,ESTABLISHED
ACCEPT all -- 192.168.122.0/24 anywhere
ACCEPT all -- anywhere anywhere
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT udp -- anywhere anywhere udp dpt:bootpc
Chain BF-cloudbr0 (2 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
BF-cloudbr0-IN all -- anywhere anywhere PHYSDEV match --physdev-is-in --physdev-is-bridged
BF-cloudbr0-OUT all -- anywhere anywhere PHYSDEV match --physdev-is-out --physdev-is-bridged
ACCEPT all -- anywhere anywhere PHYSDEV match --physdev-out eno1 --physdev-is-bridged
Chain BF-cloudbr0-IN (1 references)
target prot opt source destination
r-12-VM all -- anywhere anywhere PHYSDEV match --physdev-in vnet0 --physdev-is-bridged
s-10-VM all -- anywhere anywhere PHYSDEV match --physdev-in vnet3 --physdev-is-bridged
s-10-VM all -- anywhere anywhere PHYSDEV match --physdev-in vnet4 --physdev-is-bridged
v-9-VM all -- anywhere anywhere PHYSDEV match --physdev-in vnet6 --physdev-is-bridged
v-9-VM all -- anywhere anywhere PHYSDEV match --physdev-in vnet7 --physdev-is-bridged
i-2-21-def all -- anywhere anywhere PHYSDEV match --physdev-in vnet8 --physdev-is-bridged
Chain BF-cloudbr0-OUT (1 references)
target prot opt source destination
r-12-VM all -- anywhere anywhere PHYSDEV match --physdev-out vnet0 --physdev-is-bridged
s-10-VM all -- anywhere anywhere PHYSDEV match --physdev-out vnet3 --physdev-is-bridged
s-10-VM all -- anywhere anywhere PHYSDEV match --physdev-out vnet4 --physdev-is-bridged
v-9-VM all -- anywhere anywhere PHYSDEV match --physdev-out vnet6 --physdev-is-bridged
v-9-VM all -- anywhere anywhere PHYSDEV match --physdev-out vnet7 --physdev-is-bridged
i-2-21-def all -- anywhere anywhere PHYSDEV match --physdev-out vnet8 --physdev-is-bridged
Chain i-2-21-VM (1 references)
target prot opt source destination
DROP all -- anywhere anywhere
Chain i-2-21-VM-eg (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere
Chain i-2-21-def (2 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT udp -- anywhere anywhere PHYSDEV match --physdev-in vnet8 --physdev-is-bridged udp spt:bootpc dpt:bootps
ACCEPT udp -- anywhere anywhere PHYSDEV match --physdev-out vnet8 --physdev-is-bridged udp spt:bootps dpt:bootpc
DROP all -- anywhere anywhere PHYSDEV match --physdev-in vnet8 --physdev-is-bridged ! match-set i-2-21-VM src
RETURN udp -- anywhere anywhere PHYSDEV match --physdev-in vnet8 --physdev-is-bridged match-set i-2-21-VM src udp dpt:domain
RETURN tcp -- anywhere anywhere PHYSDEV match --physdev-in vnet8 --physdev-is-bridged match-set i-2-21-VM src tcp dpt:domain
i-2-21-VM-eg all -- anywhere anywhere PHYSDEV match --physdev-in vnet8 --physdev-is-bridged match-set i-2-21-VM src
i-2-21-VM all -- anywhere anywhere PHYSDEV match --physdev-out vnet8 --physdev-is-bridged
Chain r-12-VM (2 references)
target prot opt source destination
RETURN all -- anywhere anywhere PHYSDEV match --physdev-in vnet0 --physdev-is-bridged
ACCEPT all -- anywhere anywhere
Chain s-10-VM (4 references)
target prot opt source destination
RETURN all -- anywhere anywhere PHYSDEV match --physdev-in vnet3 --physdev-is-bridged
RETURN all -- anywhere anywhere PHYSDEV match --physdev-in vnet4 --physdev-is-bridged
ACCEPT all -- anywhere anywhere
Chain v-9-VM (4 references)
target prot opt source destination
RETURN all -- anywhere anywhere PHYSDEV match --physdev-in vnet6 --physdev-is-bridged
RETURN all -- anywhere anywhere PHYSDEV match --physdev-in vnet7 --physdev-is-bridged
ACCEPT all -- anywhere anywhere
________________________________
De : Ivan Kudryavtsev <ku...@bw-sw.com>
Envoyé : lundi 3 juin 2019 13:21
À : users
Objet : Re: Unable to ping/ssh my VMs after a stop/start
Daniel,
why you think you should be able to ping from the hypervisor? Normally, you
have to add ip in the same net to the bridge to ping tun/tap. I'm not sure,
but if it doesn't work after the step above, check your iptables/ebtables
to check every rule is ok.
Commands:
Iptables-save
Ebtables -t nat
Ipset -L
пн, 3 июн. 2019 г., 18:14 daniel.bellido@outlook.be <
daniel.bellido@outlook.be>:
> Hello,
>
> I'm a new cloudstack user. I've started with a fresh install on 2 ubuntu
> 18.04 boxes : 1 for the management server , 1 for a KVM node.
> I'm able to create ubuntu VMs to which I can connect via ssh from the KVM
> node.
> But when I stop/start the VMs, I can no longer ping or ssh the VMs.
>
> ================================
> The sequence of operations are :
> ================================
>
> 0) Check bridge on KVM node before VM creation
> ----------------------------------------------
> xxx@node001:~$ brctl show cloudbr0
> bridge name bridge id STP enabled interfaces
> cloudbr0 8000.ce1467c23351 no eno1
> vnet0
> vnet3
> vnet4
> vnet6
> vnet7
>
> 1) Create a VM instance from the web UI
> ---------------------------------------
> => IP 192.168.0.61
> => internal name i-2-21-VM
> xxx@node001:~$ brctl show cloudbr0
> bridge name bridge id STP enabled interfaces
> cloudbr0 8000.ce1467c23351 no eno1
> vnet0
> vnet3
> vnet4
> vnet6
> vnet7
> vnet8
> xxx@node001:~$ ifconfig vnet8
> vnet8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> inet6 fe80::fc00:13ff:fe00:16 prefixlen 64 scopeid 0x20<link>
> ether fe:00:13:00:00:16 txqueuelen 1000 (Ethernet)
> RX packets 22 bytes 2540 (2.5 KB)
> RX errors 0 dropped 0 overruns 0 frame 0
> TX packets 37 bytes 5398 (5.3 KB)
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
> => arp entry : 192.168.0.61 ether 1e:00:13:00:00:16 C
> cloudbr0
>
> 2) From the KVM node, ssh or ping 192.168.0.61 works perfectly
> --------------------------------------------------------------
> xxx@node001:~$ ping 192.168.0.61
> PING 192.168.0.61 (192.168.0.61) 56(84) bytes of data.
> 64 bytes from 192.168.0.61: icmp_seq=1 ttl=64 time=0.631 ms
> 64 bytes from 192.168.0.61: icmp_seq=2 ttl=64 time=0.209 ms
> 64 bytes from 192.168.0.61: icmp_seq=3 ttl=64 time=0.226 ms
> ^C
> --- 192.168.0.61 ping statistics ---
> 3 packets transmitted, 3 received, 0% packet loss, time 2056ms
> rtt min/avg/max/mdev = 0.209/0.355/0.631/0.195 ms
>
> 3) Stop the VM instance from the UI
> -----------------------------------
> xxx@node001:~$ brctl show cloudbr0
> bridge name bridge id STP enabled interfaces
> cloudbr0 8000.ce1467c23351 no eno1
> vnet0
> vnet3
> vnet4
> vnet6
> vnet7
> xxx@node001:~$ ping 192.168.0.61
> PING 192.168.0.61 (192.168.0.61) 56(84) bytes of data.
> ^C
> --- 192.168.0.61 ping statistics ---
> 3 packets transmitted, 0 received, 100% packet loss, time 2044ms
>
> => arp entry : 192.168.0.61 (incomplete)
> cloudbr0
>
> 4) Start the VM instance from the UI
> ------------------------------------
> xxx@node001:~$ brctl show cloudbr0
> bridge name bridge id STP enabled interfaces
> cloudbr0 8000.ce1467c23351 no eno1
> vnet0
> vnet3
> vnet4
> vnet6
> vnet7
> vnet8
>
> xxx@node001:~$ ifconfig vnet8
> vnet8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> inet6 fe80::fc00:13ff:fe00:16 prefixlen 64 scopeid 0x20<link>
> ether fe:00:13:00:00:16 txqueuelen 1000 (Ethernet)
> RX packets 89 bytes 4330 (4.3 KB)
> RX errors 0 dropped 0 overruns 0 frame 0
> TX packets 25 bytes 3518 (3.5 KB)
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
> 5) ISSUE : From the KVM node, ssh 192.168.0.61 no longer work
> -------------------------------------------------------------
> xxx@node001:~$ ping 192.168.0.61
> PING 192.168.0.61 (192.168.0.61) 56(84) bytes of data.
> From 192.168.0.28 icmp_seq=1 Destination Host Unreachable
> From 192.168.0.28 icmp_seq=2 Destination Host Unreachable
> From 192.168.0.28 icmp_seq=3 Destination Host Unreachable
> ^C
>
> => arp entry : 192.168.0.61 (incomplete)
> cloudbr0
>
> ======================
> Management server LOGS
> ======================
>
> VM Creation
> -----------
> 2019-06-03 12:52:21,201 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocating
> entries for VM: VM[User|i-2-21-VM]
> 2019-06-03 12:52:21,201 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocating
> nics for VM[User|i-2-21-VM]
> 2019-06-03 12:52:21,201 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocating
> nic for vm VM[User|i-2-21-VM] in network Ntwk[204|Guest|6] with requested
> profile NicProfile[0-0-null-null-null
> 2019-06-03 12:52:21,206 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocating
> disks for VM[User|i-2-21-VM]
> 2019-06-03 12:52:21,215 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocation
> completed for VM: VM[User|i-2-21-VM]
> 2019-06-03 12:52:21,215 DEBUG [c.c.v.UserVmManagerImpl]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Successfully
> allocated DB entry for VM[User|i-2-21-VM]
> 2019-06-03 12:52:23,090 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-71:ctx-57028b14 job-188/job-189 ctx-8ad63f37)
> (logid:56adfd38) Successfully transitioned to start state for
> VM[User|i-2-21-VM] reservation id = 24b9ed17-834d-4e0a-a6c7-b6521f7813b2
> 2019-06-03 12:52:23,178 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-71:ctx-57028b14 job-188/job-189 ctx-8ad63f37)
> (logid:56adfd38) Deployment found - P0=VM[User|i-2-21-VM],
> P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(21|ROOT-->Pool(2))]
> 8174: Sending { Cmd , MgmtId: 75006884826222, via: 1(node001), Ver: v1,
> Flags: 100111,
> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.
> cloudstack.storage.to.TemplateObjectTO":{"path":"ef1cda00-68e7-4958-b35a-3df95ba8c029","uuid":"ef1cda00-68e7-4958-b35a-3df95ba8c029","id":207,"format":"QCOW2","accountId":2,"hvm":true,"displayText":"ubuntu
> 16.04.6
> server","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8
> ","isManaged":false}},"name":"b3527b5d-1d1e-4b9d-bc03-308f023b655e","size":5368709120,"hypervisorType":"KVM","bootable":false,"uniqueName":"b3527b5d-1d1e-4b9d-bc03-308f023b655e","directDownload":false}},"destTO":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.
> apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8","isManaged":false}},"name":"ROOT-21","size":5368709120,"volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"hypervisorType":"KVM"}},"executeInSequence":true,"options":{},"options2":{},"wait":0}}]
> }
> 2019-06-03 12:52:26,648 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-71:ctx-57028b14 job-188/job-189 ctx-8ad63f37)
> (logid:56adfd38) Seq 1-7235595751324128175: Sending { Cmd , MgmtId:
> 75006884826222, via: 1(node001), Ver: v1, Flags: 100011,
> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":21,"name":"i-2-21-VM","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Ubuntu
> 16.04 (64-bit)","platformEmulator":"Ubuntu
> 16.04","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"COaaT8974fN86DWrTYVQrA","params":{"keyboard":"us","deployvm":"true","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"4fb4b946-5d2b-4b66-a29a-cf218886615a","disks":[{"data":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.
> apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8
> ","isManaged":false}},"name":"ROOT-21","size":5368709120,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","type":"ROOT","_details":{"storageHost":"192.168.0.34","managed":"false","storagePort":"2049","volumeSize":"5368709120"}},{"data":{"org.
> apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false,"bootable":false,"directDownload":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false"},"uuid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","ip":"192.168.0.61","netmask":"255.255.255.0","gateway":"192.168.0.1","mac":"1e:00:13:00:00:16","dns1":"1.1.1.1","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true}],"guestOsDetails":{}},"hostIp":"192.168.0.28","executeInSequence":false,"wait":0}}]
> }
> 2019-06-03 12:52:27,507 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-8:null) (logid:) Seq 1-7235595751324128175:
> Processing: { Ans: , MgmtId: 75006884826222, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":21,"name":"i-2-21-VM","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Ubuntu
> 16.04 (64-bit)","platformEmulator":"Ubuntu
> 16.04","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"COaaT8974fN86DWrTYVQrA","vncAddr":"192.168.0.28","params":{"keyboard":"us","deployvm":"true","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"4fb4b946-5d2b-4b66-a29a-cf218886615a","disks":[{"data":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.
> apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8
> ","isManaged":false}},"name":"ROOT-21","size":5368709120,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","type":"ROOT","_details":{"storageHost":"192.168.0.34","managed":"false","storagePort":"2049","volumeSize":"5368709120"}},{"data":{"org.
> apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false,"bootable":false,"directDownload":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false"},"uuid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","ip":"192.168.0.61","netmask":"255.255.255.0","gateway":"192.168.0.1","mac":"1e:00:13:00:00:16","dns1":"1.1.1.1","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true}],"guestOsDetails":{}},"result":true,"wait":0}}]
> }
> 2019-06-03 12:52:27,885 DEBUG [c.c.n.s.SecurityGroupManagerImpl]
> (SecGrp-Worker-25:ctx-1fab9c8b) (logid:89e0d0db) SecurityGroupManager v2:
> sending ruleset update for vm i-2-21-VM:ingress num rules=0:egress num
> rules=0 num cidrs=0 sig=d41d8cd98f00b204e9800998ecf8427e
> 2019-06-03 12:52:27,886 DEBUG [c.c.a.t.Request]
> (SecGrp-Worker-25:ctx-1fab9c8b) (logid:89e0d0db) Seq 1-7235595751324128176:
> Sending { Cmd , MgmtId: 75006884826222, via: 1(node001), Ver: v1, Flags:
> 100111,
> [{"com.cloud.agent.api.SecurityGroupRulesCmd":{"guestIp":"192.168.0.61","vmName":"i-2-21-VM","guestMac":"1e:00:13:00:00:16","signature":"d41d8cd98f00b204e9800998ecf8427e","seqNum":1,"vmId":21,"msId":75006884826222,"ingressRuleSet":[],"egressRuleSet":[],"wait":0}}]
> }
> 2019-06-03 12:52:28,081 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-71:ctx-57028b14 job-188/job-189 ctx-8ad63f37)
> (logid:56adfd38) Start completed for VM VM[User|i-2-21-VM]
> 2019-06-03 12:52:28,311 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-51:ctx-505e4d01 job-188 ctx-e3665401) (logid:56adfd38)
> Complete async job-188, jobStatus: SUCCEEDED, resultCode: 0, result:
> org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{"id":"4fb4b946-5d2b-4b66-a29a-cf218886615a","name":"vmu001","displayname":"vmu001","account":"admin","userid":"0d19d291-80c4-11e9-88d7-4437e6b9cc6e","username":"admin","domainid":"e20d260a-80c0-11e9-88d7-4437e6b9cc6e","domain":"ROOT","created":"2019-06-03T12:52:21+0200","state":"Running","haenable":false,"zoneid":"0d0c2a76-42e7-4ac7-9037-3712448fb0b0","zonename":"Zone1","hostid":"192c59bd-aa9a-47fa-9e9c-63df100d4449","hostname":"node001","templateid":"ef1cda00-68e7-4958-b35a-3df95ba8c029","templatename":"ubuntu
> 16.04.6 server","templatedisplaytext":"ubuntu 16.04.6
> server","passwordenabled":false,"serviceofferingid":"c1f3654e-e7c7-4db5-b064-948eb2bbe517","serviceofferingname":"Small
> Instance","cpunumber":1,"cpuspeed":500,"memory":512,"guestosid":"ed88d9c7-80c3-11e9-88d7-4437e6b9cc6e","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[{"id":"0d4ecb8f-80c4-11e9-88d7-4437e6b9cc6e","name":"default","description":"Default
> Security
> Group","account":"admin","ingressrule":[],"egressrule":[],"tags":[],"virtualmachineids":[]}],"nic":[{"id":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","networkid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","networkname":"defaultGuestNetwork","netmask":"255.255.255.0","gateway":"192.168.0.1","ipaddress":"192.168.0.61","broadcasturi":"vlan://untagged","traffictype":"Guest","type":"Shared","isdefault":true,"macaddress":"1e:00:13:00:00:16","secondaryip":[],"extradhcpoption":[]}],"hypervisor":"KVM","instancename":"i-2-21-VM","details":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"affinitygroup":[],"displayvm":true,"isdynamicallyscalable":false,"ostypeid":256,"tags":[],"jobid":"56adfd38-67d5-4472-b241-300b8baad7f3","jobstatus":0}
>
> VM Stop
> -------
> 2019-06-03 12:57:23,179 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-8:null) (logid:) Seq 1-7235595751324128197:
> Processing: { Ans: , MgmtId: 75006884826222, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.GetVmDiskStatsAnswer":{"hostName":"node001","vmDiskStatsMap":{"i-2-21-VM":[{"vmName":"i-2-21-VM","path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","ioRead":4423,"ioWrite":140,"bytesWrite":1795072,"bytesRead":144043008}]},"result":true,"details":"","wait":0}}]
> }
> 2019-06-03 12:57:23,375 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-14:null) (logid:) Seq 1-7235595751324128198:
> Processing: { Ans: , MgmtId: 75006884826222, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.GetVmNetworkStatsAnswer":{"hostName":"node001","vmNetworkStatsMap":{"i-2-21-VM":[{"vmName":"i-2-21-VM","macAddress":"1e:00:13:00:00:16","bytesSent":3134,"bytesReceived":7366}]},"result":true,"details":"","wait":0}}]
> }
> 2019-06-03 12:57:23,458 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-72:ctx-ec8a2106 job-190/job-191 ctx-72c73494)
> (logid:92805b41) Seq 1-7235595751324128199: Sending { Cmd , MgmtId:
> 75006884826222, via: 1(node001), Ver: v1, Flags: 100011,
> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"forceStop":false,"volumesToDisconnect":[],"vmName":"i-2-21-VM","executeInSequence":false,"wait":0}}]
> }
> 2019-06-03 12:59:26,120 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-72:ctx-ec8a2106 job-190/job-191 ctx-72c73494)
> (logid:92805b41) VM[User|i-2-21-VM] is stopped on the host. Proceeding to
> release resource held.
> 2019-06-03 12:59:26,261 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-72:ctx-ec8a2106 job-190/job-191 ctx-72c73494)
> (logid:92805b41) Successfully released network resources for the vm
> VM[User|i-2-21-VM]
> 2019-06-03 12:59:26,261 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-72:ctx-ec8a2106 job-190/job-191 ctx-72c73494)
> (logid:92805b41) Successfully released storage resources for the vm
> VM[User|i-2-21-VM]
> 2019-06-03 12:59:27,015 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-52:ctx-546ec715 job-190 ctx-13580c5f) (logid:92805b41)
> Complete async job-190, jobStatus: SUCCEEDED, resultCode: 0, result:
> org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{"id":"4fb4b946-5d2b-4b66-a29a-cf218886615a","name":"vmu001","displayname":"vmu001","account":"admin","userid":"0d19d291-80c4-11e9-88d7-4437e6b9cc6e","username":"admin","domainid":"e20d260a-80c0-11e9-88d7-4437e6b9cc6e","domain":"ROOT","created":"2019-06-03T12:52:21+0200","state":"Stopped","haenable":false,"zoneid":"0d0c2a76-42e7-4ac7-9037-3712448fb0b0","zonename":"Zone1","templateid":"ef1cda00-68e7-4958-b35a-3df95ba8c029","templatename":"ubuntu
> 16.04.6 server","templatedisplaytext":"ubuntu 16.04.6
> server","passwordenabled":false,"serviceofferingid":"c1f3654e-e7c7-4db5-b064-948eb2bbe517","serviceofferingname":"Small
> Instance","cpunumber":1,"cpuspeed":500,"memory":512,"cpuused":"1.99%","networkkbsread":3,"networkkbswrite":1,"diskkbsread":0,"diskkbswrite":864,"memorykbs":524288,"memoryintfreekbs":279324,"memorytargetkbs":524288,"diskioread":0,"diskiowrite":99,"guestosid":"ed88d9c7-80c3-11e9-88d7-4437e6b9cc6e","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[{"id":"0d4ecb8f-80c4-11e9-88d7-4437e6b9cc6e","name":"default","description":"Default
> Security
> Group","account":"admin","ingressrule":[],"egressrule":[],"tags":[],"virtualmachineids":[]}],"nic":[{"id":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","networkid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","networkname":"defaultGuestNetwork","netmask":"255.255.255.0","gateway":"192.168.0.1","ipaddress":"192.168.0.61","broadcasturi":"vlan://untagged","traffictype":"Guest","type":"Shared","isdefault":true,"macaddress":"1e:00:13:00:00:16","secondaryip":[],"extradhcpoption":[]}],"hypervisor":"KVM","instancename":"i-2-21-VM","details":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"affinitygroup":[],"displayvm":true,"isdynamicallyscalable":false,"ostypeid":256,"tags":[],"jobid":"92805b41-9391-4211-8eb4-3b7d06f27030","jobstatus":0}
>
> VM Start
> --------
> 2019-06-03 13:02:35,531 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-73:ctx-625e7d4f job-192/job-193 ctx-8e8437af)
> (logid:b1f99bee) Seq 1-7235595751324128220: Sending { Cmd , MgmtId:
> 75006884826222, via: 1(node001), Ver: v1, Flags: 100011,
> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":21,"name":"i-2-21-VM","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Ubuntu
> 16.04 (64-bit)","platformEmulator":"Ubuntu
> 16.04","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"COaaT8974fN86DWrTYVQrA","params":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"4fb4b946-5d2b-4b66-a29a-cf218886615a","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8
> ","isManaged":false}},"name":"ROOT-21","size":5368709120,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","type":"ROOT","_details":{"storageHost":"192.168.0.34","managed":"false","storagePort":"2049","volumeSize":"5368709120"}},{"data":{"org.
> apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false,"bootable":false,"directDownload":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false"},"uuid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","ip":"192.168.0.61","netmask":"255.255.255.0","gateway":"192.168.0.1","mac":"1e:00:13:00:00:16","dns1":"1.1.1.1","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true}],"guestOsDetails":{}},"hostIp":"192.168.0.28","executeInSequence":false,"wait":0}}]
> }
> 2019-06-03 13:02:36,231 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-6:null) (logid:) Seq 1-7235595751324128220:
> Processing: { Ans: , MgmtId: 75006884826222, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":21,"name":"i-2-21-VM","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Ubuntu
> 16.04 (64-bit)","platformEmulator":"Ubuntu
> 16.04","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"COaaT8974fN86DWrTYVQrA","vncAddr":"192.168.0.28","params":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"4fb4b946-5d2b-4b66-a29a-cf218886615a","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8
> ","isManaged":false}},"name":"ROOT-21","size":5368709120,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","type":"ROOT","_details":{"storageHost":"192.168.0.34","managed":"false","storagePort":"2049","volumeSize":"5368709120"}},{"data":{"org.
> apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false,"bootable":false,"directDownload":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false"},"uuid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","ip":"192.168.0.61","netmask":"255.255.255.0","gateway":"192.168.0.1","mac":"1e:00:13:00:00:16","dns1":"1.1.1.1","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true}],"guestOsDetails":{}},"result":true,"wait":0}}]
> }
> 2019-06-03 13:02:36,648 DEBUG [c.c.n.s.SecurityGroupManagerImpl]
> (SecGrp-Worker-34:ctx-1a1c6a04) (logid:4f0cd9ce) SecurityGroupManager v2:
> sending ruleset update for vm i-2-21-VM:ingress num rules=0:egress num
> rules=0 num cidrs=0 sig=d41d8cd98f00b204e9800998ecf8427e
> 2019-06-03 13:02:36,648 DEBUG [c.c.a.t.Request]
> (SecGrp-Worker-34:ctx-1a1c6a04) (logid:4f0cd9ce) Seq 1-7235595751324128221:
> Sending { Cmd , MgmtId: 75006884826222, via: 1(node001), Ver: v1, Flags:
> 100111,
> [{"com.cloud.agent.api.SecurityGroupRulesCmd":{"guestIp":"192.168.0.61","vmName":"i-2-21-VM","guestMac":"1e:00:13:00:00:16","signature":"d41d8cd98f00b204e9800998ecf8427e","seqNum":2,"vmId":21,"msId":75006884826222,"ingressRuleSet":[],"egressRuleSet":[],"wait":0}}]
> }
> 2019-06-03 13:02:36,778 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-73:ctx-625e7d4f job-192/job-193 ctx-8e8437af)
> (logid:b1f99bee) Start completed for VM VM[User|i-2-21-VM]
> 2019-06-03 13:02:37,012 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-53:ctx-9ad730ea job-192 ctx-b612185b) (logid:b1f99bee)
> Complete async job-192, jobStatus: SUCCEEDED, resultCode: 0, result:
> org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{"id":"4fb4b946-5d2b-4b66-a29a-cf218886615a","name":"vmu001","displayname":"vmu001","account":"admin","userid":"0d19d291-80c4-11e9-88d7-4437e6b9cc6e","username":"admin","domainid":"e20d260a-80c0-11e9-88d7-4437e6b9cc6e","domain":"ROOT","created":"2019-06-03T12:52:21+0200","state":"Running","haenable":false,"zoneid":"0d0c2a76-42e7-4ac7-9037-3712448fb0b0","zonename":"Zone1","hostid":"192c59bd-aa9a-47fa-9e9c-63df100d4449","hostname":"node001","templateid":"ef1cda00-68e7-4958-b35a-3df95ba8c029","templatename":"ubuntu
> 16.04.6 server","templatedisplaytext":"ubuntu 16.04.6
> server","passwordenabled":false,"serviceofferingid":"c1f3654e-e7c7-4db5-b064-948eb2bbe517","serviceofferingname":"Small
> Instance","cpunumber":1,"cpuspeed":500,"memory":512,"cpuused":"1.99%","networkkbsread":3,"networkkbswrite":1,"diskkbsread":0,"diskkbswrite":864,"memorykbs":524288,"memoryintfreekbs":279324,"memorytargetkbs":524288,"diskioread":0,"diskiowrite":99,"guestosid":"ed88d9c7-80c3-11e9-88d7-4437e6b9cc6e","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[{"id":"0d4ecb8f-80c4-11e9-88d7-4437e6b9cc6e","name":"default","description":"Default
> Security
> Group","account":"admin","ingressrule":[],"egressrule":[],"tags":[],"virtualmachineids":[]}],"nic":[{"id":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","networkid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","networkname":"defaultGuestNetwork","netmask":"255.255.255.0","gateway":"192.168.0.1","ipaddress":"192.168.0.61","broadcasturi":"vlan://untagged","traffictype":"Guest","type":"Shared","isdefault":true,"macaddress":"1e:00:13:00:00:16","secondaryip":[],"extradhcpoption":[]}],"hypervisor":"KVM","instancename":"i-2-21-VM","details":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"affinitygroup":[],"displayvm":true,"isdynamicallyscalable":false,"ostypeid":256,"tags":[],"jobid":"b1f99bee-ce28-4a06-8929-8549806d60ba","jobstatus":0}
> root@mgmtserver:~#
>
>
> Thanks in advance for your help
> Regards
> Daniel
>
>
RE: Unable to ping/ssh my VMs after a stop/start
Posted by daniel bellido <da...@outlook.be>.
Hello Ivan,
Thanks a lot for your reply.
Indeed the next step will be to be able to ping from elsewhere than the hypervisor but some additional network configs are required and I didn't look at it yet.
So for the moment I'm just trying to ping/ssh my VMs from the hypervisor node where the bridge cloudbr0 is located.
After the stop/start, I also noticed that I was able to open a console from the UI to the VM. From there a ifconfig returns me 192.168.0.49 as IP on NIC ens3 but I can't ping that IP from the supervisor.
By the way iptables -L on the hypervisor returns this :
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT udp -- anywhere anywhere udp dpt:domain
ACCEPT tcp -- anywhere anywhere tcp dpt:domain
ACCEPT udp -- anywhere anywhere udp dpt:bootps
ACCEPT tcp -- anywhere anywhere tcp dpt:bootps
Chain FORWARD (policy ACCEPT)
target prot opt source destination
BF-cloudbr0 all -- anywhere anywhere PHYSDEV match --physdev-is-bridged
BF-cloudbr0 all -- anywhere anywhere PHYSDEV match --physdev-is-bridged
DROP all -- anywhere anywhere
DROP all -- anywhere anywhere
ACCEPT all -- anywhere 192.168.122.0/24 ctstate RELATED,ESTABLISHED
ACCEPT all -- 192.168.122.0/24 anywhere
ACCEPT all -- anywhere anywhere
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT udp -- anywhere anywhere udp dpt:bootpc
Chain BF-cloudbr0 (2 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
BF-cloudbr0-IN all -- anywhere anywhere PHYSDEV match --physdev-is-in --physdev-is-bridged
BF-cloudbr0-OUT all -- anywhere anywhere PHYSDEV match --physdev-is-out --physdev-is-bridged
ACCEPT all -- anywhere anywhere PHYSDEV match --physdev-out eno1 --physdev-is-bridged
Chain BF-cloudbr0-IN (1 references)
target prot opt source destination
r-12-VM all -- anywhere anywhere PHYSDEV match --physdev-in vnet0 --physdev-is-bridged
s-10-VM all -- anywhere anywhere PHYSDEV match --physdev-in vnet3 --physdev-is-bridged
s-10-VM all -- anywhere anywhere PHYSDEV match --physdev-in vnet4 --physdev-is-bridged
v-9-VM all -- anywhere anywhere PHYSDEV match --physdev-in vnet6 --physdev-is-bridged
v-9-VM all -- anywhere anywhere PHYSDEV match --physdev-in vnet7 --physdev-is-bridged
i-2-21-def all -- anywhere anywhere PHYSDEV match --physdev-in vnet8 --physdev-is-bridged
Chain BF-cloudbr0-OUT (1 references)
target prot opt source destination
r-12-VM all -- anywhere anywhere PHYSDEV match --physdev-out vnet0 --physdev-is-bridged
s-10-VM all -- anywhere anywhere PHYSDEV match --physdev-out vnet3 --physdev-is-bridged
s-10-VM all -- anywhere anywhere PHYSDEV match --physdev-out vnet4 --physdev-is-bridged
v-9-VM all -- anywhere anywhere PHYSDEV match --physdev-out vnet6 --physdev-is-bridged
v-9-VM all -- anywhere anywhere PHYSDEV match --physdev-out vnet7 --physdev-is-bridged
i-2-21-def all -- anywhere anywhere PHYSDEV match --physdev-out vnet8 --physdev-is-bridged
Chain i-2-21-VM (1 references)
target prot opt source destination
DROP all -- anywhere anywhere
Chain i-2-21-VM-eg (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere
Chain i-2-21-def (2 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT udp -- anywhere anywhere PHYSDEV match --physdev-in vnet8 --physdev-is-bridged udp spt:bootpc dpt:bootps
ACCEPT udp -- anywhere anywhere PHYSDEV match --physdev-out vnet8 --physdev-is-bridged udp spt:bootps dpt:bootpc
DROP all -- anywhere anywhere PHYSDEV match --physdev-in vnet8 --physdev-is-bridged ! match-set i-2-21-VM src
RETURN udp -- anywhere anywhere PHYSDEV match --physdev-in vnet8 --physdev-is-bridged match-set i-2-21-VM src udp dpt:domain
RETURN tcp -- anywhere anywhere PHYSDEV match --physdev-in vnet8 --physdev-is-bridged match-set i-2-21-VM src tcp dpt:domain
i-2-21-VM-eg all -- anywhere anywhere PHYSDEV match --physdev-in vnet8 --physdev-is-bridged match-set i-2-21-VM src
i-2-21-VM all -- anywhere anywhere PHYSDEV match --physdev-out vnet8 --physdev-is-bridged
Chain r-12-VM (2 references)
target prot opt source destination
RETURN all -- anywhere anywhere PHYSDEV match --physdev-in vnet0 --physdev-is-bridged
ACCEPT all -- anywhere anywhere
Chain s-10-VM (4 references)
target prot opt source destination
RETURN all -- anywhere anywhere PHYSDEV match --physdev-in vnet3 --physdev-is-bridged
RETURN all -- anywhere anywhere PHYSDEV match --physdev-in vnet4 --physdev-is-bridged
ACCEPT all -- anywhere anywhere
Chain v-9-VM (4 references)
target prot opt source destination
RETURN all -- anywhere anywhere PHYSDEV match --physdev-in vnet6 --physdev-is-bridged
RETURN all -- anywhere anywhere PHYSDEV match --physdev-in vnet7 --physdev-is-bridged
ACCEPT all -- anywhere anywhere
________________________________
De : Ivan Kudryavtsev <ku...@bw-sw.com>
Envoyé : lundi 3 juin 2019 13:21
À : users
Objet : Re: Unable to ping/ssh my VMs after a stop/start
Daniel,
why you think you should be able to ping from the hypervisor? Normally, you
have to add ip in the same net to the bridge to ping tun/tap. I'm not sure,
but if it doesn't work after the step above, check your iptables/ebtables
to check every rule is ok.
Commands:
Iptables-save
Ebtables -t nat
Ipset -L
пн, 3 июн. 2019 г., 18:14 daniel.bellido@outlook.be <
daniel.bellido@outlook.be>:
> Hello,
>
> I'm a new cloudstack user. I've started with a fresh install on 2 ubuntu
> 18.04 boxes : 1 for the management server , 1 for a KVM node.
> I'm able to create ubuntu VMs to which I can connect via ssh from the KVM
> node.
> But when I stop/start the VMs, I can no longer ping or ssh the VMs.
>
> ================================
> The sequence of operations are :
> ================================
>
> 0) Check bridge on KVM node before VM creation
> ----------------------------------------------
> xxx@node001:~$ brctl show cloudbr0
> bridge name bridge id STP enabled interfaces
> cloudbr0 8000.ce1467c23351 no eno1
> vnet0
> vnet3
> vnet4
> vnet6
> vnet7
>
> 1) Create a VM instance from the web UI
> ---------------------------------------
> => IP 192.168.0.61
> => internal name i-2-21-VM
> xxx@node001:~$ brctl show cloudbr0
> bridge name bridge id STP enabled interfaces
> cloudbr0 8000.ce1467c23351 no eno1
> vnet0
> vnet3
> vnet4
> vnet6
> vnet7
> vnet8
> xxx@node001:~$ ifconfig vnet8
> vnet8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> inet6 fe80::fc00:13ff:fe00:16 prefixlen 64 scopeid 0x20<link>
> ether fe:00:13:00:00:16 txqueuelen 1000 (Ethernet)
> RX packets 22 bytes 2540 (2.5 KB)
> RX errors 0 dropped 0 overruns 0 frame 0
> TX packets 37 bytes 5398 (5.3 KB)
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
> => arp entry : 192.168.0.61 ether 1e:00:13:00:00:16 C
> cloudbr0
>
> 2) From the KVM node, ssh or ping 192.168.0.61 works perfectly
> --------------------------------------------------------------
> xxx@node001:~$ ping 192.168.0.61
> PING 192.168.0.61 (192.168.0.61) 56(84) bytes of data.
> 64 bytes from 192.168.0.61: icmp_seq=1 ttl=64 time=0.631 ms
> 64 bytes from 192.168.0.61: icmp_seq=2 ttl=64 time=0.209 ms
> 64 bytes from 192.168.0.61: icmp_seq=3 ttl=64 time=0.226 ms
> ^C
> --- 192.168.0.61 ping statistics ---
> 3 packets transmitted, 3 received, 0% packet loss, time 2056ms
> rtt min/avg/max/mdev = 0.209/0.355/0.631/0.195 ms
>
> 3) Stop the VM instance from the UI
> -----------------------------------
> xxx@node001:~$ brctl show cloudbr0
> bridge name bridge id STP enabled interfaces
> cloudbr0 8000.ce1467c23351 no eno1
> vnet0
> vnet3
> vnet4
> vnet6
> vnet7
> xxx@node001:~$ ping 192.168.0.61
> PING 192.168.0.61 (192.168.0.61) 56(84) bytes of data.
> ^C
> --- 192.168.0.61 ping statistics ---
> 3 packets transmitted, 0 received, 100% packet loss, time 2044ms
>
> => arp entry : 192.168.0.61 (incomplete)
> cloudbr0
>
> 4) Start the VM instance from the UI
> ------------------------------------
> xxx@node001:~$ brctl show cloudbr0
> bridge name bridge id STP enabled interfaces
> cloudbr0 8000.ce1467c23351 no eno1
> vnet0
> vnet3
> vnet4
> vnet6
> vnet7
> vnet8
>
> xxx@node001:~$ ifconfig vnet8
> vnet8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> inet6 fe80::fc00:13ff:fe00:16 prefixlen 64 scopeid 0x20<link>
> ether fe:00:13:00:00:16 txqueuelen 1000 (Ethernet)
> RX packets 89 bytes 4330 (4.3 KB)
> RX errors 0 dropped 0 overruns 0 frame 0
> TX packets 25 bytes 3518 (3.5 KB)
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
> 5) ISSUE : From the KVM node, ssh 192.168.0.61 no longer work
> -------------------------------------------------------------
> xxx@node001:~$ ping 192.168.0.61
> PING 192.168.0.61 (192.168.0.61) 56(84) bytes of data.
> From 192.168.0.28 icmp_seq=1 Destination Host Unreachable
> From 192.168.0.28 icmp_seq=2 Destination Host Unreachable
> From 192.168.0.28 icmp_seq=3 Destination Host Unreachable
> ^C
>
> => arp entry : 192.168.0.61 (incomplete)
> cloudbr0
>
> ======================
> Management server LOGS
> ======================
>
> VM Creation
> -----------
> 2019-06-03 12:52:21,201 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocating
> entries for VM: VM[User|i-2-21-VM]
> 2019-06-03 12:52:21,201 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocating
> nics for VM[User|i-2-21-VM]
> 2019-06-03 12:52:21,201 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocating
> nic for vm VM[User|i-2-21-VM] in network Ntwk[204|Guest|6] with requested
> profile NicProfile[0-0-null-null-null
> 2019-06-03 12:52:21,206 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocating
> disks for VM[User|i-2-21-VM]
> 2019-06-03 12:52:21,215 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocation
> completed for VM: VM[User|i-2-21-VM]
> 2019-06-03 12:52:21,215 DEBUG [c.c.v.UserVmManagerImpl]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Successfully
> allocated DB entry for VM[User|i-2-21-VM]
> 2019-06-03 12:52:23,090 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-71:ctx-57028b14 job-188/job-189 ctx-8ad63f37)
> (logid:56adfd38) Successfully transitioned to start state for
> VM[User|i-2-21-VM] reservation id = 24b9ed17-834d-4e0a-a6c7-b6521f7813b2
> 2019-06-03 12:52:23,178 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-71:ctx-57028b14 job-188/job-189 ctx-8ad63f37)
> (logid:56adfd38) Deployment found - P0=VM[User|i-2-21-VM],
> P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(21|ROOT-->Pool(2))]
> 8174: Sending { Cmd , MgmtId: 75006884826222, via: 1(node001), Ver: v1,
> Flags: 100111,
> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.
> cloudstack.storage.to.TemplateObjectTO":{"path":"ef1cda00-68e7-4958-b35a-3df95ba8c029","uuid":"ef1cda00-68e7-4958-b35a-3df95ba8c029","id":207,"format":"QCOW2","accountId":2,"hvm":true,"displayText":"ubuntu
> 16.04.6
> server","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8
> ","isManaged":false}},"name":"b3527b5d-1d1e-4b9d-bc03-308f023b655e","size":5368709120,"hypervisorType":"KVM","bootable":false,"uniqueName":"b3527b5d-1d1e-4b9d-bc03-308f023b655e","directDownload":false}},"destTO":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.
> apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8","isManaged":false}},"name":"ROOT-21","size":5368709120,"volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"hypervisorType":"KVM"}},"executeInSequence":true,"options":{},"options2":{},"wait":0}}]
> }
> 2019-06-03 12:52:26,648 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-71:ctx-57028b14 job-188/job-189 ctx-8ad63f37)
> (logid:56adfd38) Seq 1-7235595751324128175: Sending { Cmd , MgmtId:
> 75006884826222, via: 1(node001), Ver: v1, Flags: 100011,
> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":21,"name":"i-2-21-VM","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Ubuntu
> 16.04 (64-bit)","platformEmulator":"Ubuntu
> 16.04","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"COaaT8974fN86DWrTYVQrA","params":{"keyboard":"us","deployvm":"true","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"4fb4b946-5d2b-4b66-a29a-cf218886615a","disks":[{"data":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.
> apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8
> ","isManaged":false}},"name":"ROOT-21","size":5368709120,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","type":"ROOT","_details":{"storageHost":"192.168.0.34","managed":"false","storagePort":"2049","volumeSize":"5368709120"}},{"data":{"org.
> apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false,"bootable":false,"directDownload":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false"},"uuid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","ip":"192.168.0.61","netmask":"255.255.255.0","gateway":"192.168.0.1","mac":"1e:00:13:00:00:16","dns1":"1.1.1.1","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true}],"guestOsDetails":{}},"hostIp":"192.168.0.28","executeInSequence":false,"wait":0}}]
> }
> 2019-06-03 12:52:27,507 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-8:null) (logid:) Seq 1-7235595751324128175:
> Processing: { Ans: , MgmtId: 75006884826222, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":21,"name":"i-2-21-VM","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Ubuntu
> 16.04 (64-bit)","platformEmulator":"Ubuntu
> 16.04","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"COaaT8974fN86DWrTYVQrA","vncAddr":"192.168.0.28","params":{"keyboard":"us","deployvm":"true","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"4fb4b946-5d2b-4b66-a29a-cf218886615a","disks":[{"data":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.
> apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8
> ","isManaged":false}},"name":"ROOT-21","size":5368709120,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","type":"ROOT","_details":{"storageHost":"192.168.0.34","managed":"false","storagePort":"2049","volumeSize":"5368709120"}},{"data":{"org.
> apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false,"bootable":false,"directDownload":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false"},"uuid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","ip":"192.168.0.61","netmask":"255.255.255.0","gateway":"192.168.0.1","mac":"1e:00:13:00:00:16","dns1":"1.1.1.1","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true}],"guestOsDetails":{}},"result":true,"wait":0}}]
> }
> 2019-06-03 12:52:27,885 DEBUG [c.c.n.s.SecurityGroupManagerImpl]
> (SecGrp-Worker-25:ctx-1fab9c8b) (logid:89e0d0db) SecurityGroupManager v2:
> sending ruleset update for vm i-2-21-VM:ingress num rules=0:egress num
> rules=0 num cidrs=0 sig=d41d8cd98f00b204e9800998ecf8427e
> 2019-06-03 12:52:27,886 DEBUG [c.c.a.t.Request]
> (SecGrp-Worker-25:ctx-1fab9c8b) (logid:89e0d0db) Seq 1-7235595751324128176:
> Sending { Cmd , MgmtId: 75006884826222, via: 1(node001), Ver: v1, Flags:
> 100111,
> [{"com.cloud.agent.api.SecurityGroupRulesCmd":{"guestIp":"192.168.0.61","vmName":"i-2-21-VM","guestMac":"1e:00:13:00:00:16","signature":"d41d8cd98f00b204e9800998ecf8427e","seqNum":1,"vmId":21,"msId":75006884826222,"ingressRuleSet":[],"egressRuleSet":[],"wait":0}}]
> }
> 2019-06-03 12:52:28,081 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-71:ctx-57028b14 job-188/job-189 ctx-8ad63f37)
> (logid:56adfd38) Start completed for VM VM[User|i-2-21-VM]
> 2019-06-03 12:52:28,311 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-51:ctx-505e4d01 job-188 ctx-e3665401) (logid:56adfd38)
> Complete async job-188, jobStatus: SUCCEEDED, resultCode: 0, result:
> org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{"id":"4fb4b946-5d2b-4b66-a29a-cf218886615a","name":"vmu001","displayname":"vmu001","account":"admin","userid":"0d19d291-80c4-11e9-88d7-4437e6b9cc6e","username":"admin","domainid":"e20d260a-80c0-11e9-88d7-4437e6b9cc6e","domain":"ROOT","created":"2019-06-03T12:52:21+0200","state":"Running","haenable":false,"zoneid":"0d0c2a76-42e7-4ac7-9037-3712448fb0b0","zonename":"Zone1","hostid":"192c59bd-aa9a-47fa-9e9c-63df100d4449","hostname":"node001","templateid":"ef1cda00-68e7-4958-b35a-3df95ba8c029","templatename":"ubuntu
> 16.04.6 server","templatedisplaytext":"ubuntu 16.04.6
> server","passwordenabled":false,"serviceofferingid":"c1f3654e-e7c7-4db5-b064-948eb2bbe517","serviceofferingname":"Small
> Instance","cpunumber":1,"cpuspeed":500,"memory":512,"guestosid":"ed88d9c7-80c3-11e9-88d7-4437e6b9cc6e","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[{"id":"0d4ecb8f-80c4-11e9-88d7-4437e6b9cc6e","name":"default","description":"Default
> Security
> Group","account":"admin","ingressrule":[],"egressrule":[],"tags":[],"virtualmachineids":[]}],"nic":[{"id":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","networkid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","networkname":"defaultGuestNetwork","netmask":"255.255.255.0","gateway":"192.168.0.1","ipaddress":"192.168.0.61","broadcasturi":"vlan://untagged","traffictype":"Guest","type":"Shared","isdefault":true,"macaddress":"1e:00:13:00:00:16","secondaryip":[],"extradhcpoption":[]}],"hypervisor":"KVM","instancename":"i-2-21-VM","details":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"affinitygroup":[],"displayvm":true,"isdynamicallyscalable":false,"ostypeid":256,"tags":[],"jobid":"56adfd38-67d5-4472-b241-300b8baad7f3","jobstatus":0}
>
> VM Stop
> -------
> 2019-06-03 12:57:23,179 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-8:null) (logid:) Seq 1-7235595751324128197:
> Processing: { Ans: , MgmtId: 75006884826222, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.GetVmDiskStatsAnswer":{"hostName":"node001","vmDiskStatsMap":{"i-2-21-VM":[{"vmName":"i-2-21-VM","path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","ioRead":4423,"ioWrite":140,"bytesWrite":1795072,"bytesRead":144043008}]},"result":true,"details":"","wait":0}}]
> }
> 2019-06-03 12:57:23,375 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-14:null) (logid:) Seq 1-7235595751324128198:
> Processing: { Ans: , MgmtId: 75006884826222, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.GetVmNetworkStatsAnswer":{"hostName":"node001","vmNetworkStatsMap":{"i-2-21-VM":[{"vmName":"i-2-21-VM","macAddress":"1e:00:13:00:00:16","bytesSent":3134,"bytesReceived":7366}]},"result":true,"details":"","wait":0}}]
> }
> 2019-06-03 12:57:23,458 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-72:ctx-ec8a2106 job-190/job-191 ctx-72c73494)
> (logid:92805b41) Seq 1-7235595751324128199: Sending { Cmd , MgmtId:
> 75006884826222, via: 1(node001), Ver: v1, Flags: 100011,
> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"forceStop":false,"volumesToDisconnect":[],"vmName":"i-2-21-VM","executeInSequence":false,"wait":0}}]
> }
> 2019-06-03 12:59:26,120 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-72:ctx-ec8a2106 job-190/job-191 ctx-72c73494)
> (logid:92805b41) VM[User|i-2-21-VM] is stopped on the host. Proceeding to
> release resource held.
> 2019-06-03 12:59:26,261 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-72:ctx-ec8a2106 job-190/job-191 ctx-72c73494)
> (logid:92805b41) Successfully released network resources for the vm
> VM[User|i-2-21-VM]
> 2019-06-03 12:59:26,261 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-72:ctx-ec8a2106 job-190/job-191 ctx-72c73494)
> (logid:92805b41) Successfully released storage resources for the vm
> VM[User|i-2-21-VM]
> 2019-06-03 12:59:27,015 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-52:ctx-546ec715 job-190 ctx-13580c5f) (logid:92805b41)
> Complete async job-190, jobStatus: SUCCEEDED, resultCode: 0, result:
> org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{"id":"4fb4b946-5d2b-4b66-a29a-cf218886615a","name":"vmu001","displayname":"vmu001","account":"admin","userid":"0d19d291-80c4-11e9-88d7-4437e6b9cc6e","username":"admin","domainid":"e20d260a-80c0-11e9-88d7-4437e6b9cc6e","domain":"ROOT","created":"2019-06-03T12:52:21+0200","state":"Stopped","haenable":false,"zoneid":"0d0c2a76-42e7-4ac7-9037-3712448fb0b0","zonename":"Zone1","templateid":"ef1cda00-68e7-4958-b35a-3df95ba8c029","templatename":"ubuntu
> 16.04.6 server","templatedisplaytext":"ubuntu 16.04.6
> server","passwordenabled":false,"serviceofferingid":"c1f3654e-e7c7-4db5-b064-948eb2bbe517","serviceofferingname":"Small
> Instance","cpunumber":1,"cpuspeed":500,"memory":512,"cpuused":"1.99%","networkkbsread":3,"networkkbswrite":1,"diskkbsread":0,"diskkbswrite":864,"memorykbs":524288,"memoryintfreekbs":279324,"memorytargetkbs":524288,"diskioread":0,"diskiowrite":99,"guestosid":"ed88d9c7-80c3-11e9-88d7-4437e6b9cc6e","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[{"id":"0d4ecb8f-80c4-11e9-88d7-4437e6b9cc6e","name":"default","description":"Default
> Security
> Group","account":"admin","ingressrule":[],"egressrule":[],"tags":[],"virtualmachineids":[]}],"nic":[{"id":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","networkid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","networkname":"defaultGuestNetwork","netmask":"255.255.255.0","gateway":"192.168.0.1","ipaddress":"192.168.0.61","broadcasturi":"vlan://untagged","traffictype":"Guest","type":"Shared","isdefault":true,"macaddress":"1e:00:13:00:00:16","secondaryip":[],"extradhcpoption":[]}],"hypervisor":"KVM","instancename":"i-2-21-VM","details":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"affinitygroup":[],"displayvm":true,"isdynamicallyscalable":false,"ostypeid":256,"tags":[],"jobid":"92805b41-9391-4211-8eb4-3b7d06f27030","jobstatus":0}
>
> VM Start
> --------
> 2019-06-03 13:02:35,531 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-73:ctx-625e7d4f job-192/job-193 ctx-8e8437af)
> (logid:b1f99bee) Seq 1-7235595751324128220: Sending { Cmd , MgmtId:
> 75006884826222, via: 1(node001), Ver: v1, Flags: 100011,
> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":21,"name":"i-2-21-VM","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Ubuntu
> 16.04 (64-bit)","platformEmulator":"Ubuntu
> 16.04","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"COaaT8974fN86DWrTYVQrA","params":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"4fb4b946-5d2b-4b66-a29a-cf218886615a","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8
> ","isManaged":false}},"name":"ROOT-21","size":5368709120,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","type":"ROOT","_details":{"storageHost":"192.168.0.34","managed":"false","storagePort":"2049","volumeSize":"5368709120"}},{"data":{"org.
> apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false,"bootable":false,"directDownload":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false"},"uuid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","ip":"192.168.0.61","netmask":"255.255.255.0","gateway":"192.168.0.1","mac":"1e:00:13:00:00:16","dns1":"1.1.1.1","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true}],"guestOsDetails":{}},"hostIp":"192.168.0.28","executeInSequence":false,"wait":0}}]
> }
> 2019-06-03 13:02:36,231 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-6:null) (logid:) Seq 1-7235595751324128220:
> Processing: { Ans: , MgmtId: 75006884826222, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":21,"name":"i-2-21-VM","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Ubuntu
> 16.04 (64-bit)","platformEmulator":"Ubuntu
> 16.04","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"COaaT8974fN86DWrTYVQrA","vncAddr":"192.168.0.28","params":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"4fb4b946-5d2b-4b66-a29a-cf218886615a","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8
> ","isManaged":false}},"name":"ROOT-21","size":5368709120,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","type":"ROOT","_details":{"storageHost":"192.168.0.34","managed":"false","storagePort":"2049","volumeSize":"5368709120"}},{"data":{"org.
> apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false,"bootable":false,"directDownload":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false"},"uuid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","ip":"192.168.0.61","netmask":"255.255.255.0","gateway":"192.168.0.1","mac":"1e:00:13:00:00:16","dns1":"1.1.1.1","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true}],"guestOsDetails":{}},"result":true,"wait":0}}]
> }
> 2019-06-03 13:02:36,648 DEBUG [c.c.n.s.SecurityGroupManagerImpl]
> (SecGrp-Worker-34:ctx-1a1c6a04) (logid:4f0cd9ce) SecurityGroupManager v2:
> sending ruleset update for vm i-2-21-VM:ingress num rules=0:egress num
> rules=0 num cidrs=0 sig=d41d8cd98f00b204e9800998ecf8427e
> 2019-06-03 13:02:36,648 DEBUG [c.c.a.t.Request]
> (SecGrp-Worker-34:ctx-1a1c6a04) (logid:4f0cd9ce) Seq 1-7235595751324128221:
> Sending { Cmd , MgmtId: 75006884826222, via: 1(node001), Ver: v1, Flags:
> 100111,
> [{"com.cloud.agent.api.SecurityGroupRulesCmd":{"guestIp":"192.168.0.61","vmName":"i-2-21-VM","guestMac":"1e:00:13:00:00:16","signature":"d41d8cd98f00b204e9800998ecf8427e","seqNum":2,"vmId":21,"msId":75006884826222,"ingressRuleSet":[],"egressRuleSet":[],"wait":0}}]
> }
> 2019-06-03 13:02:36,778 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-73:ctx-625e7d4f job-192/job-193 ctx-8e8437af)
> (logid:b1f99bee) Start completed for VM VM[User|i-2-21-VM]
> 2019-06-03 13:02:37,012 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-53:ctx-9ad730ea job-192 ctx-b612185b) (logid:b1f99bee)
> Complete async job-192, jobStatus: SUCCEEDED, resultCode: 0, result:
> org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{"id":"4fb4b946-5d2b-4b66-a29a-cf218886615a","name":"vmu001","displayname":"vmu001","account":"admin","userid":"0d19d291-80c4-11e9-88d7-4437e6b9cc6e","username":"admin","domainid":"e20d260a-80c0-11e9-88d7-4437e6b9cc6e","domain":"ROOT","created":"2019-06-03T12:52:21+0200","state":"Running","haenable":false,"zoneid":"0d0c2a76-42e7-4ac7-9037-3712448fb0b0","zonename":"Zone1","hostid":"192c59bd-aa9a-47fa-9e9c-63df100d4449","hostname":"node001","templateid":"ef1cda00-68e7-4958-b35a-3df95ba8c029","templatename":"ubuntu
> 16.04.6 server","templatedisplaytext":"ubuntu 16.04.6
> server","passwordenabled":false,"serviceofferingid":"c1f3654e-e7c7-4db5-b064-948eb2bbe517","serviceofferingname":"Small
> Instance","cpunumber":1,"cpuspeed":500,"memory":512,"cpuused":"1.99%","networkkbsread":3,"networkkbswrite":1,"diskkbsread":0,"diskkbswrite":864,"memorykbs":524288,"memoryintfreekbs":279324,"memorytargetkbs":524288,"diskioread":0,"diskiowrite":99,"guestosid":"ed88d9c7-80c3-11e9-88d7-4437e6b9cc6e","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[{"id":"0d4ecb8f-80c4-11e9-88d7-4437e6b9cc6e","name":"default","description":"Default
> Security
> Group","account":"admin","ingressrule":[],"egressrule":[],"tags":[],"virtualmachineids":[]}],"nic":[{"id":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","networkid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","networkname":"defaultGuestNetwork","netmask":"255.255.255.0","gateway":"192.168.0.1","ipaddress":"192.168.0.61","broadcasturi":"vlan://untagged","traffictype":"Guest","type":"Shared","isdefault":true,"macaddress":"1e:00:13:00:00:16","secondaryip":[],"extradhcpoption":[]}],"hypervisor":"KVM","instancename":"i-2-21-VM","details":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"affinitygroup":[],"displayvm":true,"isdynamicallyscalable":false,"ostypeid":256,"tags":[],"jobid":"b1f99bee-ce28-4a06-8929-8549806d60ba","jobstatus":0}
> root@mgmtserver:~#
>
>
> Thanks in advance for your help
> Regards
> Daniel
>
>
Re: Unable to ping/ssh my VMs after a stop/start
Posted by Ivan Kudryavtsev <ku...@bw-sw.com>.
Daniel,
why you think you should be able to ping from the hypervisor? Normally, you
have to add ip in the same net to the bridge to ping tun/tap. I'm not sure,
but if it doesn't work after the step above, check your iptables/ebtables
to check every rule is ok.
Commands:
Iptables-save
Ebtables -t nat
Ipset -L
пн, 3 июн. 2019 г., 18:14 daniel.bellido@outlook.be <
daniel.bellido@outlook.be>:
> Hello,
>
> I'm a new cloudstack user. I've started with a fresh install on 2 ubuntu
> 18.04 boxes : 1 for the management server , 1 for a KVM node.
> I'm able to create ubuntu VMs to which I can connect via ssh from the KVM
> node.
> But when I stop/start the VMs, I can no longer ping or ssh the VMs.
>
> ================================
> The sequence of operations are :
> ================================
>
> 0) Check bridge on KVM node before VM creation
> ----------------------------------------------
> xxx@node001:~$ brctl show cloudbr0
> bridge name bridge id STP enabled interfaces
> cloudbr0 8000.ce1467c23351 no eno1
> vnet0
> vnet3
> vnet4
> vnet6
> vnet7
>
> 1) Create a VM instance from the web UI
> ---------------------------------------
> => IP 192.168.0.61
> => internal name i-2-21-VM
> xxx@node001:~$ brctl show cloudbr0
> bridge name bridge id STP enabled interfaces
> cloudbr0 8000.ce1467c23351 no eno1
> vnet0
> vnet3
> vnet4
> vnet6
> vnet7
> vnet8
> xxx@node001:~$ ifconfig vnet8
> vnet8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> inet6 fe80::fc00:13ff:fe00:16 prefixlen 64 scopeid 0x20<link>
> ether fe:00:13:00:00:16 txqueuelen 1000 (Ethernet)
> RX packets 22 bytes 2540 (2.5 KB)
> RX errors 0 dropped 0 overruns 0 frame 0
> TX packets 37 bytes 5398 (5.3 KB)
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
> => arp entry : 192.168.0.61 ether 1e:00:13:00:00:16 C
> cloudbr0
>
> 2) From the KVM node, ssh or ping 192.168.0.61 works perfectly
> --------------------------------------------------------------
> xxx@node001:~$ ping 192.168.0.61
> PING 192.168.0.61 (192.168.0.61) 56(84) bytes of data.
> 64 bytes from 192.168.0.61: icmp_seq=1 ttl=64 time=0.631 ms
> 64 bytes from 192.168.0.61: icmp_seq=2 ttl=64 time=0.209 ms
> 64 bytes from 192.168.0.61: icmp_seq=3 ttl=64 time=0.226 ms
> ^C
> --- 192.168.0.61 ping statistics ---
> 3 packets transmitted, 3 received, 0% packet loss, time 2056ms
> rtt min/avg/max/mdev = 0.209/0.355/0.631/0.195 ms
>
> 3) Stop the VM instance from the UI
> -----------------------------------
> xxx@node001:~$ brctl show cloudbr0
> bridge name bridge id STP enabled interfaces
> cloudbr0 8000.ce1467c23351 no eno1
> vnet0
> vnet3
> vnet4
> vnet6
> vnet7
> xxx@node001:~$ ping 192.168.0.61
> PING 192.168.0.61 (192.168.0.61) 56(84) bytes of data.
> ^C
> --- 192.168.0.61 ping statistics ---
> 3 packets transmitted, 0 received, 100% packet loss, time 2044ms
>
> => arp entry : 192.168.0.61 (incomplete)
> cloudbr0
>
> 4) Start the VM instance from the UI
> ------------------------------------
> xxx@node001:~$ brctl show cloudbr0
> bridge name bridge id STP enabled interfaces
> cloudbr0 8000.ce1467c23351 no eno1
> vnet0
> vnet3
> vnet4
> vnet6
> vnet7
> vnet8
>
> xxx@node001:~$ ifconfig vnet8
> vnet8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> inet6 fe80::fc00:13ff:fe00:16 prefixlen 64 scopeid 0x20<link>
> ether fe:00:13:00:00:16 txqueuelen 1000 (Ethernet)
> RX packets 89 bytes 4330 (4.3 KB)
> RX errors 0 dropped 0 overruns 0 frame 0
> TX packets 25 bytes 3518 (3.5 KB)
> TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
>
> 5) ISSUE : From the KVM node, ssh 192.168.0.61 no longer work
> -------------------------------------------------------------
> xxx@node001:~$ ping 192.168.0.61
> PING 192.168.0.61 (192.168.0.61) 56(84) bytes of data.
> From 192.168.0.28 icmp_seq=1 Destination Host Unreachable
> From 192.168.0.28 icmp_seq=2 Destination Host Unreachable
> From 192.168.0.28 icmp_seq=3 Destination Host Unreachable
> ^C
>
> => arp entry : 192.168.0.61 (incomplete)
> cloudbr0
>
> ======================
> Management server LOGS
> ======================
>
> VM Creation
> -----------
> 2019-06-03 12:52:21,201 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocating
> entries for VM: VM[User|i-2-21-VM]
> 2019-06-03 12:52:21,201 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocating
> nics for VM[User|i-2-21-VM]
> 2019-06-03 12:52:21,201 DEBUG [o.a.c.e.o.NetworkOrchestrator]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocating
> nic for vm VM[User|i-2-21-VM] in network Ntwk[204|Guest|6] with requested
> profile NicProfile[0-0-null-null-null
> 2019-06-03 12:52:21,206 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocating
> disks for VM[User|i-2-21-VM]
> 2019-06-03 12:52:21,215 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Allocation
> completed for VM: VM[User|i-2-21-VM]
> 2019-06-03 12:52:21,215 DEBUG [c.c.v.UserVmManagerImpl]
> (qtp895947612-365:ctx-778601b0 ctx-e52d5549) (logid:22cbc102) Successfully
> allocated DB entry for VM[User|i-2-21-VM]
> 2019-06-03 12:52:23,090 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-71:ctx-57028b14 job-188/job-189 ctx-8ad63f37)
> (logid:56adfd38) Successfully transitioned to start state for
> VM[User|i-2-21-VM] reservation id = 24b9ed17-834d-4e0a-a6c7-b6521f7813b2
> 2019-06-03 12:52:23,178 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-71:ctx-57028b14 job-188/job-189 ctx-8ad63f37)
> (logid:56adfd38) Deployment found - P0=VM[User|i-2-21-VM],
> P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
> : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(21|ROOT-->Pool(2))]
> 8174: Sending { Cmd , MgmtId: 75006884826222, via: 1(node001), Ver: v1,
> Flags: 100111,
> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.
> cloudstack.storage.to.TemplateObjectTO":{"path":"ef1cda00-68e7-4958-b35a-3df95ba8c029","uuid":"ef1cda00-68e7-4958-b35a-3df95ba8c029","id":207,"format":"QCOW2","accountId":2,"hvm":true,"displayText":"ubuntu
> 16.04.6
> server","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8
> ","isManaged":false}},"name":"b3527b5d-1d1e-4b9d-bc03-308f023b655e","size":5368709120,"hypervisorType":"KVM","bootable":false,"uniqueName":"b3527b5d-1d1e-4b9d-bc03-308f023b655e","directDownload":false}},"destTO":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.
> apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8","isManaged":false}},"name":"ROOT-21","size":5368709120,"volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"hypervisorType":"KVM"}},"executeInSequence":true,"options":{},"options2":{},"wait":0}}]
> }
> 2019-06-03 12:52:26,648 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-71:ctx-57028b14 job-188/job-189 ctx-8ad63f37)
> (logid:56adfd38) Seq 1-7235595751324128175: Sending { Cmd , MgmtId:
> 75006884826222, via: 1(node001), Ver: v1, Flags: 100011,
> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":21,"name":"i-2-21-VM","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Ubuntu
> 16.04 (64-bit)","platformEmulator":"Ubuntu
> 16.04","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"COaaT8974fN86DWrTYVQrA","params":{"keyboard":"us","deployvm":"true","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"4fb4b946-5d2b-4b66-a29a-cf218886615a","disks":[{"data":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.
> apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8
> ","isManaged":false}},"name":"ROOT-21","size":5368709120,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","type":"ROOT","_details":{"storageHost":"192.168.0.34","managed":"false","storagePort":"2049","volumeSize":"5368709120"}},{"data":{"org.
> apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false,"bootable":false,"directDownload":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false"},"uuid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","ip":"192.168.0.61","netmask":"255.255.255.0","gateway":"192.168.0.1","mac":"1e:00:13:00:00:16","dns1":"1.1.1.1","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true}],"guestOsDetails":{}},"hostIp":"192.168.0.28","executeInSequence":false,"wait":0}}]
> }
> 2019-06-03 12:52:27,507 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-8:null) (logid:) Seq 1-7235595751324128175:
> Processing: { Ans: , MgmtId: 75006884826222, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":21,"name":"i-2-21-VM","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Ubuntu
> 16.04 (64-bit)","platformEmulator":"Ubuntu
> 16.04","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"COaaT8974fN86DWrTYVQrA","vncAddr":"192.168.0.28","params":{"keyboard":"us","deployvm":"true","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"4fb4b946-5d2b-4b66-a29a-cf218886615a","disks":[{"data":{"org.apache.
> cloudstack.storage.to
> .VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.
> apache.cloudstack.storage.to
> .PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8
> ","isManaged":false}},"name":"ROOT-21","size":5368709120,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","type":"ROOT","_details":{"storageHost":"192.168.0.34","managed":"false","storagePort":"2049","volumeSize":"5368709120"}},{"data":{"org.
> apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false,"bootable":false,"directDownload":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false"},"uuid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","ip":"192.168.0.61","netmask":"255.255.255.0","gateway":"192.168.0.1","mac":"1e:00:13:00:00:16","dns1":"1.1.1.1","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true}],"guestOsDetails":{}},"result":true,"wait":0}}]
> }
> 2019-06-03 12:52:27,885 DEBUG [c.c.n.s.SecurityGroupManagerImpl]
> (SecGrp-Worker-25:ctx-1fab9c8b) (logid:89e0d0db) SecurityGroupManager v2:
> sending ruleset update for vm i-2-21-VM:ingress num rules=0:egress num
> rules=0 num cidrs=0 sig=d41d8cd98f00b204e9800998ecf8427e
> 2019-06-03 12:52:27,886 DEBUG [c.c.a.t.Request]
> (SecGrp-Worker-25:ctx-1fab9c8b) (logid:89e0d0db) Seq 1-7235595751324128176:
> Sending { Cmd , MgmtId: 75006884826222, via: 1(node001), Ver: v1, Flags:
> 100111,
> [{"com.cloud.agent.api.SecurityGroupRulesCmd":{"guestIp":"192.168.0.61","vmName":"i-2-21-VM","guestMac":"1e:00:13:00:00:16","signature":"d41d8cd98f00b204e9800998ecf8427e","seqNum":1,"vmId":21,"msId":75006884826222,"ingressRuleSet":[],"egressRuleSet":[],"wait":0}}]
> }
> 2019-06-03 12:52:28,081 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-71:ctx-57028b14 job-188/job-189 ctx-8ad63f37)
> (logid:56adfd38) Start completed for VM VM[User|i-2-21-VM]
> 2019-06-03 12:52:28,311 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-51:ctx-505e4d01 job-188 ctx-e3665401) (logid:56adfd38)
> Complete async job-188, jobStatus: SUCCEEDED, resultCode: 0, result:
> org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{"id":"4fb4b946-5d2b-4b66-a29a-cf218886615a","name":"vmu001","displayname":"vmu001","account":"admin","userid":"0d19d291-80c4-11e9-88d7-4437e6b9cc6e","username":"admin","domainid":"e20d260a-80c0-11e9-88d7-4437e6b9cc6e","domain":"ROOT","created":"2019-06-03T12:52:21+0200","state":"Running","haenable":false,"zoneid":"0d0c2a76-42e7-4ac7-9037-3712448fb0b0","zonename":"Zone1","hostid":"192c59bd-aa9a-47fa-9e9c-63df100d4449","hostname":"node001","templateid":"ef1cda00-68e7-4958-b35a-3df95ba8c029","templatename":"ubuntu
> 16.04.6 server","templatedisplaytext":"ubuntu 16.04.6
> server","passwordenabled":false,"serviceofferingid":"c1f3654e-e7c7-4db5-b064-948eb2bbe517","serviceofferingname":"Small
> Instance","cpunumber":1,"cpuspeed":500,"memory":512,"guestosid":"ed88d9c7-80c3-11e9-88d7-4437e6b9cc6e","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[{"id":"0d4ecb8f-80c4-11e9-88d7-4437e6b9cc6e","name":"default","description":"Default
> Security
> Group","account":"admin","ingressrule":[],"egressrule":[],"tags":[],"virtualmachineids":[]}],"nic":[{"id":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","networkid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","networkname":"defaultGuestNetwork","netmask":"255.255.255.0","gateway":"192.168.0.1","ipaddress":"192.168.0.61","broadcasturi":"vlan://untagged","traffictype":"Guest","type":"Shared","isdefault":true,"macaddress":"1e:00:13:00:00:16","secondaryip":[],"extradhcpoption":[]}],"hypervisor":"KVM","instancename":"i-2-21-VM","details":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"affinitygroup":[],"displayvm":true,"isdynamicallyscalable":false,"ostypeid":256,"tags":[],"jobid":"56adfd38-67d5-4472-b241-300b8baad7f3","jobstatus":0}
>
> VM Stop
> -------
> 2019-06-03 12:57:23,179 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-8:null) (logid:) Seq 1-7235595751324128197:
> Processing: { Ans: , MgmtId: 75006884826222, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.GetVmDiskStatsAnswer":{"hostName":"node001","vmDiskStatsMap":{"i-2-21-VM":[{"vmName":"i-2-21-VM","path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","ioRead":4423,"ioWrite":140,"bytesWrite":1795072,"bytesRead":144043008}]},"result":true,"details":"","wait":0}}]
> }
> 2019-06-03 12:57:23,375 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-14:null) (logid:) Seq 1-7235595751324128198:
> Processing: { Ans: , MgmtId: 75006884826222, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.GetVmNetworkStatsAnswer":{"hostName":"node001","vmNetworkStatsMap":{"i-2-21-VM":[{"vmName":"i-2-21-VM","macAddress":"1e:00:13:00:00:16","bytesSent":3134,"bytesReceived":7366}]},"result":true,"details":"","wait":0}}]
> }
> 2019-06-03 12:57:23,458 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-72:ctx-ec8a2106 job-190/job-191 ctx-72c73494)
> (logid:92805b41) Seq 1-7235595751324128199: Sending { Cmd , MgmtId:
> 75006884826222, via: 1(node001), Ver: v1, Flags: 100011,
> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"checkBeforeCleanup":false,"forceStop":false,"volumesToDisconnect":[],"vmName":"i-2-21-VM","executeInSequence":false,"wait":0}}]
> }
> 2019-06-03 12:59:26,120 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-72:ctx-ec8a2106 job-190/job-191 ctx-72c73494)
> (logid:92805b41) VM[User|i-2-21-VM] is stopped on the host. Proceeding to
> release resource held.
> 2019-06-03 12:59:26,261 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-72:ctx-ec8a2106 job-190/job-191 ctx-72c73494)
> (logid:92805b41) Successfully released network resources for the vm
> VM[User|i-2-21-VM]
> 2019-06-03 12:59:26,261 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-72:ctx-ec8a2106 job-190/job-191 ctx-72c73494)
> (logid:92805b41) Successfully released storage resources for the vm
> VM[User|i-2-21-VM]
> 2019-06-03 12:59:27,015 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-52:ctx-546ec715 job-190 ctx-13580c5f) (logid:92805b41)
> Complete async job-190, jobStatus: SUCCEEDED, resultCode: 0, result:
> org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{"id":"4fb4b946-5d2b-4b66-a29a-cf218886615a","name":"vmu001","displayname":"vmu001","account":"admin","userid":"0d19d291-80c4-11e9-88d7-4437e6b9cc6e","username":"admin","domainid":"e20d260a-80c0-11e9-88d7-4437e6b9cc6e","domain":"ROOT","created":"2019-06-03T12:52:21+0200","state":"Stopped","haenable":false,"zoneid":"0d0c2a76-42e7-4ac7-9037-3712448fb0b0","zonename":"Zone1","templateid":"ef1cda00-68e7-4958-b35a-3df95ba8c029","templatename":"ubuntu
> 16.04.6 server","templatedisplaytext":"ubuntu 16.04.6
> server","passwordenabled":false,"serviceofferingid":"c1f3654e-e7c7-4db5-b064-948eb2bbe517","serviceofferingname":"Small
> Instance","cpunumber":1,"cpuspeed":500,"memory":512,"cpuused":"1.99%","networkkbsread":3,"networkkbswrite":1,"diskkbsread":0,"diskkbswrite":864,"memorykbs":524288,"memoryintfreekbs":279324,"memorytargetkbs":524288,"diskioread":0,"diskiowrite":99,"guestosid":"ed88d9c7-80c3-11e9-88d7-4437e6b9cc6e","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[{"id":"0d4ecb8f-80c4-11e9-88d7-4437e6b9cc6e","name":"default","description":"Default
> Security
> Group","account":"admin","ingressrule":[],"egressrule":[],"tags":[],"virtualmachineids":[]}],"nic":[{"id":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","networkid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","networkname":"defaultGuestNetwork","netmask":"255.255.255.0","gateway":"192.168.0.1","ipaddress":"192.168.0.61","broadcasturi":"vlan://untagged","traffictype":"Guest","type":"Shared","isdefault":true,"macaddress":"1e:00:13:00:00:16","secondaryip":[],"extradhcpoption":[]}],"hypervisor":"KVM","instancename":"i-2-21-VM","details":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"affinitygroup":[],"displayvm":true,"isdynamicallyscalable":false,"ostypeid":256,"tags":[],"jobid":"92805b41-9391-4211-8eb4-3b7d06f27030","jobstatus":0}
>
> VM Start
> --------
> 2019-06-03 13:02:35,531 DEBUG [c.c.a.t.Request]
> (Work-Job-Executor-73:ctx-625e7d4f job-192/job-193 ctx-8e8437af)
> (logid:b1f99bee) Seq 1-7235595751324128220: Sending { Cmd , MgmtId:
> 75006884826222, via: 1(node001), Ver: v1, Flags: 100011,
> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":21,"name":"i-2-21-VM","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Ubuntu
> 16.04 (64-bit)","platformEmulator":"Ubuntu
> 16.04","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"COaaT8974fN86DWrTYVQrA","params":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"4fb4b946-5d2b-4b66-a29a-cf218886615a","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8
> ","isManaged":false}},"name":"ROOT-21","size":5368709120,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","type":"ROOT","_details":{"storageHost":"192.168.0.34","managed":"false","storagePort":"2049","volumeSize":"5368709120"}},{"data":{"org.
> apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false,"bootable":false,"directDownload":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false"},"uuid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","ip":"192.168.0.61","netmask":"255.255.255.0","gateway":"192.168.0.1","mac":"1e:00:13:00:00:16","dns1":"1.1.1.1","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true}],"guestOsDetails":{}},"hostIp":"192.168.0.28","executeInSequence":false,"wait":0}}]
> }
> 2019-06-03 13:02:36,231 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-6:null) (logid:) Seq 1-7235595751324128220:
> Processing: { Ans: , MgmtId: 75006884826222, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":21,"name":"i-2-21-VM","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Ubuntu
> 16.04 (64-bit)","platformEmulator":"Ubuntu
> 16.04","bootArgs":"","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"COaaT8974fN86DWrTYVQrA","vncAddr":"192.168.0.28","params":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"4fb4b946-5d2b-4b66-a29a-cf218886615a","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"57007f6f-ffe4-3350-b4e9-61fae258bad8","id":2,"poolType":"NetworkFilesystem","host":"192.168.0.34","path":"/export/primary","port":2049,"url":"NetworkFilesystem://
> 192.168.0.34/export/primary/?ROLE=Primary&STOREUUID=57007f6f-ffe4-3350-b4e9-61fae258bad8
> ","isManaged":false}},"name":"ROOT-21","size":5368709120,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","volumeId":21,"vmName":"i-2-21-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","id":21,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"hypervisorType":"KVM"}},"diskSeq":0,"path":"9537daa2-9bc3-45ca-8aaf-8ca2cc703056","type":"ROOT","_details":{"storageHost":"192.168.0.34","managed":"false","storagePort":"2049","volumeSize":"5368709120"}},{"data":{"org.
> apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false,"bootable":false,"directDownload":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false"},"uuid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","ip":"192.168.0.61","netmask":"255.255.255.0","gateway":"192.168.0.1","mac":"1e:00:13:00:00:16","dns1":"1.1.1.1","dns2":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isSecurityGroupEnabled":true}],"guestOsDetails":{}},"result":true,"wait":0}}]
> }
> 2019-06-03 13:02:36,648 DEBUG [c.c.n.s.SecurityGroupManagerImpl]
> (SecGrp-Worker-34:ctx-1a1c6a04) (logid:4f0cd9ce) SecurityGroupManager v2:
> sending ruleset update for vm i-2-21-VM:ingress num rules=0:egress num
> rules=0 num cidrs=0 sig=d41d8cd98f00b204e9800998ecf8427e
> 2019-06-03 13:02:36,648 DEBUG [c.c.a.t.Request]
> (SecGrp-Worker-34:ctx-1a1c6a04) (logid:4f0cd9ce) Seq 1-7235595751324128221:
> Sending { Cmd , MgmtId: 75006884826222, via: 1(node001), Ver: v1, Flags:
> 100111,
> [{"com.cloud.agent.api.SecurityGroupRulesCmd":{"guestIp":"192.168.0.61","vmName":"i-2-21-VM","guestMac":"1e:00:13:00:00:16","signature":"d41d8cd98f00b204e9800998ecf8427e","seqNum":2,"vmId":21,"msId":75006884826222,"ingressRuleSet":[],"egressRuleSet":[],"wait":0}}]
> }
> 2019-06-03 13:02:36,778 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Work-Job-Executor-73:ctx-625e7d4f job-192/job-193 ctx-8e8437af)
> (logid:b1f99bee) Start completed for VM VM[User|i-2-21-VM]
> 2019-06-03 13:02:37,012 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-53:ctx-9ad730ea job-192 ctx-b612185b) (logid:b1f99bee)
> Complete async job-192, jobStatus: SUCCEEDED, resultCode: 0, result:
> org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{"id":"4fb4b946-5d2b-4b66-a29a-cf218886615a","name":"vmu001","displayname":"vmu001","account":"admin","userid":"0d19d291-80c4-11e9-88d7-4437e6b9cc6e","username":"admin","domainid":"e20d260a-80c0-11e9-88d7-4437e6b9cc6e","domain":"ROOT","created":"2019-06-03T12:52:21+0200","state":"Running","haenable":false,"zoneid":"0d0c2a76-42e7-4ac7-9037-3712448fb0b0","zonename":"Zone1","hostid":"192c59bd-aa9a-47fa-9e9c-63df100d4449","hostname":"node001","templateid":"ef1cda00-68e7-4958-b35a-3df95ba8c029","templatename":"ubuntu
> 16.04.6 server","templatedisplaytext":"ubuntu 16.04.6
> server","passwordenabled":false,"serviceofferingid":"c1f3654e-e7c7-4db5-b064-948eb2bbe517","serviceofferingname":"Small
> Instance","cpunumber":1,"cpuspeed":500,"memory":512,"cpuused":"1.99%","networkkbsread":3,"networkkbswrite":1,"diskkbsread":0,"diskkbswrite":864,"memorykbs":524288,"memoryintfreekbs":279324,"memorytargetkbs":524288,"diskioread":0,"diskiowrite":99,"guestosid":"ed88d9c7-80c3-11e9-88d7-4437e6b9cc6e","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[{"id":"0d4ecb8f-80c4-11e9-88d7-4437e6b9cc6e","name":"default","description":"Default
> Security
> Group","account":"admin","ingressrule":[],"egressrule":[],"tags":[],"virtualmachineids":[]}],"nic":[{"id":"2c65c7d6-7094-4d93-a468-f9a5a0475b3f","networkid":"ee5a7886-2073-43c8-adfd-939511a7d8e9","networkname":"defaultGuestNetwork","netmask":"255.255.255.0","gateway":"192.168.0.1","ipaddress":"192.168.0.61","broadcasturi":"vlan://untagged","traffictype":"Guest","type":"Shared","isdefault":true,"macaddress":"1e:00:13:00:00:16","secondaryip":[],"extradhcpoption":[]}],"hypervisor":"KVM","instancename":"i-2-21-VM","details":{"keyboard":"us","Message.ReservedCapacityFreed.Flag":"false"},"affinitygroup":[],"displayvm":true,"isdynamicallyscalable":false,"ostypeid":256,"tags":[],"jobid":"b1f99bee-ce28-4a06-8929-8549806d60ba","jobstatus":0}
> root@mgmtserver:~#
>
>
> Thanks in advance for your help
> Regards
> Daniel
>
>